Tagged: Benchmark

109 articles

Alphabet lost $500B in a week despite shipping its best model, says Mostaque

Alphabet shed $500 billion in a week even as it released its best model yet. Emad Mostaque argues search ads will peak in Q1 2027, and that the benchmark crown comes with an asterisk. Source: finance.biggo.com

LLMs parse public health policy debates in German news

Researchers show that GPT-5 and Gemini can extract and classify policy-framing quotes from German news articles. Their F1-scores come close to 80 percent when measured against a human benchmark. Source: bioengineer.org

Kernel-Tuned ML Pushes Network Intrusion Detection to Nearly 99% Accuracy

A new study reports that Random Forest and Support Vector Machines with Gaussian and Sigmoid kernels, paired with systematic data preprocessing, reached accuracy of up to 99.87% on benchmark data. The results suggest careful kernel tuning can significantly boost network intrusion detection performance. Source: bioengineer.org

Revamped Artificial Analysis index widens gap with open models

Artificial Analysis has overhauled its Intelligence Index, reshuffling the leaderboard. As a result, Kimi K3.1 Pro slipped five places in the rankings. Source: the-decoder.com

Open-data ML framework nears perfect breast cancer diagnosis on benchmark

Researchers in China have built a fully reproducible, open-data framework that combines interpretable machine learning with quantitative cytopathology. On a benchmark dataset, it reaches near-perfect accuracy in diagnosing breast cancer. Source: bioengineer.org

Anthropic study: AI agents still can't do real-world scientific research

Cutting-edge coding agents are turning machine learning research into automatically executable workflows, but most existing studies focus on narrow benchmark settings. Anthropic's research indicates that AI still falls short of conducting genuine, real-world scientific research. Source: eu.36kr.com

HeyGen's Video 1 takes top spot on silent text-to-video leaderboard

AI video startup HeyGen announced on 22 October 2026 that Video 1, its first general-purpose video model, ranked first for text-to-video on the Artificial Analysis leaderboard for silent video. The result marks a strong debut for the company's move beyond avatar-style content. Source: app.dealroom.co

Martingale Launches Raven Alpha for AI Model Drift Detection

Martingale, founded by three Carnegie Mellon computer science students, has released Raven Alpha, a model drift detection method. It ranks first in benchmark tests. Source: usatoday.com

Prompting strategies for LLMs show no stable advantage in benchmark

A comparative benchmark found that LLM prompting strategies were competitive with logistic regression and XGBoost at some operating points. However, they showed no consistently stable advantage over those classical methods. Source: bioengineer.org

Meta debuts Muse Wonder AI model family, topping rivals on benchmark

Meta has unveiled Muse Wonder, a new family of AI models. On the Artificial Analysis leaderboard it edges past Gemini 4, GPT-Astra and Fable 5, putting Meta back in the race with OpenAI and Anthropic. Source: mashable.com

Benchmark Tests AI Agents on Writing Econometric Code

Large language models are increasingly writing the code behind empirical research. A new column reports a benchmark of 40 econometric tasks across Stata, R and other tools. Source: cepr.org

Build Your Own Cursor vs GitHub Copilot Benchmark in 12 Steps

A hands-on tutorial walks through creating a reproducible comparison of Cursor and GitHub Copilot inside your own repository. It includes 12 steps, scoring scripts, and a complete working project. Source: shattered.io

Kyutai unveils REKI, a 2.4B VLA model with built-in memory for robot manipulation

Kyutai's REKI is a 2.4B-parameter vision-language-action model trained on public data. It scores 90.4 on the LIBERO-Plus benchmark and includes built-in memory for more efficient robot manipulation. Source: marktechpost.com

Crun AI debuts leaderboard ranking LLM, video, image and audio models

Crun AI has launched a leaderboard that lets users compare leading models across LLM, video, image and audio categories. Models are ranked side by side on both performance and price. Source: financialcontent.com

Gemini 4 Argon targets robotics with physical reasoning and zero-shot control

Gemini 4 Argon is pitched for robotics, topping the ERQA benchmark. Its highlighted capabilities include physical reasoning, zero-shot control, Real2Sim and agentic workflows. Source: developers.googleblog.com

Arena raises $200M Series B at $3.1B valuation

San Francisco-based Arena, which runs an AI evaluation platform and leaderboard measuring real-world capabilities of models and agents, has raised $200 million in a Series B round at a $3.1 billion valuation. Source: finsmes.com

Best practices for building AI agents: tools, security, evaluation

The article lays out three core principles for agentic AI architecture. It also covers patterns for tool permissions, retrieval access control, prompt caching, human review and evals. Source: medium.com

ROBOTERA takes top spot on embodied AI benchmark RoboDojo

ROBOTERA's World Action Model, VPP2 (Video Prediction Policy 2), has ranked first on the RoboDojo benchmark. According to the announcement, it achieved the result without additional data or agent-based recursive self-improvement. Source: prnewswire.com

AI leaderboard Arena nearly doubles valuation to $3.1B in 10 months

Arena began in 2023 as a UC Berkeley research project that crowdsourced rankings of AI models. It has now raised a $200 million Series B at a $3.1 billion valuation. Source: techcrunch.com

Copilot wins GitHub's own AI code review benchmark, independent test disagrees

GitHub's ReviewBench is meant to be a shared standard for measuring AI code reviewers. However, the team behind it ran the initial tests for every competing tool themselves, and an independent evaluation paints a different picture. Source: thenewstack.io