Skip to content
Hippocampus's Garden
Machine learning, software, books, notes

Top 5 AI Papers of 2025

This article picks five notable AI papers from 2025 and summarizes their key ideas and limitations, including reinforcement learning for reasoning, agent benchmarks, long‑task metrics, and a statistical explanation of hallucinations.

2025-12-3111 min readEN
Table of contents

2025 was a year when post‑training for large language models (LLMs) made big steps forward. New techniques helped models think more carefully, while new benchmarks and metrics tried to measure reliability and safe use. In industry we saw strong systems like GPT‑5, Gemini 3 and Claude 4.5, and agent tools such as Cursor and Claude Code changed how people code and learn. Playing with these products is the best way to understand them, but to see what problems remain and what might happen in the next few years, it also helps to look at research. This article picks five papers from 2025 that I found important. It explains their main ideas, why they matter, and where they fall short, for engineers and people in tech.

If you are interested, you can also read my 2024 review.

DeepSeek‑R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

This paper from DeepSeek AI tries to pull out long chains of reasoning from an LLM without using any human‑written reasoning examples. The authors start with DeepSeek V3, their base model, and use group relative policy optimization (GRPO) 1 to do reinforcement learning. During training they only give a reward when the model’s answer is correct on math or programming questions. They use special tags like <think/> and <answer/> to tell the model to think first and then answer. This setup makes the model develop its own long chain‑of‑thought (CoT). For example, on the AIME 2024 math competition the pass@1 score jumps from 15.6% to 71%.

Two points make this work interesting:

  1. Learning without human annotations. Before R1, most LLM post‑training used supervised fine‑tuning (SFT) and reinforcement learning from human feedback (RLHF). These methods need labelled data and can introduce bias. The authors build R1‑zero, a model that only uses correct/incorrect rewards from verifiable tasks like math. This approach is called reinforcement learning with verifiable rewards (RLVR), which the Tulu 3 paper2 originally proposed, and shows that pure RL can reach the level of OpenAI’s o1 model. Note that R1‑zero still had problems like mixing languages and strange wording, so the authors also released R1, which adds SFT to fix these.
  2. Emergent long thinking. During RL training the model starts to spend more time thinking. It changes its plan many times until it finds the right answer. On hard problems it can generate thousands of tokens of reasoning. Before R1 came out, many people were trying to reproduce the o1 model. This paper provided a clear recipe for making a model think longer. The paper also discusses other methods, like process reward models and Monte‑Carlo tree search, that did not work as well.

DeepSeek AI released the R1 models and weights openly. In January 2025 this release triggered what some people called the “DeepSeek shock.” The DeepSeek mobile app jumped to the top of the AppStore in several countries, even above ChatGPT, and Nvidia’s share price fell. Many Chinese companies, including Alibaba, Moonshot AI and Z.ai, started releasing open‑weight models with performance close to closed models.

You can read more about this paper in my blog post from March 2025.

Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?

This paper looks at the limits of RLVR, the idea used in DeepSeek R1 and Tulu 3. The authors test several RLVR models, including DeepSeek R1 and OpenAI’s o1, on math, coding and visual reasoning benchmarks. They measure pass@k up to very large k. They find that RLVR models are better than the base model when k is small, but when k grows the base model catches up and eventually beats the RLVR model. Analysis shows that RLVR models mainly sample answers that already exist in the base model. In other words, RLVR improves sampling efficiency but does not create new reasoning patterns.

Why does this happen? At the core is how hard exploration becomes under a 0/1 reward. In RLVR, you only get reward for an exactly correct output, so when the base model cannot solve a problem, there is no gradient signal to learn from. The model therefore concentrates on familiar, high‑hit‑rate patterns, and its output distribution collapses sharply around answers that already receive reward. This reduces the diversity of reasoning patterns that the base model originally had, so at larger k the base model’s diversity can win out and you see a pass@k reversal. By contrast, distillation can inject the teacher’s reasoning patterns into the student.

As paths toward acquiring genuinely new capabilities, the authors point to approaches such as process rewards that score intermediate steps, credit assignment via value functions, curriculum learning with large and dynamic data, stronger exploration methods, and agent frameworks that incorporate tool use. Credit assignment with value functions is something Ilya Sutskever also called promising in a recent interview.

τ‑bench: A Benchmark for Tool‑Agent‑User Interaction in Real‑World Domains

Sierra AI points out that many agent benchmarks test only single turns or single API calls. In real life, agents need to talk over many turns, follow domain rules, and use tools to change the state. τ‑bench simulates these interactions with three parts: a tool (API), an agent and a user (simulated by an LLM). It judges success by checking both the conversation and the final database state.

Key features include:

  • Real conversations and tools. There are domain‑specific APIs for airline and retail tasks, and a local JSON database. A simulated user asks questions according to a scenario, creating multi‑turn dialogue.
  • Evaluation based on database state. In addition to LLM-as-a-judge, τ‑bench checks whether the agent actually changes the state correctly. This fits real applications where the correct final state matters.
  • Reliability metric pass^k. Unlike pass@k, which counts at least one success, pass^k requires success on all k runs. It measures how consistent an agent is3.

When the team evaluated τ‑bench with GPT‑4o, which was state‑of‑the‑art at the time, the average success rate was 48%, highlighting how challenging it still was to use these systems as agents. However, a closer look at the failures showed that some tasks were effectively impossible because of errors in the user simulator. In response, the Sierra AI team proposed τ²‑bench4, which improves simulation fidelity by requiring the user side to use tools as well. Since then, it has been widely referenced as a benchmark for measuring tool‑use capability in AI agents.

If you look at scores from different groups, be careful. As explained in this article, because τ‑bench is open source, results can depend on prompt tuning and time budgets, so comparisons are often not apples‑to‑apples.

Measuring AI Ability to Complete Long Tasks

The non‑profit organization METR (Model Evaluation & Threat Research) proposes a metric based on time to finish a task, not just accuracy. They look at many software tasks, from research with Wikipedia to complicated coding. Human experts record how long each task takes. Then they see how long an AI model can achieve a 50 % success rate on the same tasks. They call this the 50% task completion time horizon.

Some main observations:

  • In March 2025, the best model (Claude 3.7 Sonnet) had a 50% horizon of 59 minutes. Models do almost perfectly on tasks that take humans less than four minutes, but do very poorly on tasks that take humans more than four hours. This shows that models still struggle with long memory and recovering from errors.
  • Since 2019 the horizon has doubled roughly every seven months. If this trend continues, models might do week‑long tasks by around 2028–2031. But extrapolating is risky; algorithm breakthroughs or changes could break the trend.
  • Growth comes not just from better reasoning but from being more reliable and better at using tools. This hints that reinforcement learning and agentic approaches are important.

Image taken from METR.org.

This metric is like an “AI Moore’s Law” in the past 6 years, but there are limitations5:

  1. In late 2025 models like GPT‑5.1‑Codex‑Max reached around 2 hours 53 minutes, and Claude Opus 4.5 reached about 4 hours 49 minutes. However, the 1–4 hour range has only 14 tasks, so the estimates are noisy and easy to manipulate.
  2. Because the task topics are public, labs could train on similar tasks (“benchmaxxing”).
  3. Most tasks are about cybersecurity or machine‑learning competitions, so this metric may not reflect general long‑task ability.

Why Language Models Hallucinate

This paper from OpenAI and Georgia Tech offers a statistical answer to a long‑standing question since LLMs emerged: why they confidently generate content that is not factual. Intuitively, the reason is that current evaluation metrics often favor guessing over being wrong, so when a model does not know, it tends to guess rather than say “I don’t know.”

For example, suppose you ask for someone’s birthday during pretraining. The probability of the exact correct answer is 1/365, but replying “I don’t know” is guaranteed to be counted as incorrect. If the metric is only accuracy, then random guessing can be the better move. Language models are good at pattern‑based generation, such as fluency and grammar, but they cannot reliably learn rare proper nouns or dates. Without consulting an external knowledge base, hallucination can systematically arise.

The authors argue that to help a model recognize its own ignorance and learn to refrain from answering when uncertain (to abstain), we need to evaluate not only accuracy but also honesty.

OpenAI reported a large reduction in hallucinations in GPT‑5, released in August 2025, and it has continued to reduce them in GPT‑5.1 and 5.2. This paper does not go into concrete methods for reducing hallucinations, and the causal link to GPT‑5 is unclear, but I found its core takeaway important: the design of evaluation metrics shapes model behavior, which matters for future quality evaluation and safety.

Concluding Remarks

Overall, what I felt defined 2025 was the success of RLVR, the rapid shift toward LLMs becoming agents, and the emergence of new benchmarks to evaluate those capabilities. I’ll keep watching what developments 2026 brings. Thank you for reading, wishing you a happy new year!

Footnotes

  1. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y.K. Li, Y. Wu, Daya Guo. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. 2024. ↩

  2. Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, Pradeep Dasigi, Hannaneh Hajishirzi. Tulu 3: Pushing Frontiers in Open Language Model Post-Training. COLM. 2025. ↩

  3. For tasks that are easy to verify, pass@k is a common metric. When the cost of a wrong answer is high, pass^k is often more appropriate. See mt previous post for more. ↩

  4. Victor Barres, Honghua Dong, Soham Ray, Xujie Si, Karthik Narasimhan. τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment. 2025. ↩

  5. Shashwat Goel. How to game the METR plot. 2025. ↩

Related posts

Shion Honda

Software engineer based in Rome since February 2026. I work on bringing AI agents into products, and I write about evaluation, context engineering, and the practical edges of LLM systems.