PaperBench: Can AI Agents Actually Replicate AI Research?
Model's Replication Scores
Average Replication Scores on PaperBench
Madhu Shantan
Tool Chaos No More: How We’re Measuring Model-Tool Accuracy in the Age of MCP
Introduction
Picture this scenario: you’ve built an AI agent, given it access to dozens of tools, and deployed it to handle a complex workflow. But instead of executing queries crisply, it’s making redundant tool calls, burning API credits needlessly, and overcomplicating straightforward processes.
This isn’t just an
User Simulation in AI: From Rule-Based Models to LLM-Powered Realism
What if you could test your AI system with thousands of diverse users without recruiting a single person? User Simulation makes this possible. Simulating human users - a fundamental application of AI has driven progress in both research and industry. By allowing machines to imitate real user interactions, user simulation
Do Language Models Know That They're Being Evaluated?
Picture this scenario: You’re very new to AI, exploring chatgpt by testing its capabilities on various topics, expecting honest answers unaware that behind the scenes, it already figured out that it’s being tested and is subtly changing its behaviour to ace your tests.
This feels like a subtle
AlphaEvolve : AI for Scientific Discovery
Introduction
Consider a scenario: You're facing a complex optimization challenge with no known solution - the kind that requires inventing entirely new algorithms, not just tweaking existing ones. There's no textbook answer, no established approach. Existing coding models like Claude, Gemini 2.5 can implement known