Agentic SEER Reproducibility at Scale
An AI agent reproduces thousands of SEER-based claims and stress-tests their analytical choices, flagging fragile results for human review.
DailyUpdated 2026-11-07 12:20 UTC15 articles
Evals & reliability
Evaluation, benchmarks, observability, guardrails, reliability of agentic systems
Testing LLM Solidity with SolBehave-style oracles
SolBehave shows that compilation is a weak signal for LLM-generated Solidity and that you need behavioral test oracles wired into your agent stack.
Mechanical verification for agentic autoformalization
Using numerical and vacuity checks to keep Lean formalizations honest, beyond “LLM judge” pipelines.
Lessons from MC-EvoBench for self-improving agents
What a controlled Minecraft benchmark exposes about “learning from experience” in embodied agents.
Wiring unlearning into your MLLM stack
How Open-MMUnlearning standardizes methods and metrics for multimodal model unlearning—and what you can actually do with it.
FastBench and ProactiveFrame for high-speed video agents
What FastBench reveals about streaming VLM limits and how to design frame policies that don’t miss fast events
SciExam for ENSO: Agents as Climate Modellers
A new benchmark tests whether AI agents can build valid climate models without ever seeing the grading code.
ScienceClaw and the mechanics of self-evolving agents
How ScienceClaw turns task runs into persistent skills for AI-for-science agents.
Compiling GNNs into ground-truth explainer tests
Gracr replaces trained GNNs with compiled logic to make explainer benchmarks actually sound.
NeutronGym and physics-graded agent loops
How NeutronGym turns neutron instrument design into an executable, physics-grounded gym for LLM agents.
Building agents for HumanoidToolBench
What HumanoidToolBench actually measures and how to architect agents that don’t just pick the right tool but finish the job.
Argo-Bench and the real work of data agents
Inside a 7.5B-row simulated ERP that finally tests agents on end-to-end enterprise workflows.
Routing-Gated Visual Grounding in MoE VLMs
Using expert routing signals to catch hallucinated objects before your VLM answers.
Building Agents with the GameHorizon Suite
How GameHorizon’s multi-horizon gameplay data and evals change how you train and debug agentic systems.
EnigmaForge and the missing question in evals
A new benchmark hides the task inside a story, exposing how brittle our current evaluation and agent designs are.
Live signals on Evals & reliability
- Scientist World: Grow a Research World for AI-Scientists arXiv · 1 days ago · frontier 95%
- FrontierPhysics: Evaluating Agents for End-to-End Physics Research arXiv · 2 days ago · frontier 90%
- Airlock: Bounding the Multi-Agent Blast Radius via Selective Online Containment arXiv · 2 days ago · frontier 85%
- Multimodal-SEA: Benchmarking AI Agent Security under Multimodal Social Engineering Attacks arXiv · 1 days ago · frontier 72%
- LongBanana: An Expert-Verified Benchmark for Long-Context Multi-Reference Image Synthesis arXiv · 2 days ago · frontier 72%
- Benchmarks as Software: Terminal-Bench and the Case for Continuous Benchmarks arXiv · 1 days ago · frontier 71%
- ReproBridge: Building Reproducibility into ML Research Artifacts arXiv · 1 days ago · frontier 69%
- The Gold Rush in AI4Math, II: What are signatures of AI-assisted Proofs? arXiv · 1 days ago · frontier 81%
- @OpenAI: A new way to measure how AI agents handle the routine analyses of wet-lab science—BioBasicsBench. X · ai radar · 20 h ago · frontier 60%
- Measuring Risk, Not Harm: Detecting Psychological Harm Risk Factors in LLM Conversations arXiv · 2 days ago · frontier 67%
- Failure Prediction and Classification in Vision-Language-Action Models Via Calibrated Latent Probing arXiv · 1 days ago · frontier 79%
- XStack: A Benchmark for ML Repo Migration arXiv · 1 days ago · frontier 73%
- LLMs Don't Know What They Know: Benchmarking Temporal Self-Knowledge for Financial Backtesting arXiv · 2 days ago · frontier 72%
- LeanContext: Disentangling Mathematical Resources from Local Formal Context in Lean Whole-Proof Generation arXiv · 1 days ago · frontier 73%
- MedPhysBench: A Benchmark for Evaluating Language Models on Medical Physics Calculations and Error Detection arXiv · 3 days ago · frontier 54%