What SEAR is trying to measure

SEAR (Self-Evolving Agentic Robotics) is a framework and benchmark for robotic agents that use frontier language models to improve themselves without human-in-the-loop tuning 1.

The core loop SEAR is interested in is:

  • The agent writes or modifies robot control code.
  • It runs that code in a simulator or on hardware.
  • It retains successful behaviors as skills or memories that should matter on the next task.

The paper asks two very concrete questions 1:

  1. Is the agent actually learning, or just getting more chances to roll the dice?
  2. Does whatever it “learned” still help when:
  • The task changes,
  • The simulator changes, and
  • You go from sim to a real robot?

To get signal on this, SEAR defines:

  • A framework: how the agent is allowed to interact with robots and training runs.
  • A benchmark: tasks, modes, and controls that separate real retained experience from confounders.

Across seven frontier language models, 330 simulated tasks, five physical tasks, and over four years of agent runtime, they use this setup to stress-test what “self-evolving” actually buys you 1.

The three SEAR agent modes

SEAR constrains how the language-model agent plugs into your control stack. It supports three distinct modes of operation 1:

  1. Code-as-policy
  2. Agent-as-policy
  3. Auto-research

Each mode enforces a different control flow between “LLM thinking,” executable policy, and outer-loop learning.

1. Code-as-policy: the agent is your policy compiler

In code-as-policy, the LLM does not sit in the inner control loop. Instead 1:

  • The agent writes control code that *is* the policy.
  • That code is then executed directly by the robot or simulator.
  • When a piece of code performs well, it is kept as a skill or shared routine in the agent’s memory for future tasks.

Mechanically, this is:

  • An outer loop: propose new policy code → evaluate → keep or discard.
  • An inner loop: run that code at control frequency, no LLM calls.

This mode is good for isolating:

  • How well LLMs can synthesize reusable control programs.
  • How much task transfer you get from storing and reusing routines 1.

SEAR finds that control code and shared routines do transfer across tasks and simulators, while task-specific solutions do not 1. That is a direct constraint on how you should structure the memory: prioritize generic routines, not one-off hacks.

2. Agent-as-policy: the agent lives in the control loop

In agent-as-policy, the LLM *is* the policy: it acts live in the control loop 1.

  • At each control step (or macro-step), observations flow through the LLM.
  • The LLM outputs actions or tool calls.
  • It can use its memory to decide what to do next.

Relative to code-as-policy, the differences are:

  • No static policy code; behavior is emergent from the sequence of LLM calls.
  • Memory stores experience, instructions, and artifacts that help the LLM plan on future episodes.

SEAR uses this mode to test:

  • Whether acting live lets some models evolve more effectively 1.
  • How useful perception within the loop is, compared to pure reasoning + tools.

One of SEAR’s findings is that different models evolve more effectively in different ways: some perform best acting live in the loop, others do better when they first write code and then step out 1. For a builder, this is a warning: the optimal control architecture depends on model idiosyncrasies, not just on the task.

3. Auto-research: the agent as a VLA fine-tuner

The third mode, auto-research, shifts the agent’s role again: it works as an autonomous researcher that fine-tunes a vision-language-action (VLA) model 1.

Here:

  • The policy is a separate VLA model.
  • The LLM agent:
  • Designs experiments,
  • Chooses data,
  • Adjusts the VLA training process.

The VLA model is then deployed as the policy. The “self-evolving” part is now:

  • At the model-weight level (via fine-tuning), not just via code or memories.

SEAR uses this to probe auto-research as a realistic path to recursive self-improvement in robotics: can a general coding-and-planning agent bootstrap and keep improving a downstream control model, given only verifiable feedback 1?

Benchmark design: separating learning from re-rolling

To answer whether an agent is actually learning, SEAR’s benchmark designs explicit controls to separate retained experience from other confounders 1.

They vary:

  • Retained experience vs repeated attempts
  • By controlling what the agent is allowed to keep from past runs (skills, code, memories).
  • Task familiarity
  • Familiar vs novel tasks, to see if performance gains are narrow or broad.
  • Access to privileged simulator state
  • With vs without extra state that would not be available on the real robot.
  • Budget
  • Small vs large time budgets (e.g., numbers of episodes / tokens / calls), to see whether performance gains scale with budget.
  • Luck
  • Through multiple runs and baselines that hold experience fixed while varying randomness 1.

The central *learning-curve* result is:

The learning curve of a robotic agent depends both on the overall time budget and the task difficulty relative to model capability 1.

They observe that:

  • On harder tasks, longer sessions are worthwhile: more time → more evolution → better performance.
  • On easier tasks, simply re-rolling short attempts (i.e., more fresh restarts without retained experience) does “just as well” 1.

Operationally, this means:

  • You need budget-aware orchestration: don’t allocate long evolving runs to simple tasks that your model can already solve by brute-force exploration.
  • You should use SEAR-style controls (budgeted runs with and without memory) to convince yourself that your “self-improving” system is not just a sophisticated re-roll machine.

Transfer: what actually carries over?

One explicit SEAR finding is about transfer 1:

  • Control code and shared routines transfer across tasks and simulators.
  • Task-specific solutions do not transfer.

Within their 330 simulated and five real tasks, they also find, on a real robot:

  • A performance gap between agent-designed practice scenes and novel human-designed scenes 1.

Two implications for an agentic robotics stack:

  • If you let the agent design practice tasks or simulations, expect an overfitting gap to human-curated environments.
  • The content of what you store in memory matters: generic routines and reusable code help; ultra-specific hacks don’t.

This maps naturally to work like ChunkAct in agentic robotics, which also leans on reusable routines. ChunkAct observes that agentic robotics is often inefficient because every “primitive” action requires a full LLM reasoning step over the entire interaction history, making episode cost scale with the number of primitives 2. They show that:

  • Once a sub-goal is selected, the sequence of primitives usually follows a routine already prescribed by memory.
  • By compiling these routines into action chunks offline and running them autonomously online, they reduce LLM calls and tokens dramatically: on LIBERO-Pro, ChunkAct improves task success by 5.0 points while using 2.9× fewer LLM calls, 4.4× fewer input tokens, and 2.3× fewer output tokens compared to a prior agentic baseline 2.

SEAR’s finding that shared routines transfer across tasks and simulators lines up with the idea that you should:

  • Represent evolving knowledge at the level of chunks / routines, not unstructured logs.
  • Expect those routines to be the primary unit of transfer and efficiency.

Models: multimodal isn’t automatically better

SEAR runs seven frontier language models across its tasks and hours of runtime 1. One concrete result is about multimodality vs reasoning:

  • A text-only model keeps pace with a comparable multimodal one in their setup 1.
  • Perception helps with error assessment, but agentic reasoning and tool use can matter more 1.

Design implications:

  • For many robotic tasks in this framework, lifting perception into the LLM may not be the main bottleneck.
  • Investing in:
  • Better agentic reasoning,
  • Smarter tooling, and
  • More structured memory of code and routines,

can deliver as much or more benefit than pushing to a more expensive multimodal backbone.

This resonates with Tev’s view on long-horizon agents more broadly. Tev introduces Tev-Gym, a long-horizon training dataset with 1,170 task environments across 7 domains with dense, verifiable progress rewards 3. They construct a latent world—a hidden specification of domain, workflow, and expected state after each step—and use it to provide dense per-step reward via progress-potential reward shaping (PPRS) 3.

When they train a 27B-parameter model (Tev-27B) using this recipe, it:

  • Achieves substantial gains on the Agents’ Last Exam computing and math sciences domains, matching specific larger frontier models at Pass@5 3.
  • Generalizes to Terminal-Bench subsets, surpassing some contemporary baselines and reaching the level of a strong proprietary model on mid-horizon tasks, despite training only on ALE-related data 3.

Together, SEAR and Tev suggest:

  • Careful environment and reward design plus agentic reasoning can significantly close gaps to larger or more multimodal models in long-horizon settings—including robotics.

Self-evolving harness vs self-evolving robotics

SEAR is about robots; Self-Evolving Agent Harness looks at the agent’s own scaffolding 6.

Harness self-evolution:

  • Revises an agent’s scaffolding (prompts, memory, workflows, executable components).
  • Keeps the model weights fixed.
  • Runs a feedback-driven process where accepted revisions persist across runs 6.

The framework:

  • Separates update targets from proposal and admission mechanisms.
  • Distinguishes:
  • Changes to interpreted state (e.g., prompts, instructions),
  • Revisions to tools and code,
  • Repository-level control logic,
  • Plugin composition 6.

They analyze this as an evolutionary optimization process and, under explicit assumptions, derive:

  • An expected generalization bound,
  • A retention bound tied to historical regression budgets,
  • A high-probability admission rule using fresh, independent validation 6.

SEAR and Harness are complementary:

  • SEAR focuses on the robot-and-task level (policies, routines, VLA fine-tuning).
  • Harness focuses on the agent’s meta-structure (prompts, workflows, tooling).

A plausible architecture for a “self-evolving robotics stack” would need both:

  • SEAR-style loops for evolving robot behavior.
  • Harness-style mechanisms for evolving the agent’s own software shell, with explicit guards against regressions.

Long-horizon autonomy and dense progress

A recurring challenge in SEAR-like settings is long-horizon autonomy: robots that run for hours or days and meaningfully improve themselves.

Tev’s latent-world-first design offers a recipe relevant to SEAR-style auto-research 3:

  • Build a complete hidden specification of tasks and workflows (latent world).
  • Render agent-facing environments progressively from it, with controllable complexity and a natural progress metric 3.
  • Turn that metric into dense rewards via PPRS to make training feasible on otherwise intractable tasks 3.

SEAR doesn’t spell out similar latent-world constructions, but its emphasis on:

  • Careful benchmark design,
  • Controlled budget and difficulty, and
  • Cross-simulator and sim-to-real transfer 1,

fits naturally into this broader direction: designing environments where agents can reliably measure incremental progress and improve policies or VLAs through their own experimentation.

Why SEAR matters for practitioners

From the perspective of someone shipping robot agents, SEAR gives you:

  • A taxonomy of roles for your LLM:
  • Compiler of policies (code-as-policy).
  • Online controller (agent-as-policy).
  • Auto-researcher / trainer (auto-research).
  • A set of controls you can implement in your eval harness:
  • With vs without memory, at fixed budgets.
  • Familiar vs novel tasks.
  • With vs without privileged sim state.
  • Concrete signals about what to remember:
  • General-purpose control code and shared routines.
  • Avoid over-indexing on one-off strategies.

Its empirical findings are directly operationalized:

  • For easy tasks, don’t waste budget on long evolving sessions; preferred strategy can be short attempts and re-tries 1.
  • For hard tasks, long evolving runs with persistent memory can outperform brute-force re-rolling 1.
  • When choosing between text-only vs multimodal models:
  • Don’t assume multimodal always wins; consider whether better reasoning + tools might match or exceed it 1.

Finally, SEAR’s explicit sim-to-real gap—where agent-designed practice scenes underperform on novel human-designed scenes—reminds you that:

  • Curriculum and environment design are learnable levers, but you must continuously validate on out-of-distribution, human-designed testbeds 1.

What is not documented

The sources above do not document:

  • Specific robot platforms, simulators, or toolchains used in SEAR.
  • Exact language model names, sizes, or detailed architectures beyond “frontier language models” and, for Tev, a 27B-parameter model 1, 3.
  • Concrete implementation details of the SEAR memory system (data formats, retrieval algorithms, etc.).
  • Precise numerical performance curves, per-task metrics, or error bars for SEAR’s experiments, beyond high-level qualitative findings 1.
  • Low-level control interfaces between LLM agents and robots (action frequencies, safety constraints, fallback mechanisms).