Reading MiMo’s RL run as a systems engineer

What Xiaomi actually said

All we concretely know about Xiaomi’s current RL effort comes from a short thread by Fuli Luo about MiMo-V2.6:

• They spent “nearly half a year” focused on “one problem: how far RL can scale” 1.
• MiMo-V2.6 is “in the middle of its RL run right now” 1.
• They scaled three things 1:

1. Compute: “~2B tokens per step, 1568 prompts × 16 rollouts, fully async.”

2. Environments and harnesses: “multi-task agentic RL, mixed across multiple harnesses in one run.”

3. Grader compute: “agentic in-group credit assignment, with test-case and rubric-based rewards.”

• They plan to “open-source the details piece by piece over the coming weeks” and are livestreaming the run at mimo.xiaomi.com/rl 1.
• Separately, Luo states: “We believe RL is one of the most scalable and efficient paths toward self-improvement” and they hope the livestream “sparks the research community’s interest in the core challenges of scaling RL” 1.

That’s it for facts. Everything else in this piece is interpretation and design questions for builders, not a description of what MiMo actually does internally.

Decomposing the MiMo RL loop

Taken at face value, the description implies a fairly standard high-level RL loop, scaled along three axes.

You can mentally model it in four stages per “step”:

1. Prompt selection / environment sampling

- ~1,568 prompts are used per step 1.

- These likely correspond to tasks drawn from multiple “harnesses” (evaluation frameworks or task suites), since the run is “multi-task agentic RL, mixed across multiple harnesses in one run” 1.

2. Rollout generation (experience collection)

- Each prompt gets 16 rollouts, so you have 1,568 × 16 trajectories per step 1.

- Rollouts are “fully async” 1, which means the system is not waiting for all trajectories to finish in lockstep before proceeding.

3. Grading / reward computation

- Reward is not a simple heuristic; it comes from “grader compute” that is itself agentic 1.

- This grading uses test-case and rubric-based rewards and performs “in-group credit assignment” 1.

4. Parameter update

- The whole process uses about 2B tokens per step 1.

- The exact optimizer, policy architecture, and update schedule are not documented, but some form of policy update must exist for this to be RL.

From a systems lens, those bullet points reveal where the engineering weight went: scaling throughput (2B tokens/step, async rollouts), task diversity (multi-harness), and reward quality/complexity (agentic graders with rubrics).

Axis 1: Scaling compute and asynchronous rollouts

The first dimension is raw throughput:

• “~2B tokens per step” 1.
• “1568 prompts × 16 rollouts” 1.
• “Fully async” rollout execution 1.

If you’re building RL-for-agents yourself, these details translate into several concrete design concerns (again, as general engineering questions, not MiMo specifics):

1. Throughput vs. credit quality

Using billions of tokens per step suggests a bias toward *many* trajectories rather than a small number of carefully curated ones. The trade-off you’ll hit in your own systems is:

• more trajectories → better coverage and more gradient signal,
• but potentially more noisy or redundant supervision unless your grading is strong.

MiMo’s explicit investment in complex graders (see below) looks like a direct counterweight to that.

2. Async orchestration

“Fully async” means you *could* architect your pipeline so that:

• environments/harnesses enqueue tasks,
• rollouts are generated by a fleet of workers as capacity frees up,
• results stream back to a grading service,
• updates happen continuously as batches of graded trajectories arrive.

For your own stack, the practical questions are:

• How do you keep the policy reasonably “fresh” across all workers when updates are being applied while new trajectories are still being generated?
• How do you prevent stale policies from dominating the experience buffer?
• How do you handle variable-length trajectories without back-pressure stalling the whole system?

MiMo’s “fully async” claim 1 is notable because implementing this without instability is a major systems challenge in large-scale RL.

Axis 2: Multi-task, multi-harness agentic RL

The second axis: environments and harnesses.

The run is described as:

• “multi-task agentic RL” 1,
• “mixed across multiple harnesses in one run” 1.

For a builder, the interesting part is multi-harness in a *single* RL run. This suggests a design where:

• A “harness” is some combination of:

- task distribution,

- tools / APIs available to the agent,

- evaluation protocol,

- and maybe observation/action formatting.

• Instead of training separate specialized policies per harness, there is a shared agent being optimized across all of them concurrently.

If you tried something similar, you’d run into:

• Routing and conditioning

You need some way to let the policy know which harness it’s in (or what tools and success criteria apply). That might be through natural language system prompts, structured metadata, or both. The MiMo posts don’t document how this is done.

• Cross-harness interference vs. transfer

A single policy trained across harnesses could:

- benefit from shared skills and patterns,

- or suffer from conflicting objectives and reward scales.

Because Xiaomi explicitly calls out harness scaling as one of only three pillars 1, it’s reasonable to assume managing this interference/transfer problem is central to the project, but there are no public details yet.

Design questions for your own stack:

• Do you co-train on your code agent harness, your planning agent harness, and your retrieval agent harness in one loop, or keep them separate and distill later?
• How do you normalize reward magnitudes across harnesses so the RL algorithm doesn’t over-fit to “easy” or high-variance environments?

The MiMo description doesn’t say how they address this; it just confirms they run “multi-task agentic RL” with “multiple harnesses in one run” 1.

Axis 3: Agentic graders and in‑group credit assignment

The third axis is grader compute. This is where MiMo’s description deviates most from vanilla RL:

• Graders are agentic themselves 1.
• They perform “in-group credit assignment” 1.
• Reward is based on test cases and rubrics 1.

We don’t know what “in-group” means here in detail. It might be:

• credit assigned across different rollouts for the same prompt,
• or across sub-agents in a multi-agent setup,
• or across steps in a tool-using workflow.

But we do know the grading pipeline is itself an agentic system and that it uses both:

• test-case-based rewards – suggesting programmatic checks of outputs, and
• rubric-based rewards – suggesting more qualitative, possibly natural-language evaluation criteria 1.

If you’re building something similar, the main implications are:

1. Reward model is a full agent, not a scalar heuristic

Graders that are themselves agents can:

- read task descriptions,

- inspect intermediate artifacts,

- and reason according to a rubric.

That’s conceptually different from a fixed classifier or simple success/fail rule. It also means your RL loop is nested inside another agent loop, which raises stability and alignment questions.

2. Test cases + rubrics

Combining both forms of evaluation could give:

- sharp signals when test cases exist (e.g., code problems, structured answers),

- plus softer rubric scores when outputs aren’t easily auto-graded.

Again, we don’t know how MiMo mixes these signals numerically; we only know both exist and are used for credit assignment 1.

3. Scaling grader compute

Making grader compute one of the three main scaling axes 1 implicitly acknowledges that:

- it’s not enough to scale the policy;

- you must also scale how *well* you can tell if a trajectory was good.

For your own stack, that means budgeting not only GPU-hours for the policy, but also for the evaluators. MiMo’s thread confirms they viewed this as first-class, not an afterthought.

How it fits in an agentic stack

If you’re running agents in production, MiMo’s announced approach slots roughly into this lifecycle:

1. Base model: some pretrained model (not described in the sources).

2. Agent harnesses: your actual deployed patterns (coding agent, research agent, planning agent, etc.).

3. MiMo-style RL loop tying them together:

- use your harnesses as environments;

- run many async rollouts per step;

- grade with an agentic evaluator using a mix of test cases and rubrics;

- update the policy.

4. Deployment: ship the updated model back into those same harnesses.

The key conceptual move here is to treat your real harnesses as the RL environments, rather than training on synthetic or simplified setups. MiMo explicitly says “multi-task agentic RL, mixed across multiple harnesses in one run” 1, which is consistent with the idea of training the model directly in the same shapes you run in production.

Why this matters now (for builders)

The MiMo team states: “We believe RL is one of the most scalable and efficient paths toward self-improvement” 1. They also emphasize that they want the livestream to surface “the core challenges of scaling RL” and help refine their “training recipe” 1.

From an engineering standpoint, those “core challenges” likely include:

• Throughput engineering: running billions of tokens/step with async rollouts.
• Multi-harness stability: preventing destructive interference across task suites, tools, and workflows.
• Grader design: tuning agentic, rubric-based graders so they:

- don’t exploit loopholes in task specs,

- don’t reward degenerate strategies,

- and remain consistent enough to provide usable learning signals.

As they “open-source the details piece by piece over the coming weeks” 1, those are the aspects to watch if you’re planning to adopt similar methods.

For now, if you want to “be MiMo-pilled” in your own stack without undocumented assumptions, the safe operational lessons are conceptual:

• Treat harness design and grading as equally important to model architecture.
• Expect to spend real compute on evaluators, not just policies.
• Design your system so you *could* run many async rollouts per step, even if you don’t start at MiMo’s scale.

What is not documented

The sources do not establish:

• What model architecture, size, tokenizer, or pretraining data MiMo-V2.6 uses.
• What specific RL algorithm, optimizer, or hyperparameters are used in the run.
• How prompts are constructed, how harnesses are implemented, or how tasks are routed to harnesses.
• How rewards from test cases and rubrics are combined into a scalar signal.
• What “in-group credit assignment” precisely means (across rollouts, agents, steps, or something else).
• Any quantitative performance results, benchmarks, or comparisons to other systems.
• Any concrete failure cases, safety mechanisms, or guardrails they employed.
• Any hardware details, scaling laws, or cost figures behind “~2B tokens per step” 1.

Everything beyond the explicit quotes in 1 remains speculative until Xiaomi publishes the promised open-source details.