What ReaDy is, in system terms

ReaDy is a speculative decoding scheme designed specifically for hybrid linear attention (HLA) large language models [\[1\]](http://arxiv.org/abs/2610.20503v1).

The ingredients the paper explicitly calls out:

  • Speculative decoding: speeds up autoregressive generation “by drafting multiple tokens per step” [\[1\]](http://arxiv.org/abs/2610.20503v1).
  • Hybrid linear attention models: they “maintain compact recurrent states that accumulate historical associations” [\[1\]](http://arxiv.org/abs/2610.20503v1).
  • Problem with feature-based drafters: prior speculative methods for HLA rely on “target hidden features whose contextual emphasis may not align with the needs of subsequent draft positions” [\[1\]](http://arxiv.org/abs/2610.20503v1).

ReaDy’s core idea is to use those compact recurrent states as a dynamic, position-specific context source during drafting:

  • Each draft position gets its own learned query that reads from the target model’s committed recurrent states.
  • A lightweight state reader integrates that retrieved context into the draft representation via residual updates.
  • The reading is kept cheap with low-rank state reads and fused operators to limit compute and I/O overhead [\[1\]](http://arxiv.org/abs/2610.20503v1).

The authors describe ReaDy as, to their knowledge, the first speculative decoding method to leverage HLA recurrent states for position-specific context retrieval during drafting [\[1\]](http://arxiv.org/abs/2610.20503v1).

On top of that, ReaDy is not a full re-architecture; it is layered onto DSpark’s parallel backbone and sequential prediction head [\[1\]](http://arxiv.org/abs/2610.20503v1). It uses a lightweight two-stage adaptation strategy, and the paper runs ablations over:

  • reader capacity,
  • state sources, and
  • residual placement [\[1\]](http://arxiv.org/abs/2610.20503v1).

So if you are already in a DSpark-style speculative setup with an HLA model, ReaDy looks like an additional reader module and training procedure, not a replacement of your decoding stack.

How speculative decoding and state management interact

Two other pieces in the provided sources give context on why this is interesting for agentic systems:

  • Generic speculative decoding is characterized as an inference method that “has demonstrated its efficacy in accelerating large language model inference” [\[4\]](http://arxiv.org/abs/2610.18735v1).
  • State-space models (SSMs) and hybrids pay a cost for “reading and writing [state] at every decoding step”, which “can bottleneck hybrid models” due to “large state I/O with low arithmetic intensity”, difficult rollback, and error accumulation [\[7\]](http://arxiv.org/abs/2610.13778v1).

ReplaySSM explicitly targets these issues by:

  • representing state as a checkpoint + buffer of recent SSM inputs,
  • separating output computation from checkpoint updates,
  • avoiding state writes on most steps and “roughly halv[ing] the dominant state memory traffic”, and
  • making speculative rollback a buffer-pointer operation while enabling “output-only verification without” materializing full state [\[7\]](http://arxiv.org/abs/2610.13778v1).

ReaDy addresses a different but related bottleneck: for HLA models that already maintain compact recurrent state, it tries to turn that state into a better context source for the drafter, rather than rethinking how the state is stored or updated.

Put differently:

  • ReplaySSM optimizes *how* recurrence is represented and manipulated for hybrid/SSM models, with benefits to speculative decoding via cheaper rollback and less state I/O [\[7\]](http://arxiv.org/abs/2610.13778v1).
  • ReaDy optimizes *what* context is fed into each drafted token in an HLA model, by exposing the target’s recurrent states directly to the drafter via learned queries and a state reader [\[1\]](http://arxiv.org/abs/2610.20503v1).

Both are trying to better align recurrent machinery with speculative workflows, but at different layers of the stack.

ReaDy’s mechanism, step by step (as documented)

The paper’s abstract is high-level, but it does spell out the main control-flow components of ReaDy [\[1\]](http://arxiv.org/abs/2610.20503v1). What is documented:

  1. Baseline: DSpark-style speculative pipeline

ReaDy is “building on DSpark’s parallel backbone and sequential prediction head” [\[1\]](http://arxiv.org/abs/2610.20503v1). That implies a two-component speculative setup:

  • a parallel backbone that can process multiple draft positions in parallel, and
  • a sequential head that runs step-wise prediction / verification.

The details of DSpark’s internals are not in the supplied text, only that ReaDy uses it as a foundation [\[1\]](http://arxiv.org/abs/2610.20503v1).

  1. HLA recurrent state as a context source

Hybrid linear attention models “maintain compact recurrent states that accumulate historical associations” [\[1\]](http://arxiv.org/abs/2610.20503v1). ReaDy reads from the target model’s committed recurrent states during drafting [\[1\]](http://arxiv.org/abs/2610.20503v1).

The key point is that, instead of only using hidden features at the current step (which “may not align with the needs of subsequent draft positions”), the drafter can look back into these accumulated states [\[1\]](http://arxiv.org/abs/2610.20503v1).

  1. Per-position learned queries

ReaDy “equips each draft position with a learned query to retrieve supplementary context directly from the target’s committed recurrent states” [\[1\]](http://arxiv.org/abs/2610.20503v1).

Mechanistically, this means:

  • Each speculative position in the draft has its own query vector (or parameterized function of that position) learned during adaptation.
  • Those queries index into the target’s recurrent state, returning some context representation per drafted token [\[1\]](http://arxiv.org/abs/2610.20503v1).

The abstract does not specify the query architecture, dimensionality, or how many layers it touches.

  1. Lightweight state reader with residual updates

The per-position queries feed a state reader, which is described as [\[1\]](http://arxiv.org/abs/2610.20503v1):

  • Lightweight, suggesting small relative parameter/compute overhead, but no numbers are provided.
  • Responsible for integrating “this context into draft representations through residual updates”.

That last phrase implies the reader output is added back (residually) into the draft token embeddings or intermediate representations, but the exact layer placement is not specified; instead the paper notes that “residual placement” is one of the axes studied in ablations [\[1\]](http://arxiv.org/abs/2610.20503v1).

  1. Low-rank state reads and fused operators

To keep this extra machinery efficient, ReaDy uses:

  • Low-rank state reads, and
  • Fused operators,

specifically to “keep additional computation and I/O overhead low” [\[1\]](http://arxiv.org/abs/2610.20503v1).

The abstract does not document how low the rank is, what gets fused, or how much overhead remains; only the qualitative claim that the design is lightweight.

  1. Two-stage adaptation strategy

ReaDy “employs a lightweight two-stage adaptation strategy” [\[1\]](http://arxiv.org/abs/2610.20503v1). The abstract doesn’t elaborate, beyond mentioning that:

  • Reader capacity,
  • State sources, and
  • Residual placement

are varied in ablation studies [\[1\]](http://arxiv.org/abs/2610.20503v1).

Without more detail in the provided text, we only know that the training / fine-tuning procedure is staged and that these axes were explored.

In other words, from a systems viewpoint, ReaDy adds a position-aware, low-rank state-reader module that taps into the HLA recurrent state and feeds residual context into the drafter, all sitting atop a DSpark-like speculative backbone.

How it fits in a practical stack

The abstract gives one concrete deployment configuration:

  • Base model: Qwen3.8-27B (a 27B-parameter LLM) [\[1\]](http://arxiv.org/abs/2610.20503v1).
  • Hardware: single NVIDIA B200 GPU [\[1\]](http://arxiv.org/abs/2610.20503v1).
  • Decoding baselines:
  • Autoregressive decoding with no speculation.
  • DSpark speculative decoding without ReaDy’s state reader [\[1\]](http://arxiv.org/abs/2610.20503v1).

Under that setup, the reported results are:

  • ReaDy achieves an average 3.16× decoding speedup over plain autoregressive decoding on Qwen3.8-27B on a single B200 [\[1\]](http://arxiv.org/abs/2610.20503v1).
  • Across seven benchmarks spanning mathematics, code generation, and dialogue, ReaDy outperforms DSpark on every benchmark, improving:
  • Throughput by 7.97%, and
  • Acceptance length by 8.50%,

on average relative to DSpark [\[1\]](http://arxiv.org/abs/2610.20503v1).

The abstract does not define throughput or acceptance length in detail, but in typical speculative decoding literature these refer, respectively, to:

  • tokens per second (or a similar performance measure), and
  • how many drafted tokens are accepted before a mismatch triggers rollback.

However, those operational definitions are not present in the supplied ReaDy text, so they should be treated as conventional but not documented here.

For an agent system with long-running tool calls, code synthesis, or mathematical reasoning, the domains tested (math, code, dialogue) match common workloads [\[1\]](http://arxiv.org/abs/2610.20503v1). If your target model is an HLA architecture similar to Qwen3.8-27B, the paper’s empirical claim is that adding ReaDy on top of DSpark yields:

  • a material latency/throughput improvement over naive autoregressive decoding (3.16×), and
  • an incremental improvement over an existing sophisticated speculative baseline (≈8% on two important spec metrics) [\[1\]](http://arxiv.org/abs/2610.20503v1).

The code is reported as publicly available (“Our code is available at this https URL”) [\[1\]](http://arxiv.org/abs/2610.20503v1), but the exact repository link is not provided in the abstract snippet.

Why it matters now, especially for agentic systems

Several trends reflected in the sources make ReaDy-like work relevant for builders:

  • Speculative decoding as a standard speedup: SP²ec notes that speculative decoding “has demonstrated its efficacy in accelerating large language model inference” [\[4\]](http://arxiv.org/abs/2610.18735v1). Agentic systems chained over multiple tools and LLM calls are especially sensitive to per-call latency.
  • State-heavy hybrid architectures: ReplaySSM highlights that in hybrid models, naïve recurrent state management “can bottleneck” due to “large state I/O with low arithmetic intensity” and poor rollback characteristics [\[7\]](http://arxiv.org/abs/2610.13778v1). HLA models are similarly stateful; ReaDy shows those same states can be active assets for drafting, not just overhead.
  • Growing reliance on large, capable models: ReaDy’s reported experiments use a 27B model on a single high-end GPU [\[1\]](http://arxiv.org/abs/2610.20503v1). That is in the range many production shops target when they want strong reasoning but still need single-device serving for cost and deployment simplicity.

In that context, ReaDy suggests an emerging pattern:

  • Don’t just speed up decoding with external drafters.
  • Exploit the model’s own recurrent or linear-attention state as a cheap, compact memory bank for drafting decisions.

ReplaySSM does this by changing how state is stored and updated [\[7\]](http://arxiv.org/abs/2610.13778v1); ReaDy does it by letting each draft position read from the accumulated recurrent state via low-rank queries [\[1\]](http://arxiv.org/abs/2610.20503v1).

For agent builders, this points toward a decoding stack where:

  • the architecture (HLA/SSM),
  • the state representation and caching (e.g., ReplaySSM-style),
  • and the speculative drafting logic (e.g., DSpark + ReaDy)

are co-designed rather than bolted together independently.

What to watch next and practical questions

Given only the abstract, several practical questions remain open but important:

  • Integration constraints: ReaDy is tuned for HLA models; the abstract does not document whether it can be ported to standard softmax attention transformers, or whether its gains rely on properties unique to HLA recurrent states [\[1\]](http://arxiv.org/abs/2610.20503v1).
  • Memory vs. speed trade-offs: The authors emphasize that low-rank reads and fused ops keep overhead low [\[1\]](http://arxiv.org/abs/2610.20503v1), but there are no reported memory overhead numbers in the supplied text. ReplaySSM, by contrast, claims to “roughly half” the dominant state memory traffic by reducing state writes [\[7\]](http://arxiv.org/abs/2610.13778v1); similar explicit statements about ReaDy’s memory footprint are absent.
  • Draft depth and configuration: SP²ec devotes attention to adapting draft configuration during generation using a bandit algorithm, motivated by a “single-peak throughput pattern” in self-speculative methods [\[4\]](http://arxiv.org/abs/2610.18735v1). ReaDy’s abstract does not say how draft length or tree shape is chosen, or whether it could benefit from adaptive schemes of that kind.
  • Effect on quality: The ReaDy abstract reports throughput and acceptance-length improvements over DSpark, plus overall decoding speedup relative to autoregressive decoding, but does not state whether there is any change in task accuracy or output quality on the math/code/dialogue benchmarks [\[1\]](http://arxiv.org/abs/2610.20503v1).
  • Applicability beyond Qwen3.8-27B and a single B200: All the reported ReaDy numbers are for that specific model/hardware combo [\[1\]](http://arxiv.org/abs/2610.20503v1). It is not documented how performance scales with smaller models, multi-GPU setups, or different hardware classes.

If you are designing a speculative decoding layer for an HLA or hybrid model today, the documented ReaDy mechanisms suggest concrete design hooks:

  • Add a position-aware reader that can query the model’s own recurrent state per draft position.
  • Keep that reader low-rank and fuse its ops as much as possible to avoid turning state reads into a new bottleneck [\[1\]](http://arxiv.org/abs/2610.20503v1).
  • Treat state representation choices (ReplaySSM-type ideas) and speculative drafting as linked design problems, not independent knobs [\[7\]](http://arxiv.org/abs/2610.13778v1).

What is not documented

Based on the provided sources, the following are not established:

  • The exact architecture of the ReaDy state reader (layers, dimensions, which transformer blocks it touches).
  • How many additional parameters ReaDy adds to Qwen3.8-27B, or its precise memory and latency overhead relative to DSpark alone.
  • The implementation details of DSpark’s backbone and prediction head, beyond that ReaDy builds on them.
  • The precise definition and measurement units for “throughput” and “acceptance length” in the ReaDy experiments.
  • Any effect of ReaDy on task accuracy or output quality for the seven benchmarks, beyond speed and acceptance metrics.
  • Any ReaDy results on models other than Qwen3.8-27B, or hardware other than a single NVIDIA B200 GPU.
  • Any direct comparison or integration between ReaDy and ReplaySSM or SP²ec; their relationship here is inferred by topic, not documented as combined systems.