FuTransformer’s FP8/INT4 path on Hopper, decoded
How FuTransformer restructures Llama-style inference into FP8 prefill and INT4 decode kernels on NVIDIA Hopper.
DailyUpdated 2026-11-07 12:20 UTC6 articles
Infra & inference
Serving, inference optimisation, GPUs, routing, cost and latency engineering
ReaDy: speculative drafting with hybrid linear attention
How ReaDy taps hybrid linear attention states to push speculative decoding faster than DSpark on Qwen3.8-27B.
Training drafters by minimizing decode rounds
A Markov-reward view of speculative decoding that trains parallel drafters directly on end-to-end efficiency.
ReplaySSM for hybrid SSM–Transformer stacks
How caching SSM inputs instead of state cuts memory traffic and unlocks faster speculative decoding.
Action-guided KV eviction for agentic LLM stacks
How ActKV squeezes KV memory in agent loops without killing action quality
KernelOPT’s agentic loop for GPU kernel search
A multi-agent LLM system optimizes Triton kernels while preserving compiler structure and end-to-end correctness.
Live signals on Infra & inference
- ServerBench: How can your coding agents design specialized LLM serving systems? arXiv · 2 days ago · frontier 88%
- Enabling AWS Cloud FPGA Development Using High-Level Vitis HLS arXiv · 2 days ago · frontier 79%
- StrataFlux: Memory-Hierarchy-Native Low-Bit LLM Inference arXiv · 1 days ago · frontier 86%
- Systolic Linear Attention: Co-Designing Tile-Grouped Gated Delta Attention for TPUs arXiv · 2 days ago · frontier 82%
- AutoCompress: Automated Hardware-Guided Model Compression arXiv · 1 days ago · frontier 75%
- Adversarial Speculative Reasoning Improves Long Horizon Agents arXiv · 3 days ago · frontier 83%
- @XFreeze: Elon Musk just revealed that SpaceX will begin launching StarMind SI computers into orbit in Q3… X · ai radar · 19 h ago · frontier 72%