NVIDIA and Princeton's Dao AI Lab release ReplaySSM for hybrid model decoding
Blockchain.News reports NVIDIA and Princeton's Dao AI Lab released ReplaySSM, which caches recent inputs instead of full SSM state to speed up decoding in hybrid language models.
Why it matters
The claim concerns serving cost rather than any crypto market: if the reported speedups hold, hybrid models such as GLM-5.3-Flash and Kimi-K3 would decode fewer tokens per GPU-second at long context, lowering the cost of hosting those open-weight models. Only one outlet has reported it, and no third-party benchmark has been cited.
Key facts
NVIDIA and the Dao AI Lab at Princeton University released ReplaySSM, which caches recent inputs instead of storing the full SSM state at every decoding step.
Reported by Blockchain.News
The paper ReplaySSM: Cache SSM Inputs, Not State was published on arXiv on October 12, 2026.
Reported by Blockchain.News
Michelle Horton wrote a post about the work on the NVIDIA Developer site.
Reported by Blockchain.News
ReplaySSM is integrated into NVIDIA TensorRT-LLM, vLLM, SGLang and FlashInfer, according to Blockchain.News.
Reported by Blockchain.News
ReplaySSM runs in production in GLM-5.3-Flash and Kimi-K3, according to Blockchain.News.
Reported by Blockchain.News
Reported kernel speedups are 1.46x to 1.80x and whole-decode-step speedups are 1.20x to 1.47x across four models at batch size 256.
Reported by Blockchain.News
What happened
Blockchain.News reports that NVIDIA and the Dao AI Lab at Princeton University released ReplaySSM, a method that caches recent inputs rather than storing the full state-space model state at every decoding step, with the paper ReplaySSM: Cache SSM Inputs, Not State posted to arXiv on October 12, 2026 alongside a NVIDIA Developer blog post by Michelle Horton. Blockchain.News says the technique is already integrated into NVIDIA TensorRT-LLM, vLLM, SGLang and FlashInfer and runs in production in GLM-5.3-Flash and Kimi-K3. The outlet traces the problem to the structure of hybrid models, where Nemotron 3, Qwen3.5 through Qwen3.8, Kimi-K3 and GLM-5.3-Flash interleave SSM layers such as Mamba-2, Gated DeltaNet or Kimi Delta Attention with fewer attention layers, and reports that on Nemotron-3-Super-NVFP4 at batch size 256 on one NVIDIA Blackwell Ultra B300 GPU the SSM layers accounted for the largest share of decode-step latency out to 100,000 tokens of context. The same outlet reports kernel speedups of 1.46x to 1.80x and whole-decode-step speedups of 1.20x to 1.47x across four models at batch size 256, and attributes the latency share to the model carrying five times as many SSM layers as attention layers, with one FP32 state of 4 MiB per sequence, equal to an FP8 key-value cache for 8,000 tokens per attention layer. Both accounts come from Blockchain.News, so the measurements and the production deployment claims have not been confirmed by a second outlet.
How the story developed
- Official statement
ReplaySSM paper appears on arXiv with NVIDIA Developer blog post
The paper ReplaySSM: Cache SSM Inputs, Not State is dated October 12, 2026, and Michelle Horton's NVIDIA Developer post covers the same work.
- First report
First report by Blockchain.News
NVIDIA and Princeton Introduce ReplaySSM to Accelerate Hybrid AI Model Decoding
- Update
Blockchain.News reports the ReplaySSM release
First Blockchain.News write-up of the NVIDIA and Princeton Dao AI Lab method.
- Update
ReplaySSM reported as live in two production model deployments
The method is reported as integrated into TensorRT-LLM, vLLM, SGLang and FlashInfer and running in GLM-5.3-Flash and Kimi-K3.
How coverage built up
Independent sources over time, counted the way the consensus panel counts them: a republication of a wire story does not move the line.
- 13 Oct 2026, 13:47 UTC: Blockchain.News — 1 independent source, 1 reports
- 13 Oct 2026, 13:56 UTC: Blockchain.News — 2 independent sources, 2 reports
Affected entities
Original sources
| Publisher | Report | Role | Published |
|---|---|---|---|
| Blockchain.News Crypto media | NVIDIA and Princeton Introduce ReplaySSM to Accelerate Hybrid AI Model Decoding | Original | |
| Blockchain.News Crypto media | NVIDIA and Princeton Unveil ReplaySSM to Accelerate Hybrid-Model Inference | Syndicated |