DiffusionGemma Integrated into vLLM: A New Approach to Response Generation Without Traditional Token Generation
Developers integrated DiffusionGemma into vLLM, implementing the System One Models approach where responses are generated in a single denoising step instead of sequential token generation.
Diogo Almeida, co-developer of RLHF and one of the authors of the InstructGPT paper (the technology that laid the foundation for ChatGPT in 2022), introduced a new class of models called System One Models. These models differ fundamentally from traditional LLMs as they do not generate text sequentially.
Matt Mastracci, a contributor to the vLLM project, took Google's DiffusionGemma 26B model and implemented a similar approach without fine-tuning the model.
How it Works
Instead of traditional text generation, an answer template is specified. After a single denoising step, answers and their probabilities are immediately extracted from the logits. The method supports formats:
- yes/no
- multiple-choice
- scoring
Integration into vLLM
This mode has now been added to vLLM. According to initial tests, the quality of answers is close to the original Jev approach. A key advantage is that the model in NVFP4 format can be run on a single GPU with 24GB of memory.
Additional materials:
Why it matters
AnalysisThis opens up new possibilities for fast and efficient response generation in classification tasks, allowing powerful models to run on consumer hardware.
Discuss in community
Share your questions and insights with developers
Related in Ecosystem
Practical guides, core terminology, and agent skills
vLLM (High-Performance Inference Engine)
Leading open-source inference engine and LLM servicing framework that revolutionizes throughput with the PagedAttention memory virtualization algorithm and continuous batching.
KV-Cache Offloading & Compression
Hardware and algorithmic methods for temporarily offloading Key-Value Cache (KV-Cache) from expensive GPU VRAM to system RAM or fast NVMe SSDs.
Unlimited Figma MCP for AI agents via an open-source plugin
figma-console-mcp lets AI agents use Figma's Plugin API as an MCP server, bypassing the 20-call monthly limit on Figma's free tier.
Prompting in 2026: ChatGPT, Claude, image generation and work assignments
Most people use AI as a search engine: type a short query, skim the answer and close. For reference it works. For everything that is really important - no.