Google’s Multi-Token Prediction Boosts AI Speed Without Hardware Changes
- Google released Multi-Token Prediction (MTP) drafters for Gemma 4, enhancing inference speed by up to three times without quality loss.
- Speculative decoding allows a lightweight drafter model to predict multiple tokens simultaneously, verified by the main model in parallel.
- MTP drafters are available on platforms like Hugging Face, Kaggle, and Ollama under Apache 2.0 license.
- A Gemma 4 model on an Nvidia RTX Pro can achieve double the tokens per second with MTP enabled, while Apple Silicon sees a speedup of about two times.
- The drafter models share the target model’s KV cache to avoid recalculating known data, improving efficiency further.
Google’s MTP drafters enhance AI performance by optimizing existing hardware capabilities rather than requiring new hardware investments. This approach leverages speculative decoding to increase processing speed significantly without compromising output quality.
With MTP drafters, users experience faster response times in applications like chat and voice interfaces, making AI tools more practical for everyday use on consumer devices without additional costs or upgrades.( Source)
