Google DeepMind 发布 Gemma 4 技术报告

meng shao (@shao__meng) 转发了 Sumanth (@Sumanth_077) 的帖子： Google DeepMind just dropped the Gemma 4 Technical Report! Gemma 4 is a family of open-weight, natively multimodal models ranging from 2.3B to 31B parameters, covering both dense and Mixture-of-Experts architectures. The most interesting architectural decision is in the 12B model. Instead of separate vision and audio encoders, it uses a unified encoder-free design. The 550M vision encoder is replaced by a single matrix multiplication at 35M parameters. The audio encoder is discarded entirely. Raw image patches and audio chunks are projected directly into the LLM embedding space. No separate encoder pipelines, no memory fragmentation. The efficiency numbers are compelling. A 2.3B effective parameter model (E2B) roughly matches Gemma 3 27B performance with 10x fewer parameters. The quantized 12B fits in 7.65 GB. KV cache footprint is reduced by up to 37.5% through local-to-global attention ratios, positional encoding, and key-value reuse in global attention layers. The audio encoder shrank from 390MB to 87MB after quantization, a 78% reduction, with improved transcription and translation performance. Gemma 4 31B sits at rank 43 on Arena Text as of June 2026, the top open dense model on the leaderboard. Models ranked above it are all MoE architectures with significantly more total parameters. All models ship with a thinking mode that generates a reasoning trace before responding. Benchmark results in thinking mode for the 31B: AIME 2026 at 89.2%, GPQA Diamond at 84.3%, LiveCodeBench v6 at 80.0%, and a Codeforces Elo of 2150. I've shared the link to the report the replies! Also put together a complete guide on fine-tuning Gemma 4 locally with Unsloth. I've quoted the article!  > **引用原帖 Sumanth (@Sumanth_077):** > https://t.co/zQeFLsziGf > https://x.com/Sumanth_077/status/2041875179392135559