PyTorch 2.15 Supercharges Mac AI: FP8, MoE Speedups, and Metal Native Kernels
This summary and analysis were generated by AI from the original article at Hugging Face Blog and may contain errors (how Viqus works). Read the source for full details.
7
What is the Viqus Verdict?
We evaluate each news story based on its real impact versus its media hype to offer a clear and objective perspective.
AI Analysis:
While the hype is high due to the impressive benchmarks, the actual impact is a concrete, measurable performance uplift for a specific, high-value user base (Mac developers/researchers).
Article Summary
The PyTorch team at Hugging Face announced major improvements in the MPS backend for PyTorch 2.15, making Mac hardware a much more capable platform for running and training AI models. Key upgrades include support for FP8 tensors, which halves memory usage compared to BF16. Performance gains are substantial, with Mixture-of-Experts (MoE) models decoding 5–6.5x faster on M3 Ultra chips due to native grouped matmul support. Furthermore, specialized kernels for linear layers and softmax accelerate general LLM inference, while video decoding memory usage drops dramatically from 270 GiB to 8 GiB. These changes require macOS 15 or later.
Key Points
- PyTorch 2.15 introduces FP8 support on MPS, halving memory requirements for tensor operations.
- MoE model decoding speed sees massive gains (up to 6.5x faster) on Apple Silicon due to native Metal kernels.
- Numerous operations, including softmax and video decoding, are moved from the less efficient MPSGraph to dedicated Metal kernels.
Why It Matters
This release represents a significant maturation of running complex AI workloads on Apple Silicon. By moving core operations like attention and MoE decoding directly to optimized Metal kernels and introducing FP8 quantization, PyTorch is closing the performance gap between Mac hardware and dedicated accelerators. This lowers the barrier to entry for running state-of-the-art models locally, potentially shifting enterprise adoption patterns for on-device AI.