ArticleAI Coding
The best 4-bit format in MLX is the one its own converter sets up to lose
NVFP4 has a second scaling level. mlx-lm never sets it, so NVFP4 converts to the worst 4-bit result on the table — worse than a format using fewer bits. Set it and NVFP4 wins. I nearly published this as a defect in NVIDIA's spec; it was my own unit error.
Mihai Perdum9 min read