MIT HAN Lab company blog
SVDQuant: 4-Bit Quantization Powers 12B Flux on a 16GB 4090 GPU with 3x Speedup
A new post-training training quantization paradigm for diffusion models, which quantize both the weights and activations of FLUX.1 to 4 bits, achieving 3.5× memory and 8.7× latency reduction on a 16GB laptop 4090 GPU. Code: https://www.github.com/mit-han-lab/nunchaku
- Points
- 179
- HN comments
- 65
- Submissions
- 1
- Cited by
- 0 commenters
AI & Machine LearningSystems & Performance2024
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| Nov 9, 2024 | SVDQuant: 4-Bit Quantization Powers 12B Flux on a 16GB 4090 GPU with 3x Speedup by lmxyy | 179 | 65 |
