mirror of
https://github.com/kamranahmedse/developer-roadmap.git
synced 2026-10-25 09:34:00 +08:00
Javier CanalesandClaude Opus 5.5
4f7e267d15
Add curated papers, official docs, open-source repos and books to 64 topics of the Inference Engineering roadmap, following the content style guide (typed links, ordered by type, max 8 per topic). Also normalise the heading on the Post-Training Quantization and Multi-Cloud topics. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1.1 KiB
1.1 KiB
Quantization
Quantization converts model weights and other values from their native precision to a lower-precision format. Cutting precision in half doubles memory bandwidth for decode and enables twice the FLOPS for prefill on compatible Tensor Cores. The tradeoff is potential quality loss from reduced numerical precision, which can compound across the many calculations in a forward pass.
Visit the following resources to learn more:
- @opensource@AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
- @opensource@bitsandbytes
- @opensource@SmoothQuant
- @article@GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- @article@LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
- @article@SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot