mirror of
https://github.com/kamranahmedse/developer-roadmap.git
synced 2026-10-25 09:34:00 +08:00
Javier CanalesandClaude Opus 5.5
4f7e267d15
Add curated papers, official docs, open-source repos and books to 64 topics of the Inference Engineering roadmap, following the content style guide (typed links, ordered by type, max 8 per topic). Also normalise the heading on the Post-Training Quantization and Multi-Cloud topics. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1.2 KiB
1.2 KiB
Post-Training Quantization
Post-training quantization converts finished model weights to lower precision after training is complete, using a small calibration dataset to compute per-layer scale factors that minimize the gap between quantized and original outputs. It is the standard approach for most deployments since it requires no retraining and can be applied to any existing model. The leading tool is NVIDIA ModelOpt, which exports quantized models in formats compatible with all major inference engines.
Visit the following resources to learn more:
- @opensource@AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
- @opensource@bitsandbytes
- @opensource@SmoothQuant
- @article@GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- @article@LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
- @article@SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot