chore: sync content to repo

roadmaps/inference-engineering/content/aiperf@zffK61pNUJDwkX7Ukql8y.md

@@ -1,10 +1,3 @@
-# AIPerf
+# NVIDIA GenAI-Perf
 
-AIPerf is an open-source benchmarking framework designed to evaluate the performance of artificial intelligence hardware and software stacks by measuring their ability to execute large-scale neural network models. It simulates real-world workloads to provide standardized metrics, such as throughput and latency, allowing developers to compare how different systems handle complex computational tasks.
-
-Visit the following resources to learn more:
-
-- [@opensource@aiperf](https://github.com/ai-dynamo/aiperf)
-- [@official@AIPerf Docs](https://docs.nvidia.com/aiperf/welcome-to-ai-perf-documentation)
-- [@official@Profiling with AIPerf](https://docs.nvidia.com/aiperf/getting-started/profiling-with-ai-perf)
-- [@video@AI Perf benchmarking - Dynamo and other LLM endpoints](https://www.youtube.com/watch?v=OuQCSOIR-Y8)
+GenAI-Perf is NVIDIA's inference benchmarking tool, designed to work across TensorRT-LLM, Triton Inference Server, and OpenAI-compatible endpoints. It generates synthetic workloads with configurable sequence length distributions and concurrency levels and outputs standard latency and throughput metrics including TTFT, ITL, and TPS percentiles.

roadmaps/inference-engineering/content/auto-speech-recognition@6PRICRK6aRQCoThxnevB0.md

@@ -5,6 +5,7 @@ ASR models like Whisper take audio input and produce text transcripts. Whisper i
 Visit the following resources to learn more:
 
 - [@official@OpenAI Whisper Models on Hugging Face](https://huggingface.co/openai)
-- [@official@OpenAI Whisper](https://openai.com/index/whisper/)
-- [@opensource@openai/whisper](https://github.com/openai/whisper)
-- [@article@Robust Speech Recognition via Large-Scale Weak Supervision (Whisper)](https://arxiv.org/abs/2212.04356)
+- [@article@What is automatic speech recognition (ASR)?](https://www.ionos.com/digitalguide/websites/web-development/automatic-speech-recognition/)
+- [@article@What is Automatic Speech Recognition? A Comprehensive Overview of ASR Technology](https://www.assemblyai.com/blog/what-is-asr)
+- [@video@How streaming ASR inference differs from LLM serving](https://www.youtube.com/watch?v=8HJWwD5Uacc)
+- [@video@Automatic Speech Recognition - An Overview](https://www.youtube.com/watch?v=q67z7PTGRi8)

roadmaps/inference-engineering/content/autoscaling@o_gotSHKIPtV5SgGN3Vja.md

@@ -4,6 +4,7 @@ Autoscaling dynamically adjusts the number of active model replicas to match inc
 
 Visit the following resources to learn more:
 
-- [@book@Designing Data-Intensive Applications](https://dataintensive.net/)
-- [@book@Site Reliability Engineering](https://sre.google/books/)
-- [@article@The Llama 3 Herd of Models](https://arxiv.org/abs/2407.21783)
+- [@article@Autoscaling endpoints for LLM inference](https://www.together.ai/blog/autoscaling-endpoints-for-llm-inference)
+- [@article@Autoscaling Inference Endpoints](https://apxml.com/courses/mlops-for-large-models-llmops/chapter-4-llm-deployment-serving-optimization/autoscaling-inference-endpoints)
+- [@article@Best practices for autoscaling large language model (LLM) inference workloads](https://docs.cloud.google.com/kubernetes-engine/docs/best-practices/machine-learning/inference/autoscaling)
+- [@video@Autoscaling your AI agent under load](https://www.youtube.com/watch?v=ESIL0Rzl5VQ&t=91s)

roadmaps/inference-engineering/content/cache-aware-routing@3DR0Qvqu2qUTi9niU_63n.md

@@ -1,3 +1,10 @@
 # Cache-Aware Routing
 
-In deployments with multiple replicas, cache-aware routing directs requests to the replica most likely to have a matching prefix already in its KV cache. Without this, a user in a long conversation may hit different replicas each turn, getting cache misses that increase latency. A global KV cache stored on networked SSD allows all replicas to eventually access any cached sequence.
+In deployments with multiple replicas, cache-aware routing directs requests to the replica most likely to have a matching prefix already in its KV cache. Without this, a user in a long conversation may hit different replicas each turn, getting cache misses that increase latency. A global KV cache stored on networked SSD allows all replicas to eventually access any cached sequence.
+
+Visit the following resources to learn more:
+
+- [@article@Why Agentic Inference Needs Prefix-Aware Routing Infrastructure](https://www.coreweave.com/blog/why-agentic-inference-needs-prefix-aware-routing-infrastructure)
+- [@article@Enabling inference at massive scale with hybrid storage for KV cache offloading.](https://www.seagate.com/es/es/resources/enabling-inference-at-massive-scale-with-hybrid-storage-for-kv-cache-offloading/)
+- [@article@KV Cache Aware Routing](https://docs.vllm.ai/projects/production-stack/en/vllm-stack-0.1.5/tutorials/kvaware.html)
+- [@article@Prefix-Cache Aware Routing](https://llm-d.ai/docs/architecture/advanced/kv-management/prefix-cache-aware-routing)

roadmaps/inference-engineering/content/caching-approaches@2nGv7_8PTWYr7uqbudCcX.md

@@ -5,6 +5,6 @@ KV cache reuse is the primary caching strategy in LLM inference, covering both p
 Visit the following resources to learn more:
 
 - [@opensource@LMCache](https://github.com/LMCache/LMCache)
-- [@article@CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion](https://arxiv.org/abs/2405.16444)
 - [@article@What is Prompt Caching?](https://www.ibm.com/think/topics/prompt-caching)
-- [@video@What is Prompt Caching? Optimize LLM Latency with AI Transformers](https://www.youtube.com/watch?v=u57EnkQaUTY)
+- [@video@What is Prompt Caching? Optimize LLM Latency with AI Transformers](https://www.youtube.com/watch?v=u57EnkQaUTY)
+- [@video@How KV Cache Speeds Up LLMs for Faster AI Models on GPUs](https://www.youtube.com/watch?v=o0gkdZBtwEg&t=1s)

roadmaps/inference-engineering/content/caching@PzEj5yq1lDQ324RWYpDdQ.md

@@ -1,3 +1,10 @@
 # Caching
 
-Caching reuses previously computed KV values to avoid redundant prefill computation across requests that share common context. It is one of the most impactful techniques for reducing TTFT in workloads with long system prompts or multi-turn conversations. This group covers the approaches, storage tiers, routing implications, and long-context strategies that make caching effective at scale.
+Caching reuses previously computed KV values to avoid redundant prefill computation across requests that share common context. It is one of the most impactful techniques for reducing TTFT in workloads with long system prompts or multi-turn conversations. This group covers the approaches, storage tiers, routing implications, and long-context strategies that make caching effective at scale.
+
+Visit the following resources to learn more:
+
+- [@article@KV Caching Explained: Optimizing Transformer Inference Efficiency](https://huggingface.co/blog/not-lain/kv-caching)
+- [@article@KV Caching in LLMs: A Guide for Developers](https://machinelearningmastery.com/kv-caching-in-llms-a-guide-for-developers/)
+- [@video@KV Cache: The Trick That Makes LLMs Faster](https://www.youtube.com/watch?v=gpp57x_z_Jg)
+- [@video@The KV Cache: Memory Usage in Transformers](https://www.youtube.com/watch?v=80bIUggRJf4&t=247s)

roadmaps/inference-engineering/content/capacity-management@xrr5Y1sbN2eyQyuA5LlAW.md

@@ -1,3 +1,8 @@
 # Capacity Management
 
-Multi-cloud inference pools GPU capacity from multiple cloud providers and regions into a single schedulable resource, treating distinct compute pools as fungible. This requires a global control plane for deployment and scaling decisions alongside per-cluster workload planes that serve traffic and report utilization. True multi-cloud unlocks greater total capacity, improved reliability through redundancy, lower latency via geographic proximity, and data sovereignty compliance.
+Multi-cloud inference pools GPU capacity from multiple cloud providers and regions into a single schedulable resource, treating distinct compute pools as fungible. This requires a global control plane for deployment and scaling decisions alongside per-cluster workload planes that serve traffic and report utilization. True multi-cloud unlocks greater total capacity, improved reliability through redundancy, lower latency via geographic proximity, and data sovereignty compliance.
+
+Visit the following resources to learn more:
+
+- [@article@How we built Multi-cloud Capacity Management (MCM)](https://www.baseten.co/blog/how-we-built-multi-cloud-capacity-management/)
+- [@article@Multi-cloud capacity management](https://learn-inference.com/chapters/production/multi-cloud)

roadmaps/inference-engineering/content/cold-starts@Q3zM9MN5lmHRAUWjdLdXR.md

@@ -1,3 +1,9 @@
 # Cold Starts
 
-A cold start is the time from zero replicas to serving the first request, including GPU provisioning, container image loading, model weight loading, and inference engine startup or compilation. Reducing cold start time is essential for autoscaling systems to scale down confidently and scale up responsively. Key techniques include quantized weights, local weight caching near the GPU instances, and caching compiled inference engines to disk.
+A cold start is the time from zero replicas to serving the first request, including GPU provisioning, container image loading, model weight loading, and inference engine startup or compilation. Reducing cold start time is essential for autoscaling systems to scale down confidently and scale up responsively. Key techniques include quantized weights, local weight caching near the GPU instances, and caching compiled inference engines to disk.
+
+Visit the following resources to learn more:
+
+- [@article@Autoscaling endpoints for LLM inference](https://www.together.ai/blog/autoscaling-endpoints-for-llm-inference)
+- [@article@Autoscaling for LLM Inference Serving and Cold Starts](https://www.onesourcecloud.net/cms/autoscaling-llm-inference-serving-cold-starts.html)
+- [@article@GPU Cold Start on Serverless LLM Inference: 4 Fixes That Actually Work (2026)](https://www.spheron.network/blog/gpu-cold-start-llm-inference-2026/)

roadmaps/inference-engineering/content/continuous-batching@h_PDOMA-V-n7Pab3epuP3.md

@@ -1,3 +1,10 @@
 # Continuous Batching
 
-Continuous batching operates at the token level, swapping requests in and out of the batch as they complete individual tokens rather than waiting for a full batch to finish. This keeps GPU utilization high and minimizes the latency penalty of batching compared to static or dynamic batching. All major inference engines implement continuous batching by default.
+Continuous batching operates at the token level, swapping requests in and out of the batch as they complete individual tokens rather than waiting for a full batch to finish. This keeps GPU utilization high and minimizes the latency penalty of batching compared to static or dynamic batching. All major inference engines implement continuous batching by default.
+
+Visit the following resources to learn more:
+
+- [@article@Continuous batching](https://huggingface.co/blog/continuous_batching)
+- [@article@Continuous Batching: Optimizing LLM Inference Throughput](https://mbrenndoerfer.com/writing/continuous-batching)
+- [@video@How to Scale LLM Applications With Continuous Batching!](https://www.youtube.com/watch?v=PqHVpvvHLpE)
+- [@video@Continuous Batching - How LLM Servers Keep the GPU Full](https://www.youtube.com/watch?v=X8CwAqYxH1E)

roadmaps/inference-engineering/content/cost-estimation@Gof4kW2LbkNCtH68c7Xvq.md

@@ -4,4 +4,7 @@ Dedicated inference cost depends on batch sizing, traffic patterns, and sequence
 
 Visit the following resources to learn more:
 
-- [@article@The LLM Inference Trilemma: Throughput, Latency, Cost](https://www.digitalocean.com/blog/llm-inference-tradeoffs)
+- [@article@The Hidden Economics of LLM Inference](https://www.hebbia.com/blog/the-hidden-economics-of-llm-inference)
+- [@article@LLM Inference Benchmarking: How Much Does Your LLM Inference Cost?](https://developer.nvidia.com/blog/llm-inference-benchmarking-how-much-does-your-llm-inference-cost/)
+- [@article@LLM inference cost: what drives it and how to lower it](https://www.cloudzero.com/blog/inference-cost/)
+- [@article@LLM Cost Calculation Guide for Enterprise AI Teams in 2026](https://www.digitalocean.com/resources/articles/llm-cost-calculation-guide)

roadmaps/inference-engineering/content/dependency-management@wEaM8VyAo6MFsc3RXQ7Ar.md

@@ -4,4 +4,6 @@ Inference dependency chains are long and fragile due to the tight coupling betwe
 
 Visit the following resources to learn more:
 
-- [@official@Kubernetes Documentation](https://kubernetes.io/docs/home/)
+- [@official@Kubernetes Documentation](https://kubernetes.io/docs/home/)
+- [@article@Dependency Management](https://www.baseten.co/inference-engineering/book/07-production/containerization/#dependency-management)
+- [@article@Production in Inference Engineering](https://inferenceengineering.tech/chapters/production/)

roadmaps/inference-engineering/content/disaggregation@f6PWIqf4LubR7lQztaoB5.md

@@ -4,4 +4,7 @@ Disaggregation separates the prefill and decode phases onto independent GPU inst
 
 Visit the following resources to learn more:
 
-- [@article@Beyond the Buzz: A Pragmatic Take on Inference Disaggregation](https://arxiv.org/abs/2506.05508)
+- [@article@Beyond the Buzz: A Pragmatic Take on Inference Disaggregation](https://arxiv.org/abs/2506.05508)
+- [@article@Disaggregated Serving](https://llm-d.ai/docs/architecture/advanced/disaggregation)
+- [@article@Disaggregated Serving in LLM Inference, Explained](https://handbook.modular.com/inference-optimization/prefill-decode-disaggregation/)
+- [@video@Why Separating Prefill and Decode Makes LLMs Faster | vLLM, LLM-D and NIXL](https://www.youtube.com/watch?v=BaD3CTYf6V0)

roadmaps/inference-engineering/content/draft-target-decoding@m0EIHovHWGkWULN0a3C3v.md

@@ -4,4 +4,6 @@ Draft-target speculation uses a small draft model, typically at least ten times
 
 Visit the following resources to learn more:
 
-- [@article@Fast Inference from Transformers via Speculative Decoding](https://arxiv.org/abs/2211.17192)
+- [@article@Fast Inference from Transformers via Speculative Decoding](https://arxiv.org/abs/2211.17192)
+- [@article@An Introduction to Speculative Decoding for Reducing Latency in AI Inference](https://developer.nvidia.com/blog/an-introduction-to-speculative-decoding-for-reducing-latency-in-ai-inference/)
+- [@article@Draft-Target Speculative Decoding Simulation](https://www.deep-ml.com/problems/430)

roadmaps/inference-engineering/content/eagle@0FZTg3_ZnTpMOQ078OOxE.md

@@ -6,4 +6,6 @@ Visit the following resources to learn more:
 
 - [@article@EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty](https://arxiv.org/abs/2401.15077)
 - [@article@EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees](https://arxiv.org/abs/2406.16858)
-- [@article@EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test](https://arxiv.org/abs/2503.01840)
+- [@article@EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test](https://arxiv.org/abs/2503.01840)
+- [@video@EAGLE and EAGLE-2: Lossless Inference Acceleration for LLMs - Hongyang Zhang](https://www.youtube.com/watch?v=oXRSorx-Llg)
+- [@video@Speculative Decoding, Explained: Why EAGLE-3 Finally Scales | 20-Min Deep Dive](https://www.youtube.com/watch?v=aXdQ3JbRb98)

roadmaps/inference-engineering/content/embedding-model-inference@GmCkfSeVDJ8T8WLZ7DoOp.md

@@ -1,3 +1,9 @@
 # Embedding Model Inference
 
-TensorRT-LLM with its optimized XQA attention kernel offers the best performance for embedding models with LLM backbones. FP8 quantization of weights provides performance gains with minimal quality loss, verified by checking cosine similarity between quantized and original output vectors. Embedding models are small enough that horizontal scaling across individual GPU replicas is more effective than multi-GPU parallelism.
+TensorRT-LLM with its optimized XQA attention kernel offers the best performance for embedding models with LLM backbones. FP8 quantization of weights provides performance gains with minimal quality loss, verified by checking cosine similarity between quantized and original output vectors. Embedding models are small enough that horizontal scaling across individual GPU replicas is more effective than multi-GPU parallelism.
+
+Visit the following resources to learn more:
+
+- [@article@High-performance embedding model inference](https://www.baseten.co/resources/guide/high-performance-embedding-model-inference/#high-performance-embedding-model-inference-how-to-scale-semantic-search-rag-and-agents)
+- [@article@What Actually Makes Embedding Model Inference Fast?](https://filipmakraduli.substack.com/p/what-actually-makes-embedding-model)
+- [@article@Text Embeddings Inference - HuggingFace](https://huggingface.co/docs/text-embeddings-inference/en/index)

roadmaps/inference-engineering/content/embedding-models@Tvt-vX1gWuJZJBHh5vfKu.md

@@ -8,5 +8,5 @@ Visit the following resources to learn more:
 - [@article@BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/abs/1810.04805)
 - [@article@Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084)
 - [@article@Matryoshka Representation Learning](https://arxiv.org/abs/2205.13147)
-- [@article@MTEB: Massive Text Embedding Benchmark](https://arxiv.org/abs/2210.07316)
-- [@article@What Are Embedding Models? Benefits and Best Practices](https://cohere.com/blog/embedding-models)
+- [@article@What Are Embedding Models? Benefits and Best Practices](https://cohere.com/blog/embedding-models)
+- [@video@What is an embedding model?](https://www.youtube.com/watch?v=0U1S0WSsPuE)

roadmaps/inference-engineering/content/expert-parallelism@62y78y-RXZT_QCUfd-YFE.md

@@ -4,5 +4,6 @@ Expert Parallelism distributes MoE experts across GPUs so each GPU hosts multipl
 
 Visit the following resources to learn more:
 
-- [@opensource@NVIDIA/Megatron-LM](https://github.com/NVIDIA/Megatron-LM)
-- [@article@Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://arxiv.org/abs/1909.08053)
+- [@article@Expert parallelism](https://huggingface.co/docs/transformers/en/expert_parallelism)
+- [@article@Expert Parallelism: Scaling Mixture-of-Experts Models](https://www.digitalocean.com/community/tutorials/expert-parallelism-in-deep-learning)
+- [@article@Expert Parallelism: Distributing Experts Across Devices](https://apxml.com/courses/mixture-of-experts/chapter-4-scaling-moe-distributed-training/expert-parallelism-moe)

roadmaps/inference-engineering/content/geo-aware-load-balancing@mzF2mHSj8cvr9mvr16R0N.md

@@ -1,3 +1,8 @@
 # Geo-Aware Load Balancing
 
-A global load balancer routes requests to the GPU cluster geographically closest to the requesting user. Network latency accumulates at roughly 5 milliseconds per time zone traversed, so routing a request from Singapore to San Francisco adds 50 to 100 milliseconds of unnecessary overhead. Geo-aware routing ensures that spare capacity in a distant region is used only when nearby capacity is exhausted.
+A global load balancer routes requests to the GPU cluster geographically closest to the requesting user. Network latency accumulates at roughly 5 milliseconds per time zone traversed, so routing a request from Singapore to San Francisco adds 50 to 100 milliseconds of unnecessary overhead. Geo-aware routing ensures that spare capacity in a distant region is used only when nearby capacity is exhausted.
+
+Visit the following resources to learn more:
+
+- [@article@How Geographic Load Balancing Improves Performance](https://www.serverion.com/uncategorized/how-geographic-load-balancing-improves-performance/)
+- [@article@Geo-aware load balancing](https://www.baseten.co/resources/guide/the-baseten-inference-stack/#geo-aware-load-balancing)

roadmaps/inference-engineering/content/gpu-procurement@4mbGBgTu8AtrFHVYiffkB.md

@@ -1,3 +1,8 @@
 # GPU Procurement
 
-Cloud GPUs are available from hyperscalers like AWS and GCP, GPU-focused neoclouds like CoreWeave and Nebius, and secondary markets. Procurement mechanisms include reserved instances for predictable baseline load at discounted rates, on-demand instances for flexibility, and spot instances for cost-sensitive batch workloads that tolerate preemption. Large-scale deployments typically combine all three across multiple providers.
+Cloud GPUs are available from hyperscalers like AWS and GCP, GPU-focused neoclouds like CoreWeave and Nebius, and secondary markets. Procurement mechanisms include reserved instances for predictable baseline load at discounted rates, on-demand instances for flexibility, and spot instances for cost-sensitive batch workloads that tolerate preemption. Large-scale deployments typically combine all three across multiple providers.
+
+Visit the following resources to learn more:
+
+- [@article@Multi-Cloud GPU Strategy: How to Avoid AI Infrastructure Vendor Lock-In](https://lyceum.technology/magazine/multi-cloud-gpu-avoid-vendor-lock-in/)
+- [@article@Multi-Cloud GPU Strategy: Definition, Benefits, and Implementation](https://compux.net/docs/concepts/multi-cloud-gpu-strategy/)

roadmaps/inference-engineering/content/image--video-generation@U0l_xCUiizbJfjfrlBaJ0.md

@@ -1,3 +1,8 @@
 # Image and Video Generation
 
-Image and video generation inference is compute-bound rather than memory-bound, making it a different optimization problem from LLM serving. This group covers the production patterns, kernel choices, and parallelism techniques specific to diffusion model deployment.
+Image and video generation inference is compute-bound rather than memory-bound, making it a different optimization problem from LLM serving. This group covers the production patterns, kernel choices, and parallelism techniques specific to diffusion model deployment.
+
+Visit the following resources to learn more:
+
+- [@article@Understanding AI Image Generation: Models, Tools, and Techniques](https://www.digitalocean.com/community/tutorials/understanding-ai-image-generation-models-tools-and-techniques)
+- [@article@Understanding AI Video Generation History: Key Concepts and Impact](https://blog.prodia.com/post/understanding-ai-video-generation-history-key-concepts-and-impact)

roadmaps/inference-engineering/content/image-generation-inference@lnIHjq4nvCs3zvUln-jHR.md

@@ -4,5 +4,7 @@ Production image generation inference uses SGLang Diffusion, TensorRT, or hand-o
 
 Visit the following resources to learn more:
 
-- [@opensource@ComfyUI](https://github.com/comfyanonymous/ComfyUI)
-- [@article@Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://arxiv.org/abs/2310.04378)
+- [@opensource@Image Generation Inference Mechanics](https://github.com/comfyanonymous/ComfyUI)
+- [@article@What Is Inference in Generative AI: How It Differs From Traditional Model Inference](https://www.gmicloud.ai/en/blog/what-is-inference-in-generative-ai)
+- [@video@Diffusion Models for AI Image Generation](https://www.youtube.com/watch?v=x2GRE-RzmD8)
+- [@video@Why Does Diffusion Work Better than Auto-Regression?](https://www.youtube.com/watch?v=zc5NTeJbk-k)

roadmaps/inference-engineering/content/indep-component-scaling@SKs0yuBEAyeTPQxLll3EM.md

@@ -1,3 +1,9 @@
 # Independent Component Scaling
 
-Multi-model pipelines, like a voice agent that runs VAD, ASR, an LLM, and TTS in sequence, have components with different hardware requirements and traffic patterns. Each component should autoscale independently based on its own load. Running all components within the same Kubernetes cluster minimizes inter-component network latency, which can otherwise consume a significant fraction of a tight latency budget.
+Multi-model pipelines, like a voice agent that runs VAD, ASR, an LLM, and TTS in sequence, have components with different hardware requirements and traffic patterns. Each component should autoscale independently based on its own load. Running all components within the same Kubernetes cluster minimizes inter-component network latency, which can otherwise consume a significant fraction of a tight latency budget.
+
+Visit the following resources to learn more:
+
+- [@official@Autoscaling Workloads](https://kubernetes.io/docs/concepts/workloads/autoscaling/)
+- [@article@Autoscaling endpoints for LLM inference](https://www.together.ai/blog/autoscaling-endpoints-for-llm-inference)
+- [@article@Intro to Kubernetes Autoscaling and Best Practices for Implementations](https://www.cloudbolt.io/kubernetes-autoscaling/)

roadmaps/inference-engineering/content/kv-cache-storage@gkt2Gr34RJJf1NZgbmtXf.md

@@ -5,4 +5,6 @@ KV caches can be stored in GPU VRAM (fastest, tens to hundreds of gigabytes), CP
 Visit the following resources to learn more:
 
 - [@opensource@LMCache](https://github.com/LMCache/LMCache)
+- [@article@KV Cache Storage Is the New AI Inference Bottleneck](https://xinnor.io/blog/kv-cache-storage-is-the-new-ai-inference-bottleneck-how-lustre-ro-pcc-and-xiraid-make-shared-storage-competitive-with-local-nvme-for-long-context-llm-serving/)
+- [@article@Stop Calling It KV Cache: It’s Something Much Bigger](https://blog.lmcache.ai/en/2026/04/28/stop-calling-it-kv-cache-its-something-much-bigger/)
 - [@article@CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion](https://arxiv.org/abs/2405.16444)

roadmaps/inference-engineering/content/llm-based-models@M7K6d-Qw4BpM724LfM1u6.md

@@ -1,3 +1,9 @@
 # LLM-based Models
 
-LLM-based embedding models use a decoder-only transformer backbone, typically between one and eight billion parameters, to produce higher-quality semantic representations than BERT-style models. They apply a pooling operation to the final layer hidden states and are trained with contrastive objectives on large-scale retrieval datasets. The quality advantage is most pronounced on complex or cross-lingual retrieval tasks; the cost is higher compute per embedding and the need for GPU inference rather than CPU.
+LLM-based embedding models use a decoder-only transformer backbone, typically between one and eight billion parameters, to produce higher-quality semantic representations than BERT-style models. They apply a pooling operation to the final layer hidden states and are trained with contrastive objectives on large-scale retrieval datasets. The quality advantage is most pronounced on complex or cross-lingual retrieval tasks; the cost is higher compute per embedding and the need for GPU inference rather than CPU.
+
+Visit the following resources to learn more:
+
+- [@article@LLM Embeddings Explained: A Visual and Intuitive Guide](https://huggingface.co/spaces/hesamation/primer-llm-embedding?section=what_are_embeddings?)
+- [@article@LLMs are Also Effective Embedding Models: An In-depth Overview](https://arxiv.org/abs/2412.12591)
+- [@article@Understanding Embedding Models in the Context of Large Language Models](https://medium.com/about-ai/understanding-embedding-models-in-the-context-of-large-language-models-02da706ee9b3)

roadmaps/inference-engineering/content/lora-aware-routing@KdceJEDBdy3TbzG5suE38.md

@@ -1,3 +1,9 @@
 # LoRA-Aware Routing
 
-LoRA fine-tune weights are small adapter layers loaded on top of a base model to specialize its behavior for a specific task or customer. A deployment serving many LoRA variants for the same base model must route each request to a replica that already has the required LoRA loaded in GPU memory. Routing without this awareness forces constant LoRA swapping, adding latency and reducing throughput on each affected replica.
+LoRA fine-tune weights are small adapter layers loaded on top of a base model to specialize its behavior for a specific task or customer. A deployment serving many LoRA variants for the same base model must route each request to a replica that already has the required LoRA loaded in GPU memory. Routing without this awareness forces constant LoRA swapping, adding latency and reducing throughput on each affected replica.
+
+Visit the following resources to learn more:
+
+- [@article@Unsupervised LoRA Routing](https://www.emergentmind.com/topics/unsupervised-lora-routing)
+- [@article@Intelligent LoRA Routing](https://vllm-sr.ai/docs/v0.1/tutorials/intelligent-route/lora-routing/)
+- [@video@What is Low-Rank Adaptation (LoRA)](https://www.youtube.com/watch?v=DhRoTONcyZE)

roadmaps/inference-engineering/content/measuring-quality-impact@mK_4R8liXtl64iG7ONvgz.md

@@ -1,3 +1,10 @@
 # Measuring Quality Impact
 
-After quantizing a model, quality must be validated against the original. Three methods are used: perplexity scoring (lower is better, with minimal increase after quantization), standard intelligence benchmarks like MMLU or SWE-bench, and product-specific evals tailored to the actual use case. All three should show differences indistinguishable from noise before a quantized model is deployed.
+After quantizing a model, quality must be validated against the original. Three methods are used: perplexity scoring (lower is better, with minimal increase after quantization), standard intelligence benchmarks like MMLU or SWE-bench, and product-specific evals tailored to the actual use case. All three should show differences indistinguishable from noise before a quantized model is deployed.
+
+Visit the following resources to learn more:
+
+- [@article@Perplexity for LLM Evaluation](https://www.comet.com/site/blog/perplexity-for-llm-evaluation/)
+- [@article@AI Benchmarks Explained: MMLU, ARC-AGI, and SWE-bench](https://lumichats.com/blog/ai-benchmarks-explained-mmlu-arc-agi-swe-bench-2026)
+- [@video@What is perplexity?](https://www.youtube.com/watch?v=NURcDHhYe98&t=10s)
+- [@video@What do AI Benchmarks Actually Mean?! A Fast Breakdown (MMLU, SWE-bench, & More Explained)](https://www.youtube.com/watch?v=fSWg4HAJjBo)

roadmaps/inference-engineering/content/model-parallelism@CiXsdHHL0e2DN4ICI5jeX.md

@@ -1,3 +1,9 @@
 # Model Parallelism
 
-When a model is too large to fit on a single GPU, or when serving latency requires more compute than one device can provide, model parallelism distributes the workload across multiple GPUs or nodes. This group covers the three parallelism strategies and their trade-offs in terms of communication overhead, latency, and hardware requirements.
+When a model is too large to fit on a single GPU, or when serving latency requires more compute than one device can provide, model parallelism distributes the workload across multiple GPUs or nodes. This group covers the three parallelism strategies and their trade-offs in terms of communication overhead, latency, and hardware requirements.
+
+Visit the following resources to learn more:
+
+- [@article@Model Parallelism](https://huggingface.co/docs/transformers/v4.13.0/en/parallelism)
+- [@video@LLM Model Parallelism](https://www.youtube.com/watch?v=DryXbg48bLc)
+- [@video@How LLMs use multiple GPUs](https://www.youtube.com/watch?v=4i76hmmnJEo)

roadmaps/inference-engineering/content/modelopt@zhiKpdZm8zOg4e3bm1OgN.md

@@ -1,3 +1,10 @@
 # ModelOpt
 
-ModelOpt is NVIDIA's toolkit for post-training quantization, supporting FP8, INT8, FP4, MXFP8, MXFP4, and NVFP4 output formats. It integrates with PyTorch and exports quantized models directly to TensorRT-LLM, vLLM, and SGLang. Calibration runs a small set of representative inputs through the model to compute per-layer scale factors that minimize the gap between quantized and original outputs.
+ModelOpt is NVIDIA's toolkit for post-training quantization, supporting FP8, INT8, FP4, MXFP8, MXFP4, and NVFP4 output formats. It integrates with PyTorch and exports quantized models directly to TensorRT-LLM, vLLM, and SGLang. Calibration runs a small set of representative inputs through the model to compute per-layer scale factors that minimize the gap between quantized and original outputs.
+
+Visit the following resources to learn more:
+
+- [@official@Optimize Models with NVIDIA Model Optimizer](https://www.jetson-ai-lab.com/tutorials/modelopt-on-jetson/)
+- [@official@Model Quantization: Post-Training Quantization Using NVIDIA Model Optimizer](https://developer.nvidia.com/blog/model-quantization-post-training-quantization-using-nvidia-model-optimizer/)
+- [@opensource@Model-Optimizer](https://github.com/NVIDIA/Model-Optimizer)
+- [@article@NVIDIA ModelOpt](https://huggingface.co/docs/diffusers/en/quantization/modelopt)

roadmaps/inference-engineering/content/multi-cloud@d78UiI_FZyLsU6O52btXg.md

@@ -4,7 +4,7 @@ Multi-cloud inference distributes GPU capacity across providers and regions to m
 
 Visit the following resources to learn more:
 
-- [@book@Designing Data-Intensive Applications](https://dataintensive.net/)
-- [@book@Site Reliability Engineering](https://sre.google/books/)
-- [@article@SemiAnalysis](https://semianalysis.com/)
-- [@article@The Llama 3 Herd of Models](https://arxiv.org/abs/2407.21783)
+- [@article@How to Do Agentic AI Inference in a Multicloud, Multi-Model World](https://blog.equinix.com/blog/2025/03/19/how-to-do-agentic-ai-inference-in-a-multicloud-multi-model-world/)
+- [@article@What is multicloud?](https://cloud.google.com/learn/what-is-multicloud?hl=en)
+- [@article@Multi-Cloud GPU Strategy: How to Avoid AI Infrastructure Vendor Lock-In](https://lyceum.technology/magazine/multi-cloud-gpu-avoid-vendor-lock-in/)
+- [@video@What is Multicloud? How Do You Manage It?](https://www.youtube.com/watch?v=AjtdZ3gFRjU)

roadmaps/inference-engineering/content/multi-node-inference@FZx2xsqw_cmYLFS3GMAl-.md

@@ -1,3 +1,8 @@
 # Multi-Node Inference
 
-Multi-node inference spreads a model across more than eight GPUs using InfiniBand for inter-node communication. It is necessary for the largest models where a single node lacks sufficient VRAM for weights and KV cache. InfiniBand bandwidth is an order of magnitude lower than NVLink, so parallelism strategies must minimize cross-node communication. For MoE models, Expert Parallelism is preferred over Tensor Parallelism between nodes.
+Multi-node inference spreads a model across more than eight GPUs using InfiniBand for inter-node communication. It is necessary for the largest models where a single node lacks sufficient VRAM for weights and KV cache. InfiniBand bandwidth is an order of magnitude lower than NVLink, so parallelism strategies must minimize cross-node communication. For MoE models, Expert Parallelism is preferred over Tensor Parallelism between nodes.
+
+Visit the following resources to learn more:
+
+- [@article@How multi-node inference works for massive LLMs like DeepSeek-R1](https://www.baseten.co/blog/how-multi-node-inference-works-llms-deepseek-r1/#from-single-node-to-multi-node-infrastructure)
+- [@video@AI Models Are Huge, but Your GPUs Aren’t: Mastering Multi-Node Distributed Inference](https://www.youtube.com/watch?v=MQR8jyTR5QE&xstg=CAMSEBUJ_b-oH-PhF0yjBgb_zzE%3D)

roadmaps/inference-engineering/content/n-gram@ZOclQMd7JASGXTRyctfW2.md

@@ -4,4 +4,6 @@ N-gram speculation builds a dictionary during prefill that maps token prefixes t
 
 Visit the following resources to learn more:
 
+- [@article@Introducing N-gram Speculative Decoding: Faster Inference for Structured Tasks](https://friendli.ai/blog/n-gram-speculative-decoding)
+- [@article@What is N-gram Speculation in LLMs and How Does It Speed Up Generation?](https://outcomeschool.com/blog/n-gram-speculation-in-llms)
 - [@article@Break the Sequential Dependency of LLM Inference Using Lookahead Decoding](https://arxiv.org/abs/2402.02057)

roadmaps/inference-engineering/content/nims@KB7ExU9X49omOuk9D_Qal.md

@@ -1,3 +1,10 @@
 # NIMs
 
-NIMs are pre-built Docker containers provided by NVIDIA for popular open models, available as flexible multi-LLM containers or as GPU- and model-specific optimized containers. They provide a ready-made starting point for serving common architectures without building a custom container from scratch. For maximum control and custom configurations, starting from a less opinionated base image and building up is generally preferable.
+NIMs are pre-built Docker containers provided by NVIDIA for popular open models, available as flexible multi-LLM containers or as GPU- and model-specific optimized containers. They provide a ready-made starting point for serving common architectures without building a custom container from scratch. For maximum control and custom configurations, starting from a less opinionated base image and building up is generally preferable.
+
+Visit the following resources to learn more:
+
+- [@official@NVIDIA NIM for Developers](https://developer.nvidia.com/nim)
+- [@official@NVIDIA NIM Microservices](https://www.nvidia.com/en-us/ai-data-science/products/nim-microservices/)
+- [@official@NVIDIA NIM in Docker](https://docs.docker.com/ai/docker-agent/providers/nvidia/)
+- [@video@How to Self-Host LLMs and Multi-Modal AI Models with NVIDIA NIM in 5 Minutes](https://www.youtube.com/watch?v=087spL8hMvM&t=59s)

roadmaps/inference-engineering/content/nvidia-dynamo@KhTFecXEkHR4WX-_TYY3g.md

@@ -4,4 +4,7 @@ Dynamo supports conditional disaggregation, where requests are first sent to the
 
 Visit the following resources to learn more:
 
-- [@official@NVIDIA Dynamo Documentation](https://docs.nvidia.com/dynamo/latest/index.html)
+- [@official@NVIDIA Dynamo Documentation](https://docs.nvidia.com/dynamo/latest/index.html)
+- [@official@Disaggregated Serving](https://docs.nvidia.com/dynamo/kubernetes/disaggregated-serving/overview)
+- [@article@NVIDIA Dynamo 1.0: Disaggregated LLM Inference Deployment Guide (2026)](https://www.spheron.network/blog/nvidia-dynamo-disaggregated-inference-guide/)
+- [@video@NVIDIA Dynamo in 5 Minutes: What Is It and Why Now?](https://www.youtube.com/watch?v=mXYFcz27eDw&t=7s)

roadmaps/inference-engineering/content/observability@w-dVv5dFNxD7Rqy16m4uL.md

@@ -4,4 +4,7 @@ Inference observability tracks total request volume, input and output sequence l
 
 Visit the following resources to learn more:
 
-- [@official@NVIDIA Nsight Systems](https://developer.nvidia.com/nsight-systems)
+- [@official@NVIDIA Nsight Systems](https://developer.nvidia.com/nsight-systems)
+- [@article@LLM Observability: A Complete Guide to Monitoring Production Deployments](https://inference.net/content/llm-observability-monitoring-production-deployments/)
+- [@article@What is LLM Observability? The Ultimate Guide for AI Developers](https://www.comet.com/site/blog/llm-observability/)
+- [@article@Understanding LLM Observability](https://clickhouse.com/resources/engineering/llm-observability)

roadmaps/inference-engineering/content/omni-modal-models@plw2WiOoGxg64P-Kv-Xm7.md

@@ -1,3 +1,10 @@
 # Omni-Modal Models
 
-Omni models accept multiple input modalities and produce multiple output types in a single model. They avoid the latency of running separate specialized models in sequence but often lag behind dedicated models in quality within each modality. Production pipelines using VLMs typically coordinate preprocessing steps like OCR, audio transcription, and PDF extraction as separate independently scaling components.
+Omni models accept multiple input modalities and produce multiple output types in a single model. They avoid the latency of running separate specialized models in sequence but often lag behind dedicated models in quality within each modality. Production pipelines using VLMs typically coordinate preprocessing steps like OCR, audio transcription, and PDF extraction as separate independently scaling components.
+
+Visit the following resources to learn more:
+
+- [@article@What Is an Omni-Model?](https://www.nvidia.com/en-us/glossary/omni-model/)
+- [@article@Omni-Modal Models](https://www.baseten.co/inference-engineering/book/06-modalities/vision-language-models/#omni-modal-models)
+- [@article@OmniModels: The Unified Architecture for Intelligence](https://medium.com/@frinktyler1445/omnimodels-the-unified-architecture-for-intelligence-89856a63e975)
+- [@video@Qwen3 Omni | Multimodality, Grounding, LLM](https://www.youtube.com/watch?v=tRqEAN61qpk)

roadmaps/inference-engineering/content/pipeline-parallelism@9KLS3gWArBofLEULu3sQc.md

@@ -5,4 +5,6 @@ Pipeline Parallelism splits an LLM's layers into sequential stages and assigns e
 Visit the following resources to learn more:
 
 - [@opensource@NVIDIA/Megatron-LM](https://github.com/NVIDIA/Megatron-LM)
-- [@article@Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://arxiv.org/abs/1909.08053)
+- [@article@LLM Training — Fundamentals of Pipeline Parallelism](https://medium.com/byte-sized-ai/pipeline-parallelism-explained-in-2-mins-6bdf1ab29053)
+- [@article@Pipeline Parallelism](https://docs.pytorch.org/docs/2.14/distributed.pipelining.html)
+- [@article@What Is Pipeline Parallelism and How to Use It](https://www.newline.co/@Dipen/what-is-pipeline-parallelism-and-how-to-use-it--8db3dd11)

roadmaps/inference-engineering/content/post-training-quantization@Zvvs8cB98IRee1okBILsS.md

@@ -7,6 +7,6 @@ Visit the following resources to learn more:
 - [@opensource@AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https://github.com/mit-han-lab/llm-awq)
 - [@opensource@bitsandbytes](https://github.com/TimDettmers/bitsandbytes)
 - [@opensource@SmoothQuant](https://github.com/mit-han-lab/smoothquant)
-- [@article@GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://arxiv.org/abs/2210.17323)
-- [@article@LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://arxiv.org/abs/2208.07339)
-- [@article@SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot](https://arxiv.org/abs/2301.00774)
+- [@article@Optimizing LLMs for Performance and Accuracy with Post-Training Quantization](https://developer.nvidia.com/blog/optimizing-llms-for-performance-and-accuracy-with-post-training-quantization/)
+- [@article@Post-training quantization](https://developers.google.com/edge/litert/conversion/tensorflow/quantization/post_training_quantization)
+- [@article@Reverse-engineering GGUF | Post-Training Quantization](https://www.youtube.com/watch?v=vW30o4U9BFE)

roadmaps/inference-engineering/content/quantization-aware-training@QCXV4Utq0Ajs1krLM3fhg.md

@@ -1,3 +1,9 @@
 # Quantization-Aware Training
 
-Quantization-aware training computes scale factors and optimizes model weights jointly during training so the final model is already calibrated for the target low-precision format. This produces better quality at low precision than post-training quantization but requires retraining the model from scratch or from a checkpoint. Some labs release models trained this way, such as GPT-OSS in MXFP4.
+Quantization-aware training computes scale factors and optimizes model weights jointly during training so the final model is already calibrated for the target low-precision format. This produces better quality at low precision than post-training quantization but requires retraining the model from scratch or from a checkpoint. Some labs release models trained this way, such as GPT-OSS in MXFP4.
+
+Visit the following resources to learn more:
+
+- [@article@What is quantization aware training?](https://www.ibm.com/think/topics/quantization-aware-training)
+- [@article@How Quantization Aware Training Enables Low-Precision Accuracy Recovery](https://developer.nvidia.com/blog/how-quantization-aware-training-enables-low-precision-accuracy-recovery/)
+- [@video@Quantization Aware Training (QAT) With a Custom DataLoader](https://www.youtube.com/watch?v=s3tqqBaRuHE)

roadmaps/inference-engineering/content/quantization@DaSqcLHfnJ4_n9utVVTSk.md

@@ -7,6 +7,5 @@ Visit the following resources to learn more:
 - [@opensource@AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https://github.com/mit-han-lab/llm-awq)
 - [@opensource@bitsandbytes](https://github.com/TimDettmers/bitsandbytes)
 - [@opensource@SmoothQuant](https://github.com/mit-han-lab/smoothquant)
-- [@article@GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://arxiv.org/abs/2210.17323)
-- [@article@LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://arxiv.org/abs/2208.07339)
-- [@article@SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot](https://arxiv.org/abs/2301.00774)
+- [@article@What is quantization?](https://www.ibm.com/think/topics/quantization)
+- [@video@How LLMs survive in low precision | Quantization Fundamentals](https://www.youtube.com/watch?v=qoQJq5UwV1c)

roadmaps/inference-engineering/content/request-queueing@k-zXOGynI0B0yyT-n3raa.md

@@ -1,3 +1,9 @@
 # Request Queueing
 
-A request queue holds incoming traffic when all active replicas are at capacity while autoscaling brings new replicas online. A first-in, first-out queue is the standard implementation; priority queues can give paid users better position. Queues must be visible to newly started replicas so that requests are not held waiting for existing replicas when new capacity becomes available.
+A request queue holds incoming traffic when all active replicas are at capacity while autoscaling brings new replicas online. A first-in, first-out queue is the standard implementation; priority queues can give paid users better position. Queues must be visible to newly started replicas so that requests are not held waiting for existing replicas when new capacity becomes available.
+
+Visit the following resources to learn more:
+
+- [@article@Efficient Request Queueing – Optimizing LLM Performance](https://huggingface.co/blog/tngtech/llm-performance-request-queueing)
+- [@article@LLM Request Queuing](https://gigagpu.com/llm-request-queuing-concurrent-users/)
+- [@video@Why Do AI Requests Queue? | Inside vLLM](https://www.youtube.com/watch?v=MmRalL3oHMM)

roadmaps/inference-engineering/content/routing--load-balancing@DSyTAtFC1TvC1P7yD10Nb.md

@@ -4,6 +4,7 @@ Routers make request-level decisions about which replica to send a request to, c
 
 Visit the following resources to learn more:
 
-- [@book@Designing Data-Intensive Applications](https://dataintensive.net/)
-- [@book@Site Reliability Engineering](https://sre.google/books/)
-- [@article@The Llama 3 Herd of Models](https://arxiv.org/abs/2407.21783)
+- [@article@LLM Inference Serving: Architecture, Routing & Auto-Scaling](https://mbrenndoerfer.com/writing/llm-inference-serving-architecture-scaling-optimization)
+- [@article@LLM Model Routing: Route Queries to the Right Model Automatically](https://neuraltrust.ai/blog/llm-model-routing)
+- [@article@LLM Load Balancing: Concepts, Strategies & Best Practices](https://www.truefoundry.com/blog/llm-load-balancing)
+- [@video@Auto Routing Fully Explained](https://www.youtube.com/watch?v=2M47nZbyhac)

roadmaps/inference-engineering/content/scale-to-zero@Pi87dWgu5NTIJRQUc169s.md

@@ -1,3 +1,9 @@
 # Scale to Zero
 
-Scale to zero allows the system to run zero replicas during periods of no traffic and spin up on demand. It requires fast cold starts and robust queuing to hold incoming requests while a replica starts. Scale to zero is well-suited for development environments and periodically accessed workloads, but not for latency-sensitive production applications with unpredictable light traffic.
+Scale to zero allows the system to run zero replicas during periods of no traffic and spin up on demand. It requires fast cold starts and robust queuing to hold incoming requests while a replica starts. Scale to zero is well-suited for development environments and periodically accessed workloads, but not for latency-sensitive production applications with unpredictable light traffic.
+
+Visit the following resources to learn more:
+
+- [@article@Should LLM Serving Scale to Zero for Cost](https://www.onesourcecloud.net/cms/should-llm-serving-scale-to-zero-cost.html)
+- [@article@Runpod Secrets: Scaling LLM Inference to Zero Cost During Downtime](https://www.runpod.io/articles/guides/runpod-secrets-scale-llm-inference-zero-cost)
+- [@article@Zero Redundancy Optimizer (ZeRO): Scaling LLM Training Without Breaking the Bank](https://medium.com/@adilmaqsood501/zero-redundancy-optimizer-zero-scaling-llm-training-without-breaking-the-bank-ba045b6a805e)

roadmaps/inference-engineering/content/security-and-compliance@4x-4BFZH7bfNteD5ZAwTI.md

@@ -1,3 +1,9 @@
 # Security and Compliance
 
-Inference infrastructure must protect user inputs, model outputs, and model weights. Standard practices include data encryption, container security, network isolation, and access controls validated by third-party penetration testing. Regulated industries require compliance with frameworks like SOC 2 and HIPAA, which also apply to underlying cloud providers. Multi-cloud deployments support data residency requirements by routing each country's data to clusters in that country.
+Inference infrastructure must protect user inputs, model outputs, and model weights. Standard practices include data encryption, container security, network isolation, and access controls validated by third-party penetration testing. Regulated industries require compliance with frameworks like SOC 2 and HIPAA, which also apply to underlying cloud providers. Multi-cloud deployments support data residency requirements by routing each country's data to clusters in that country.
+
+Visit the following resources to learn more:
+
+- [@article@Inference security: How to protect APIs in production AI systems](https://www.gmicloud.ai/zh-tw/blog/inference-security-how-to-protect-apis-in-production-ai-systems)
+- [@article@How to achieve compliance and security in AI inference](https://gcore.com/blog/compliance-and-security-in-ai-inference)
+- [@article@Are your AI Inference and GenAI Environments Secure? These Five Essentials Will Help](https://www.a10networks.com/blog/are-your-ai-inference-and-genai-environments-secure/)

roadmaps/inference-engineering/content/sglang-genai-bench@KHDnY7byKgmaPtC_0uDBM.md

@@ -4,5 +4,5 @@ SGLang Genai-bench is a benchmarking tool built into the SGLang project for meas
 
 Visit the following resources to learn more:
 
-- [@opensource@genai-bench](https://github.com/sgl-project/genai-bench)
-- [@official@GenAI Bench](https://sgl-project.github.io/genai-bench/)
+- [@official@GenAI Bench](https://sgl-project.github.io/genai-bench/)
+- [@opensource@genai-bench](https://github.com/sgl-project/genai-bench)

roadmaps/inference-engineering/content/speculative-decoding@gMWgd10cxcQG_RG2oSAEH.md

@@ -4,5 +4,8 @@ Speculative decoding takes advantage of spare compute during memory-bound decode
 
 Visit the following resources to learn more:
 
+- [@article@An Introduction to Speculative Decoding for Reducing Latency in AI Inference](https://developer.nvidia.com/blog/an-introduction-to-speculative-decoding-for-reducing-latency-in-ai-inference/)
+- [@article@Speculative Decoding - vLLM](https://docs.vllm.ai/en/latest/features/speculative_decoding/)
 - [@article@Fast Inference from Transformers via Speculative Decoding](https://arxiv.org/abs/2211.17192)
-- [@article@Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads](https://arxiv.org/abs/2401.10774)
+- [@video@Faster LLMs: Accelerate Inference with Speculative Decoding](https://www.youtube.com/watch?v=VkWlLSTdHs8)
+- [@video@Speculative Decoding: When Two LLMs are Faster than One](https://www.youtube.com/watch?v=S-8yr_RibJ4)

roadmaps/inference-engineering/content/speech-to-speech-models@moyS740HQoUUovilx1Yyt.md

@@ -1,3 +1,10 @@
 # Speech-to-Speech Models
 
-Speech-to-speech models unify ASR, reasoning, and TTS into a single model by expanding the LLM vocabulary with audio input and output tokens, eliminating the latency of cascading three separate models in sequence. Current closed models like OpenAI's gpt-4o-realtime exist but remain more expensive and less capable than well-tuned cascading pipelines of dedicated ASR, LLM, and TTS models. Open speech-to-speech models are an active research area without commercially viable options at the time of writing.
+Speech-to-speech models unify ASR, reasoning, and TTS into a single model by expanding the LLM vocabulary with audio input and output tokens, eliminating the latency of cascading three separate models in sequence. Current closed models like OpenAI's gpt-4o-realtime exist but remain more expensive and less capable than well-tuned cascading pipelines of dedicated ASR, LLM, and TTS models. Open speech-to-speech models are an active research area without commercially viable options at the time of writing.
+
+Visit the following resources to learn more:
+
+- [@article@What Is Speech-to-Speech? Uses, Benefits & How It Works](https://deepgram.com/learn/what-is-speech-to-speech)
+- [@article@Speech-to-Speech LLMs](https://www.emergentmind.com/topics/speech-to-speech-large-language-models-sllms)
+- [@article@Best Speech-to-Speech Model](https://inworld.ai/resources/best-speech-to-speech-model)
+- [@video@Speech-to-Speech Model Research at Google DeepMind](https://www.youtube.com/watch?v=18Um2VjMM_g)

roadmaps/inference-engineering/content/speech@Lk8906Mpi3yxiSt8EC_BC.md

@@ -1,3 +1,9 @@
 # Speech
 
-Speech inference covers the pipeline from raw audio to text and back, including the detection, transcription, diarization, and synthesis steps that make up voice applications. Each component has distinct inference characteristics and often runs on separate hardware to avoid resource contention.
+Speech inference covers the pipeline from raw audio to text and back, including the detection, transcription, diarization, and synthesis steps that make up voice applications. Each component has distinct inference characteristics and often runs on separate hardware to avoid resource contention.
+
+Visit the following resources to learn more:
+
+- [@article@TTS Models](https://www.baseten.co/inference-engineering/book/06-modalities/tts-models/)
+- [@video@How streaming ASR inference differs from LLM serving](https://www.youtube.com/watch?v=8HJWwD5Uacc)
+- [@video@Speech LLMs: Models that listen and talk back](https://www.youtube.com/watch?v=MyxgEx4_Moo)

roadmaps/inference-engineering/content/streaming-tts@3VbPNaSnzWZ-pC-lGFMrh.md

@@ -1,3 +1,9 @@
 # Streaming TTS
 
-Real-time TTS streams audio continuously over WebSockets rather than returning a complete file after generation finishes. The audio decoder converts token streams to waveforms as they are generated, implemented in compiled PyTorch with dynamic batching. After tuning the inference engine and decoder, the primary performance lever is how many concurrent real-time streams a single GPU replica can sustain.
+Real-time TTS streams audio continuously over WebSockets rather than returning a complete file after generation finishes. The audio decoder converts token streams to waveforms as they are generated, implemented in compiled PyTorch with dynamic batching. After tuning the inference engine and decoder, the primary performance lever is how many concurrent real-time streams a single GPU replica can sustain.
+
+Visit the following resources to learn more:
+
+- [@article@Streaming TTS](https://soniox.com/wiki/streaming-tts)
+- [@article@Streaming Text-to-Speech](https://www.emergentmind.com/topics/streaming-text-to-speech-tts)
+- [@video@Streaming Speech to Text Models - Kyutai vs Whisper](https://www.youtube.com/watch?v=TTx6M4CCbXk)

roadmaps/inference-engineering/content/tensor-parallelism@csghcNLDywO8oTIZNXeyP.md

@@ -5,4 +5,6 @@ Tensor Parallelism splits each layer's weight matrices across multiple GPUs and
 Visit the following resources to learn more:
 
 - [@opensource@NVIDIA/Megatron-LM](https://github.com/NVIDIA/Megatron-LM)
-- [@article@Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://arxiv.org/abs/1909.08053)
+- [@article@Tensor Parallelism](https://huggingface.co/docs/text-generation-inference/en/conceptual/tensor_parallelism)
+- [@article@Demystifying Tensor Parallelism](https://robotchinwag.com/posts/demystifying-tensor-parallelism/)
+- [@article@What Is Tensor Parallelism and How to Apply It](https://www.newline.co/@Dipen/what-is-tensor-parallelism-and-how-to-apply-it--b4633fa4)

roadmaps/inference-engineering/content/tensorrt-llm@n2_zIGkQgyYCMYPypz3vb.md

@@ -4,7 +4,7 @@ TensorRT-LLM is NVIDIA's inference engine combining PyTorch-based model executio
 
 Visit the following resources to learn more:
 
-- [@opensource@NVIDIA/TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM)
 - [@official@TensorRT-LLM](https://developer.nvidia.com/tensorrt#inference)
+- [@opensource@NVIDIA/TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM)
 - [@video@TensorRT-LLM is Game Changer](https://www.youtube.com/watch?v=FWhK4O1RXxs)
 - [@video@TensorRT LLM 1.0 Livestream: New Easy-To-Use Pythonic Runtime](https://www.youtube.com/watch?v=sTpQQSbta1k)

roadmaps/inference-engineering/content/testing--deployment@ZlOtQtxJST10nuHjhbUx6.md

@@ -1,3 +1,7 @@
 # Testing and Deployment
 
-Deploying inference services safely requires tools for rolling out changes without downtime, estimating costs before committing to a configuration, and maintaining visibility into system health once live. This group covers the deployment, cost, observability, and compliance concerns that apply once a service is ready for production traffic.
+Deploying inference services safely requires tools for rolling out changes without downtime, estimating costs before committing to a configuration, and maintaining visibility into system health once live. This group covers the deployment, cost, observability, and compliance concerns that apply once a service is ready for production traffic.
+
+Visit the following resources to learn more:
+
+- [@article@Testing and Deployment AI Models](https://www.baseten.co/inference-engineering/book/07-production/testing-and-deployment/)

roadmaps/inference-engineering/content/text-to-speech-tts@XnhhN1QVWzlp9DUwkii9-.md

@@ -1,3 +1,10 @@
 # Text-to-Speech (TTS)
 
-Modern TTS models like Orpheus TTS are fine-tuned LLMs with an expanded vocabulary that includes audio tokens. They generate audio token sequences decoded to waveforms by a small audio decoder. The key performance metrics are time to first byte of audio (TTFB) and the number of concurrent real-time streams a single GPU replica can sustain, rather than raw token throughput.
+Modern TTS models like Orpheus TTS are fine-tuned LLMs with an expanded vocabulary that includes audio tokens. They generate audio token sequences decoded to waveforms by a small audio decoder. The key performance metrics are time to first byte of audio (TTFB) and the number of concurrent real-time streams a single GPU replica can sustain, rather than raw token throughput.
+
+Visit the following resources to learn more:
+
+- [@article@What is text to speech?](https://www.ibm.com/think/topics/text-to-speech)
+- [@article@Text-to-Speech](https://huggingface.co/tasks/text-to-speech)
+- [@article@TTS Models](https://www.baseten.co/inference-engineering/book/06-modalities/tts-models/)
+- [@video@Create your own voices with Gemini 3.8 text-to-speech](https://www.youtube.com/watch?v=FL6mI_Br-mc)

roadmaps/inference-engineering/content/transformers--diffusers@H_e3KZ7p7enUCYE5M5Lr7.md

@@ -4,7 +4,7 @@ The transformers and diffusers libraries provide reference implementations of po
 
 Visit the following resources to learn more:
 
+- [@course@Hugging Face Course](https://www.youtube.com/playlist?list=PLo2EIpI_JMQvWfQndUesu0nPBAtZ9gP1o)
 - [@official@Hugging Face Diffusers](https://huggingface.co/docs/diffusers/index)
 - [@official@Hugging Face Transformers](https://huggingface.co/docs/transformers/index)
-- [@course@Hugging Face Course](https://www.youtube.com/playlist?list=PLo2EIpI_JMQvWfQndUesu0nPBAtZ9gP1o)
 - [@video@Getting Started With Hugging Face in 15 Minutes](https://www.youtube.com/watch?v=QEaBAZQCtwE)

roadmaps/inference-engineering/content/video-generation-inference@PqEH1ez5dg5tEsspgfAK1.md

@@ -6,4 +6,6 @@ Visit the following resources to learn more:
 
 - [@opensource@Cache-DiT](https://github.com/vipshop/cache-dit)
 - [@opensource@TeaCache: Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model](https://github.com/ali-vilab/TeaCache)
-- [@article@SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration](https://arxiv.org/abs/2410.02367)
+- [@article@Video Generation Models](https://www.baseten.co/inference-engineering/book/06-modalities/video-generation-models/)
+- [@article@Video generation - HuggingFace](https://huggingface.co/docs/diffusers/en/using-diffusers/text-img2vid)
+- [@video@How do AI video generation models work?](https://www.youtube.com/watch?v=Y5zHHoLrvAE)

roadmaps/inference-engineering/content/video-processing-for-vlms@7pRZu7xa2_43HVwKty9px.md

@@ -4,4 +4,8 @@ One second of video contains 24 frames, each requiring roughly 1,000 tokens to r
 
 Visit the following resources to learn more:
 
-- [@opensource@Awesome LLMs for Video Understanding](https://github.com/yunlong10/Awesome-LLMs-for-Video-Understanding)
+- [@opensource@Awesome LLMs for Video Understanding](https://github.com/yunlong10/Awesome-LLMs-for-Video-Understanding)
+- [@article@Vision Language Model Prompt Engineering Guide for Image and Video Understanding](https://developer.nvidia.com/blog/vision-language-model-prompt-engineering-guide-for-image-and-video-understanding/)
+- [@article@Exploring the Limits of VLMs: A Dataset for Evaluating Text-to-Video Generation](https://dl.acm.org/doi/full/10.1145/3702250.3702298)
+- [@article@Vision Language Models](https://www.baseten.co/inference-engineering/book/06-modalities/vision-language-models/)
+- [@video@Vision Transformer](https://www.youtube.com/watch?v=vJF3TBI8esQ)

roadmaps/inference-engineering/content/vision-language-models@JsjmQrN1V_dM1Kw6UnYLc.md

@@ -5,7 +5,6 @@ VLMs combine a large language model with a small vision encoder that converts in
 Visit the following resources to learn more:
 
 - [@course@A Multimodal World - Hugging Face Computer Vision Course](https://huggingface.co/learn/computer-vision-course/en/unit4/multimodal-models/a_multimodal_world)
-- [@article@BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://arxiv.org/abs/2301.12597)
-- [@article@Visual Instruction Tuning](https://arxiv.org/abs/2304.08485)
-- [@article@Segment Anything](https://arxiv.org/abs/2304.02643)
-- [@article@SpecVLM: Fast Speculative Decoding in Vision-Language Models](https://arxiv.org/abs/2509.11815)
+- [@article@Vision Language Models Explained](https://huggingface.co/blog/vlms)
+- [@video@What Are Vision Language Models? How AI Sees & Understands Images](https://www.youtube.com/watch?v=lOD_EE96jhM)
+- [@video@Vision Language Models (VLMs)](https://www.youtube.com/playlist?list=PLg9z7TXgYiH3ZOyOENDy72vLMLrmD50Ze)

roadmaps/inference-engineering/content/voice-activity-detection@P0BA_p3xqpxkQsFHqM0AZ.md

@@ -1,3 +1,9 @@
 # Voice Activity Detection (VAD)
 
-VAD is a lightweight model that monitors an audio stream or file and identifies segments containing speech, filtering out silence. It is a required preprocessing component in both real-time ASR pipelines and long-file transcription pipelines, splitting audio into discrete chunks at natural speech boundaries rather than fixed time intervals. VAD runs on dedicated hardware to avoid competing with the ASR model for GPU resources.
+VAD is a lightweight model that monitors an audio stream or file and identifies segments containing speech, filtering out silence. It is a required preprocessing component in both real-time ASR pipelines and long-file transcription pipelines, splitting audio into discrete chunks at natural speech boundaries rather than fixed time intervals. VAD runs on dedicated hardware to avoid competing with the ASR model for GPU resources.
+
+Visit the following resources to learn more:
+
+- [@article@Voice Activity Detection (VAD): The Complete 2026 Guide to Speech Detection](https://picovoice.ai/blog/complete-guide-voice-activity-detection-vad/)
+- [@article@What is voice activity detection (VAD)?](https://soniox.com/wiki/voice-activity-detection)
+- [@video@Voice Activity Detection | The key to realtime voice chat](https://www.youtube.com/watch?v=HUbYXGeR8_c)

roadmaps/inference-engineering/content/zero-downtime-deplo@le4sPQ0h9l-vlmCB4G8ZY.md

@@ -1,3 +1,9 @@
 # Zero-Downtime Deployment
 
-Canary deployments update inference services without interrupting live traffic by routing a small percentage of requests to the new deployment, monitoring for issues, and gradually shifting traffic over. Blue-green deployments, which cut all traffic at once to a parallel environment, are impractical for large inference workloads because they require doubling GPU capacity during the transition. With autoscaling, canary deployments are low cost since the old service scales down as the new one takes traffic.
+Canary deployments update inference services without interrupting live traffic by routing a small percentage of requests to the new deployment, monitoring for issues, and gradually shifting traffic over. Blue-green deployments, which cut all traffic at once to a parallel environment, are impractical for large inference workloads because they require doubling GPU capacity during the transition. With autoscaling, canary deployments are low cost since the old service scales down as the new one takes traffic.
+
+Visit the following resources to learn more:
+
+- [@article@Zero-Downtime Deployments: A Practical Guide for Busy Engineers](https://medium.com/codetodeploy/zero-downtime-deployments-a-practical-guide-for-busy-engineers-2d7b43f1a2bc)
+- [@article@How do software systems achieve zero downtime on new deploys?](https://www.youtube.com/watch?v=BiuQ-sVbWGU)
+- [@article@Zero Downtime Deployments in Kubernetes Explained](https://www.youtube.com/watch?v=Tb1x1KtyVww)