chore: sync content to repo (#10413)

Co-authored-by: nilbuild <4921183+nilbuild@users.noreply.github.com>

roadmaps/inference-engineering/content/arithmetic-intensity@bPO6uwod30-ewwbo8i1eU.md

@@ -1,3 +1,10 @@
 # Arithmetic Intensity
 
-Arithmetic intensity is a property of an algorithm: the number of floating-point operations performed per byte of memory read or written. A large matrix multiplication has high arithmetic intensity because the same weights participate in many operations per load; a decode forward pass has low arithmetic intensity because each weight is loaded from VRAM and used once per token. Comparing a workload's arithmetic intensity against the GPU's ops:byte ratio is the fastest way to diagnose whether adding compute or adding bandwidth would help.
+Arithmetic intensity is a property of an algorithm: the number of floating-point operations performed per byte of memory read or written. A large matrix multiplication has high arithmetic intensity because the same weights participate in many operations per load; a decode forward pass has low arithmetic intensity because each weight is loaded from VRAM and used once per token. Comparing a workload's arithmetic intensity against the GPU's ops:byte ratio is the fastest way to diagnose whether adding compute or adding bandwidth would help.
+
+Visit the following resources to learn more:
+
+- [@article@A guide to LLM inference and performance](https://www.baseten.co/blog/llm-transformer-inference-guide/)
+- [@article@Arithmetic Intensity : Understand Op limits — Memory or Compute](https://medium.com/better-ml/arithmetic-intensity-understand-op-limits-memory-or-compute-342fd15342bb)
+- [@article@Arithmetic Intensity Explained: The Formula and Why It Predicts GPU Speedup](https://ayoob.ai/blog/webgpu-matrix-multiplication-gemm-arithmetic-intensity)
+- [@video@Compute-Bound or Memory-Bound? Arithmetic Intensity Explained](https://www.youtube.com/watch?v=jCUWDXVohTY)

roadmaps/inference-engineering/content/attention-mechanism@2KY3T5JVHQBR8pmAiuq0V.md

@@ -4,4 +4,7 @@ Attention is the mechanism transformers use to relate each token to all previous
 
 Visit the following resources to learn more:
 
-- [@article@Attention Is All You Need](https://arxiv.org/abs/1706.03762)
+- [@article@Attention Is All You Need](https://arxiv.org/abs/1706.03762)
+- [@article@What is an attention mechanism?](https://www.ibm.com/think/topics/attention-mechanism)
+- [@video@Attention mechanism: Overview](https://www.youtube.com/watch?v=fjJOgb-E41w&t=1s)
+- [@video@Attention in transformers, step-by-step](https://www.youtube.com/watch?v=eMlx5fFNoYc)

roadmaps/inference-engineering/content/attention-optimization@KWM3V7_15_Aur0He9j2Va.md

@@ -1,3 +1,9 @@
 # Attention Optimization
 
-Attention optimization refers to techniques designed to improve the computational efficiency and memory usage of the attention mechanism within transformer models. Because standard attention scales quadratically with input sequence length, these optimizations aim to reduce computation and memory overhead. By streamlining how tokens interact with one another, these methods allow models to process longer inputs faster and run on hardware with less available memory.
+Attention optimization refers to techniques designed to improve the computational efficiency and memory usage of the attention mechanism within transformer models. Because standard attention scales quadratically with input sequence length, these optimizations aim to reduce computation and memory overhead. By streamlining how tokens interact with one another, these methods allow models to process longer inputs faster and run on hardware with less available memory.
+
+Visit the following resources to learn more:
+
+- [@article@Attention Mechanism Innovation: Next-Generation Focus](https://translated.com/resources/attention-mechanism-innovation-next-generation-focus)
+- [@article@Advanced Attention Mechanisms in Transformer LLMs](https://pub.towardsai.net/advanced-attention-mechanisms-in-transformer-llms-44cac04ec356?gi=22c6a00772a8)
+- [@article@​Sequential Attention: Making AI models leaner and faster without sacrificing accuracy](https://research.google/blog/sequential-attention-making-ai-models-leaner-and-faster-without-sacrificing-accuracy/)

roadmaps/inference-engineering/content/blackwell@LihLABERuKBw30G_j48Fj.md

@@ -4,4 +4,6 @@ The Blackwell architecture, released in late 2024 with the B200 and B300, added
 
 Visit the following resources to learn more:
 
-- [@official@NVIDIA Blackwell Architecture Technical Brief](https://resources.nvidia.com/en-us-blackwell-architecture)
+- [@official@NVIDIA Blackwell Architecture Technical Brief](https://resources.nvidia.com/en-us-blackwell-architecture)
+- [@official@NVIDIA Blackwell Architecture](https://www.nvidia.com/en-us/data-center/technologies/blackwell-architecture/)
+- [@article@NVIDIA Blackwell GPUs: Architecture, Features, Specs](https://www.nexgencloud.com/blog/performance-benchmarks/nvidia-blackwell-gpus-architecture-features-specs)

roadmaps/inference-engineering/content/bottleneck-analysis@Gtd4g4MtB9zFfahRpbE2K.md

@@ -1,3 +1,10 @@
 # Bottleneck Analysis
 
-Bottleneck analysis identifies whether a given inference workload is limited by compute or by memory bandwidth, which determines which optimizations will actually improve performance. The core tool is the ops:byte ratio: comparing a GPU's peak FLOPS against its peak memory bandwidth reveals the hardware's balance point, and comparing a model operation's arithmetic intensity against that balance point reveals whether it is compute-bound or memory-bound. LLM prefill and image generation are compute-bound; LLM decode is memory-bound.
+Bottleneck analysis identifies whether a given inference workload is limited by compute or by memory bandwidth, which determines which optimizations will actually improve performance. The core tool is the ops:byte ratio: comparing a GPU's peak FLOPS against its peak memory bandwidth reveals the hardware's balance point, and comparing a model operation's arithmetic intensity against that balance point reveals whether it is compute-bound or memory-bound. LLM prefill and image generation are compute-bound; LLM decode is memory-bound.
+
+Visit the following resources to learn more:
+
+- [@article@The Next AI Bottleneck Isn’t the Model: It’s the Inference System](https://towardsdatascience.com/the-next-ai-bottleneck-isnt-the-model-its-the-inference-system/)
+- [@article@Performance Analysis and Bottleneck Identification in AI Workflows](https://multicorewareinc.com/performance-analysis-and-bottleneck-identification-in-ai-workflows/)
+- [@article@Distributed AI Inference: Why Placement Is the New Bottleneck](https://www.akamai.com/blog/cloud/distributed-ai-inference-why-placement-new-bottleneck)
+- [@video@Inference Optimization Course](https://www.youtube.com/playlist?list=PLXV1pCewfApE)

roadmaps/inference-engineering/content/compute@o_tknzA4tqcpbU6t5HO-T.md

@@ -1,3 +1,10 @@
 # Compute
 
-A GPU's compute capability is determined by its Streaming Multiprocessors (SMs), each containing CUDA Cores for general arithmetic and Tensor Cores for matrix multiply-accumulate operations. Tensor Cores are the primary resource for inference, executing the matmuls that make up the majority of forward pass computation. GPU compute is measured in peak Tensor Core FLOPS at a given precision, and roughly doubles with each new NVIDIA architecture generation.
+A GPU's compute capability is determined by its Streaming Multiprocessors (SMs), each containing CUDA Cores for general arithmetic and Tensor Cores for matrix multiply-accumulate operations. Tensor Cores are the primary resource for inference, executing the matmuls that make up the majority of forward pass computation. GPU compute is measured in peak Tensor Core FLOPS at a given precision, and roughly doubles with each new NVIDIA architecture generation.
+
+Visit the following resources to learn more:
+
+- [@article@Understanding GPU Architecture: Structure, Layers, and Performance Explained](https://www.scalecomputing.com/resources/understanding-gpu-architecture)
+- [@article@What Is GPU Computing and How Does It Work? A Complete Guide](https://factory.fpt.ai/ai-insights/what-is-gpu-computing)
+- [@video@Understanding NVIDIA GPU Hardware as a CUDA C Programmer](https://www.youtube.com/watch?v=1Goq8Yc3dfo)
+- [@video@Fundamentals of GPU Architecture](https://www.youtube.com/playlist?list=PLxNPSjHT5qvscDTMaIAY9boOOXAJAS7y4)

roadmaps/inference-engineering/content/desktop-inference@fQDl7_3nRGa8NFivOIgTP.md

@@ -6,4 +6,5 @@ Visit the following resources to learn more:
 
 - [@official@Ollama](https://ollama.com/)
 - [@official@LM Studio](https://lmstudio.ai/)
-- [@opensource@ComfyUI](https://github.com/comfyanonymous/ComfyUI)
+- [@opensource@ComfyUI](https://github.com/comfyanonymous/ComfyUI)
+- [@article@What Is Local AI Inference — and Why It Might Change How You Use AI](https://www.merciaai.com/post/what-is-local-ai-inference-and-why-it-might-change-how-you-use-ai)

roadmaps/inference-engineering/content/diffusion-models@aagMwPPmi6ngfkIqZL-Mg.md

@@ -4,9 +4,6 @@ Diffusion models generate images by starting from random noise and iteratively d
 
 Visit the following resources to learn more:
 
-- [@article@Denoising Diffusion Probabilistic Models](https://arxiv.org/abs/2006.11239)
-- [@article@High-Resolution Image Synthesis with Latent Diffusion Models](https://arxiv.org/abs/2112.10752)
-- [@article@SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://arxiv.org/abs/2307.01952)
-- [@article@Scalable Diffusion Models with Transformers (DiT)](https://arxiv.org/abs/2212.09748)
-- [@article@How DALL-E 2 Actually Works](https://www.assemblyai.com/blog/how-dall-e-2-actually-works/)
+- [@article@What are diffusion models?](https://www.ibm.com/think/topics/diffusion-models)
+- [@video@Diffusion Models for AI Image Generation](https://www.youtube.com/watch?v=x2GRE-RzmD8)
 - [@video@How AI Image Generators Work (Stable Diffusion / DALL-E)](https://www.youtube.com/watch?v=1CIpzeNxIhU)

roadmaps/inference-engineering/content/encoder-decoder-arch@nORbxBOAtl9sNU4sIJgP3.md

@@ -4,7 +4,7 @@ Neural networks can be organized as encoder-only, decoder-only, or encoder-decod
 
 Visit the following resources to learn more:
 
-- [@article@What is an encoder-decoder model?](https://www.ibm.com/think/topics/encoder-decoder-model)
 - [@course@Encoder-Decoder Architecture](https://www.skills.google/course_templates/543)
+- [@article@What is an encoder-decoder model?](https://www.ibm.com/think/topics/encoder-decoder-model)
 - [@video@Encoder-decoder architecture: Overview](https://www.youtube.com/watch?v=zbdong_h-x4)
 - [@video@Sequence-to-Sequence (seq2seq) Encoder-Decoder Neural Networks,](https://www.youtube.com/watch?v=L8HKweZIOmg)

roadmaps/inference-engineering/content/few-step-image-generation@biVjpCA3bsKI7sa_aBIUE.md

@@ -4,4 +4,5 @@ Few-step image generation models produce usable images in eight or fewer denoisi
 
 Visit the following resources to learn more:
 
-- [@article@Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://arxiv.org/abs/2310.04378)
+- [@article@Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://arxiv.org/abs/2310.04378)
+- [@article@Few-Step Diffusion Models](https://www.emergentmind.com/topics/few-step-diffusion-model)

roadmaps/inference-engineering/content/flashattention@mQcj-H3R789o6nDh8tG9k.md

@@ -6,5 +6,5 @@ Visit the following resources to learn more:
 
 - [@opensource@Dao-AILab/flash-attention](https://github.com/Dao-AILab/flash-attention)
 - [@article@FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://arxiv.org/abs/2205.14135)
-- [@article@FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://arxiv.org/abs/2307.08691)
-- [@article@FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision](https://arxiv.org/abs/2407.08608)
+- [@video@Flash Attention: The Fastest Attention Mechanism?](https://www.youtube.com/watch?v=eJn6zsbrqUM)
+- [@video@FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://www.youtube.com/playlist?list=PLBWdTDczuFNrxnetcH-5CVLRP1u4Ua87m)

roadmaps/inference-engineering/content/gpu-architecture@00JLMvn8DUJBCw6-BxO0y.md

@@ -5,5 +5,6 @@ GPU architecture determines the hard limits of what inference can achieve on a g
 Visit the following resources to learn more:
 
 - [@book@Programming Massively Parallel Processors](https://www.goodreads.com/work/editions/10244675)
-- [@article@NVIDIA Tesla: A Unified Graphics and Computing Architecture](https://ieeexplore.ieee.org/document/4523358)
-- [@article@GPU Glossary](https://modal.com/gpu-glossary)
+- [@article@Intro to GPUs](https://max.modular.com/gpu/architecture/)
+- [@article@GPU Glossary](https://modal.com/gpu-glossary)
+- [@video@How do Graphics Cards Work? Exploring GPU Architecture](https://www.youtube.com/watch?v=h9Z4oGN89MU)

roadmaps/inference-engineering/content/gpu-models@umz3jScWQViO9-lACUU4f.md

@@ -1,3 +1,9 @@
 # GPU Models
 
-Knowing the GPU landscape — which devices are available, what they offer, and how they compare — is practical knowledge for procurement decisions, capacity planning, and understanding which optimizations apply to which hardware. This group covers the NVIDIA architectures that dominate inference deployments and the alternatives worth knowing.
+Knowing the GPU landscape — which devices are available, what they offer, and how they compare — is practical knowledge for procurement decisions, capacity planning, and understanding which optimizations apply to which hardware. This group covers the NVIDIA architectures that dominate inference deployments and the alternatives worth knowing.
+
+Visit the following resources to learn more:
+
+- [@article@12 best GPUs for AI and machine learning in 2026](https://northflank.com/blog/best-gpu-for-ai)
+- [@article@GPU machine types](https://docs.cloud.google.com/compute/docs/gpus)
+- [@video@Every 2026 GPU Brand Explained: What’s the Real Difference?](https://www.youtube.com/watch?v=namWLTrbA1A)

roadmaps/inference-engineering/content/grace--vera@kPT86kA0WwE_fWUAAKCTT.md

@@ -4,4 +4,7 @@ NVIDIA's ARM-based Grace and Vera CPUs connect to their paired GPUs via NVLink C
 
 Visit the following resources to learn more:
 
-- [@official@NVIDIA Grace CPU](https://www.nvidia.com/en-us/data-center/grace-cpu/)
+- [@official@NVIDIA Grace CPU](https://www.nvidia.com/en-us/data-center/grace-cpu/)
+- [@official@NVIDIA Vera CPU](https://www.nvidia.com/en-us/data-center/vera-cpu/)
+- [@article@NVIDIA Vera/Rubin](https://cb4uhost.com/gpus/vera-rubin)
+- [@article@NVIDIA Vera Rubin: What is it, what's new, and when you can get it](https://www.civo.com/blog/nvidia-vera-rubin)

roadmaps/inference-engineering/content/hopper@XQMbaW1tFdO8Os6mxcR2M.md

@@ -4,4 +4,7 @@ The Hopper architecture, released in 2022 with the H100 and later the H200, intr
 
 Visit the following resources to learn more:
 
-- [@official@NVIDIA H100 Tensor Core GPU Architecture](https://resources.nvidia.com/en-us-hopper-architecture/nvidia-h100-tensor-c)
+- [@official@NVIDIA H100 Tensor Core GPU Architecture](https://resources.nvidia.com/en-us-hopper-architecture/nvidia-h100-tensor-c)
+- [@official@NVIDIA Hopper Architecture](https://www.nvidia.com/en-us/data-center/technologies/hopper-architecture/)
+- [@article@What is a Hopper GPU?](https://www.ionos.com/digitalguide/server/know-how/hopper-gpu/)
+- [@article@Nvidia Hopper Explained: Why a 2022 GPU Is Still the Smart Money in 2026](https://www.moduledge.com/blog/nvidia-hopper)

roadmaps/inference-engineering/content/image--video-bottlenecks@0lRP9cmoS-_wxQSsIcRQg.md

@@ -1,3 +1,10 @@
 # Image and Video Generation Bottlenecks
 
-Image and video generation models are compute-bound because each denoising step applies attention over the entire latent space in parallel, similar to LLM prefill. The models are also relatively small, so memory bandwidth is not the limiting factor. Optimizations focus on FLOPS efficiency rather than memory traffic reduction.
+Image and video generation models are compute-bound because each denoising step applies attention over the entire latent space in parallel, similar to LLM prefill. The models are also relatively small, so memory bandwidth is not the limiting factor. Optimizations focus on FLOPS efficiency rather than memory traffic reduction.
+
+Visit the following resources to learn more:
+
+- [@article@How to Overcome Visual Data Bottlenecks in GenAI Development](https://wirestock.io/gen-ai-resources/how-to-overcome-visual-data-bottlenecks-in-genai-development)
+- [@article@6 Limitations of Current AI Video Generation Technology and Workarounds](https://kling.ai/blog/limitations-of-current-ai-video-generation-technology)
+- [@article@The biggest bottleneck in AI video generation is not the model, but the cost.](https://en.eeworld.com.cn/news/wltx/eic737186.html)
+- [@video@But how do AI images and videos actually work?](https://www.youtube.com/watch?v=iv-5mZ_9CPY&t=315s)

roadmaps/inference-engineering/content/image--video-models@Mebx7HS-I-eNP5i3f6Clf.md

@@ -1,3 +1,10 @@
 # Image & Video Models
 
-Image and video models are specialized artificial intelligence architectures designed to process, interpret, and generate visual data. These models utilize techniques like convolutional neural networks (CNNs), transformers, and diffusion processes to analyze pixel patterns for tasks such as object detection, image classification, and video frame synthesis.
+Image and video models are specialized artificial intelligence architectures designed to process, interpret, and generate visual data. These models utilize techniques like convolutional neural networks (CNNs), transformers, and diffusion processes to analyze pixel patterns for tasks such as object detection, image classification, and video frame synthesis.
+
+Visit the following resources to learn more:
+
+- [@article@Understanding AI Image Generation: Models, Tools, and Techniques](https://www.digitalocean.com/community/tutorials/understanding-ai-image-generation-models-tools-and-techniques)
+- [@article@How do AI models generate videos?](https://www.technologyreview.com/2025/09/12/1123562/how-do-ai-models-generate-videos/)
+- [@video@What Are Vision Language Models? How AI Sees & Understands Images](https://www.youtube.com/watch?v=lOD_EE96jhM)
+- [@video@But how do AI images and videos actually work?](https://www.youtube.com/watch?v=iv-5mZ_9CPY)

roadmaps/inference-engineering/content/image-generation-arch@xsV4WevPzhfydyvxVzGcw.md

@@ -4,7 +4,5 @@ Image generation models are pipelines of multiple components: a text encoder tha
 
 Visit the following resources to learn more:
 
-- [@article@Denoising Diffusion Probabilistic Models](https://arxiv.org/abs/2006.11239)
-- [@article@High-Resolution Image Synthesis with Latent Diffusion Models](https://arxiv.org/abs/2112.10752)
-- [@article@SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://arxiv.org/abs/2307.01952)
-- [@article@Scalable Diffusion Models with Transformers (DiT)](https://arxiv.org/abs/2212.09748)
+- [@article@Image Generation Guide](https://ai-guide.future.mozilla.org/content/image-models/)
+- [@video@How AI Image Generators Work (Stable Diffusion / Dall-E)](https://www.youtube.com/watch?v=1CIpzeNxIhU)

roadmaps/inference-engineering/content/instances--interconnects@dAo4PMB_VnRFMSjyDrLj0.md

@@ -5,4 +5,6 @@ Within a node, NVLink connects GPUs one-to-one at up to 1800 GB/s on Blackwell,
 Visit the following resources to learn more:
 
 - [@official@NVLink and NVSwitch](https://www.nvidia.com/en-us/data-center/nvlink/)
-- [@official@NVIDIA InfiniBand](https://www.nvidia.com/en-us/networking/products/infiniband/)
+- [@official@NVIDIA InfiniBand](https://www.nvidia.com/en-us/networking/products/infiniband/)
+- [@article@Instances](https://www.baseten.co/inference-engineering/book/03-hardware/instances/)
+- [@video@GPUs: Explained](https://www.youtube.com/watch?v=LfdK-v0SbGI)

roadmaps/inference-engineering/content/kv-cache@V678kX8FPChTZCe7Nqwhv.md

@@ -4,4 +4,7 @@ The KV cache stores the key and value tensors computed during attention for each
 
 Visit the following resources to learn more:
 
-- [@article@Efficient Memory Management for Large Language Model Serving with PagedAttention](https://arxiv.org/abs/2309.06180)
+- [@article@Efficient Memory Management for Large Language Model Serving with PagedAttention](https://arxiv.org/abs/2309.06180)
+- [@article@KV Caching Explained: Optimizing Transformer Inference Efficiency](https://huggingface.co/blog/not-lain/kv-caching)
+- [@video@How KV Cache Speeds Up LLMs for Faster AI Models on GPUs](https://www.youtube.com/watch?v=o0gkdZBtwEg)
+- [@video@The KV Cache: Memory Usage in Transformers](https://www.youtube.com/watch?v=80bIUggRJf4)

roadmaps/inference-engineering/content/large-language-models@irgGvEnY1_WW1EPtuMKgC.md

@@ -4,7 +4,7 @@ An LLM is a decoder-only transformer with three main components: an embedding la
 
 Visit the following resources to learn more:
 
-- [@book@Build a Large Language Model From Scratch](https://www.manning.com/books/build-a-large-language-model-from-scratch)
+- [@book@Build a Large Language Model From Scratch](https://dl.tehrandata.org/book/Build-a-large-language-models.pdf)
 - [@article@What is a Large Language Model (LLM)?](https://www.cloudflare.com/en-gb/learning/ai/what-is-large-language-model/)
 - [@article@New to LLMs? Start Here](https://towardsdatascience.com/new-to-llms-start-here/?utm_source=roadmap&utm_medium=Referral&utm_campaign=TDS+roadmap+integration)
 - [@video@How Large Language Models Work](https://www.youtube.com/watch?v=5sLYAQS9sWQ)

roadmaps/inference-engineering/content/llm-bottlenecks@QVadKzBfLdegat-4wgOMX.md

@@ -5,4 +5,6 @@ LLM prefill is compute-bound because it processes the entire input sequence as o
 Visit the following resources to learn more:
 
 - [@book@AI Systems Performance Engineering](https://www.oreilly.com/library/view/ai-systems-performance/9781098170202/)
-- [@article@GPU Glossary](https://modal.com/gpu-glossary)
+- [@article@Common Bottlenecks in LLM Inference at Scale (And How to Fix Them)](https://www.yottalabs.ai/post/common-bottlenecks-in-llm-inference-at-scale-and-how-to-fix-them)
+- [@article@GPU Glossary](https://modal.com/gpu-glossary)
+- [@article@The Hidden Bottlenecks in LLM Inference and How to Fix Them](https://www.digitalocean.com/community/conceptual-articles/bottlenecks-llm-inference-optimization)

roadmaps/inference-engineering/content/local--edge-inference@FilQGz3geyyvHZGyKPW3G.md

@@ -1,3 +1,10 @@
 # Local and Edge Inference
 
-Not all inference runs in a datacenter. Desktop and mobile deployments have different hardware constraints, tooling ecosystems, and use cases from cloud inference. This group covers the patterns and tools for running models on consumer hardware and mobile devices.
+Not all inference runs in a datacenter. Desktop and mobile deployments have different hardware constraints, tooling ecosystems, and use cases from cloud inference. This group covers the patterns and tools for running models on consumer hardware and mobile devices.
+
+Visit the following resources to learn more:
+
+- [@article@What Is Edge Inference?](https://aws.amazon.com/what-is/edge-inference/)
+- [@article@AI-Focused Edge Inference: Use Cases And Guide for Enterprise](https://www.mirantis.com/blog/ai-focused-edge-inference-use-cases-and-guide-for-enterprise/)
+- [@article@What Is Local Inference? Definition & Requirements](https://ltx.io/glossary/local-inference)
+- [@video@What is Edge AI?](https://www.youtube.com/watch?v=QXuvUW4B7jU&t=232s)

roadmaps/inference-engineering/content/memory@5829d5dvXq9qlMSD1CG0N.md

@@ -1,3 +1,10 @@
 # Memory
 
-GPU memory determines what fits on a single device and how fast data can move through the compute units. VRAM capacity sets the upper limit on model size and KV cache; VRAM bandwidth determines how quickly the decode phase can load weights per token. Modern datacenter GPUs use high-bandwidth memory (HBM) in generations HBM3, HBM3e, and HBM4, stacked directly on the GPU die to maximize bandwidth.
+GPU memory determines what fits on a single device and how fast data can move through the compute units. VRAM capacity sets the upper limit on model size and KV cache; VRAM bandwidth determines how quickly the decode phase can load weights per token. Modern datacenter GPUs use high-bandwidth memory (HBM) in generations HBM3, HBM3e, and HBM4, stacked directly on the GPU die to maximize bandwidth.
+
+Visit the following resources to learn more:
+
+- [@article@What is GPU Memory and Why it Matters for LLM Inference](https://www.bentoml.com/blog/what-is-gpu-memory-and-why-it-matters-for-llm-inference)
+- [@article@What is GPU Memory (VRAM)?](https://www.ertas.ai/glossary/gpu-memory)
+- [@video@NPU vs. CPU vs. GPU vs. TPU: AI Hardware Compared](https://www.youtube.com/watch?v=d3SqH0UBLEY&t=58s)
+- [@video@Memory Hierarchy | GPU Programming](https://www.youtube.com/watch?v=Zrbw0zajhJM)

roadmaps/inference-engineering/content/mixture-of-experts-moe@7BddTfOkWmlL4HAkFBXtq.md

@@ -4,4 +4,7 @@ Mixture of Experts models replace dense linear layers with a set of smaller expe
 
 Visit the following resources to learn more:
 
-- [@article@Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://arxiv.org/abs/1701.06538)
+- [@article@Mixture of Experts Explained](https://huggingface.co/blog/moe)
+- [@article@Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://arxiv.org/abs/1701.06538)
+- [@video@What is Mixture of Experts?](https://www.youtube.com/watch?v=sYDlVVyJYn4)
+- [@video@A Visual Guide to Mixture of Experts (MoE) in LLMs](https://www.youtube.com/watch?v=sOPDGQjFcuM)

roadmaps/inference-engineering/content/mobile-inference@kqzHHoiZ9y4tCS6MMlLHQ.md

@@ -1,3 +1,9 @@
 # Mobile Inference
 
-Mobile devices run inference using vendor frameworks: Google's AI Edge SDK for Android and Apple's Foundation Models and Core ML for iOS. Hardware limitations restrict practical model sizes to one to two billion parameters on current phones. Small fine-tuned models for tasks like transcription, translation, and on-device classification are the strongest fit for mobile edge inference.
+Mobile devices run inference using vendor frameworks: Google's AI Edge SDK for Android and Apple's Foundation Models and Core ML for iOS. Hardware limitations restrict practical model sizes to one to two billion parameters on current phones. Small fine-tuned models for tasks like transcription, translation, and on-device classification are the strongest fit for mobile edge inference.
+
+Visit the following resources to learn more:
+
+- [@article@Shifting AI inference from the cloud to your phone can reduce AI costs](https://www.qualcomm.com/news/onq/2025/09/shifting-ai-inference-from-the-cloud-to-your-phone-can-reduce-ai-costs)
+- [@article@LLM Inference guide for Android](https://developers.google.com/edge/mediapipe/solutions/genai/llm_inference/android)
+- [@article@Cactus: High-Performance AI Inference on Any Smartphone](https://huggingface.co/blog/rshemet/cactus-on-device-inference)

roadmaps/inference-engineering/content/nvidia@Uq3mE-0WIASHfcBDEnbLj.md

@@ -1,3 +1,8 @@
 # NVIDIA
 
-NVIDIA GPUs are the dominant hardware for inference due to their combination of Tensor Core compute, HBM bandwidth, NVLink interconnects, and the CUDA software ecosystem. The three most relevant architectures for current inference deployments are Hopper, Blackwell, and the Grace and Vera CPU platforms that pair with them.
+NVIDIA GPUs are the dominant hardware for inference due to their combination of Tensor Core compute, HBM bandwidth, NVLink interconnects, and the CUDA software ecosystem. The three most relevant architectures for current inference deployments are Hopper, Blackwell, and the Grace and Vera CPU platforms that pair with them.
+
+Visit the following resources to learn more:
+
+- [@article@Top 12 NVIDIA GPUs for AI Training & Inference in 2026](https://www.atlantic.net/gpu-server-hosting/top-nvidia-gpus-for-ai-training-and-inference/)
+- [@video@Every Nvidia Graphics Card Explained as Simply as Possible](https://www.youtube.com/watch?v=UwPezhj2zoY)

roadmaps/inference-engineering/content/opsbyte-ratio@uc2enkdtRqA5K7kbgrYJF.md

@@ -1,3 +1,8 @@
 # Ops:Byte Ratio
 
-The ops:byte ratio is a property of GPU hardware that expresses the balance between compute throughput and memory bandwidth. It is calculated by dividing peak FLOPS by peak memory bandwidth, and sets the threshold that separates compute-bound from memory-bound workloads on that device. Different GPU generations have very different ops:byte ratios, which is why an optimization that works well on one generation may have no effect on another.
+The ops:byte ratio is a property of GPU hardware that expresses the balance between compute throughput and memory bandwidth. It is calculated by dividing peak FLOPS by peak memory bandwidth, and sets the threshold that separates compute-bound from memory-bound workloads on that device. Different GPU generations have very different ops:byte ratios, which is why an optimization that works well on one generation may have no effect on another.
+
+Visit the following resources to learn more:
+
+- [@article@A guide to LLM inference and performance](https://www.baseten.co/blog/llm-transformer-inference-guide/)
+- [@article@Understanding Performance](https://docs.nvidia.com/deeplearning/performance/dl-performance-gpu-background/index.html#understand-perf)

roadmaps/inference-engineering/content/other-accelerators@am4Hh-14g6Awk0WcFmVN7.md

@@ -1,3 +1,9 @@
 # Other Accelerators
 
-Alternatives to NVIDIA GPUs include AMD MI350, Google TPUs, AWS Inferentia and Trainium, Cerebras WSE-3, and Groq LPUs. Each offers a specific advantage: AMD provides CUDA-compatible tooling via ROCm; TPUs integrate deeply with Google Cloud and JAX; Inferentia and Trainium offer cost advantages within AWS. All face the shared challenge of building an ecosystem to match the breadth of NVIDIA's CUDA libraries and inference engine support.
+Alternatives to NVIDIA GPUs include AMD MI350, Google TPUs, AWS Inferentia and Trainium, Cerebras WSE-3, and Groq LPUs. Each offers a specific advantage: AMD provides CUDA-compatible tooling via ROCm; TPUs integrate deeply with Google Cloud and JAX; Inferentia and Trainium offer cost advantages within AWS. All face the shared challenge of building an ecosystem to match the breadth of NVIDIA's CUDA libraries and inference engine support.
+
+Visit the following resources to learn more:
+
+- [@article@On-Device AI Without NVIDIA](https://www.buildmvpfast.com/blog/on-device-ai-without-nvidia-apple-silicon-intel-alternatives-2026)
+- [@article@Top 12 Cloud GPU Providers for AI and Machine Learning in 2026](https://www.runpod.io/articles/guides/top-cloud-gpu-providers)
+- [@video@Every 2026 GPU Brand Explained: What’s the Real Difference?](https://www.youtube.com/watch?v=namWLTrbA1A)

roadmaps/inference-engineering/content/pagedattention@VcIRY4vSi7JkcV6g13vkN.md

@@ -4,4 +4,6 @@ PagedAttention stores the KV cache in fixed-size pages rather than a single cont
 
 Visit the following resources to learn more:
 
-- [@article@Efficient Memory Management for Large Language Model Serving with PagedAttention](https://arxiv.org/abs/2309.06180)
+- [@article@Efficient Memory Management for Large Language Model Serving with PagedAttention](https://arxiv.org/abs/2309.06180)
+- [@article@PagedAttention](https://handbook.modular.com/inference-optimization/pagedattention/)
+- [@video@PagedAttention: Behind vLLM's Insane Speed](https://www.youtube.com/watch?v=6uPnLkCiy5g)

roadmaps/inference-engineering/content/roofline-model@XF-wMQCqPZiWv1XmryPBP.md

@@ -1,3 +1,9 @@
 # Roofline Model
 
-The roofline model is a visual framework for reasoning about hardware performance limits. It plots arithmetic intensity on the x-axis against achievable performance on the y-axis, with two ceilings: a diagonal memory bandwidth ceiling for low-intensity workloads and a horizontal compute ceiling for high-intensity ones. A workload's position relative to these ceilings shows not just whether it is compute-bound or memory-bound, but how far it sits from the hardware's theoretical maximum — which tells you how much headroom remains before an optimization hits a hard limit.
+The roofline model is a visual framework for reasoning about hardware performance limits. It plots arithmetic intensity on the x-axis against achievable performance on the y-axis, with two ceilings: a diagonal memory bandwidth ceiling for low-intensity workloads and a horizontal compute ceiling for high-intensity ones. A workload's position relative to these ceilings shows not just whether it is compute-bound or memory-bound, but how far it sits from the hardware's theoretical maximum — which tells you how much headroom remains before an optimization hits a hard limit.
+
+Visit the following resources to learn more:
+
+- [@article@What is the roofline model?](https://modal.com/gpu-glossary/perf/roofline-model)
+- [@article@Calculating inference bottlenecks](https://learn-inference.com/chapters/models/bottlenecks)
+- [@video@A very short intro to the Roofline model](https://www.youtube.com/watch?v=IrkNZG8MJ64)

roadmaps/inference-engineering/content/sizing--procurement@da0HGfSDr6_DV4ZJcrcIB.md

@@ -6,4 +6,6 @@ Visit the following resources to learn more:
 
 - [@book@Designing Data-Intensive Applications](https://dataintensive.net/)
 - [@book@Site Reliability Engineering](https://sre.google/books/)
-- [@article@SemiAnalysis](https://semianalysis.com/)
+- [@article@SemiAnalysis](https://semianalysis.com/)
+- [@article@GPU Requirements for LLM Inference: Memory, Throughput, and Sizing](https://www.onesourcecloud.net/cms/gpu-requirements-llm-inference-sizing.html)
+- [@article@Inference Is the New Bottleneck: How to Plan GPU Capacity for Production AI](https://clear.ml/blog/inference-is-the-new-bottleneck-how-to-plan-gpu-capacity-for-production-ai)

roadmaps/inference-engineering/content/transformer-blocks@xHM7qjigMmXEwxeMEeMWl.md

@@ -4,4 +4,7 @@ Transformer blocks are the repeated hidden layers of an LLM. Each block contains
 
 Visit the following resources to learn more:
 
-- [@article@Attention Is All You Need](https://arxiv.org/abs/1706.03762)
+- [@article@Attention Is All You Need](https://arxiv.org/abs/1706.03762)
+- [@article@What Are Transformer Models and How Do They Work?](https://cohere.com/llmu/what-are-transformer-models)
+- [@article@A Simplified Explanation of the Transformer Block](https://medium.com/@akhileshkapse/a-simplified-explanation-of-the-transformer-block-must-read-blog-for-nlp-enthusiasts-12ef240a62ac)
+- [@video@Transformers, the tech behind LLMs](https://www.youtube.com/watch?v=wjZofJX0v4M&t=183s)

roadmaps/inference-engineering/content/video-generation-models@yTSrQfJsE-Mf4ZU_hJwYT.md

@@ -5,4 +5,6 @@ Video generation models extend image generation to three-dimensional latent spac
 Visit the following resources to learn more:
 
 - [@article@Video Diffusion Models](https://arxiv.org/abs/2204.03458)
-- [@article@Imagen Video: High Definition Video Generation with Diffusion Models](https://arxiv.org/abs/2210.02303)
+- [@article@How do AI models generate videos?](https://www.technologyreview.com/2025/09/12/1123562/how-do-ai-models-generate-videos/)
+- [@article@Demystifying Video Generation Models](https://adityang5.medium.com/demystifying-video-generation-models-cfac8dbe1913)
+- [@video@How do AI video generation models work?](https://www.youtube.com/watch?v=Y5zHHoLrvAE)