From 1be6b1160f008e4b73d2430c15086c3d4c7d951a Mon Sep 17 00:00:00 2001 From: H23 Date: Thu, 1 Oct 2026 14:38:32 +0800 Subject: [PATCH 1/8] docs: fix broken external resource links (#10399) - Escape parentheses in the Column/Row composable anchor URLs (android) and the Bash Wikipedia URL (shell-bash) so the links are not cut off at the first unescaped parenthesis, matching #10397 - Remove stray '](' in the DNS video resource link (devsecops) All fixed URLs verified to resolve (HTTP 200). Co-authored-by: Weijian He <230282609+Madin-H23@users.noreply.github.com> --- roadmaps/android/content/column--row@yE0qAQZiEC9R8WvCdskpr.md | 4 ++-- roadmaps/devsecops/content/dns@TnDsOr1iJX3hY7vJRpp7S.md | 2 +- roadmaps/shell-bash/content/bash@EqgAG1t05RuiAO1TXRq4Y.md | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/roadmaps/android/content/column--row@yE0qAQZiEC9R8WvCdskpr.md b/roadmaps/android/content/column--row@yE0qAQZiEC9R8WvCdskpr.md index 81a7a2f50..32783d317 100644 --- a/roadmaps/android/content/column--row@yE0qAQZiEC9R8WvCdskpr.md +++ b/roadmaps/android/content/column--row@yE0qAQZiEC9R8WvCdskpr.md @@ -4,5 +4,5 @@ Column and Row are fundamental layout composables in Jetpack Compose used to arr Visit the following resources to learn more: -- [@official@Column](https://developer.android.com/reference/kotlin/androidx/compose/foundation/layout/Column.composable#Column(androidx.compose.ui.Modifier,androidx.compose.foundation.layout.Arrangement.Vertical,androidx.compose.ui.Alignment.Horizontal,kotlin.Function1)(androidx.compose.ui.Modifier,androidx.compose.foundation.layout.Arrangement.Vertical,androidx.compose.ui.Alignment.Horizontal,kotlin.Function1)) -- [@official@Row](https://developer.android.com/reference/kotlin/androidx/compose/foundation/layout/Row.composable#Row(androidx.compose.ui.Modifier,androidx.compose.foundation.layout.Arrangement.Horizontal,androidx.compose.ui.Alignment.Vertical,kotlin.Function1)(androidx.compose.ui.Modifier,androidx.compose.foundation.layout.Arrangement.Horizontal,androidx.compose.ui.Alignment.Vertical,kotlin.Function1)) \ No newline at end of file +- [@official@Column](https://developer.android.com/reference/kotlin/androidx/compose/foundation/layout/Column.composable#Column%28androidx.compose.ui.Modifier,androidx.compose.foundation.layout.Arrangement.Vertical,androidx.compose.ui.Alignment.Horizontal,kotlin.Function1%29%28androidx.compose.ui.Modifier,androidx.compose.foundation.layout.Arrangement.Vertical,androidx.compose.ui.Alignment.Horizontal,kotlin.Function1%29) +- [@official@Row](https://developer.android.com/reference/kotlin/androidx/compose/foundation/layout/Row.composable#Row%28androidx.compose.ui.Modifier,androidx.compose.foundation.layout.Arrangement.Horizontal,androidx.compose.ui.Alignment.Vertical,kotlin.Function1%29%28androidx.compose.ui.Modifier,androidx.compose.foundation.layout.Arrangement.Horizontal,androidx.compose.ui.Alignment.Vertical,kotlin.Function1%29) \ No newline at end of file diff --git a/roadmaps/devsecops/content/dns@TnDsOr1iJX3hY7vJRpp7S.md b/roadmaps/devsecops/content/dns@TnDsOr1iJX3hY7vJRpp7S.md index 5062ce0ac..4cf283974 100644 --- a/roadmaps/devsecops/content/dns@TnDsOr1iJX3hY7vJRpp7S.md +++ b/roadmaps/devsecops/content/dns@TnDsOr1iJX3hY7vJRpp7S.md @@ -7,5 +7,5 @@ Visit the following resources to learn more: - [@article@What is DNS?](https://www.cloudflare.com/en-gb/learning/dns/what-is-dns/) - [@article@Mess with DNS - DNS Playground](https://messwithdns.net/) - [@article@How DNS works (comic)](https://howdns.works/) -- [@video@DNS and How does it Work?](](https://www.youtube.com/watch?v=Wj0od2ag5sk) +- [@video@DNS and How does it Work?](https://www.youtube.com/watch?v=Wj0od2ag5sk) - [@feed@Explore top posts about DNS](https://app.daily.dev/tags/dns?ref=roadmapsh) \ No newline at end of file diff --git a/roadmaps/shell-bash/content/bash@EqgAG1t05RuiAO1TXRq4Y.md b/roadmaps/shell-bash/content/bash@EqgAG1t05RuiAO1TXRq4Y.md index 851631eec..8454bcd2b 100644 --- a/roadmaps/shell-bash/content/bash@EqgAG1t05RuiAO1TXRq4Y.md +++ b/roadmaps/shell-bash/content/bash@EqgAG1t05RuiAO1TXRq4Y.md @@ -4,7 +4,7 @@ Bash, short for Bourne Again Shell, is a command-line interpreter and a shell sc Visit the following resources to learn more: -- [@article@https://en.wikipedia.org/wiki/Bash_(Unix_shell)#:~:text=Bash (short for "Bourne Again,Chet Ramey)](https://en.wikipedia.org/wiki/Bash_(Unix_shell)#:~:text=Bash%20(short%20for%20%22Bourne%20Again,Chet%20Ramey)) +- [@article@https://en.wikipedia.org/wiki/Bash_(Unix_shell)#:~:text=Bash (short for "Bourne Again,Chet Ramey)](https://en.wikipedia.org/wiki/Bash_%28Unix_shell%29#:~:text=Bash%20%28short%20for%20%22Bourne%20Again,Chet%20Ramey%29) - [@article@Bash Tutorial](https://www.w3schools.com/bash/) - [@video@Bash Scripting Full Course 3 Hours](https://www.youtube.com/watch?v=e7BufAVwDiM) - [@video@Bash Scripting Tutorial for Beginners](https://www.youtube.com/watch?v=tK9Oc6AEnR4) \ No newline at end of file From d11ed4a4afc3b1599d96d2390caa9debedfb5798 Mon Sep 17 00:00:00 2001 From: H23 Date: Thu, 1 Oct 2026 14:42:08 +0800 Subject: [PATCH 2/8] docs: fix dead and corrupted resource links (#10400) Fixes 20 resource links across roadmaps: - Repair corrupted/truncated URLs (trailing junk, missing slashes, wrong slugs): vscode, goperf, laminar, pkg.go.dev/std, pandas query, Scala Days YouTube, CircleCI discuss, JUnit, PostgreSQL DBA roadmap - Update moved official docs: Elixir processes guide (hexdocs), Kotlin Flow, Cloudflare Calls, Infura, TestNG - Remove dead daily.dev tag feed links (pages no longer exist) - Unlink the username.github.io placeholder in the GitHub Pages intro All replacement URLs verified to resolve (HTTP 200). Co-authored-by: Weijian He <230282609+Madin-H23@users.noreply.github.com> --- roadmaps/android/content/flow@W-WTIiQml8dLK6i_V69JK.md | 2 +- roadmaps/aspnet-core/content/c@fxANnSiTb2VzA9u-YeBL1.md | 3 +-- .../aspnet-core/content/circle-ci@KyGjXgG6JFo42qOV0wqfB.md | 2 +- .../content/learn-the-basics-of-c@NvODRFR0DLINB0RlPSsvt.md | 3 +-- roadmaps/blockchain/content/infura@hJmNuEMHaMSM0QQlABFRS.md | 2 +- roadmaps/blockchain/content/vs-code@J4ev2h5cipcwh3OcuBdMU.md | 2 +- roadmaps/cloudflare/content/calls@vHQdMgaL2EEr2o_eJmOuV.md | 2 +- .../content/postgresql@YJ3cNcQGBifxTLY_X7Rsb.md | 2 +- .../frontend/content/github-pages@vpzh9bNXCxgdDFQLsot2_.md | 2 +- .../content/compiler--linker-flags@xsjUeyxweN2GbcjknDjT8.md | 2 +- .../golang/content/standard-library@-Qy9DEWVYUNRHWiXXhIWU.md | 2 +- roadmaps/java/content/junit@hY1-sEpTmpaj1PregdkFf.md | 2 +- roadmaps/java/content/testng@XU2C8bF9ICej8LS7ZGTTv.md | 2 +- .../what-are-relational-databases@soar-NBWCr4xVKj7ttfnc.md | 3 +-- .../content/filtering--querying@wxO7f8wh6JTXHYDiPcbmy.md | 2 +- roadmaps/scala/content/laminar@ZH55uB8H48i9mKBtCDciZ.md | 2 +- roadmaps/scala/content/youtube@ToGJzk3ZpvQsUwOFicxW2.md | 2 +- .../content/channel@SXOEMkcVYBsRza6BPmmwy.md | 2 +- .../object-oriented-programming@VZrERRRYhmqDx4slnZtdc.md | 3 +-- ...ganize-code-by-actor-it-belongs-to@kp86Vc3uue3IxTN9B9p59.md | 3 +-- 20 files changed, 20 insertions(+), 25 deletions(-) diff --git a/roadmaps/android/content/flow@W-WTIiQml8dLK6i_V69JK.md b/roadmaps/android/content/flow@W-WTIiQml8dLK6i_V69JK.md index 94006e383..1555a022b 100644 --- a/roadmaps/android/content/flow@W-WTIiQml8dLK6i_V69JK.md +++ b/roadmaps/android/content/flow@W-WTIiQml8dLK6i_V69JK.md @@ -5,4 +5,4 @@ Visit the following resources to learn more: - [@official@Flow](https://kotlinlang.org/docs/flow.html) -- [@official@Flow: Coroutines](https://kotlinlang.org/docs/flow-coroutines.html) \ No newline at end of file +- [@official@Flow: Coroutines](https://kotlinlang.org/docs/flow.html) \ No newline at end of file diff --git a/roadmaps/aspnet-core/content/c@fxANnSiTb2VzA9u-YeBL1.md b/roadmaps/aspnet-core/content/c@fxANnSiTb2VzA9u-YeBL1.md index 8090b0124..07e763e97 100644 --- a/roadmaps/aspnet-core/content/c@fxANnSiTb2VzA9u-YeBL1.md +++ b/roadmaps/aspnet-core/content/c@fxANnSiTb2VzA9u-YeBL1.md @@ -5,5 +5,4 @@ C# is a modern coding language that was developed by Microsoft that focuses on a Visit the following resources to learn more: - [@article@C# official website?](https://learn.microsoft.com/en-us/dotnet/csharp/) -- [@video@C# Full Course - Learn C# 10 and .NET 6 in 7 hours](https://www.youtube.com/watch?v=q_F4PyW8GTg) -- [@feed@Explore top posts about C#](https://app.daily.dev/tags/csharp?ref=roadmapsh) \ No newline at end of file +- [@video@C# Full Course - Learn C# 10 and .NET 6 in 7 hours](https://www.youtube.com/watch?v=q_F4PyW8GTg) \ No newline at end of file diff --git a/roadmaps/aspnet-core/content/circle-ci@KyGjXgG6JFo42qOV0wqfB.md b/roadmaps/aspnet-core/content/circle-ci@KyGjXgG6JFo42qOV0wqfB.md index 0e25f62e6..e85490ab9 100644 --- a/roadmaps/aspnet-core/content/circle-ci@KyGjXgG6JFo42qOV0wqfB.md +++ b/roadmaps/aspnet-core/content/circle-ci@KyGjXgG6JFo42qOV0wqfB.md @@ -5,5 +5,5 @@ CircleCI is a cloud-based continuous integration and continuous delivery platfor Visit the following resources to learn more: - [@official@Building and testing an ASP.NET apps with CircleCI](https://circleci.com/blog/building-and-testing-an-asp-net-core-application/) -- [@article@How to Setup CircleCI for ASP.NET project](https://discuss.circleci.com/t/setup-circleci-for-asp-net-project/33796s) +- [@article@How to Setup CircleCI for ASP.NET project](https://discuss.circleci.com/t/setup-circleci-for-asp-net-project/33796) - [@feed@Explore top posts about CI/CD](https://app.daily.dev/tags/cicd?ref=roadmapsh) \ No newline at end of file diff --git a/roadmaps/aspnet-core/content/learn-the-basics-of-c@NvODRFR0DLINB0RlPSsvt.md b/roadmaps/aspnet-core/content/learn-the-basics-of-c@NvODRFR0DLINB0RlPSsvt.md index 343caf8fe..0370ab94c 100644 --- a/roadmaps/aspnet-core/content/learn-the-basics-of-c@NvODRFR0DLINB0RlPSsvt.md +++ b/roadmaps/aspnet-core/content/learn-the-basics-of-c@NvODRFR0DLINB0RlPSsvt.md @@ -6,5 +6,4 @@ Visit the following resources to learn more: - [@article@Introduction to C#](https://learn.microsoft.com/en-us/dotnet/csharp/tour-of-csharp/tutorials/) - [@article@Basics Of C#](https://www.c-sharpcorner.com/UploadFile/e9fdcd/basics-of-C-Sharp/) -- [@article@C# Tutorials](https://dotnettutorials.net/course/csharp-dot-net-tutorials/) -- [@feed@Explore top posts about C#](https://app.daily.dev/tags/csharp?ref=roadmapsh) \ No newline at end of file +- [@article@C# Tutorials](https://dotnettutorials.net/course/csharp-dot-net-tutorials/) \ No newline at end of file diff --git a/roadmaps/blockchain/content/infura@hJmNuEMHaMSM0QQlABFRS.md b/roadmaps/blockchain/content/infura@hJmNuEMHaMSM0QQlABFRS.md index f5e16b4b2..f58b325c0 100644 --- a/roadmaps/blockchain/content/infura@hJmNuEMHaMSM0QQlABFRS.md +++ b/roadmaps/blockchain/content/infura@hJmNuEMHaMSM0QQlABFRS.md @@ -5,4 +5,4 @@ Infura provides a suite of tools and infrastructure that allows developers to ea Visit the following resources to learn more: - [@official@Infura](https://infura.io/) -- [@official@Infura Documentation](https://docs.infura.io/api) \ No newline at end of file +- [@official@Infura Documentation](https://docs.infura.io/) \ No newline at end of file diff --git a/roadmaps/blockchain/content/vs-code@J4ev2h5cipcwh3OcuBdMU.md b/roadmaps/blockchain/content/vs-code@J4ev2h5cipcwh3OcuBdMU.md index ded67a58e..ebc5c187c 100644 --- a/roadmaps/blockchain/content/vs-code@J4ev2h5cipcwh3OcuBdMU.md +++ b/roadmaps/blockchain/content/vs-code@J4ev2h5cipcwh3OcuBdMU.md @@ -4,7 +4,7 @@ VS Code (Visual Studio Code) is a free and popular source code editor developed Visit the following resources to learn more: -- [@official@Visual Studio Code](https://code.visualstudio.com/o) +- [@official@Visual Studio Code](https://code.visualstudio.com/) - [@official@Docs - Visual Studio Code](https://code.visualstudio.com/docs) - [@article@VSCode Basics](https://www.youtube.com/watch?v=B-s71n0dHUk) - [@article@VSCode in 100 Seconds](https://www.youtube.com/watch?v=KMxo3T_MTvY) diff --git a/roadmaps/cloudflare/content/calls@vHQdMgaL2EEr2o_eJmOuV.md b/roadmaps/cloudflare/content/calls@vHQdMgaL2EEr2o_eJmOuV.md index 2c05da8a6..38d685c82 100644 --- a/roadmaps/cloudflare/content/calls@vHQdMgaL2EEr2o_eJmOuV.md +++ b/roadmaps/cloudflare/content/calls@vHQdMgaL2EEr2o_eJmOuV.md @@ -5,4 +5,4 @@ Cloudflare Calls is a platform for building real-time audio and video applicatio Visit the following resources to learn more: - [@official@Cloudflare Calls](https://developers.cloudflare.com/calls/) -- [@official@Introduction · Cloudflare Calls](https://developers.cloudflare.com/calls/introduction/) \ No newline at end of file +- [@official@Introduction · Cloudflare Calls](https://developers.cloudflare.com/calls/) \ No newline at end of file diff --git a/roadmaps/forward-deployed-engineer/content/postgresql@YJ3cNcQGBifxTLY_X7Rsb.md b/roadmaps/forward-deployed-engineer/content/postgresql@YJ3cNcQGBifxTLY_X7Rsb.md index 710965f82..e2ada2889 100644 --- a/roadmaps/forward-deployed-engineer/content/postgresql@YJ3cNcQGBifxTLY_X7Rsb.md +++ b/roadmaps/forward-deployed-engineer/content/postgresql@YJ3cNcQGBifxTLY_X7Rsb.md @@ -4,6 +4,6 @@ PostgreSQL is a powerful, open-source relational database system that supports a Visit the following resources to learn more: -- [@roadmap@Visit the Dedicated PostgreSQL DBA Roadmap](https://roadmap.sh/postgresql) +- [@roadmap@Visit the Dedicated PostgreSQL DBA Roadmap](https://roadmap.sh/postgresql-dba) - [@official@PostgreSQL Docs](https://www.postgresql.org/docs/) - [@video@Learn PostgreSQL Tutorial - Full Course for Beginners](https://www.youtube.com/watch?v=qw--VYLpxG4) \ No newline at end of file diff --git a/roadmaps/frontend/content/github-pages@vpzh9bNXCxgdDFQLsot2_.md b/roadmaps/frontend/content/github-pages@vpzh9bNXCxgdDFQLsot2_.md index 0660a84bb..dc2f307c7 100644 --- a/roadmaps/frontend/content/github-pages@vpzh9bNXCxgdDFQLsot2_.md +++ b/roadmaps/frontend/content/github-pages@vpzh9bNXCxgdDFQLsot2_.md @@ -1,6 +1,6 @@ # GitHub Pages -GitHub Pages is a free static site hosting service that takes HTML, CSS, and JavaScript files directly from a GitHub repository and publishes them as a website. It's ideal for hosting project documentation, portfolios, blogs, and other static websites. Pages supports custom domains, HTTPS, and Jekyll for automatic site generation. Each repository can have one project site, while user and organization accounts get one primary site at [username.github.io](http://username.github.io). +GitHub Pages is a free static site hosting service that takes HTML, CSS, and JavaScript files directly from a GitHub repository and publishes them as a website. It's ideal for hosting project documentation, portfolios, blogs, and other static websites. Pages supports custom domains, HTTPS, and Jekyll for automatic site generation. Each repository can have one project site, while user and organization accounts get one primary site at username.github.io. Visit the following resources to learn more: diff --git a/roadmaps/golang/content/compiler--linker-flags@xsjUeyxweN2GbcjknDjT8.md b/roadmaps/golang/content/compiler--linker-flags@xsjUeyxweN2GbcjknDjT8.md index 3f1c7ffc5..c796be2c4 100644 --- a/roadmaps/golang/content/compiler--linker-flags@xsjUeyxweN2GbcjknDjT8.md +++ b/roadmaps/golang/content/compiler--linker-flags@xsjUeyxweN2GbcjknDjT8.md @@ -5,5 +5,5 @@ Build flags control compilation and linking. Common flags include `-ldflags` for Visit the following resources to learn more: - [@official@Flag Package](https://pkg.go.dev/flag) -- [@article@Leveraging Compiler Optimization Flags](https://goperf.dev/01-common-patterns/comp-flags/o) +- [@article@Leveraging Compiler Optimization Flags](https://goperf.dev/01-common-patterns/comp-flags/) - [@article@Compiler Optimization Flags](https://diginode.in/go/compiler-optimization-flags/) \ No newline at end of file diff --git a/roadmaps/golang/content/standard-library@-Qy9DEWVYUNRHWiXXhIWU.md b/roadmaps/golang/content/standard-library@-Qy9DEWVYUNRHWiXXhIWU.md index c9b4a40f8..8520f897a 100644 --- a/roadmaps/golang/content/standard-library@-Qy9DEWVYUNRHWiXXhIWU.md +++ b/roadmaps/golang/content/standard-library@-Qy9DEWVYUNRHWiXXhIWU.md @@ -4,6 +4,6 @@ Comprehensive collection of packages providing core functionality. Includes I/O, Visit the following resources to learn more: -- [@official@std package](https://pkg.go.dev/stds) +- [@official@std package](https://pkg.go.dev/std) - [@article@Building Robust APIs with Go's Standard Library](https://dev.to/aaravjoshi/building-robust-apis-with-gos-standard-library-a-comprehensive-guide-3036) - [@article@How to use standard library packages in Golang](https://labex.io/tutorials/go-how-to-use-standard-library-packages-in-golang-446140) \ No newline at end of file diff --git a/roadmaps/java/content/junit@hY1-sEpTmpaj1PregdkFf.md b/roadmaps/java/content/junit@hY1-sEpTmpaj1PregdkFf.md index 557b02156..9d3e5f212 100644 --- a/roadmaps/java/content/junit@hY1-sEpTmpaj1PregdkFf.md +++ b/roadmaps/java/content/junit@hY1-sEpTmpaj1PregdkFf.md @@ -4,7 +4,7 @@ JUnit is a widely-used open-source testing framework for the Java programming la Visit the following resources to learn more: -- [@official@JUnit](https://junit.org/junit5) +- [@official@JUnit](https://junit.org/junit5/) - [@official@JUnit Documentation](https://junit.org/junit5/docs/current/user-guide/) - [@article@JUnit tutorial](https://www.guru99.com/junit-tutorial.html) - [@article@Basic JUnit tutorial](https://www.baeldung.com/junit-5) \ No newline at end of file diff --git a/roadmaps/java/content/testng@XU2C8bF9ICej8LS7ZGTTv.md b/roadmaps/java/content/testng@XU2C8bF9ICej8LS7ZGTTv.md index fcb1f970b..2690088c0 100644 --- a/roadmaps/java/content/testng@XU2C8bF9ICej8LS7ZGTTv.md +++ b/roadmaps/java/content/testng@XU2C8bF9ICej8LS7ZGTTv.md @@ -5,5 +5,5 @@ TestNG is a testing framework inspired from JUnit and NUnit but introducing some Visit the following resources to learn more: - [@official@Testng](https://testng.org) -- [@official@Testng Documentation](https://testng.org/doc/documentation-main.html) +- [@official@Testng Documentation](https://testng.org/) - [@article@Testng tutorial](https://www.guru99.com/all-about-testng-and-selenium.html) \ No newline at end of file diff --git a/roadmaps/postgresql-dba/content/what-are-relational-databases@soar-NBWCr4xVKj7ttfnc.md b/roadmaps/postgresql-dba/content/what-are-relational-databases@soar-NBWCr4xVKj7ttfnc.md index eb3ac7016..9ad62599c 100644 --- a/roadmaps/postgresql-dba/content/what-are-relational-databases@soar-NBWCr4xVKj7ttfnc.md +++ b/roadmaps/postgresql-dba/content/what-are-relational-databases@soar-NBWCr4xVKj7ttfnc.md @@ -7,5 +7,4 @@ Visit the following resources to learn more: - [@course@Databases and SQL](https://www.edx.org/course/databases-5-sql) - [@article@Relational Databases](https://www.ibm.com/cloud/learn/relational-databases) - [@article@Intro To Relational Databases](https://www.udacity.com/course/intro-to-relational-databases--ud197) -- [@article@Relational Databases: Concept and History](https://www.ibm.com/topics/relational-databases) -- [@feed@Explore top posts about Relational Databases](https://app.daily.dev/tags/relational-databases?ref=roadmapsh) \ No newline at end of file +- [@article@Relational Databases: Concept and History](https://www.ibm.com/topics/relational-databases) \ No newline at end of file diff --git a/roadmaps/python-data-analysis/content/filtering--querying@wxO7f8wh6JTXHYDiPcbmy.md b/roadmaps/python-data-analysis/content/filtering--querying@wxO7f8wh6JTXHYDiPcbmy.md index c2f108d5e..eac99d9fa 100644 --- a/roadmaps/python-data-analysis/content/filtering--querying@wxO7f8wh6JTXHYDiPcbmy.md +++ b/roadmaps/python-data-analysis/content/filtering--querying@wxO7f8wh6JTXHYDiPcbmy.md @@ -4,7 +4,7 @@ Filtering in Pandas selects rows that meet specified conditions. Boolean masks, Visit the following resources to learn more: -- [@official@pandas.DataFrame.query](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.query.htmlme.query) +- [@official@pandas.DataFrame.query](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.query.html) - [@official@How do I select a subset of a DataFrame?](https://pandas.pydata.org/docs/getting_started/intro_tutorials/03_subset_data.html) - [@article@10 Elegant Ways to Filter Pandas DataFrames](https://towardsdatascience.com/stop-writing-messy-boolean-masks-10-elegant-ways-to-filter-pandas-dataframes/) - [@video@Filtering Columns and Rows in Pandas](https://www.youtube.com/watch?v=kB7FV-ijdqE) \ No newline at end of file diff --git a/roadmaps/scala/content/laminar@ZH55uB8H48i9mKBtCDciZ.md b/roadmaps/scala/content/laminar@ZH55uB8H48i9mKBtCDciZ.md index 91fe62252..a67f7115d 100644 --- a/roadmaps/scala/content/laminar@ZH55uB8H48i9mKBtCDciZ.md +++ b/roadmaps/scala/content/laminar@ZH55uB8H48i9mKBtCDciZ.md @@ -4,6 +4,6 @@ Laminar is a UI library for Scala.js that focuses on simplicity, expressiveness, Visit the following resources to learn more: -- [@official@Official Laminar Website](https://laminar.dev/v) +- [@official@Official Laminar Website](https://laminar.dev/) - [@official@Laminar Documentation](https://laminar.dev/documentation) - [@article@Build UIs with Laminar - Scala.js Tutorial](https://www.scala-js.org/doc/tutorial/laminar.html) \ No newline at end of file diff --git a/roadmaps/scala/content/youtube@ToGJzk3ZpvQsUwOFicxW2.md b/roadmaps/scala/content/youtube@ToGJzk3ZpvQsUwOFicxW2.md index df112cad3..5ef036891 100644 --- a/roadmaps/scala/content/youtube@ToGJzk3ZpvQsUwOFicxW2.md +++ b/roadmaps/scala/content/youtube@ToGJzk3ZpvQsUwOFicxW2.md @@ -7,6 +7,6 @@ Visit the following resources to learn more: - [@video@Rock the JVM](https://www.youtube.com/@rockth) - [@video@Software Mill - YouTube channel](https://www.youtube.com/@SoftwareMillCom) - [@video@Dev Inside You](https://www.youtube.com/@DevInsideYou) -- [@video@Scala Days - YouTube channel](https://roadmap.sh/r/www.youtube.com/@ScalaDaysConferences) +- [@video@Scala Days - YouTube channel](https://www.youtube.com/@ScalaDaysConferences) - [@video@IntelliJ IDEA x Scala](https://youtube.com/playlist?list=PLPZy-hmwOdEVDwhWoNSyT7OejCBxgjfsL&si=Jvr-ltV3xUud1jpH) - [@video@FP Tower](https://www.youtube.com/@fptower-programming) \ No newline at end of file diff --git a/roadmaps/server-side-game-developer/content/channel@SXOEMkcVYBsRza6BPmmwy.md b/roadmaps/server-side-game-developer/content/channel@SXOEMkcVYBsRza6BPmmwy.md index 209fa6b3c..3deebaf9e 100644 --- a/roadmaps/server-side-game-developer/content/channel@SXOEMkcVYBsRza6BPmmwy.md +++ b/roadmaps/server-side-game-developer/content/channel@SXOEMkcVYBsRza6BPmmwy.md @@ -6,4 +6,4 @@ Visit the following resources to learn more: - [@official@Go Channel Documentation](https://golang.org/doc/effective_go.html#channels) - [@official@Rust Channels for Concurrency](https://doc.rust-lang.org/book/ch16-02-message-passing.html) -- [@article@Comprehensive Guide to Channel](https://elixir-lang.org/getting-started/processes.html#using-processes-and-messages) \ No newline at end of file +- [@article@Comprehensive Guide to Channel](https://hexdocs.pm/elixir/processes.html) \ No newline at end of file diff --git a/roadmaps/software-design-architecture/content/object-oriented-programming@VZrERRRYhmqDx4slnZtdc.md b/roadmaps/software-design-architecture/content/object-oriented-programming@VZrERRRYhmqDx4slnZtdc.md index ca807c114..085bf0c94 100644 --- a/roadmaps/software-design-architecture/content/object-oriented-programming@VZrERRRYhmqDx4slnZtdc.md +++ b/roadmaps/software-design-architecture/content/object-oriented-programming@VZrERRRYhmqDx4slnZtdc.md @@ -5,5 +5,4 @@ Object-oriented programming (OOP) is a programming paradigm that uses objects an Visit the following resources to learn more: - [@article@What is Object Oriented Programming?](https://www.freecodecamp.org/news/what-is-object-oriented-programming/) -- [@article@OOP introduction](https://www.geeksforgeeks.org/introduction-of-object-oriented-programming/) -- [@feed@Explore top posts about OOP](https://app.daily.dev/tags/oop?ref=roadmapsh) \ No newline at end of file +- [@article@OOP introduction](https://www.geeksforgeeks.org/introduction-of-object-oriented-programming/) \ No newline at end of file diff --git a/roadmaps/software-design-architecture/content/organize-code-by-actor-it-belongs-to@kp86Vc3uue3IxTN9B9p59.md b/roadmaps/software-design-architecture/content/organize-code-by-actor-it-belongs-to@kp86Vc3uue3IxTN9B9p59.md index e968bc24b..b0937e0d0 100644 --- a/roadmaps/software-design-architecture/content/organize-code-by-actor-it-belongs-to@kp86Vc3uue3IxTN9B9p59.md +++ b/roadmaps/software-design-architecture/content/organize-code-by-actor-it-belongs-to@kp86Vc3uue3IxTN9B9p59.md @@ -5,5 +5,4 @@ Organizing code by the actor it belongs to involves grouping components, classes Visit the following resources to learn more: - [@article@Package by Feature vs Package by Layer](https://www.baeldung.com/java-packaging-structures) -- [@article@Screaming Architecture](https://blog.cleancoder.com/uncle-bob/2011/09/30/Screaming-Architecture.html) -- [@feed@Explore top posts about Software Architecture](https://app.daily.dev/tags/software-architecture?ref=roadmapsh) \ No newline at end of file +- [@article@Screaming Architecture](https://blog.cleancoder.com/uncle-bob/2011/09/30/Screaming-Architecture.html) \ No newline at end of file From 82c5279c142ce349752bd129ba81e2d4bf6fc356 Mon Sep 17 00:00:00 2001 From: H23 Date: Thu, 1 Oct 2026 15:23:53 +0800 Subject: [PATCH 3/8] docs: fix moved official documentation links (#10401) - NIST glossary terms: add trailing slashes (404 without them) - Swift Package Manager docs: update to swift.org/docs hub (old docs.swift.org/swiftpm pages removed) - Apple: about Apple File System moved to the File System docs group; WWDC24 community page folded into the WWDC24 hub - Cloudflare Workers: service bindings moved under Runtime APIs bindings All replacement URLs verified to resolve (HTTP 200). Co-authored-by: Weijian He <230282609+Madin-H23@users.noreply.github.com> --- .../content/service-bindings@JfpVexcbuWCx_R3EjFmbo.md | 2 +- .../content/service-bindings@WZSi9inWPhqZQvDN-C8BV.md | 2 +- .../cyber-security/content/guestos@LocGETHz6ANYinNd5ZLsS.md | 2 +- .../content/stakeholders@lv6fI3WeJawuCbwKtMRIh.md | 2 +- roadmaps/ios/content/core-os@IdGdLNgJI3WmONEFsMq-d.md | 2 +- roadmaps/ios/content/file-system@_bkRbqVXKNCnRvEFCM6mN.md | 2 +- .../content/new-feature-announcements@0dbdUmWvTIJ0fkUuLp9gR.md | 2 +- .../qa/content/vulnerability-scanning@mKM27KrX1CEViOwz1EDtQ.md | 2 +- .../swift-ui/content/creating-packages@03OPOqR5K4H-oy2KBx744.md | 2 +- roadmaps/swift-ui/content/plugins@ygVGffnYXG6j6JzM5q9mO.md | 2 +- .../content/swift-package-manager@YNfVAMdAK9vI0bUkvdVyc.md | 2 +- .../swift-ui/content/using-packages@NGyvq-w9rGEqjkxoWYoqA.md | 2 +- 12 files changed, 12 insertions(+), 12 deletions(-) diff --git a/roadmaps/cloudflare/content/service-bindings@JfpVexcbuWCx_R3EjFmbo.md b/roadmaps/cloudflare/content/service-bindings@JfpVexcbuWCx_R3EjFmbo.md index 5ba71d192..6bedabf78 100644 --- a/roadmaps/cloudflare/content/service-bindings@JfpVexcbuWCx_R3EjFmbo.md +++ b/roadmaps/cloudflare/content/service-bindings@JfpVexcbuWCx_R3EjFmbo.md @@ -4,4 +4,4 @@ Service Bindings allow Cloudflare Workers to seamlessly integrate with other Clo Visit the following resources to learn more: -- [@official@Service Bindings · Cloudflare](https://developers.cloudflare.com/workers/platform/service-bindings/) \ No newline at end of file +- [@official@Service Bindings · Cloudflare](https://developers.cloudflare.com/workers/runtime-apis/bindings/service-bindings/) \ No newline at end of file diff --git a/roadmaps/cloudflare/content/service-bindings@WZSi9inWPhqZQvDN-C8BV.md b/roadmaps/cloudflare/content/service-bindings@WZSi9inWPhqZQvDN-C8BV.md index 80811dc4b..ade45ed6b 100644 --- a/roadmaps/cloudflare/content/service-bindings@WZSi9inWPhqZQvDN-C8BV.md +++ b/roadmaps/cloudflare/content/service-bindings@WZSi9inWPhqZQvDN-C8BV.md @@ -5,4 +5,4 @@ Service Bindings allow Cloudflare Workers to seamlessly integrate with other Clo Visit the following resources to learn more: - [@official@Service Bindings - Runtime APIs](https://developers.cloudflare.com/workers/runtime-apis/bindings/service-bindings/) -- [@official@Service Bindings · Cloudflare](https://developers.cloudflare.com/workers/platform/service-bindings/) \ No newline at end of file +- [@official@Service Bindings · Cloudflare](https://developers.cloudflare.com/workers/runtime-apis/bindings/service-bindings/) \ No newline at end of file diff --git a/roadmaps/cyber-security/content/guestos@LocGETHz6ANYinNd5ZLsS.md b/roadmaps/cyber-security/content/guestos@LocGETHz6ANYinNd5ZLsS.md index 028dec65f..fcf46b77f 100644 --- a/roadmaps/cyber-security/content/guestos@LocGETHz6ANYinNd5ZLsS.md +++ b/roadmaps/cyber-security/content/guestos@LocGETHz6ANYinNd5ZLsS.md @@ -4,5 +4,5 @@ A Guest Operating System (GuestOS) is an operating system installed within a vir Visit the following resources to learn more: -- [@article@Guest Operating System - NIST Glossary](https://csrc.nist.gov/glossary/term/guest_operating_system) +- [@article@Guest Operating System - NIST Glossary](https://csrc.nist.gov/glossary/term/guest_operating_system/) - [@article@Guest Operating System](https://nordvpn.com/cybersecurity/glossary/guest-operating-system/?srsltid=AfmBOop0L-VFCtuYvEBQgHy7dCIa3sfzNVa-Zn6l0SniAYDpftfOgH7N) \ No newline at end of file diff --git a/roadmaps/cyber-security/content/stakeholders@lv6fI3WeJawuCbwKtMRIh.md b/roadmaps/cyber-security/content/stakeholders@lv6fI3WeJawuCbwKtMRIh.md index 603aff3c2..78da56385 100644 --- a/roadmaps/cyber-security/content/stakeholders@lv6fI3WeJawuCbwKtMRIh.md +++ b/roadmaps/cyber-security/content/stakeholders@lv6fI3WeJawuCbwKtMRIh.md @@ -6,4 +6,4 @@ Visit the following resources to learn more: - [@course@TryHackMe room on Cyber Governance and regulation](https://tryhackme.com/r/room/cybergovernanceregulation) - [@official@NIST Publication on Engineering Trustworthy Secure Systems](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-160v1r1.pdf) -- [@official@NIST Glossary](https://csrc.nist.gov/glossary/term/stakeholder) \ No newline at end of file +- [@official@NIST Glossary](https://csrc.nist.gov/glossary/term/stakeholder/) \ No newline at end of file diff --git a/roadmaps/ios/content/core-os@IdGdLNgJI3WmONEFsMq-d.md b/roadmaps/ios/content/core-os@IdGdLNgJI3WmONEFsMq-d.md index 14dc15315..8a505e71a 100644 --- a/roadmaps/ios/content/core-os@IdGdLNgJI3WmONEFsMq-d.md +++ b/roadmaps/ios/content/core-os@IdGdLNgJI3WmONEFsMq-d.md @@ -6,5 +6,5 @@ Visit the following resources to learn more: - [@article@Kernel API](https://developer.apple.com/documentation/kernel) - [@article@System Configuration](https://developer.apple.com/documentation/systemconfiguration) -- [@article@Apple File System Guide](https://developer.apple.com/documentation/foundation/file_system/about_apple_file_system) +- [@article@Apple File System Guide](https://developer.apple.com/documentation/foundation/file-system) - [@article@Keychain Services](https://developer.apple.com/documentation/security/keychain_services) \ No newline at end of file diff --git a/roadmaps/ios/content/file-system@_bkRbqVXKNCnRvEFCM6mN.md b/roadmaps/ios/content/file-system@_bkRbqVXKNCnRvEFCM6mN.md index 612ac4c6a..b68f96530 100644 --- a/roadmaps/ios/content/file-system@_bkRbqVXKNCnRvEFCM6mN.md +++ b/roadmaps/ios/content/file-system@_bkRbqVXKNCnRvEFCM6mN.md @@ -5,5 +5,5 @@ The iOS file system sandboxes each app's data into private directories. The Docu Visit the following resources to learn more: - [@official@File System Basics](https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html) -- [@official@About File System](https://developer.apple.com/documentation/foundation/file_system/about_apple_file_system/) +- [@official@About File System](https://developer.apple.com/documentation/foundation/file-system) - [@official@FileManager](https://developer.apple.com/documentation/foundation/filemanager) \ No newline at end of file diff --git a/roadmaps/ios/content/new-feature-announcements@0dbdUmWvTIJ0fkUuLp9gR.md b/roadmaps/ios/content/new-feature-announcements@0dbdUmWvTIJ0fkUuLp9gR.md index 86fd1696c..060674199 100644 --- a/roadmaps/ios/content/new-feature-announcements@0dbdUmWvTIJ0fkUuLp9gR.md +++ b/roadmaps/ios/content/new-feature-announcements@0dbdUmWvTIJ0fkUuLp9gR.md @@ -5,4 +5,4 @@ Apple announces new platform features at WWDC each year that affect how iOS apps Visit the following resources to learn more: - [@official@Apple Developer What's New](https://developer.apple.com/whats-new/) -- [@official@Apple WDDC Community](https://developer.apple.com/wwdc24/community/) \ No newline at end of file +- [@official@Apple WDDC Community](https://developer.apple.com/wwdc24/) \ No newline at end of file diff --git a/roadmaps/qa/content/vulnerability-scanning@mKM27KrX1CEViOwz1EDtQ.md b/roadmaps/qa/content/vulnerability-scanning@mKM27KrX1CEViOwz1EDtQ.md index 6a8db52d7..c1470ebdd 100644 --- a/roadmaps/qa/content/vulnerability-scanning@mKM27KrX1CEViOwz1EDtQ.md +++ b/roadmaps/qa/content/vulnerability-scanning@mKM27KrX1CEViOwz1EDtQ.md @@ -4,4 +4,4 @@ Vulnerability scanning uses automated tools to detect known security weaknesses Visit the following resources to learn more: -- [@official@NIST Website](https://csrc.nist.gov/glossary/term/vulnerability_scanning) \ No newline at end of file +- [@official@NIST Website](https://csrc.nist.gov/glossary/term/vulnerability_scanning/) \ No newline at end of file diff --git a/roadmaps/swift-ui/content/creating-packages@03OPOqR5K4H-oy2KBx744.md b/roadmaps/swift-ui/content/creating-packages@03OPOqR5K4H-oy2KBx744.md index a8b921c35..70fc8045d 100644 --- a/roadmaps/swift-ui/content/creating-packages@03OPOqR5K4H-oy2KBx744.md +++ b/roadmaps/swift-ui/content/creating-packages@03OPOqR5K4H-oy2KBx744.md @@ -4,7 +4,7 @@ The Swift Package Manager is a tool for managing the distribution of Swift code. Visit the following resources to learn more: -- [@official@Creating a Library Package](https://docs.swift.org/swiftpm/documentation/packagemanagerdocs/gettingstarted#Creating-a-Library-Package) +- [@official@Creating a Library Package](https://www.swift.org/documentation/package-manager/) - [@official@Creating a standalone Swift package with Xcode](https://developer.apple.com/documentation/xcode/creating-a-standalone-swift-package-with-xcode) - [@article@Swift Package Manager framework creation in Xcode](https://www.avanderlee.com/swift/creating-swift-package-manager-framework/) - [@video@Creating and Publishing Swift Packages (Swift Package Manager)](https://www.youtube.com/watch?v=4Rxuc4BcW8o) diff --git a/roadmaps/swift-ui/content/plugins@ygVGffnYXG6j6JzM5q9mO.md b/roadmaps/swift-ui/content/plugins@ygVGffnYXG6j6JzM5q9mO.md index 33fc3eabc..74df7f329 100644 --- a/roadmaps/swift-ui/content/plugins@ygVGffnYXG6j6JzM5q9mO.md +++ b/roadmaps/swift-ui/content/plugins@ygVGffnYXG6j6JzM5q9mO.md @@ -4,7 +4,7 @@ Swift Package Manager plugins allow you to extend the build process of your Swif Visit the following resources to learn more: -- [@official@Plugins](https://docs.swift.org/swiftpm/documentation/packagemanagerdocs/plugins/) +- [@official@Plugins](https://www.swift.org/documentation/package-manager/) - [@article@Meet Swift Package plugins](https://wwdcnotes.com/documentation/wwdcnotes/wwdc22-110359-meet-swift-package-plugins/) - [@video@WWDC22: Create Swift Package plugins | Apple](https://www.youtube.com/watch?v=JiyZmB6aX30) - [@video@WWDC22: Meet Swift Package plugins | Apple](https://www.youtube.com/watch?v=Oe5JPnVNhRo) \ No newline at end of file diff --git a/roadmaps/swift-ui/content/swift-package-manager@YNfVAMdAK9vI0bUkvdVyc.md b/roadmaps/swift-ui/content/swift-package-manager@YNfVAMdAK9vI0bUkvdVyc.md index 5fe51b5cc..eee16c927 100644 --- a/roadmaps/swift-ui/content/swift-package-manager@YNfVAMdAK9vI0bUkvdVyc.md +++ b/roadmaps/swift-ui/content/swift-package-manager@YNfVAMdAK9vI0bUkvdVyc.md @@ -4,6 +4,6 @@ The Swift Package Manager is a tool for managing dependencies in your Swift proj Visit the following resources to learn more: -- [@official@Swift Package Manager](https://docs.swift.org/swiftpm/documentation/packagemanagerdocs/) +- [@official@Swift Package Manager](https://www.swift.org/documentation/package-manager/) - [@opensource@swift-package-manager](https://github.com/swiftlang/swift-package-manager) - [@article@Mastering Swift Package Manager: A Comprehensive Guide](https://medium.com/@dipenapanchasara/mastering-swift-package-manager-a-comprehensive-guide-5e06f29d812d) \ No newline at end of file diff --git a/roadmaps/swift-ui/content/using-packages@NGyvq-w9rGEqjkxoWYoqA.md b/roadmaps/swift-ui/content/using-packages@NGyvq-w9rGEqjkxoWYoqA.md index 7819100d5..61682cd62 100644 --- a/roadmaps/swift-ui/content/using-packages@NGyvq-w9rGEqjkxoWYoqA.md +++ b/roadmaps/swift-ui/content/using-packages@NGyvq-w9rGEqjkxoWYoqA.md @@ -5,6 +5,6 @@ Swift Package Manager lets you add external libraries and tools to your Swift pr Visit the following resources to learn more: - [@official@Swift packages](https://developer.apple.com/documentation/xcode/swift-packages) -- [@official@Introducing Packages](https://docs.swift.org/swiftpm/documentation/packagemanagerdocs/introducingpackages) +- [@official@Introducing Packages](https://www.swift.org/documentation/package-manager/) - [@video@Adding Swift package dependencies in Xcode](https://www.hackingwithswift.com/books/ios-swiftui/adding-swift-package-dependencies-in-xcode) - [@video@How to use Third Party Swift Packages in SwiftUI | Swift Packages #0](https://www.youtube.com/watch?v=yp9n5oYONDs) \ No newline at end of file From 93dda66d387a8479034796a605c6012b93e8f847 Mon Sep 17 00:00:00 2001 From: H23 Date: Thu, 1 Oct 2026 15:39:11 +0800 Subject: [PATCH 4/8] docs: replace dead links with archived snapshots (#10402) - boffin.education restructured its site; the three technical-writer articles now live in the Internet Archive - Apple removed the uifont scaling_fonts_automatically page; archived snapshot linked - MDN removed the text-level_semantics page; link updated to the HTML element index (Text content / Inline text semantics sections) - MDN HTML5 notes page has no replacement or archive; dropped the resource line (html/case-insensitivity) All snapshot URLs verified to resolve (HTTP 200). Co-authored-by: Weijian He <230282609+Madin-H23@users.noreply.github.com> --- .../content/case-insensitivity@qXcn_AGcGWjT8GgjaW5WN.md | 1 - roadmaps/html/content/textual-tags@ksKI4TDz4_9IcmNORczWo.md | 2 +- roadmaps/ios/content/dynamic-type@0nei6iwP4Pgi_j4vVi_Qt.md | 2 +- .../who-is-a-technical-writer@ZRGsokU313Ky-anWbWK6q.md | 6 +++--- 4 files changed, 5 insertions(+), 6 deletions(-) diff --git a/roadmaps/html/content/case-insensitivity@qXcn_AGcGWjT8GgjaW5WN.md b/roadmaps/html/content/case-insensitivity@qXcn_AGcGWjT8GgjaW5WN.md index 667dad462..7e5baa71d 100644 --- a/roadmaps/html/content/case-insensitivity@qXcn_AGcGWjT8GgjaW5WN.md +++ b/roadmaps/html/content/case-insensitivity@qXcn_AGcGWjT8GgjaW5WN.md @@ -4,6 +4,5 @@ HTML isn't picky about capitalization. You can write tags and attributes using u Visit the following resources to learn more: -- [@article@HTML Style Guide - Capitalization](https://developer.mozilla.org/en-US/docs/Web/HTML/HTML5/HTML5_notes) - [@article@HTML Case Sensitivity Explained](https://www.w3schools.com/html/html5_syntax.asp) - [@article@What is Case Sensitivity in HTML?](https://www.educative.io/answers/what-is-case-sensitivity-in-html) \ No newline at end of file diff --git a/roadmaps/html/content/textual-tags@ksKI4TDz4_9IcmNORczWo.md b/roadmaps/html/content/textual-tags@ksKI4TDz4_9IcmNORczWo.md index c17f87e81..d37c2a1d7 100644 --- a/roadmaps/html/content/textual-tags@ksKI4TDz4_9IcmNORczWo.md +++ b/roadmaps/html/content/textual-tags@ksKI4TDz4_9IcmNORczWo.md @@ -4,6 +4,6 @@ Textual tags in HTML are used to structure and format text content on a webpage. Visit the following resources to learn more: -- [@article@HTML Text Formatting Elements](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/text-level_semantics) +- [@article@HTML Text Formatting Elements](https://developer.mozilla.org/en-US/docs/Web/HTML/Element) - [@article@HTML Text Tags Overview](https://www.w3schools.com/html/html_formatting.asp) - [@article@Semantic HTML for Text](https://www.freecodecamp.org/news/semantic-html5-elements/) \ No newline at end of file diff --git a/roadmaps/ios/content/dynamic-type@0nei6iwP4Pgi_j4vVi_Qt.md b/roadmaps/ios/content/dynamic-type@0nei6iwP4Pgi_j4vVi_Qt.md index e868b5dc6..9445f3e47 100644 --- a/roadmaps/ios/content/dynamic-type@0nei6iwP4Pgi_j4vVi_Qt.md +++ b/roadmaps/ios/content/dynamic-type@0nei6iwP4Pgi_j4vVi_Qt.md @@ -4,5 +4,5 @@ The Dynamic Type feature allows users to choose the size of textual content disp Visit the following resources to learn more: -- [@official@Dynamic Type](https://developer.apple.com/documentation/uikit/uifont/scaling_fonts_automatically/) +- [@official@Dynamic Type](https://web.archive.org/web/20241219200814/https://developer.apple.com/documentation/uikit/uifont/scaling_fonts_automatically/) - [@official@WWDC24: Get started with Dynamic Type](https://www.youtube.com/watch?v=ZqDZjW9TpFw) \ No newline at end of file diff --git a/roadmaps/technical-writer/content/who-is-a-technical-writer@ZRGsokU313Ky-anWbWK6q.md b/roadmaps/technical-writer/content/who-is-a-technical-writer@ZRGsokU313Ky-anWbWK6q.md index 6ca1d8db8..6cd292d6e 100644 --- a/roadmaps/technical-writer/content/who-is-a-technical-writer@ZRGsokU313Ky-anWbWK6q.md +++ b/roadmaps/technical-writer/content/who-is-a-technical-writer@ZRGsokU313Ky-anWbWK6q.md @@ -4,6 +4,6 @@ A technical writer produces documentation that helps users understand and work w Visit the following resources to learn more: -- [@article@What Is a Technical Writer?](https://boffin.education/introduction-to-technical-writing/#2-what-is-a-technical-writer) -- [@article@A Day in the Life of a Technical Writer](https://boffin.education/technical-writing-roles-and-responsibilities/#4-a-day-in-the-life-of-a-technical-writer) -- [@article@Glossary of Technical Writing Terms: "Technical Writer"](https://boffin.education/glossary-of-technical-writing-terms/#technical-writer) \ No newline at end of file +- [@article@What Is a Technical Writer?](https://web.archive.org/web/20250114032850/https://boffin.education/introduction-to-technical-writing/#2-what-is-a-technical-writer) +- [@article@A Day in the Life of a Technical Writer](https://web.archive.org/web/20250114022932/https://boffin.education/technical-writing-roles-and-responsibilities/#4-a-day-in-the-life-of-a-technical-writer) +- [@article@Glossary of Technical Writing Terms: "Technical Writer"](https://web.archive.org/web/20250214134718/https://boffin.education/glossary-of-technical-writing-terms/#technical-writer) \ No newline at end of file From 9592f78d8db4f275935483ee00b7cee254a54469 Mon Sep 17 00:00:00 2001 From: H23 Date: Thu, 1 Oct 2026 17:15:16 +0800 Subject: [PATCH 5/8] docs: replace dead article links with archived snapshots (#10403) * docs: replace dead article links with archived snapshots Six dead third-party article links replaced with Internet Archive snapshots, and one resource line dropped: - codedigest ASP.NET REST articles (2 files): site gone, archived - awesome-architecture back-pressure: site gone, archived - roblox forum stencil shadows thread: thread deleted, archived - cglab.ca rust BTreeMap case study: page gone, archived - levity.ai AI bias article: page gone, archived - ignitetech synthetic voices article: page gone, archived - alistapart brand strategy article: no live page and no archive snapshot; resource line dropped (design-system/vision) All snapshot URLs verified to resolve (HTTP 200). * Update bias-and-fairness@lhIU0ulpvDAn1Xc3ooYz_.md * Update text-to-speech resources in documentation * Update REST resources in ASP.NET Core documentation * Update rest@KJCtxH3mYsZq3hfBZ6cJF.md * Update backpressure.md by removing a resource link Removed a resource link and updated the formatting. * Update resource links for stencil shadows * Remove outdated BTreeMap article link Removed outdated article link for BTreeMap. --------- Co-authored-by: Weijian He <230282609+Madin-H23@users.noreply.github.com> Co-authored-by: Javier Canales <56018501+jcanalesluna@users.noreply.github.com> --- .../content/bias-and-fairness@lhIU0ulpvDAn1Xc3ooYz_.md | 4 ++-- .../content/text-to-speech@GCERpLz5BcRtWPpv-asUz.md | 2 +- roadmaps/aspnet-core/content/rest@KJCtxH3mYsZq3hfBZ6cJF.md | 4 ++-- roadmaps/aspnet-core/content/rest@dZ9fYWMBa1OemTmLOI8_q.md | 2 +- .../backend/content/backpressure@JansCqGDyXecQkD1K7E7e.md | 3 +-- .../design-system/content/vision@cMfsRtvzvDZZJ0TqeUOxm.md | 3 +-- .../content/stencil-shadow@AEAVc8Ih4fctSGGVJG0Np.md | 2 +- roadmaps/rust/content/btreemap@LWgaZWTJVWLDes3HzflRh.md | 1 - 8 files changed, 9 insertions(+), 12 deletions(-) diff --git a/roadmaps/ai-engineer/content/bias-and-fairness@lhIU0ulpvDAn1Xc3ooYz_.md b/roadmaps/ai-engineer/content/bias-and-fairness@lhIU0ulpvDAn1Xc3ooYz_.md index 98326d1c4..0ee6b89f5 100644 --- a/roadmaps/ai-engineer/content/bias-and-fairness@lhIU0ulpvDAn1Xc3ooYz_.md +++ b/roadmaps/ai-engineer/content/bias-and-fairness@lhIU0ulpvDAn1Xc3ooYz_.md @@ -5,5 +5,5 @@ Bias and fairness in AI refer to the challenges of ensuring that machine learnin Visit the following resources to learn more: - [@article@What Do We Do About the Biases in AI?](https://hbr.org/2019/10/what-do-we-do-about-the-biases-in-ai) -- [@article@AI Bias - What Is It and How to Avoid It?](https://levity.ai/blog/ai-bias-how-to-avoid) -- [@article@What about fairness, bias and discrimination?](https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/artificial-intelligence/guidance-on-ai-and-data-protection/how-do-we-ensure-fairness-in-ai/what-about-fairness-bias-and-discrimination/) \ No newline at end of file +- [@article@AI Bias - What Is It and How to Prevent It?]([https://web.archive.org/web/20230921120254/https://levity.ai/blog/ai-bias-how-to-avoid](https://builtin.com/artificial-intelligence/ai-bias)) +- [@article@What about fairness, bias and discrimination?](https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/artificial-intelligence/guidance-on-ai-and-data-protection/how-do-we-ensure-fairness-in-ai/what-about-fairness-bias-and-discrimination/) diff --git a/roadmaps/ai-engineer/content/text-to-speech@GCERpLz5BcRtWPpv-asUz.md b/roadmaps/ai-engineer/content/text-to-speech@GCERpLz5BcRtWPpv-asUz.md index 744569236..8de67bf1f 100644 --- a/roadmaps/ai-engineer/content/text-to-speech@GCERpLz5BcRtWPpv-asUz.md +++ b/roadmaps/ai-engineer/content/text-to-speech@GCERpLz5BcRtWPpv-asUz.md @@ -5,4 +5,4 @@ In the context of multimodal AI, text-to-speech (TTS) technology converts writte Visit the following resources to learn more: - [@article@What is Text-to-Speech?](https://aws.amazon.com/polly/what-is-text-to-speech/) -- [@article@From Text to Speech: The Evolution of Synthetic Voices](https://ignitetech.ai/about/blogs/text-speech-evolution-synthetic-voices) \ No newline at end of file +- [@article@From Text to Speech: A Deep Dive into TTS Technologies](https://medium.com/@zilliz_learn/from-text-to-speech-a-deep-dive-into-tts-technologies-18ea409f20e8) diff --git a/roadmaps/aspnet-core/content/rest@KJCtxH3mYsZq3hfBZ6cJF.md b/roadmaps/aspnet-core/content/rest@KJCtxH3mYsZq3hfBZ6cJF.md index 8fafab5ad..a646cae0c 100644 --- a/roadmaps/aspnet-core/content/rest@KJCtxH3mYsZq3hfBZ6cJF.md +++ b/roadmaps/aspnet-core/content/rest@KJCtxH3mYsZq3hfBZ6cJF.md @@ -4,7 +4,7 @@ REST, which stands for Representational State Transfer, is an architectural styl Visit the following resources to learn more: -- [@article@What is REST Services in ASP.NET?](http://www.codedigest.com/quick-start/16/what-is-rest-services-how-to-create-rest-services-in-aspnet) +- [@article@APIs with ASP.NET Core](https://dotnet.microsoft.com/en-us/apps/aspnet/apis) - [@article@What are RESTful APIs?](https://www.pragimtech.com/blog/blazor/what-are-restful-apis/) - [@video@Tutorial of Rest and Restful API](https://www.youtube.com/watch?v=4r1CIUs5s2I) -- [@feed@Explore top posts about REST API](https://app.daily.dev/tags/rest-api?ref=roadmapsh) \ No newline at end of file +- [@feed@Explore top posts about REST API](https://app.daily.dev/tags/rest-api?ref=roadmapsh) diff --git a/roadmaps/aspnet-core/content/rest@dZ9fYWMBa1OemTmLOI8_q.md b/roadmaps/aspnet-core/content/rest@dZ9fYWMBa1OemTmLOI8_q.md index 098e309f0..a7d7a91f6 100644 --- a/roadmaps/aspnet-core/content/rest@dZ9fYWMBa1OemTmLOI8_q.md +++ b/roadmaps/aspnet-core/content/rest@dZ9fYWMBa1OemTmLOI8_q.md @@ -4,7 +4,7 @@ REST (Representational State Transfer) is an architectural style for building we Visit the following resources to learn more: -- [@article@What is REST Services?](http://www.codedigest.com/quick-start/16/what-is-rest-services-how-to-create-rest-services-in-aspnet) +- [@article@What is REST Services?](https://web.archive.org/web/20231205224220/http://www.codedigest.com/quick-start/16/what-is-rest-services-how-to-create-rest-services-in-aspnet) - [@article@Restful API In ASP.NET: Introduction of REST & Web API](https://www.c-sharpcorner.com/UploadFile/4b0136/restful-api-in-Asp-Net-introduction-of-rest-web-api/) - [@article@What are RESTful APIs](https://www.pragimtech.com/blog/blazor/what-are-restful-apis/) - [@feed@Explore top posts about REST API](https://app.daily.dev/tags/rest-api?ref=roadmapsh) \ No newline at end of file diff --git a/roadmaps/backend/content/backpressure@JansCqGDyXecQkD1K7E7e.md b/roadmaps/backend/content/backpressure@JansCqGDyXecQkD1K7E7e.md index 1805720a4..c01b53ee7 100644 --- a/roadmaps/backend/content/backpressure@JansCqGDyXecQkD1K7E7e.md +++ b/roadmaps/backend/content/backpressure@JansCqGDyXecQkD1K7E7e.md @@ -4,7 +4,6 @@ Backpressure is a flow control mechanism where receivers signal their capacity t Visit the following resources to learn more: -- [@article@Awesome Architecture: Backpressure](https://awesome-architecture.com/back-pressure/) - [@article@Backpressure explained — the flow of data through software](https://medium.com/@jayphelps/backpressure-explained-the-flow-of-data-through-software-2350b3e77ce7) - [@article@Handling Backpressure in Node.js Streams](https://nodejs.org/learn/modules/backpressuring-in-streams) -- [@video@What is Back Pressure](https://www.youtube.com/watch?v=viTGm_cV7lE) \ No newline at end of file +- [@video@What is Back Pressure](https://www.youtube.com/watch?v=viTGm_cV7lE) diff --git a/roadmaps/design-system/content/vision@cMfsRtvzvDZZJ0TqeUOxm.md b/roadmaps/design-system/content/vision@cMfsRtvzvDZZJ0TqeUOxm.md index 482a50f21..0955237f3 100644 --- a/roadmaps/design-system/content/vision@cMfsRtvzvDZZJ0TqeUOxm.md +++ b/roadmaps/design-system/content/vision@cMfsRtvzvDZZJ0TqeUOxm.md @@ -5,5 +5,4 @@ Brand vision serves as the foundation for design system decisions, providing pur Visit the following resources to learn more: - [@article@Creating Brand Vision](https://www.interaction-design.org/literature/article/branding-in-ux) -- [@article@Vision-Driven Design Systems](https://bradfrost.com/blog/post/the-design-system-ecosystem/) -- [@article@Brand Strategy for Design](https://alistapart.com/article/creating-a-brand-identity/) \ No newline at end of file +- [@article@Vision-Driven Design Systems](https://bradfrost.com/blog/post/the-design-system-ecosystem/) \ No newline at end of file diff --git a/roadmaps/game-developer/content/stencil-shadow@AEAVc8Ih4fctSGGVJG0Np.md b/roadmaps/game-developer/content/stencil-shadow@AEAVc8Ih4fctSGGVJG0Np.md index 3243c571a..53d6158c7 100644 --- a/roadmaps/game-developer/content/stencil-shadow@AEAVc8Ih4fctSGGVJG0Np.md +++ b/roadmaps/game-developer/content/stencil-shadow@AEAVc8Ih4fctSGGVJG0Np.md @@ -4,4 +4,4 @@ Stencil shadows, also called shadow volumes, are a technique where the shadow bo Visit the following resources to learn more: -- [@article@Stencil Shadows Implementation](https://devforum.roblox.com/t/stencil-shadows-implementation/2079287) \ No newline at end of file +- [@article@The Mechanics of Robust Stencil Shadows](https://www.gamedeveloper.com/programming/the-mechanics-of-robust-stencil-shadows) diff --git a/roadmaps/rust/content/btreemap@LWgaZWTJVWLDes3HzflRh.md b/roadmaps/rust/content/btreemap@LWgaZWTJVWLDes3HzflRh.md index f509826d8..37ef325d7 100644 --- a/roadmaps/rust/content/btreemap@LWgaZWTJVWLDes3HzflRh.md +++ b/roadmaps/rust/content/btreemap@LWgaZWTJVWLDes3HzflRh.md @@ -5,4 +5,3 @@ Visit the following resources to learn more: - [@official@BTreeMap](https://doc.rust-lang.org/std/collections/struct.BTreeMap.html) -- [@article@BTreeMap](https://cglab.ca/~abeinges/blah/rust-btree-case/) \ No newline at end of file From 0c84aafda1aa9ce61c668770a14da1f33553ec1f Mon Sep 17 00:00:00 2001 From: H23 Date: Thu, 1 Oct 2026 19:11:54 +0800 Subject: [PATCH 6/8] docs: replace dead article links with live sources (#10404) Per maintainer guidance to prefer fresh content over archived files: - gafferongames udp-vs-tcp (2 files): updated to the article's current location at gafferongames.com/post/udp_vs_tcp/ - transaction script: the personal blog post is gone; linked Martin Fowler's canonical Transaction Script page instead - postgres buffer manager: the dev.to summary was deleted; linked the original book chapter it summarized (The Internals of PostgreSQL, ch.8 Buffer Manager) - dropped two resource lines whose sources are gone entirely with no replacement: businessdesign.org Business Model Inspirator (section removed from the site) and a digger.dev Terraform/Jenkins post (domain dead, label was also corrupted) All replacement URLs verified to resolve (HTTP 200). Co-authored-by: Weijian He <230282609+Madin-H23@users.noreply.github.com> --- .../content/buffer-management@KeBUzfrkorgFWpR8A-xmJ.md | 2 +- .../content/transaction-script@tyReIY4iO8kmyc_LPafp1.md | 2 +- roadmaps/system-design/content/tcp@2nF5uC6fYKbf0RFgGNHiP.md | 2 +- roadmaps/system-design/content/udp@LC5aTmUKNiw9RuSUt3fSE.md | 2 +- roadmaps/terraform/content/jenkins@qFpaqOSoefTyKxmI-FRZt.md | 1 - .../content/business-model-inspirator@GI06-DbGyJlQXq5Tyi-aH.md | 2 -- 6 files changed, 4 insertions(+), 7 deletions(-) diff --git a/roadmaps/postgresql-dba/content/buffer-management@KeBUzfrkorgFWpR8A-xmJ.md b/roadmaps/postgresql-dba/content/buffer-management@KeBUzfrkorgFWpR8A-xmJ.md index 475ef1305..391367755 100644 --- a/roadmaps/postgresql-dba/content/buffer-management@KeBUzfrkorgFWpR8A-xmJ.md +++ b/roadmaps/postgresql-dba/content/buffer-management@KeBUzfrkorgFWpR8A-xmJ.md @@ -6,4 +6,4 @@ Visit the following resources to learn more: - [@official@pg_buffercache](https://www.postgresql.org/docs/current/pgbuffercache.html) - [@official@Write Ahead Logging](https://www.postgresql.org/docs/current/wal-intro.html) -- [@article@Buffer Manager](https://dev.to/vkt1271/summary-of-chapter-8-buffer-manager-from-the-book-the-internals-of-postgresql-part-2-4f6o) \ No newline at end of file +- [@article@Buffer Manager](https://www.interdb.jp/pg/pgsql08.html) \ No newline at end of file diff --git a/roadmaps/software-design-architecture/content/transaction-script@tyReIY4iO8kmyc_LPafp1.md b/roadmaps/software-design-architecture/content/transaction-script@tyReIY4iO8kmyc_LPafp1.md index 90a5c5284..10d63d8ba 100644 --- a/roadmaps/software-design-architecture/content/transaction-script@tyReIY4iO8kmyc_LPafp1.md +++ b/roadmaps/software-design-architecture/content/transaction-script@tyReIY4iO8kmyc_LPafp1.md @@ -4,5 +4,5 @@ Transaction Script is a pattern used in enterprise application development that Visit the following resources to learn more: -- [@article@Transaction Script Pattern](https://gunnarpeipman.com/transaction-script-pattern/) +- [@article@Transaction Script Pattern](https://martinfowler.com/eaaCatalog/transactionScript.html) - [@video@Tutorial - Transaction Script Design Pattern](https://www.youtube.com/watch?v=fnsU9cqcY3I) \ No newline at end of file diff --git a/roadmaps/system-design/content/tcp@2nF5uC6fYKbf0RFgGNHiP.md b/roadmaps/system-design/content/tcp@2nF5uC6fYKbf0RFgGNHiP.md index a94246f15..4daf21fe6 100644 --- a/roadmaps/system-design/content/tcp@2nF5uC6fYKbf0RFgGNHiP.md +++ b/roadmaps/system-design/content/tcp@2nF5uC6fYKbf0RFgGNHiP.md @@ -6,5 +6,5 @@ Visit the following resources to learn more: - [@opensource@What Is TCP?](https://github.com/donnemartin/system-design-primer#transmission-control-protocol-tcp) - [@article@What is the difference between HTTP protocol and TCP protocol?](https://www.quora.com/What-is-the-difference-between-HTTP-protocol-and-TCP-protocol) -- [@article@Networking for game programming](http://gafferongames.com/networking-for-game-programmers/udp-vs-tcp/) +- [@article@Networking for game programming](https://gafferongames.com/post/udp_vs_tcp/) - [@article@Key differences between TCP and UDP protocols](http://www.cyberciti.biz/faq/key-differences-between-tcp-and-udp-protocols/) \ No newline at end of file diff --git a/roadmaps/system-design/content/udp@LC5aTmUKNiw9RuSUt3fSE.md b/roadmaps/system-design/content/udp@LC5aTmUKNiw9RuSUt3fSE.md index 18a9e24e3..c6a2d17d5 100644 --- a/roadmaps/system-design/content/udp@LC5aTmUKNiw9RuSUt3fSE.md +++ b/roadmaps/system-design/content/udp@LC5aTmUKNiw9RuSUt3fSE.md @@ -4,6 +4,6 @@ UDP, or User Datagram Protocol, is a connectionless protocol that sends data wit Visit the following resources to learn more: -- [@article@Networking for game programming](http://gafferongames.com/networking-for-game-programmers/udp-vs-tcp/) +- [@article@Networking for game programming](https://gafferongames.com/post/udp_vs_tcp/) - [@article@Key differences between TCP and UDP protocols](http://www.cyberciti.biz/faq/key-differences-between-tcp-and-udp-protocols/) - [@article@Difference between TCP and UDP](http://stackoverflow.com/questions/5970383/difference-between-tcp-and-udp) \ No newline at end of file diff --git a/roadmaps/terraform/content/jenkins@qFpaqOSoefTyKxmI-FRZt.md b/roadmaps/terraform/content/jenkins@qFpaqOSoefTyKxmI-FRZt.md index 1372ea10a..b0a05994a 100644 --- a/roadmaps/terraform/content/jenkins@qFpaqOSoefTyKxmI-FRZt.md +++ b/roadmaps/terraform/content/jenkins@qFpaqOSoefTyKxmI-FRZt.md @@ -5,5 +5,4 @@ Jenkins can orchestrate Terraform workflows using Pipeline scripts written in Gr Visit the following resources to learn more: - [@article@Terraform with Jenkins – How to Manage Workflows](https://spacelift.io/blog/terraform-jenkins) -- [@article@£article@How to run Terraform in your Jenkins CI/CD pipeline.](https://blog.digger.dev/how-to-run-terraform-in-jenkins/) - [@video@How to Use Terraform and Jenkins to Automate Infrastructure Setup](https://www.youtube.com/watch?v=kIDiP3Unj7Y) \ No newline at end of file diff --git a/roadmaps/ux-design/content/business-model-inspirator@GI06-DbGyJlQXq5Tyi-aH.md b/roadmaps/ux-design/content/business-model-inspirator@GI06-DbGyJlQXq5Tyi-aH.md index f64ce0849..305a7f688 100644 --- a/roadmaps/ux-design/content/business-model-inspirator@GI06-DbGyJlQXq5Tyi-aH.md +++ b/roadmaps/ux-design/content/business-model-inspirator@GI06-DbGyJlQXq5Tyi-aH.md @@ -3,5 +3,3 @@ A Business Model Inspirator is a tool or method that helps you to generate new or creative ideas for the strategic, operational, and financial aspects of a business. It helps entrepreneurs, startups, and established companies to explore different ways of designing or improving their business models by drawing inspiration from various sources. Visit the following resources to learn more: - -- [@article@Business Model Inspirator](https://businessdesign.org/knowledge-base/business-model-inspirator) \ No newline at end of file From 1e31884c0f3685668bdec7e2526788cd1a30cc76 Mon Sep 17 00:00:00 2001 From: H23 Date: Thu, 1 Oct 2026 19:13:37 +0800 Subject: [PATCH 7/8] docs: update moved organization and course links (#10405) - OWASP API Security project: the www-project page was removed; the project now lives at owasp.org/API-Security/ (3 files) - OWASP ZAP: the project moved off OWASP pages to its own site, zaproxy.org - NPTEL: the course URL scheme changed from /courses/106/106// to /courses/ (2 files) All replacement URLs verified to resolve (HTTP 200). Co-authored-by: Weijian He <230282609+Madin-H23@users.noreply.github.com> --- .../content/api-protection@Tszl26iNBnQBdBEWOueDA.md | 2 +- .../content/authorization@JQ3bR8odXJfd-1RCEf3-Q.md | 2 +- .../content/testing-platforms@c8n8FcYKDOgPLQvV9xF5J.md | 2 +- .../content/unauthorized-access@DQeOavZCoXpF3k_qRDABs.md | 2 +- .../content/basic-math-skills@cdNi0EYrQ5nsgNxFVnXL8.md | 2 +- .../mlops/content/maths--statistics@C4KbkZ7c8TspXa3RBOCug.md | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/roadmaps/ai-red-teaming/content/api-protection@Tszl26iNBnQBdBEWOueDA.md b/roadmaps/ai-red-teaming/content/api-protection@Tszl26iNBnQBdBEWOueDA.md index c70888036..4709d52a2 100644 --- a/roadmaps/ai-red-teaming/content/api-protection@Tszl26iNBnQBdBEWOueDA.md +++ b/roadmaps/ai-red-teaming/content/api-protection@Tszl26iNBnQBdBEWOueDA.md @@ -6,4 +6,4 @@ Visit the following resources to learn more: - [@article@Securing APIs with AI for Advanced Threat Protection](https://adevait.com/artificial-intelligence/securing-apis-with-ai) - [@article@Securing Machine Learning APIs (IBM)](https://developer.ibm.com/articles/se-securing-machine-learning-apis/) -- [@article@OWASP API Security Project (Top 10 2023)](https://owasp.org/www-project-api-security/) \ No newline at end of file +- [@article@OWASP API Security Project (Top 10 2023)](https://owasp.org/API-Security/) \ No newline at end of file diff --git a/roadmaps/ai-red-teaming/content/authorization@JQ3bR8odXJfd-1RCEf3-Q.md b/roadmaps/ai-red-teaming/content/authorization@JQ3bR8odXJfd-1RCEf3-Q.md index e0c6519f7..edf70a31c 100644 --- a/roadmaps/ai-red-teaming/content/authorization@JQ3bR8odXJfd-1RCEf3-Q.md +++ b/roadmaps/ai-red-teaming/content/authorization@JQ3bR8odXJfd-1RCEf3-Q.md @@ -6,4 +6,4 @@ Visit the following resources to learn more: - [@article@What is Authentication vs Authorization?](https://auth0.com/intro-to-iam/authentication-vs-authorization) - [@article@Identity and access management (IAM) fundamental concepts](https://learn.microsoft.com/en-us/entra/fundamentals/identity-fundamental-concepts) -- [@article@OWASP API Security Project](https://owasp.org/www-project-api-security/) \ No newline at end of file +- [@article@OWASP API Security Project](https://owasp.org/API-Security/) \ No newline at end of file diff --git a/roadmaps/ai-red-teaming/content/testing-platforms@c8n8FcYKDOgPLQvV9xF5J.md b/roadmaps/ai-red-teaming/content/testing-platforms@c8n8FcYKDOgPLQvV9xF5J.md index 35c58a1fd..62b309d10 100644 --- a/roadmaps/ai-red-teaming/content/testing-platforms@c8n8FcYKDOgPLQvV9xF5J.md +++ b/roadmaps/ai-red-teaming/content/testing-platforms@c8n8FcYKDOgPLQvV9xF5J.md @@ -6,6 +6,6 @@ Visit the following resources to learn more: - [@article@AI Red Teaming Agent - Azure AI Foundry | Microsoft Learn](https://learn.microsoft.com/en-us/azure/ai-foundry/concepts/ai-red-teaming-agent) - [@article@Kali Linux](https://www.kali.org/) -- [@article@OWASP Zed Attack Proxy (ZAP)](https://owasp.org/www-project-zap/) +- [@article@OWASP Zed Attack Proxy (ZAP)](https://www.zaproxy.org/) - [@article@Promptfoo](https://www.promptfoo.dev/) - [@article@PyRIT (Python Risk Identification Tool for generative AI)](https://github.com/Azure/PyRIT) \ No newline at end of file diff --git a/roadmaps/ai-red-teaming/content/unauthorized-access@DQeOavZCoXpF3k_qRDABs.md b/roadmaps/ai-red-teaming/content/unauthorized-access@DQeOavZCoXpF3k_qRDABs.md index 49e64704e..0be9db948 100644 --- a/roadmaps/ai-red-teaming/content/unauthorized-access@DQeOavZCoXpF3k_qRDABs.md +++ b/roadmaps/ai-red-teaming/content/unauthorized-access@DQeOavZCoXpF3k_qRDABs.md @@ -5,5 +5,5 @@ AI Red Teamers test if vulnerabilities in the AI system or its interfaces allow Visit the following resources to learn more: - [@article@Defending Model Files from Unauthorized Access](https://developer.nvidia.com/blog/defending-ai-model-files-from-unauthorized-access-with-canaries/) -- [@article@OWASP API Security Project](https://owasp.org/www-project-api-security/) +- [@article@OWASP API Security Project](https://owasp.org/API-Security/) - [@article@Detecting Unauthorized Usage](https://www.unr.edu/digital-learning/instructional-strategies/understanding-and-integrating-generative-ai-in-teaching/how-can-i-detect-unauthorized-ai-usage) \ No newline at end of file diff --git a/roadmaps/computer-science/content/basic-math-skills@cdNi0EYrQ5nsgNxFVnXL8.md b/roadmaps/computer-science/content/basic-math-skills@cdNi0EYrQ5nsgNxFVnXL8.md index 1d3f97ca6..47b1251d2 100644 --- a/roadmaps/computer-science/content/basic-math-skills@cdNi0EYrQ5nsgNxFVnXL8.md +++ b/roadmaps/computer-science/content/basic-math-skills@cdNi0EYrQ5nsgNxFVnXL8.md @@ -5,6 +5,6 @@ Basic math skills for computer science cover the mathematical foundations that a Visit the following resources to learn more: - [@article@Computer Science 70, 001 - Spring 2015 - Discrete Mathematics and Probability Theory](http://www.infocobuild.com/education/audio-video-courses/computer-science/cs70-spring2015-berkeley.html) -- [@article@Discrete Mathematics By IIT Ropar NPTEL](https://nptel.ac.in/courses/106/106/106106183/) +- [@article@Discrete Mathematics By IIT Ropar NPTEL](https://nptel.ac.in/courses/106106183) - [@video@Lec 1 | MIT 6.042J Mathematics for Computer Science, Fall 2010](https://www.youtube.com/watch?v=L3LMbpZIKhQ&list=PLB7540DEDD482705B) - [@video@Integer Arithmetic, Karatsuba Multiplication](https://www.youtube.com/watch?v=eCaXlAaN2uE&index=11&list=PLUl4u3cNGP61Oq3tWYp6V_F-5jb5L2iHb) \ No newline at end of file diff --git a/roadmaps/mlops/content/maths--statistics@C4KbkZ7c8TspXa3RBOCug.md b/roadmaps/mlops/content/maths--statistics@C4KbkZ7c8TspXa3RBOCug.md index a2f40faf3..226d669d4 100644 --- a/roadmaps/mlops/content/maths--statistics@C4KbkZ7c8TspXa3RBOCug.md +++ b/roadmaps/mlops/content/maths--statistics@C4KbkZ7c8TspXa3RBOCug.md @@ -6,7 +6,7 @@ Visit the following resources to learn more: - [@book@Introductory Statistics](https://assets.openstax.org/oscms-prodcms/media/documents/IntroductoryStatistics-OP_i6tAI7e.pdf) - [@article@Computer Science 70, 001 - Spring 2015 - Discrete Mathematics and Probability Theory](http://www.infocobuild.com/education/audio-video-courses/computer-science/cs70-spring2015-berkeley.html) -- [@article@Discrete Mathematics By IIT Ropar NPTEL](https://nptel.ac.in/courses/106/106/106106183/) +- [@article@Discrete Mathematics By IIT Ropar NPTEL](https://nptel.ac.in/courses/106106183) - [@article@Introduction to Statistics](https://imp.i384100.net/3eRv4v) - [@article@How to Learn the Math Needed for Data Science](https://towardsdatascience.com/how-to-learn-the-math-needed-for-data-science-86c6643b0c59/?utm_source=roadmap&utm_medium=Referral&utm_campaign=TDS+roadmap+integration) - [@video@Lec 1 | MIT 6.042J Mathematics for Computer Science, Fall 2010](https://www.youtube.com/watch?v=L3LMbpZIKhQ&list=PLB7540DEDD482705B) From a670d7fbee5afc391bd61a46ee3d78572c859bfe Mon Sep 17 00:00:00 2001 From: Javier Canales <56018501+jcanalesluna@users.noreply.github.com> Date: Thu, 1 Oct 2026 13:28:17 +0200 Subject: [PATCH 8/8] feat(inference-engineering): add learning resources to roadmap topics (#10406) Add curated papers, official docs, open-source repos and books to 64 topics of the Inference Engineering roadmap, following the content style guide (typed links, ordered by type, max 8 per topic). Also normalise the heading on the Post-Training Quantization and Multi-Cloud topics. Co-authored-by: Claude Opus 5.5 --- .../attention-mechanism@2KY3T5JVHQBR8pmAiuq0V.md | 6 +++++- .../attention-variants@6oTnNEsX_6Cu6lowEz9vg.md | 9 ++++++++- ...auto-speech-recognition@6PRICRK6aRQCoThxnevB0.md | 7 ++++++- .../content/autoscaling@o_gotSHKIPtV5SgGN3Vja.md | 8 +++++++- .../bert-style-models@ef3EneMibOaDQkKcpipGY.md | 7 ++++++- .../content/blackwell@LihLABERuKBw30G_j48Fj.md | 6 +++++- .../caching-approaches@2nGv7_8PTWYr7uqbudCcX.md | 7 ++++++- .../containerization@--ohScNmF5LR04P8e95AP.md | 6 +++++- .../content/cuda@DwJJLFLPHGn9zdccTlLY2.md | 8 +++++++- .../dependency-management@wEaM8VyAo6MFsc3RXQ7Ar.md | 6 +++++- .../desktop-inference@fQDl7_3nRGa8NFivOIgTP.md | 6 +++++- .../diffusion-models@aagMwPPmi6ngfkIqZL-Mg.md | 9 ++++++++- .../content/disaggregation@f6PWIqf4LubR7lQztaoB5.md | 6 +++++- .../content/distillation@IdNkeL9pzy4lFv17ak1Vy.md | 6 +++++- .../draft-target-decoding@m0EIHovHWGkWULN0a3C3v.md | 6 +++++- .../content/eagle@0FZTg3_ZnTpMOQ078OOxE.md | 8 +++++++- .../embedding-models@Tvt-vX1gWuJZJBHh5vfKu.md | 9 ++++++++- .../expert-parallelism@62y78y-RXZT_QCUfd-YFE.md | 7 ++++++- ...w-step-image-generation@biVjpCA3bsKI7sa_aBIUE.md | 6 +++++- .../content/fine-tuning@fWJplrAuUStEMNNLQc75G.md | 6 +++++- .../content/flashattention@mQcj-H3R789o6nDh8tG9k.md | 9 ++++++++- .../gpu-architecture@00JLMvn8DUJBCw6-BxO0y.md | 8 +++++++- .../content/grace--vera@kPT86kA0WwE_fWUAAKCTT.md | 6 +++++- .../content/hopper@XQMbaW1tFdO8Os6mxcR2M.md | 6 +++++- .../image-generation-arch@xsV4WevPzhfydyvxVzGcw.md | 9 ++++++++- ...ge-generation-inference@lnIHjq4nvCs3zvUln-jHR.md | 7 ++++++- .../inference-vs-training@fVUfQYb8BP_hFMmd1S2-J.md | 6 +++++- ...nstances--interconnects@dAo4PMB_VnRFMSjyDrLj0.md | 7 ++++++- .../kv-cache-storage@gkt2Gr34RJJf1NZgbmtXf.md | 7 ++++++- .../content/kv-cache@V678kX8FPChTZCe7Nqwhv.md | 6 +++++- .../large-language-models@irgGvEnY1_WW1EPtuMKgC.md | 6 +++++- .../llm-bottlenecks@QVadKzBfLdegat-4wgOMX.md | 7 ++++++- .../long-context-handling@uL3Fn8XfS1Yv_PJd9R8wi.md | 7 ++++++- .../mixture-of-experts-moe@7BddTfOkWmlL4HAkFBXtq.md | 6 +++++- .../model-evaluation@lVf5GoJF_PVTkTiQFTNVg.md | 7 ++++++- .../model-selection@jQAWZ1plw4_wPE1UvE_Ux.md | 6 +++++- .../content/multi-cloud@d78UiI_FZyLsU6O52btXg.md | 11 +++++++++-- .../content/n-gram@ZOclQMd7JASGXTRyctfW2.md | 6 +++++- .../neural-networks@p2S0W3EDn768jbqzc7CE7.md | 8 +++++++- .../content/nvidia-dynamo@KhTFecXEkHR4WX-_TYY3g.md | 6 +++++- .../content/nvidia-dynamo@ctdMz6rHa4Cwl1x19V_5k.md | 6 +++++- .../content/observability@w-dVv5dFNxD7Rqy16m4uL.md | 6 +++++- .../content/onnx-runtime@Vmtbdre3SZzwq3FxOO-Bc.md | 6 +++++- .../open-vs-closed-models@KAt5FzglOUtH7jLlzDn-I.md | 6 +++++- .../content/pagedattention@VcIRY4vSi7JkcV6g13vkN.md | 6 +++++- .../pipeline-parallelism@9KLS3gWArBofLEULu3sQc.md | 7 ++++++- ...t-training-quantization@Zvvs8cB98IRee1okBILsS.md | 13 +++++++++++-- .../profiling-performance@S2GZw9etrrvEJzV9V-1C_.md | 7 ++++++- .../content/pytorch@AHVfxQRXmRoiTxC1eeMme.md | 6 +++++- .../content/quantization@DaSqcLHfnJ4_n9utVVTSk.md | 11 ++++++++++- ...routing--load-balancing@DSyTAtFC1TvC1P7yD10Nb.md | 8 +++++++- .../content/sglang@YtlAf6qloKMmaSr9Bkkl_.md | 6 +++++- .../sizing--procurement@da0HGfSDr6_DV4ZJcrcIB.md | 8 +++++++- .../speculative-decoding@gMWgd10cxcQG_RG2oSAEH.md | 7 ++++++- .../tensor-parallelism@csghcNLDywO8oTIZNXeyP.md | 7 ++++++- .../content/tensorrt-llm@n2_zIGkQgyYCMYPypz3vb.md | 6 +++++- .../content/tensorrt@M4qjJSEAnsjrmP3A_O8HC.md | 6 +++++- .../transformer-blocks@xHM7qjigMmXEwxeMEeMWl.md | 6 +++++- ...transformers--diffusers@H_e3KZ7p7enUCYE5M5Lr7.md | 7 ++++++- ...eo-generation-inference@PqEH1ez5dg5tEsspgfAK1.md | 8 +++++++- ...video-generation-models@yTSrQfJsE-Mf4ZU_hJwYT.md | 7 ++++++- .../vision-language-models@JsjmQrN1V_dM1Kw6UnYLc.md | 9 ++++++++- .../content/vllm@wcAtbab81IJdpFh6L-heP.md | 6 +++++- .../writing-custom-kernels@1dWPVaSQ4TfVU3Obx3m19.md | 11 ++++++++++- 64 files changed, 390 insertions(+), 66 deletions(-) diff --git a/roadmaps/inference-engineering/content/attention-mechanism@2KY3T5JVHQBR8pmAiuq0V.md b/roadmaps/inference-engineering/content/attention-mechanism@2KY3T5JVHQBR8pmAiuq0V.md index 690ebb8d5..801149e68 100644 --- a/roadmaps/inference-engineering/content/attention-mechanism@2KY3T5JVHQBR8pmAiuq0V.md +++ b/roadmaps/inference-engineering/content/attention-mechanism@2KY3T5JVHQBR8pmAiuq0V.md @@ -1,3 +1,7 @@ # Attention Mechanism -Attention is the mechanism transformers use to relate each token to all previous tokens via query, key, and value projections. During decode, each new token attends to every prior token, and the KV cache stores precomputed keys and values to avoid recomputation. Attention is the most expensive and most heavily optimized operation in LLM inference. \ No newline at end of file +Attention is the mechanism transformers use to relate each token to all previous tokens via query, key, and value projections. During decode, each new token attends to every prior token, and the KV cache stores precomputed keys and values to avoid recomputation. Attention is the most expensive and most heavily optimized operation in LLM inference. + +Visit the following resources to learn more: + +- [@article@Attention Is All You Need](https://arxiv.org/abs/1706.03762) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/attention-variants@6oTnNEsX_6Cu6lowEz9vg.md b/roadmaps/inference-engineering/content/attention-variants@6oTnNEsX_6Cu6lowEz9vg.md index 630466a14..f3aa78434 100644 --- a/roadmaps/inference-engineering/content/attention-variants@6oTnNEsX_6Cu6lowEz9vg.md +++ b/roadmaps/inference-engineering/content/attention-variants@6oTnNEsX_6Cu6lowEz9vg.md @@ -1,3 +1,10 @@ # Attention Variants -Newer attention algorithms reduce the quadratic time complexity of standard attention. Sliding window attention limits each token to attending to the nearest w tokens, turning O(N²) into O(Nw). Linear attention approximates the softmax with a linear-time kernel. Compressed attention periodically compresses earlier context. These variants trade off some quality for better scaling on long sequences. \ No newline at end of file +Newer attention algorithms reduce the quadratic time complexity of standard attention. Sliding window attention limits each token to attending to the nearest w tokens, turning O(N²) into O(Nw). Linear attention approximates the softmax with a linear-time kernel. Compressed attention periodically compresses earlier context. These variants trade off some quality for better scaling on long sequences. + +Visit the following resources to learn more: + +- [@article@Longformer: The Long-Document Transformer](https://arxiv.org/abs/2004.05150) +- [@article@Reformer: The Efficient Transformer](https://arxiv.org/abs/2001.04451) +- [@article@Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://arxiv.org/abs/2108.12409) +- [@article@Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://arxiv.org/abs/2312.00752) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/auto-speech-recognition@6PRICRK6aRQCoThxnevB0.md b/roadmaps/inference-engineering/content/auto-speech-recognition@6PRICRK6aRQCoThxnevB0.md index f6c51eaea..db87f0d5a 100644 --- a/roadmaps/inference-engineering/content/auto-speech-recognition@6PRICRK6aRQCoThxnevB0.md +++ b/roadmaps/inference-engineering/content/auto-speech-recognition@6PRICRK6aRQCoThxnevB0.md @@ -1,3 +1,8 @@ # Automatic Speech Recognition (ASR) -ASR models like Whisper take audio input and produce text transcripts. Whisper is an encoder-decoder model where the decoder, an autoregressive transformer, dominates inference time. TensorRT-LLM with in-flight batching for the decoder delivers the best performance. For real-time transcription, streaming via WebSockets with VAD-segmented chunks achieves round-trip targets of around 200 milliseconds per chunk on optimized deployments. \ No newline at end of file +ASR models like Whisper take audio input and produce text transcripts. Whisper is an encoder-decoder model where the decoder, an autoregressive transformer, dominates inference time. TensorRT-LLM with in-flight batching for the decoder delivers the best performance. For real-time transcription, streaming via WebSockets with VAD-segmented chunks achieves round-trip targets of around 200 milliseconds per chunk on optimized deployments. + +Visit the following resources to learn more: + +- [@official@OpenAI Whisper Models on Hugging Face](https://huggingface.co/openai) +- [@article@Robust Speech Recognition via Large-Scale Weak Supervision (Whisper)](https://arxiv.org/abs/2212.04356) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/autoscaling@o_gotSHKIPtV5SgGN3Vja.md b/roadmaps/inference-engineering/content/autoscaling@o_gotSHKIPtV5SgGN3Vja.md index 94757738d..f387ee650 100644 --- a/roadmaps/inference-engineering/content/autoscaling@o_gotSHKIPtV5SgGN3Vja.md +++ b/roadmaps/inference-engineering/content/autoscaling@o_gotSHKIPtV5SgGN3Vja.md @@ -1,3 +1,9 @@ # Autoscaling -Autoscaling dynamically adjusts the number of active model replicas to match incoming traffic, maintaining latency SLAs while minimizing idle GPU spend. Kubernetes is the standard orchestration layer, scaling replicas within a cluster based on traffic signals and utilization metrics. Key configuration parameters are min replicas, max replicas, the autoscaling window, the scale-down delay, and the concurrency target per replica. \ No newline at end of file +Autoscaling dynamically adjusts the number of active model replicas to match incoming traffic, maintaining latency SLAs while minimizing idle GPU spend. Kubernetes is the standard orchestration layer, scaling replicas within a cluster based on traffic signals and utilization metrics. Key configuration parameters are min replicas, max replicas, the autoscaling window, the scale-down delay, and the concurrency target per replica. + +Visit the following resources to learn more: + +- [@article@The Llama 3 Herd of Models](https://arxiv.org/abs/2407.21783) +- [@book@Designing Data-Intensive Applications](https://dataintensive.net/) +- [@book@Site Reliability Engineering](https://sre.google/books/) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/bert-style-models@ef3EneMibOaDQkKcpipGY.md b/roadmaps/inference-engineering/content/bert-style-models@ef3EneMibOaDQkKcpipGY.md index adb912cc8..0e2705e90 100644 --- a/roadmaps/inference-engineering/content/bert-style-models@ef3EneMibOaDQkKcpipGY.md +++ b/roadmaps/inference-engineering/content/bert-style-models@ef3EneMibOaDQkKcpipGY.md @@ -1,3 +1,8 @@ # BERT-style Models -BERT-style encoder-only models are the traditional architecture for text embeddings, producing a fixed-size vector by pooling the final hidden states of a bidirectional transformer. They are compact, fast, and well-suited for high-throughput indexing workloads where embedding quality is sufficient for the task. Models like BGE-small and all-MiniLM run efficiently on CPU or fractional GPU instances, making them the lowest-cost embedding option. \ No newline at end of file +BERT-style encoder-only models are the traditional architecture for text embeddings, producing a fixed-size vector by pooling the final hidden states of a bidirectional transformer. They are compact, fast, and well-suited for high-throughput indexing workloads where embedding quality is sufficient for the task. Models like BGE-small and all-MiniLM run efficiently on CPU or fractional GPU instances, making them the lowest-cost embedding option. + +Visit the following resources to learn more: + +- [@article@BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/abs/1810.04805) +- [@article@Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/blackwell@LihLABERuKBw30G_j48Fj.md b/roadmaps/inference-engineering/content/blackwell@LihLABERuKBw30G_j48Fj.md index 6f1509663..fcb57a33b 100644 --- a/roadmaps/inference-engineering/content/blackwell@LihLABERuKBw30G_j48Fj.md +++ b/roadmaps/inference-engineering/content/blackwell@LihLABERuKBw30G_j48Fj.md @@ -1,3 +1,7 @@ # Blackwell -The Blackwell architecture, released in late 2024 with the B200 and B300, added FP4 Tensor Cores and new microscaling number formats including MXFP8, MXFP4, and NVFP4. B200 GPUs offer approximately 5 petaFLOPS of FP8 compute and up to 8 TB/s of memory bandwidth. They are the current gold standard for demanding inference workloads including large LLMs and video generation. \ No newline at end of file +The Blackwell architecture, released in late 2024 with the B200 and B300, added FP4 Tensor Cores and new microscaling number formats including MXFP8, MXFP4, and NVFP4. B200 GPUs offer approximately 5 petaFLOPS of FP8 compute and up to 8 TB/s of memory bandwidth. They are the current gold standard for demanding inference workloads including large LLMs and video generation. + +Visit the following resources to learn more: + +- [@official@NVIDIA Blackwell Architecture Technical Brief](https://resources.nvidia.com/en-us-blackwell-architecture) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/caching-approaches@2nGv7_8PTWYr7uqbudCcX.md b/roadmaps/inference-engineering/content/caching-approaches@2nGv7_8PTWYr7uqbudCcX.md index 7a1ffdee1..10959d4d9 100644 --- a/roadmaps/inference-engineering/content/caching-approaches@2nGv7_8PTWYr7uqbudCcX.md +++ b/roadmaps/inference-engineering/content/caching-approaches@2nGv7_8PTWYr7uqbudCcX.md @@ -1,3 +1,8 @@ # Caching Approaches -KV cache reuse is the primary caching strategy in LLM inference, covering both prefix caching (reusing cached KV values across requests that share a common prefix) and KV cache offloading (moving less-used cache blocks to slower storage tiers to free GPU VRAM). A related concern is cache-aware routing, which directs requests to replicas most likely to already hold the relevant cached prefix. \ No newline at end of file +KV cache reuse is the primary caching strategy in LLM inference, covering both prefix caching (reusing cached KV values across requests that share a common prefix) and KV cache offloading (moving less-used cache blocks to slower storage tiers to free GPU VRAM). A related concern is cache-aware routing, which directs requests to replicas most likely to already hold the relevant cached prefix. + +Visit the following resources to learn more: + +- [@opensource@LMCache](https://github.com/LMCache/LMCache) +- [@article@CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion](https://arxiv.org/abs/2405.16444) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/containerization@--ohScNmF5LR04P8e95AP.md b/roadmaps/inference-engineering/content/containerization@--ohScNmF5LR04P8e95AP.md index af056fc81..050dc2c49 100644 --- a/roadmaps/inference-engineering/content/containerization@--ohScNmF5LR04P8e95AP.md +++ b/roadmaps/inference-engineering/content/containerization@--ohScNmF5LR04P8e95AP.md @@ -1,3 +1,7 @@ # Containerization -Containerization packages an inference service and all its dependencies into a Docker image that runs consistently across environments. Inference containers include the CUDA toolkit, Python packages, the inference engine, and any system libraries like ffmpeg for audio processing. Using a pinned dependency tree prevents breaking changes from affecting running services and ensures reproducible builds. \ No newline at end of file +Containerization packages an inference service and all its dependencies into a Docker image that runs consistently across environments. Inference containers include the CUDA toolkit, Python packages, the inference engine, and any system libraries like ffmpeg for audio processing. Using a pinned dependency tree prevents breaking changes from affecting running services and ensures reproducible builds. + +Visit the following resources to learn more: + +- [@official@Kubernetes Documentation](https://kubernetes.io/docs/home/) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/cuda@DwJJLFLPHGn9zdccTlLY2.md b/roadmaps/inference-engineering/content/cuda@DwJJLFLPHGn9zdccTlLY2.md index 54cf7ecbf..fb437281b 100644 --- a/roadmaps/inference-engineering/content/cuda@DwJJLFLPHGn9zdccTlLY2.md +++ b/roadmaps/inference-engineering/content/cuda@DwJJLFLPHGn9zdccTlLY2.md @@ -1,3 +1,9 @@ # CUDA -CUDA is NVIDIA's proprietary platform for programming GPUs, consisting of a kernel language (C++), a driver for hardware management, and a runtime API for launching kernels. Every inference engine, framework, and library in the NVIDIA ecosystem depends on CUDA. Understanding CUDA is the foundation for diagnosing performance problems that tools and engines cannot solve automatically. \ No newline at end of file +CUDA is NVIDIA's proprietary platform for programming GPUs, consisting of a kernel language (C++), a driver for hardware management, and a runtime API for launching kernels. Every inference engine, framework, and library in the NVIDIA ecosystem depends on CUDA. Understanding CUDA is the foundation for diagnosing performance problems that tools and engines cannot solve automatically. + +Visit the following resources to learn more: + +- [@official@CUDA Programming Guide](https://docs.nvidia.com/cuda/cuda-programming-guide/) +- [@official@cuBLAS Documentation](https://docs.nvidia.com/cuda/cublas/) +- [@book@CUDA by Example: An Introduction to General-Purpose GPU Programming](https://developer.nvidia.com/cuda-example) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/dependency-management@wEaM8VyAo6MFsc3RXQ7Ar.md b/roadmaps/inference-engineering/content/dependency-management@wEaM8VyAo6MFsc3RXQ7Ar.md index 6d2d14816..cf672ef34 100644 --- a/roadmaps/inference-engineering/content/dependency-management@wEaM8VyAo6MFsc3RXQ7Ar.md +++ b/roadmaps/inference-engineering/content/dependency-management@wEaM8VyAo6MFsc3RXQ7Ar.md @@ -1,3 +1,7 @@ # Dependency Management -Inference dependency chains are long and fragile due to the tight coupling between CUDA versions, framework versions, and hardware architectures. Pinning exact versions of every dependency prevents unexpected breakage when upstream packages release updates. Containers for newly released models often initially depend on pre-release software builds and need to be rebuilt on stable releases within days of the model launch. \ No newline at end of file +Inference dependency chains are long and fragile due to the tight coupling between CUDA versions, framework versions, and hardware architectures. Pinning exact versions of every dependency prevents unexpected breakage when upstream packages release updates. Containers for newly released models often initially depend on pre-release software builds and need to be rebuilt on stable releases within days of the model launch. + +Visit the following resources to learn more: + +- [@official@Kubernetes Documentation](https://kubernetes.io/docs/home/) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/desktop-inference@fQDl7_3nRGa8NFivOIgTP.md b/roadmaps/inference-engineering/content/desktop-inference@fQDl7_3nRGa8NFivOIgTP.md index d2846bde7..c309427bc 100644 --- a/roadmaps/inference-engineering/content/desktop-inference@fQDl7_3nRGa8NFivOIgTP.md +++ b/roadmaps/inference-engineering/content/desktop-inference@fQDl7_3nRGa8NFivOIgTP.md @@ -1,3 +1,7 @@ # Desktop Inference -Desktop inference runs on consumer hardware, primarily Apple M-series unified memory systems and high-end NVIDIA consumer GPUs. Apple hardware offers very large unified memory capacity at moderate bandwidth, enabling larger models than discrete consumer GPUs; NVIDIA consumer GPUs offer higher bandwidth with smaller capacity. Tools like Ollama and llama.cpp make it practical to run aggressively quantized models on consumer hardware with minimal setup. \ No newline at end of file +Desktop inference runs on consumer hardware, primarily Apple M-series unified memory systems and high-end NVIDIA consumer GPUs. Apple hardware offers very large unified memory capacity at moderate bandwidth, enabling larger models than discrete consumer GPUs; NVIDIA consumer GPUs offer higher bandwidth with smaller capacity. Tools like Ollama and llama.cpp make it practical to run aggressively quantized models on consumer hardware with minimal setup. + +Visit the following resources to learn more: + +- [@opensource@ComfyUI](https://github.com/comfyanonymous/ComfyUI) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/diffusion-models@aagMwPPmi6ngfkIqZL-Mg.md b/roadmaps/inference-engineering/content/diffusion-models@aagMwPPmi6ngfkIqZL-Mg.md index e28f6c80d..d9b7a49a2 100644 --- a/roadmaps/inference-engineering/content/diffusion-models@aagMwPPmi6ngfkIqZL-Mg.md +++ b/roadmaps/inference-engineering/content/diffusion-models@aagMwPPmi6ngfkIqZL-Mg.md @@ -1,3 +1,10 @@ # Diffusion Models -Diffusion models generate images by starting from random noise and iteratively denoising it over 30 to 50 steps, guided by a text prompt. Each step runs the denoiser twice, once with conditioning and once without, to apply classifier-free guidance. The entire process operates in a low-dimensional latent space rather than full pixel space to make attention computationally feasible. \ No newline at end of file +Diffusion models generate images by starting from random noise and iteratively denoising it over 30 to 50 steps, guided by a text prompt. Each step runs the denoiser twice, once with conditioning and once without, to apply classifier-free guidance. The entire process operates in a low-dimensional latent space rather than full pixel space to make attention computationally feasible. + +Visit the following resources to learn more: + +- [@article@Denoising Diffusion Probabilistic Models](https://arxiv.org/abs/2006.11239) +- [@article@High-Resolution Image Synthesis with Latent Diffusion Models](https://arxiv.org/abs/2112.10752) +- [@article@SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://arxiv.org/abs/2307.01952) +- [@article@Scalable Diffusion Models with Transformers (DiT)](https://arxiv.org/abs/2212.09748) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/disaggregation@f6PWIqf4LubR7lQztaoB5.md b/roadmaps/inference-engineering/content/disaggregation@f6PWIqf4LubR7lQztaoB5.md index cbfbeefd6..2ea1e4cdc 100644 --- a/roadmaps/inference-engineering/content/disaggregation@f6PWIqf4LubR7lQztaoB5.md +++ b/roadmaps/inference-engineering/content/disaggregation@f6PWIqf4LubR7lQztaoB5.md @@ -1,3 +1,7 @@ # Disaggregation -Disaggregation separates the prefill and decode phases onto independent GPU instances that can scale separately. The prefill engine processes the input sequence and sends the resulting KV cache to the decode engine via the hardware interconnect. This removes the resource contention between compute-heavy prefill and memory-heavy decode. Disaggregation is most effective at high traffic volumes, for large models, and with prefill-heavy workloads like code editors. \ No newline at end of file +Disaggregation separates the prefill and decode phases onto independent GPU instances that can scale separately. The prefill engine processes the input sequence and sends the resulting KV cache to the decode engine via the hardware interconnect. This removes the resource contention between compute-heavy prefill and memory-heavy decode. Disaggregation is most effective at high traffic volumes, for large models, and with prefill-heavy workloads like code editors. + +Visit the following resources to learn more: + +- [@article@Beyond the Buzz: A Pragmatic Take on Inference Disaggregation](https://arxiv.org/abs/2506.05508) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/distillation@IdNkeL9pzy4lFv17ak1Vy.md b/roadmaps/inference-engineering/content/distillation@IdNkeL9pzy4lFv17ak1Vy.md index ffe8dae24..76082df12 100644 --- a/roadmaps/inference-engineering/content/distillation@IdNkeL9pzy4lFv17ak1Vy.md +++ b/roadmaps/inference-engineering/content/distillation@IdNkeL9pzy4lFv17ak1Vy.md @@ -1,3 +1,7 @@ # Distillation -Distillation trains a smaller student model to emulate a larger teacher model by exposing it to the teacher's probability distributions, not just its final outputs. Unlike fine-tuning on synthetic data, distillation transfers the teacher's reasoning behavior. The resulting student model is smaller and faster while retaining much of the teacher's quality on the same tasks. \ No newline at end of file +Distillation trains a smaller student model to emulate a larger teacher model by exposing it to the teacher's probability distributions, not just its final outputs. Unlike fine-tuning on synthetic data, distillation transfers the teacher's reasoning behavior. The resulting student model is smaller and faster while retaining much of the teacher's quality on the same tasks. + +Visit the following resources to learn more: + +- [@article@Adversarial Diffusion Distillation](https://arxiv.org/abs/2311.17042) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/draft-target-decoding@m0EIHovHWGkWULN0a3C3v.md b/roadmaps/inference-engineering/content/draft-target-decoding@m0EIHovHWGkWULN0a3C3v.md index 3b5bdd4fc..f5c20953f 100644 --- a/roadmaps/inference-engineering/content/draft-target-decoding@m0EIHovHWGkWULN0a3C3v.md +++ b/roadmaps/inference-engineering/content/draft-target-decoding@m0EIHovHWGkWULN0a3C3v.md @@ -1,3 +1,7 @@ # Draft-Target Speculative Decoding -Draft-target speculation uses a small draft model, typically at least ten times smaller than the target, to generate draft tokens for the target model to verify. The draft model should share the same tokenizer and general behavior as the target, often being a smaller member of the same model family. It is the simplest speculation method to set up but introduces the most overhead due to running a separate model. \ No newline at end of file +Draft-target speculation uses a small draft model, typically at least ten times smaller than the target, to generate draft tokens for the target model to verify. The draft model should share the same tokenizer and general behavior as the target, often being a smaller member of the same model family. It is the simplest speculation method to set up but introduces the most overhead due to running a separate model. + +Visit the following resources to learn more: + +- [@article@Fast Inference from Transformers via Speculative Decoding](https://arxiv.org/abs/2211.17192) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/eagle@0FZTg3_ZnTpMOQ078OOxE.md b/roadmaps/inference-engineering/content/eagle@0FZTg3_ZnTpMOQ078OOxE.md index 6524045c1..f64f3334a 100644 --- a/roadmaps/inference-engineering/content/eagle@0FZTg3_ZnTpMOQ078OOxE.md +++ b/roadmaps/inference-engineering/content/eagle@0FZTg3_ZnTpMOQ078OOxE.md @@ -1,3 +1,9 @@ # EAGLE -EAGLE is a purpose-built speculation method where a small draft model is trained to accept hidden states from the target model as input. By seeing the target's internal representations, EAGLE achieves higher acceptance rates and longer draft sequences (up to eight tokens) than using a general-purpose small model. It integrates into the same forward pass as the target model, eliminating the orchestration overhead of separate draft and target model runs. \ No newline at end of file +EAGLE is a purpose-built speculation method where a small draft model is trained to accept hidden states from the target model as input. By seeing the target's internal representations, EAGLE achieves higher acceptance rates and longer draft sequences (up to eight tokens) than using a general-purpose small model. It integrates into the same forward pass as the target model, eliminating the orchestration overhead of separate draft and target model runs. + +Visit the following resources to learn more: + +- [@article@EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty](https://arxiv.org/abs/2401.15077) +- [@article@EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees](https://arxiv.org/abs/2406.16858) +- [@article@EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test](https://arxiv.org/abs/2503.01840) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/embedding-models@Tvt-vX1gWuJZJBHh5vfKu.md b/roadmaps/inference-engineering/content/embedding-models@Tvt-vX1gWuJZJBHh5vfKu.md index ffde49cfe..5e057244f 100644 --- a/roadmaps/inference-engineering/content/embedding-models@Tvt-vX1gWuJZJBHh5vfKu.md +++ b/roadmaps/inference-engineering/content/embedding-models@Tvt-vX1gWuJZJBHh5vfKu.md @@ -1,3 +1,10 @@ # Embedding Models -Embedding models convert variable-length text or image input into a fixed-length vector that encodes semantic meaning. These vectors are used in RAG, search, recommendation, and agent memory systems. Embedding inference has two distinct traffic profiles: high-throughput backfills for bulk indexing and low-latency lookups for real-time user queries, which are best served by separate deployments. \ No newline at end of file +Embedding models convert variable-length text or image input into a fixed-length vector that encodes semantic meaning. These vectors are used in RAG, search, recommendation, and agent memory systems. Embedding inference has two distinct traffic profiles: high-throughput backfills for bulk indexing and low-latency lookups for real-time user queries, which are best served by separate deployments. + +Visit the following resources to learn more: + +- [@article@BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/abs/1810.04805) +- [@article@Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084) +- [@article@Matryoshka Representation Learning](https://arxiv.org/abs/2205.13147) +- [@article@MTEB: Massive Text Embedding Benchmark](https://arxiv.org/abs/2210.07316) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/expert-parallelism@62y78y-RXZT_QCUfd-YFE.md b/roadmaps/inference-engineering/content/expert-parallelism@62y78y-RXZT_QCUfd-YFE.md index 97b5ecf00..c4844f5f8 100644 --- a/roadmaps/inference-engineering/content/expert-parallelism@62y78y-RXZT_QCUfd-YFE.md +++ b/roadmaps/inference-engineering/content/expert-parallelism@62y78y-RXZT_QCUfd-YFE.md @@ -1,3 +1,8 @@ # Expert Parallelism -Expert Parallelism distributes MoE experts across GPUs so each GPU hosts multiple complete experts. Tokens are routed to the GPU holding their activated experts, and each expert runs independently within a single GPU without splitting across multiple. EP has lower inter-GPU communication overhead than Tensor Parallelism and scales well to multi-node deployments via InfiniBand. \ No newline at end of file +Expert Parallelism distributes MoE experts across GPUs so each GPU hosts multiple complete experts. Tokens are routed to the GPU holding their activated experts, and each expert runs independently within a single GPU without splitting across multiple. EP has lower inter-GPU communication overhead than Tensor Parallelism and scales well to multi-node deployments via InfiniBand. + +Visit the following resources to learn more: + +- [@opensource@NVIDIA/Megatron-LM](https://github.com/NVIDIA/Megatron-LM) +- [@article@Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://arxiv.org/abs/1909.08053) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/few-step-image-generation@biVjpCA3bsKI7sa_aBIUE.md b/roadmaps/inference-engineering/content/few-step-image-generation@biVjpCA3bsKI7sa_aBIUE.md index 9517449ff..86272102e 100644 --- a/roadmaps/inference-engineering/content/few-step-image-generation@biVjpCA3bsKI7sa_aBIUE.md +++ b/roadmaps/inference-engineering/content/few-step-image-generation@biVjpCA3bsKI7sa_aBIUE.md @@ -1,3 +1,7 @@ # Few-Step Image Generation -Few-step image generation models produce usable images in eight or fewer denoising steps by training the model to collapse the standard 30 to 50 step process into a fraction of the iterations. Techniques include latent consistency distillation and adversarial training, both of which teach the model to reach high-quality outputs in fewer passes. They are a strong fit for latency-sensitive use cases like real-time filters, but produce noticeably lower quality than full-step models and are a model selection decision rather than an inference optimization technique. \ No newline at end of file +Few-step image generation models produce usable images in eight or fewer denoising steps by training the model to collapse the standard 30 to 50 step process into a fraction of the iterations. Techniques include latent consistency distillation and adversarial training, both of which teach the model to reach high-quality outputs in fewer passes. They are a strong fit for latency-sensitive use cases like real-time filters, but produce noticeably lower quality than full-step models and are a model selection decision rather than an inference optimization technique. + +Visit the following resources to learn more: + +- [@article@Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://arxiv.org/abs/2310.04378) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/fine-tuning@fWJplrAuUStEMNNLQc75G.md b/roadmaps/inference-engineering/content/fine-tuning@fWJplrAuUStEMNNLQc75G.md index 86dc5d576..15d97e342 100644 --- a/roadmaps/inference-engineering/content/fine-tuning@fWJplrAuUStEMNNLQc75G.md +++ b/roadmaps/inference-engineering/content/fine-tuning@fWJplrAuUStEMNNLQc75G.md @@ -1,3 +1,7 @@ # Fine-Tuning -Fine-tuning takes a pretrained foundation model and adapts it to a specific domain by training on additional labeled data. A small fine-tuned model can match a much larger general model on constrained tasks like text-to-SQL. Smaller models are faster and cheaper to serve, so fine-tuning is one of the highest-leverage tools for hitting latency and cost targets. \ No newline at end of file +Fine-tuning takes a pretrained foundation model and adapts it to a specific domain by training on additional labeled data. A small fine-tuned model can match a much larger general model on constrained tasks like text-to-SQL. Smaller models are faster and cheaper to serve, so fine-tuning is one of the highest-leverage tools for hitting latency and cost targets. + +Visit the following resources to learn more: + +- [@article@Adversarial Diffusion Distillation](https://arxiv.org/abs/2311.17042) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/flashattention@mQcj-H3R789o6nDh8tG9k.md b/roadmaps/inference-engineering/content/flashattention@mQcj-H3R789o6nDh8tG9k.md index a25edfa3b..655ffbbed 100644 --- a/roadmaps/inference-engineering/content/flashattention@mQcj-H3R789o6nDh8tG9k.md +++ b/roadmaps/inference-engineering/content/flashattention@mQcj-H3R789o6nDh8tG9k.md @@ -1,3 +1,10 @@ # FlashAttention -FlashAttention is a series of hand-optimized CUDA kernels that compute attention with fewer reads and writes to GPU memory by fusing the attention algorithm into a single kernel. Standard attention stores intermediate matrices to memory between steps; FlashAttention eliminates those round-trips. FlashAttention 3 targets Hopper GPUs and FlashAttention 4 targets Blackwell GPUs. \ No newline at end of file +FlashAttention is a series of hand-optimized CUDA kernels that compute attention with fewer reads and writes to GPU memory by fusing the attention algorithm into a single kernel. Standard attention stores intermediate matrices to memory between steps; FlashAttention eliminates those round-trips. FlashAttention 3 targets Hopper GPUs and FlashAttention 4 targets Blackwell GPUs. + +Visit the following resources to learn more: + +- [@opensource@Dao-AILab/flash-attention](https://github.com/Dao-AILab/flash-attention) +- [@article@FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://arxiv.org/abs/2205.14135) +- [@article@FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://arxiv.org/abs/2307.08691) +- [@article@FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision](https://arxiv.org/abs/2407.08608) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/gpu-architecture@00JLMvn8DUJBCw6-BxO0y.md b/roadmaps/inference-engineering/content/gpu-architecture@00JLMvn8DUJBCw6-BxO0y.md index d2489002e..1b22c166f 100644 --- a/roadmaps/inference-engineering/content/gpu-architecture@00JLMvn8DUJBCw6-BxO0y.md +++ b/roadmaps/inference-engineering/content/gpu-architecture@00JLMvn8DUJBCw6-BxO0y.md @@ -1,3 +1,9 @@ # GPU Architecture -GPU architecture determines the hard limits of what inference can achieve on a given device. This group covers compute, memory, interconnects, and sizing — the four dimensions an inference engineer needs to understand to make informed decisions about hardware selection, parallelism strategy, and optimization priorities. \ No newline at end of file +GPU architecture determines the hard limits of what inference can achieve on a given device. This group covers compute, memory, interconnects, and sizing — the four dimensions an inference engineer needs to understand to make informed decisions about hardware selection, parallelism strategy, and optimization priorities. + +Visit the following resources to learn more: + +- [@article@NVIDIA Tesla: A Unified Graphics and Computing Architecture](https://ieeexplore.ieee.org/document/4523358) +- [@article@GPU Glossary](https://modal.com/gpu-glossary) +- [@book@Programming Massively Parallel Processors](https://www.goodreads.com/work/editions/10244675) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/grace--vera@kPT86kA0WwE_fWUAAKCTT.md b/roadmaps/inference-engineering/content/grace--vera@kPT86kA0WwE_fWUAAKCTT.md index 68e0c6f68..0112073f7 100644 --- a/roadmaps/inference-engineering/content/grace--vera@kPT86kA0WwE_fWUAAKCTT.md +++ b/roadmaps/inference-engineering/content/grace--vera@kPT86kA0WwE_fWUAAKCTT.md @@ -1,3 +1,7 @@ # Grace and Vera CPUs -NVIDIA's ARM-based Grace and Vera CPUs connect to their paired GPUs via NVLink Chip-to-Chip at up to 900 GB/s, far higher than standard PCIe connections. This high-bandwidth CPU-GPU link makes offloading KV caches and LoRA weights to CPU memory practical, since data can be retrieved quickly enough to remain on the critical path. Grace pairs with Hopper and Blackwell; Vera pairs with the upcoming Rubin architecture. \ No newline at end of file +NVIDIA's ARM-based Grace and Vera CPUs connect to their paired GPUs via NVLink Chip-to-Chip at up to 900 GB/s, far higher than standard PCIe connections. This high-bandwidth CPU-GPU link makes offloading KV caches and LoRA weights to CPU memory practical, since data can be retrieved quickly enough to remain on the critical path. Grace pairs with Hopper and Blackwell; Vera pairs with the upcoming Rubin architecture. + +Visit the following resources to learn more: + +- [@official@NVIDIA Grace CPU](https://www.nvidia.com/en-us/data-center/grace-cpu/) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/hopper@XQMbaW1tFdO8Os6mxcR2M.md b/roadmaps/inference-engineering/content/hopper@XQMbaW1tFdO8Os6mxcR2M.md index fca29252e..88ff4a4f2 100644 --- a/roadmaps/inference-engineering/content/hopper@XQMbaW1tFdO8Os6mxcR2M.md +++ b/roadmaps/inference-engineering/content/hopper@XQMbaW1tFdO8Os6mxcR2M.md @@ -1,3 +1,7 @@ # Hopper -The Hopper architecture, released in 2022 with the H100 and later the H200, introduced FP8 Tensor Cores offering twice the compute throughput of FP16 and asynchronous data transfer features used by FlashAttention 3. H100 and H200 GPUs remain among the most widely deployed inference accelerators due to their maturity, broad software support, and relative availability compared to newer generations. \ No newline at end of file +The Hopper architecture, released in 2022 with the H100 and later the H200, introduced FP8 Tensor Cores offering twice the compute throughput of FP16 and asynchronous data transfer features used by FlashAttention 3. H100 and H200 GPUs remain among the most widely deployed inference accelerators due to their maturity, broad software support, and relative availability compared to newer generations. + +Visit the following resources to learn more: + +- [@official@NVIDIA H100 Tensor Core GPU Architecture](https://resources.nvidia.com/en-us-hopper-architecture/nvidia-h100-tensor-c) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/image-generation-arch@xsV4WevPzhfydyvxVzGcw.md b/roadmaps/inference-engineering/content/image-generation-arch@xsV4WevPzhfydyvxVzGcw.md index f5ebf793e..3f2b22ff0 100644 --- a/roadmaps/inference-engineering/content/image-generation-arch@xsV4WevPzhfydyvxVzGcw.md +++ b/roadmaps/inference-engineering/content/image-generation-arch@xsV4WevPzhfydyvxVzGcw.md @@ -1,3 +1,10 @@ # Image Generation Architecture -Image generation models are pipelines of multiple components: a text encoder that converts the prompt into conditioning, a denoising model that iteratively refines a latent representation from noise into an image, and a variational autoencoder that converts the final latent to pixels. The denoising model is the largest and most compute-intensive component and runs up to 100 forward passes per image generation. \ No newline at end of file +Image generation models are pipelines of multiple components: a text encoder that converts the prompt into conditioning, a denoising model that iteratively refines a latent representation from noise into an image, and a variational autoencoder that converts the final latent to pixels. The denoising model is the largest and most compute-intensive component and runs up to 100 forward passes per image generation. + +Visit the following resources to learn more: + +- [@article@Denoising Diffusion Probabilistic Models](https://arxiv.org/abs/2006.11239) +- [@article@High-Resolution Image Synthesis with Latent Diffusion Models](https://arxiv.org/abs/2112.10752) +- [@article@SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://arxiv.org/abs/2307.01952) +- [@article@Scalable Diffusion Models with Transformers (DiT)](https://arxiv.org/abs/2212.09748) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/image-generation-inference@lnIHjq4nvCs3zvUln-jHR.md b/roadmaps/inference-engineering/content/image-generation-inference@lnIHjq4nvCs3zvUln-jHR.md index df9ed063d..18a6ee6c0 100644 --- a/roadmaps/inference-engineering/content/image-generation-inference@lnIHjq4nvCs3zvUln-jHR.md +++ b/roadmaps/inference-engineering/content/image-generation-inference@lnIHjq4nvCs3zvUln-jHR.md @@ -1,3 +1,8 @@ # Image Generation Inference -Production image generation inference uses SGLang Diffusion, TensorRT, or hand-optimized PyTorch. The most important optimization target is the attention kernel, where FlashAttention 3 or 4 outperforms the default on Hopper and Blackwell respectively. GEMM kernels for linear layers can be quantized to FP8 safely. Torch compilation with manual kernel plugins caches the compiled engine to avoid multi-minute recompilation on every cold start. \ No newline at end of file +Production image generation inference uses SGLang Diffusion, TensorRT, or hand-optimized PyTorch. The most important optimization target is the attention kernel, where FlashAttention 3 or 4 outperforms the default on Hopper and Blackwell respectively. GEMM kernels for linear layers can be quantized to FP8 safely. Torch compilation with manual kernel plugins caches the compiled engine to avoid multi-minute recompilation on every cold start. + +Visit the following resources to learn more: + +- [@opensource@ComfyUI](https://github.com/comfyanonymous/ComfyUI) +- [@article@Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://arxiv.org/abs/2310.04378) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/inference-vs-training@fVUfQYb8BP_hFMmd1S2-J.md b/roadmaps/inference-engineering/content/inference-vs-training@fVUfQYb8BP_hFMmd1S2-J.md index 67d74abfe..7fc1c4c8b 100644 --- a/roadmaps/inference-engineering/content/inference-vs-training@fVUfQYb8BP_hFMmd1S2-J.md +++ b/roadmaps/inference-engineering/content/inference-vs-training@fVUfQYb8BP_hFMmd1S2-J.md @@ -1,3 +1,7 @@ # Inference vs. Training -Training is the process of learning model weights from data. Inference is serving those weights in production to generate outputs. Training runs are compute-intensive batch jobs; inference is a continuous, latency-sensitive service. Where training success is measured in loss curves, inference success is measured in time to first token, tokens per second, and uptime. \ No newline at end of file +Training is the process of learning model weights from data. Inference is serving those weights in production to generate outputs. Training runs are compute-intensive batch jobs; inference is a continuous, latency-sensitive service. Where training success is measured in loss curves, inference success is measured in time to first token, tokens per second, and uptime. + +Visit the following resources to learn more: + +- [@article@Language Models Are Few-Shot Learners](https://arxiv.org/abs/2005.14165) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/instances--interconnects@dAo4PMB_VnRFMSjyDrLj0.md b/roadmaps/inference-engineering/content/instances--interconnects@dAo4PMB_VnRFMSjyDrLj0.md index 6ae437c1a..4d70d52fe 100644 --- a/roadmaps/inference-engineering/content/instances--interconnects@dAo4PMB_VnRFMSjyDrLj0.md +++ b/roadmaps/inference-engineering/content/instances--interconnects@dAo4PMB_VnRFMSjyDrLj0.md @@ -1,3 +1,8 @@ # Instances and Interconnects -Within a node, NVLink connects GPUs one-to-one at up to 1800 GB/s on Blackwell, and NVSwitch creates an all-to-all fabric across all eight GPUs. Between nodes, InfiniBand provides inter-node connectivity at substantially lower bandwidth than NVLink. Multi-Instance GPU (MIG) partitions a single GPU into up to seven isolated slices, each with dedicated compute and memory, allowing small models to run on fractions of a high-end GPU without resource contention. \ No newline at end of file +Within a node, NVLink connects GPUs one-to-one at up to 1800 GB/s on Blackwell, and NVSwitch creates an all-to-all fabric across all eight GPUs. Between nodes, InfiniBand provides inter-node connectivity at substantially lower bandwidth than NVLink. Multi-Instance GPU (MIG) partitions a single GPU into up to seven isolated slices, each with dedicated compute and memory, allowing small models to run on fractions of a high-end GPU without resource contention. + +Visit the following resources to learn more: + +- [@official@NVLink and NVSwitch](https://www.nvidia.com/en-us/data-center/nvlink/) +- [@official@NVIDIA InfiniBand](https://www.nvidia.com/en-us/networking/products/infiniband/) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/kv-cache-storage@gkt2Gr34RJJf1NZgbmtXf.md b/roadmaps/inference-engineering/content/kv-cache-storage@gkt2Gr34RJJf1NZgbmtXf.md index 3472c860c..68c41d093 100644 --- a/roadmaps/inference-engineering/content/kv-cache-storage@gkt2Gr34RJJf1NZgbmtXf.md +++ b/roadmaps/inference-engineering/content/kv-cache-storage@gkt2Gr34RJJf1NZgbmtXf.md @@ -1,3 +1,8 @@ # KV Cache Storage -KV caches can be stored in GPU VRAM (fastest, tens to hundreds of gigabytes), CPU RAM (slower, hundreds of gigabytes to terabytes), local SSD, or networked SSD. Moving less frequently accessed KV blocks to slower storage frees VRAM for active requests. NVIDIA Dynamo's KV Block Manager provides APIs for managing KV blocks across these storage tiers transparently. \ No newline at end of file +KV caches can be stored in GPU VRAM (fastest, tens to hundreds of gigabytes), CPU RAM (slower, hundreds of gigabytes to terabytes), local SSD, or networked SSD. Moving less frequently accessed KV blocks to slower storage frees VRAM for active requests. NVIDIA Dynamo's KV Block Manager provides APIs for managing KV blocks across these storage tiers transparently. + +Visit the following resources to learn more: + +- [@opensource@LMCache](https://github.com/LMCache/LMCache) +- [@article@CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion](https://arxiv.org/abs/2405.16444) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/kv-cache@V678kX8FPChTZCe7Nqwhv.md b/roadmaps/inference-engineering/content/kv-cache@V678kX8FPChTZCe7Nqwhv.md index b921c7003..b2c02c6e6 100644 --- a/roadmaps/inference-engineering/content/kv-cache@V678kX8FPChTZCe7Nqwhv.md +++ b/roadmaps/inference-engineering/content/kv-cache@V678kX8FPChTZCe7Nqwhv.md @@ -1,3 +1,7 @@ # KV Cache -The KV cache stores the key and value tensors computed during attention for each previously processed token. Without it, attention would require recomputing every prior token on each decode step, making inference prohibitively slow. The KV cache lives in GPU memory and its size grows with sequence length and batch size, making it a major target for memory management techniques. \ No newline at end of file +The KV cache stores the key and value tensors computed during attention for each previously processed token. Without it, attention would require recomputing every prior token on each decode step, making inference prohibitively slow. The KV cache lives in GPU memory and its size grows with sequence length and batch size, making it a major target for memory management techniques. + +Visit the following resources to learn more: + +- [@article@Efficient Memory Management for Large Language Model Serving with PagedAttention](https://arxiv.org/abs/2309.06180) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/large-language-models@irgGvEnY1_WW1EPtuMKgC.md b/roadmaps/inference-engineering/content/large-language-models@irgGvEnY1_WW1EPtuMKgC.md index 4133a9a98..9092ef62d 100644 --- a/roadmaps/inference-engineering/content/large-language-models@irgGvEnY1_WW1EPtuMKgC.md +++ b/roadmaps/inference-engineering/content/large-language-models@irgGvEnY1_WW1EPtuMKgC.md @@ -1,3 +1,7 @@ # Large Language Models -An LLM is a decoder-only transformer with three main components: an embedding layer that converts input tokens to vectors, a stack of transformer blocks that process those vectors, and an output language modeling head that converts the final hidden states into a probability distribution over the vocabulary. The configuration file for an open model specifies all architectural details including layer count, hidden dimensions, and attention heads. \ No newline at end of file +An LLM is a decoder-only transformer with three main components: an embedding layer that converts input tokens to vectors, a stack of transformer blocks that process those vectors, and an output language modeling head that converts the final hidden states into a probability distribution over the vocabulary. The configuration file for an open model specifies all architectural details including layer count, hidden dimensions, and attention heads. + +Visit the following resources to learn more: + +- [@book@Build a Large Language Model From Scratch](https://www.manning.com/books/build-a-large-language-model-from-scratch) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/llm-bottlenecks@QVadKzBfLdegat-4wgOMX.md b/roadmaps/inference-engineering/content/llm-bottlenecks@QVadKzBfLdegat-4wgOMX.md index 3eb73dde1..8975a26a8 100644 --- a/roadmaps/inference-engineering/content/llm-bottlenecks@QVadKzBfLdegat-4wgOMX.md +++ b/roadmaps/inference-engineering/content/llm-bottlenecks@QVadKzBfLdegat-4wgOMX.md @@ -1,3 +1,8 @@ # LLM Inference Bottlenecks -LLM prefill is compute-bound because it processes the entire input sequence as one large matrix multiplication, performing many operations per byte loaded. LLM decode is memory-bound because it generates tokens one at a time via vector-matrix multiplications that require loading all model weights from memory for each token while performing relatively few arithmetic operations. \ No newline at end of file +LLM prefill is compute-bound because it processes the entire input sequence as one large matrix multiplication, performing many operations per byte loaded. LLM decode is memory-bound because it generates tokens one at a time via vector-matrix multiplications that require loading all model weights from memory for each token while performing relatively few arithmetic operations. + +Visit the following resources to learn more: + +- [@article@GPU Glossary](https://modal.com/gpu-glossary) +- [@book@AI Systems Performance Engineering](https://www.oreilly.com/library/view/ai-systems-performance/9781098170202/) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/long-context-handling@uL3Fn8XfS1Yv_PJd9R8wi.md b/roadmaps/inference-engineering/content/long-context-handling@uL3Fn8XfS1Yv_PJd9R8wi.md index 4d827c5f6..dc5aba8c7 100644 --- a/roadmaps/inference-engineering/content/long-context-handling@uL3Fn8XfS1Yv_PJd9R8wi.md +++ b/roadmaps/inference-engineering/content/long-context-handling@uL3Fn8XfS1Yv_PJd9R8wi.md @@ -1,3 +1,8 @@ # Long Context Handling -Long sequences produce KV caches large enough to strain GPU memory, slow down decode, and increase prefill time. Techniques for handling long context include FlashAttention for memory-efficient attention computation, PagedAttention for non-contiguous KV storage, chunked prefill to interleave long prefill with decode, and model-level approaches like sliding window attention that limit how many prior tokens each token attends to. \ No newline at end of file +Long sequences produce KV caches large enough to strain GPU memory, slow down decode, and increase prefill time. Techniques for handling long context include FlashAttention for memory-efficient attention computation, PagedAttention for non-contiguous KV storage, chunked prefill to interleave long prefill with decode, and model-level approaches like sliding window attention that limit how many prior tokens each token attends to. + +Visit the following resources to learn more: + +- [@article@Ring Attention with Blockwise Transformers for Near-Infinite Context](https://arxiv.org/abs/2310.01889) +- [@article@RoFormer: Enhanced Transformer with Rotary Position Embedding](https://arxiv.org/abs/2104.09864) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/mixture-of-experts-moe@7BddTfOkWmlL4HAkFBXtq.md b/roadmaps/inference-engineering/content/mixture-of-experts-moe@7BddTfOkWmlL4HAkFBXtq.md index 5062fa767..3464eef3d 100644 --- a/roadmaps/inference-engineering/content/mixture-of-experts-moe@7BddTfOkWmlL4HAkFBXtq.md +++ b/roadmaps/inference-engineering/content/mixture-of-experts-moe@7BddTfOkWmlL4HAkFBXtq.md @@ -1,3 +1,7 @@ # Mixture of Experts (MoE) -Mixture of Experts models replace dense linear layers with a set of smaller expert matrices, activating only a subset of experts per token via a learned router. A model like Qwen3-235B activates only 22 billion of its 235 billion parameters per forward pass, making single-user inference efficient. In batched production serving, different tokens activate different experts, so nearly all parameters are active across the batch. \ No newline at end of file +Mixture of Experts models replace dense linear layers with a set of smaller expert matrices, activating only a subset of experts per token via a learned router. A model like Qwen3-235B activates only 22 billion of its 235 billion parameters per forward pass, making single-user inference efficient. In batched production serving, different tokens activate different experts, so nearly all parameters are active across the batch. + +Visit the following resources to learn more: + +- [@article@Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://arxiv.org/abs/1701.06538) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/model-evaluation@lVf5GoJF_PVTkTiQFTNVg.md b/roadmaps/inference-engineering/content/model-evaluation@lVf5GoJF_PVTkTiQFTNVg.md index 37ab15cca..9fbd6f1b5 100644 --- a/roadmaps/inference-engineering/content/model-evaluation@lVf5GoJF_PVTkTiQFTNVg.md +++ b/roadmaps/inference-engineering/content/model-evaluation@lVf5GoJF_PVTkTiQFTNVg.md @@ -1,3 +1,8 @@ # Model Evaluation -Model evaluation, or evals, is the practice of systematically measuring model quality for a specific task or domain. Standard intelligence benchmarks like MMLU measure general capabilities but can be gamed; product-specific evals that test real use cases provide more reliable signal. High-confidence evals are a prerequisite before investing in inference optimization, ensuring the model is actually useful before making it fast. \ No newline at end of file +Model evaluation, or evals, is the practice of systematically measuring model quality for a specific task or domain. Standard intelligence benchmarks like MMLU measure general capabilities but can be gamed; product-specific evals that test real use cases provide more reliable signal. High-confidence evals are a prerequisite before investing in inference optimization, ensuring the model is actually useful before making it fast. + +Visit the following resources to learn more: + +- [@article@How to Fine-Tune Qwen3 to GPT-4o Level Performance](https://ghost.oxen.ai/how-to-fine-tune-qwen3-to-gpt-4o-level-performance/) +- [@book@Evals for AI Engineers](https://www.oreilly.com/library/view/evals-for-ai/9798341660717/) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/model-selection@jQAWZ1plw4_wPE1UvE_Ux.md b/roadmaps/inference-engineering/content/model-selection@jQAWZ1plw4_wPE1UvE_Ux.md index ec0140381..4d05c2e9a 100644 --- a/roadmaps/inference-engineering/content/model-selection@jQAWZ1plw4_wPE1UvE_Ux.md +++ b/roadmaps/inference-engineering/content/model-selection@jQAWZ1plw4_wPE1UvE_Ux.md @@ -1,3 +1,7 @@ # Model Selection -The most important decision in model performance optimization is which model to use. Smaller models with fewer parameters are faster and cheaper to serve than larger ones. The goal is to find the smallest model that meets quality requirements for the task. Model architecture also determines which inference engines and optimization techniques are available, so popular architectures with broad tooling support are preferable. \ No newline at end of file +The most important decision in model performance optimization is which model to use. Smaller models with fewer parameters are faster and cheaper to serve than larger ones. The goal is to find the smallest model that meets quality requirements for the task. Model architecture also determines which inference engines and optimization techniques are available, so popular architectures with broad tooling support are preferable. + +Visit the following resources to learn more: + +- [@book@AI Engineering: Building Applications with Foundation Models](https://www.oreilly.com/library/view/ai-engineering/9781098166298/) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/multi-cloud@d78UiI_FZyLsU6O52btXg.md b/roadmaps/inference-engineering/content/multi-cloud@d78UiI_FZyLsU6O52btXg.md index 497240a0c..0f678708d 100644 --- a/roadmaps/inference-engineering/content/multi-cloud@d78UiI_FZyLsU6O52btXg.md +++ b/roadmaps/inference-engineering/content/multi-cloud@d78UiI_FZyLsU6O52btXg.md @@ -1,3 +1,10 @@ -# Multi-Cloud +# Multi-Cloud -Multi-cloud inference distributes GPU capacity across providers and regions to maximize availability, minimize latency, and meet data residency requirements. This group covers procurement strategy, capacity management, geographic routing, and the reliability patterns that make large-scale inference resilient to hardware and provider failures. \ No newline at end of file +Multi-cloud inference distributes GPU capacity across providers and regions to maximize availability, minimize latency, and meet data residency requirements. This group covers procurement strategy, capacity management, geographic routing, and the reliability patterns that make large-scale inference resilient to hardware and provider failures. + +Visit the following resources to learn more: + +- [@article@SemiAnalysis](https://semianalysis.com/) +- [@article@The Llama 3 Herd of Models](https://arxiv.org/abs/2407.21783) +- [@book@Designing Data-Intensive Applications](https://dataintensive.net/) +- [@book@Site Reliability Engineering](https://sre.google/books/) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/n-gram@ZOclQMd7JASGXTRyctfW2.md b/roadmaps/inference-engineering/content/n-gram@ZOclQMd7JASGXTRyctfW2.md index c2661df62..3dbc4580e 100644 --- a/roadmaps/inference-engineering/content/n-gram@ZOclQMd7JASGXTRyctfW2.md +++ b/roadmaps/inference-engineering/content/n-gram@ZOclQMd7JASGXTRyctfW2.md @@ -1,3 +1,7 @@ # N-gram Speculation -N-gram speculation builds a dictionary during prefill that maps token prefixes to observed n-gram continuations from the input. During decode, generated tokens are looked up in this dictionary and matching suffixes become draft tokens with no separate model required. It is most effective for code completion, where output frequently mirrors input syntax, and can produce very long draft sequences with high acceptance rates in that domain. \ No newline at end of file +N-gram speculation builds a dictionary during prefill that maps token prefixes to observed n-gram continuations from the input. During decode, generated tokens are looked up in this dictionary and matching suffixes become draft tokens with no separate model required. It is most effective for code completion, where output frequently mirrors input syntax, and can produce very long draft sequences with high acceptance rates in that domain. + +Visit the following resources to learn more: + +- [@article@Break the Sequential Dependency of LLM Inference Using Lookahead Decoding](https://arxiv.org/abs/2402.02057) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/neural-networks@p2S0W3EDn768jbqzc7CE7.md b/roadmaps/inference-engineering/content/neural-networks@p2S0W3EDn768jbqzc7CE7.md index bcf41deed..1df6cd1e0 100644 --- a/roadmaps/inference-engineering/content/neural-networks@p2S0W3EDn768jbqzc7CE7.md +++ b/roadmaps/inference-engineering/content/neural-networks@p2S0W3EDn768jbqzc7CE7.md @@ -1,3 +1,9 @@ # Neural Networks -A neural network is a layered structure of nodes where each node applies a learned weight matrix and bias to its input. Multiple layers connected in sequence form the foundation for all generative AI models. Inference engineers need a working model of how neural networks compute outputs to understand where performance bottlenecks arise. \ No newline at end of file +A neural network is a layered structure of nodes where each node applies a learned weight matrix and bias to its input. Multiple layers connected in sequence form the foundation for all generative AI models. Inference engineers need a working model of how neural networks compute outputs to understand where performance bottlenecks arise. + +Visit the following resources to learn more: + +- [@book@Deep Learning](https://www.deeplearningbook.org/) +- [@book@Deep Learning with Python](https://www.manning.com/books/deep-learning-with-python-second-edition) +- [@book@Build a Large Language Model From Scratch](https://www.manning.com/books/build-a-large-language-model-from-scratch) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/nvidia-dynamo@KhTFecXEkHR4WX-_TYY3g.md b/roadmaps/inference-engineering/content/nvidia-dynamo@KhTFecXEkHR4WX-_TYY3g.md index 75dcbc6e8..b873fe678 100644 --- a/roadmaps/inference-engineering/content/nvidia-dynamo@KhTFecXEkHR4WX-_TYY3g.md +++ b/roadmaps/inference-engineering/content/nvidia-dynamo@KhTFecXEkHR4WX-_TYY3g.md @@ -1,3 +1,7 @@ # NVIDIA Dynamo -Dynamo supports conditional disaggregation, where requests are first sent to the decode engine and only routed to a prefill engine when the input is long or not already cached. The ratio of prefill to decode engines is configurable at runtime and can shift as traffic patterns change. This dynamic allocation prevents the prefill queue from becoming a bottleneck and avoids wasting decode engine resources on prefill-heavy traffic. \ No newline at end of file +Dynamo supports conditional disaggregation, where requests are first sent to the decode engine and only routed to a prefill engine when the input is long or not already cached. The ratio of prefill to decode engines is configurable at runtime and can shift as traffic patterns change. This dynamic allocation prevents the prefill queue from becoming a bottleneck and avoids wasting decode engine resources on prefill-heavy traffic. + +Visit the following resources to learn more: + +- [@official@NVIDIA Dynamo Documentation](https://docs.nvidia.com/dynamo/latest/index.html) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/nvidia-dynamo@ctdMz6rHa4Cwl1x19V_5k.md b/roadmaps/inference-engineering/content/nvidia-dynamo@ctdMz6rHa4Cwl1x19V_5k.md index ec30a5544..7d5dccca1 100644 --- a/roadmaps/inference-engineering/content/nvidia-dynamo@ctdMz6rHa4Cwl1x19V_5k.md +++ b/roadmaps/inference-engineering/content/nvidia-dynamo@ctdMz6rHa4Cwl1x19V_5k.md @@ -1,3 +1,7 @@ # NVIDIA Dynamo -Dynamo is NVIDIA's open-source distributed serving platform that sits above inference engines and provides orchestration for large-scale deployments. It adds KV-aware routing, disaggregated prefill/decode serving, multi-node parallelism, and an SLA-based planner that dynamically adjusts the number of prefill and decode workers based on TTFT and TPS targets. \ No newline at end of file +Dynamo is NVIDIA's open-source distributed serving platform that sits above inference engines and provides orchestration for large-scale deployments. It adds KV-aware routing, disaggregated prefill/decode serving, multi-node parallelism, and an SLA-based planner that dynamically adjusts the number of prefill and decode workers based on TTFT and TPS targets. + +Visit the following resources to learn more: + +- [@official@NVIDIA Dynamo Documentation](https://docs.nvidia.com/dynamo/latest/index.html) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/observability@w-dVv5dFNxD7Rqy16m4uL.md b/roadmaps/inference-engineering/content/observability@w-dVv5dFNxD7Rqy16m4uL.md index f836629ba..b66f800cb 100644 --- a/roadmaps/inference-engineering/content/observability@w-dVv5dFNxD7Rqy16m4uL.md +++ b/roadmaps/inference-engineering/content/observability@w-dVv5dFNxD7Rqy16m4uL.md @@ -1,3 +1,7 @@ # Observability -Inference observability tracks total request volume, input and output sequence lengths, response codes, latency percentiles (P50, P90, P99 of TTFT, TPS, and end-to-end latency), replica count, GPU utilization, and queue depth. These metrics are interdependent; a latency spike could come from long input sequences, high concurrency, or a hardware issue. Observability must integrate with existing tooling like Grafana, Datadog, and PagerDuty to put inference metrics in context. \ No newline at end of file +Inference observability tracks total request volume, input and output sequence lengths, response codes, latency percentiles (P50, P90, P99 of TTFT, TPS, and end-to-end latency), replica count, GPU utilization, and queue depth. These metrics are interdependent; a latency spike could come from long input sequences, high concurrency, or a hardware issue. Observability must integrate with existing tooling like Grafana, Datadog, and PagerDuty to put inference metrics in context. + +Visit the following resources to learn more: + +- [@official@NVIDIA Nsight Systems](https://developer.nvidia.com/nsight-systems) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/onnx-runtime@Vmtbdre3SZzwq3FxOO-Bc.md b/roadmaps/inference-engineering/content/onnx-runtime@Vmtbdre3SZzwq3FxOO-Bc.md index 931ee23f2..8c409ae95 100644 --- a/roadmaps/inference-engineering/content/onnx-runtime@Vmtbdre3SZzwq3FxOO-Bc.md +++ b/roadmaps/inference-engineering/content/onnx-runtime@Vmtbdre3SZzwq3FxOO-Bc.md @@ -1,3 +1,7 @@ # ONNX Runtime -ONNX Runtime is an open-source inference runtime that executes models exported to the ONNX format, which stores both weights and a computation graph for cross-platform portability. It supports multiple hardware backends including CUDA, DirectML, and CPU. It is widely used for embedding models, vision models, and pipelines that need to run across hardware environments without engine-specific configuration. \ No newline at end of file +ONNX Runtime is an open-source inference runtime that executes models exported to the ONNX format, which stores both weights and a computation graph for cross-platform portability. It supports multiple hardware backends including CUDA, DirectML, and CPU. It is widely used for embedding models, vision models, and pipelines that need to run across hardware environments without engine-specific configuration. + +Visit the following resources to learn more: + +- [@official@ONNX Runtime](https://onnxruntime.ai/) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/open-vs-closed-models@KAt5FzglOUtH7jLlzDn-I.md b/roadmaps/inference-engineering/content/open-vs-closed-models@KAt5FzglOUtH7jLlzDn-I.md index 3fc3fea0d..0c7606d17 100644 --- a/roadmaps/inference-engineering/content/open-vs-closed-models@KAt5FzglOUtH7jLlzDn-I.md +++ b/roadmaps/inference-engineering/content/open-vs-closed-models@KAt5FzglOUtH7jLlzDn-I.md @@ -1,3 +1,7 @@ # Open vs. Closed Models -Closed models are accessed exclusively through a vendor API; the weights are never released. Open models publish their weights for download and self-hosting. For inference engineers, this distinction determines the entire problem space: closed models require no infrastructure but offer no control over latency, cost, or configuration, while open models require building and operating a full inference stack but give complete control over every layer of it. \ No newline at end of file +Closed models are accessed exclusively through a vendor API; the weights are never released. Open models publish their weights for download and self-hosting. For inference engineers, this distinction determines the entire problem space: closed models require no infrastructure but offer no control over latency, cost, or configuration, while open models require building and operating a full inference stack but give complete control over every layer of it. + +Visit the following resources to learn more: + +- [@book@AI Engineering: Building Applications with Foundation Models](https://www.oreilly.com/library/view/ai-engineering/9781098166298/) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/pagedattention@VcIRY4vSi7JkcV6g13vkN.md b/roadmaps/inference-engineering/content/pagedattention@VcIRY4vSi7JkcV6g13vkN.md index cb06cce3e..e95d3254c 100644 --- a/roadmaps/inference-engineering/content/pagedattention@VcIRY4vSi7JkcV6g13vkN.md +++ b/roadmaps/inference-engineering/content/pagedattention@VcIRY4vSi7JkcV6g13vkN.md @@ -1,3 +1,7 @@ # PagedAttention -PagedAttention stores the KV cache in fixed-size pages rather than a single contiguous memory block, allowing it to be distributed across GPU memory without fragmentation. It uses a lookup table to access pages on demand, reducing memory waste from over-allocation and enabling more requests to share GPU memory. It is the memory management technique that made vLLM's high throughput possible. \ No newline at end of file +PagedAttention stores the KV cache in fixed-size pages rather than a single contiguous memory block, allowing it to be distributed across GPU memory without fragmentation. It uses a lookup table to access pages on demand, reducing memory waste from over-allocation and enabling more requests to share GPU memory. It is the memory management technique that made vLLM's high throughput possible. + +Visit the following resources to learn more: + +- [@article@Efficient Memory Management for Large Language Model Serving with PagedAttention](https://arxiv.org/abs/2309.06180) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/pipeline-parallelism@9KLS3gWArBofLEULu3sQc.md b/roadmaps/inference-engineering/content/pipeline-parallelism@9KLS3gWArBofLEULu3sQc.md index d2534a0d2..7ca6319aa 100644 --- a/roadmaps/inference-engineering/content/pipeline-parallelism@9KLS3gWArBofLEULu3sQc.md +++ b/roadmaps/inference-engineering/content/pipeline-parallelism@9KLS3gWArBofLEULu3sQc.md @@ -1,3 +1,8 @@ # Pipeline Parallelism -Pipeline Parallelism splits an LLM's layers into sequential stages and assigns each stage to a different GPU or node. Unlike Tensor Parallelism, each GPU holds complete layers rather than layer fragments. It introduces pipeline bubbles where GPUs wait for preceding stages, reducing efficiency. It is used for multi-node inference with dense models, typically combined with Tensor Parallelism within each node. \ No newline at end of file +Pipeline Parallelism splits an LLM's layers into sequential stages and assigns each stage to a different GPU or node. Unlike Tensor Parallelism, each GPU holds complete layers rather than layer fragments. It introduces pipeline bubbles where GPUs wait for preceding stages, reducing efficiency. It is used for multi-node inference with dense models, typically combined with Tensor Parallelism within each node. + +Visit the following resources to learn more: + +- [@opensource@NVIDIA/Megatron-LM](https://github.com/NVIDIA/Megatron-LM) +- [@article@Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://arxiv.org/abs/1909.08053) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/post-training-quantization@Zvvs8cB98IRee1okBILsS.md b/roadmaps/inference-engineering/content/post-training-quantization@Zvvs8cB98IRee1okBILsS.md index 639431c98..682c6c536 100644 --- a/roadmaps/inference-engineering/content/post-training-quantization@Zvvs8cB98IRee1okBILsS.md +++ b/roadmaps/inference-engineering/content/post-training-quantization@Zvvs8cB98IRee1okBILsS.md @@ -1,3 +1,12 @@ -Post-Training Quantization +# Post-Training Quantization -Post-training quantization converts finished model weights to lower precision after training is complete, using a small calibration dataset to compute per-layer scale factors that minimize the gap between quantized and original outputs. It is the standard approach for most deployments since it requires no retraining and can be applied to any existing model. The leading tool is NVIDIA ModelOpt, which exports quantized models in formats compatible with all major inference engines. \ No newline at end of file +Post-training quantization converts finished model weights to lower precision after training is complete, using a small calibration dataset to compute per-layer scale factors that minimize the gap between quantized and original outputs. It is the standard approach for most deployments since it requires no retraining and can be applied to any existing model. The leading tool is NVIDIA ModelOpt, which exports quantized models in formats compatible with all major inference engines. + +Visit the following resources to learn more: + +- [@opensource@AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https://github.com/mit-han-lab/llm-awq) +- [@opensource@bitsandbytes](https://github.com/TimDettmers/bitsandbytes) +- [@opensource@SmoothQuant](https://github.com/mit-han-lab/smoothquant) +- [@article@GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://arxiv.org/abs/2210.17323) +- [@article@LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://arxiv.org/abs/2208.07339) +- [@article@SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot](https://arxiv.org/abs/2301.00774) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/profiling-performance@S2GZw9etrrvEJzV9V-1C_.md b/roadmaps/inference-engineering/content/profiling-performance@S2GZw9etrrvEJzV9V-1C_.md index 120c93c12..efc6f483c 100644 --- a/roadmaps/inference-engineering/content/profiling-performance@S2GZw9etrrvEJzV9V-1C_.md +++ b/roadmaps/inference-engineering/content/profiling-performance@S2GZw9etrrvEJzV9V-1C_.md @@ -1,3 +1,8 @@ # Profiling Performance -Profiling reveals where time is spent within a benchmark, identifying which operations consume the most compute or memory. Tools include PyTorch Profiler, NVIDIA Nsight Systems for system-wide GPU and CPU analysis, and NVIDIA Nsight Compute for per-kernel analysis. Profiling is most useful when writing custom inference code or contributing to inference frameworks, rather than configuring existing engines. \ No newline at end of file +Profiling reveals where time is spent within a benchmark, identifying which operations consume the most compute or memory. Tools include PyTorch Profiler, NVIDIA Nsight Systems for system-wide GPU and CPU analysis, and NVIDIA Nsight Compute for per-kernel analysis. Profiling is most useful when writing custom inference code or contributing to inference frameworks, rather than configuring existing engines. + +Visit the following resources to learn more: + +- [@official@PyTorch Profiler](https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html) +- [@official@NVIDIA Nsight Systems](https://developer.nvidia.com/nsight-systems) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/pytorch@AHVfxQRXmRoiTxC1eeMme.md b/roadmaps/inference-engineering/content/pytorch@AHVfxQRXmRoiTxC1eeMme.md index 753411aeb..0bbb52b09 100644 --- a/roadmaps/inference-engineering/content/pytorch@AHVfxQRXmRoiTxC1eeMme.md +++ b/roadmaps/inference-engineering/content/pytorch@AHVfxQRXmRoiTxC1eeMme.md @@ -1,3 +1,7 @@ # PyTorch -PyTorch is the standard Python framework for building and running neural networks. It abstracts CUDA into a tensor operation API and provides automatic differentiation for training. For inference, PyTorch supports torch.compile for automatic kernel optimization and serves as the foundation that all major inference engines build on top of. \ No newline at end of file +PyTorch is the standard Python framework for building and running neural networks. It abstracts CUDA into a tensor operation API and provides automatic differentiation for training. For inference, PyTorch supports torch.compile for automatic kernel optimization and serves as the foundation that all major inference engines build on top of. + +Visit the following resources to learn more: + +- [@official@PyTorch Performance Tuning Guide](https://pytorch.org/tutorials/recipes/recipes/tuning_guide.html) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/quantization@DaSqcLHfnJ4_n9utVVTSk.md b/roadmaps/inference-engineering/content/quantization@DaSqcLHfnJ4_n9utVVTSk.md index 62e4bf0e9..264447515 100644 --- a/roadmaps/inference-engineering/content/quantization@DaSqcLHfnJ4_n9utVVTSk.md +++ b/roadmaps/inference-engineering/content/quantization@DaSqcLHfnJ4_n9utVVTSk.md @@ -1,3 +1,12 @@ # Quantization -Quantization converts model weights and other values from their native precision to a lower-precision format. Cutting precision in half doubles memory bandwidth for decode and enables twice the FLOPS for prefill on compatible Tensor Cores. The tradeoff is potential quality loss from reduced numerical precision, which can compound across the many calculations in a forward pass. \ No newline at end of file +Quantization converts model weights and other values from their native precision to a lower-precision format. Cutting precision in half doubles memory bandwidth for decode and enables twice the FLOPS for prefill on compatible Tensor Cores. The tradeoff is potential quality loss from reduced numerical precision, which can compound across the many calculations in a forward pass. + +Visit the following resources to learn more: + +- [@opensource@AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https://github.com/mit-han-lab/llm-awq) +- [@opensource@bitsandbytes](https://github.com/TimDettmers/bitsandbytes) +- [@opensource@SmoothQuant](https://github.com/mit-han-lab/smoothquant) +- [@article@GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://arxiv.org/abs/2210.17323) +- [@article@LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://arxiv.org/abs/2208.07339) +- [@article@SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot](https://arxiv.org/abs/2301.00774) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/routing--load-balancing@DSyTAtFC1TvC1P7yD10Nb.md b/roadmaps/inference-engineering/content/routing--load-balancing@DSyTAtFC1TvC1P7yD10Nb.md index 36bb07050..9acda65b9 100644 --- a/roadmaps/inference-engineering/content/routing--load-balancing@DSyTAtFC1TvC1P7yD10Nb.md +++ b/roadmaps/inference-engineering/content/routing--load-balancing@DSyTAtFC1TvC1P7yD10Nb.md @@ -1,3 +1,9 @@ # Routing and Load Balancing -Routers make request-level decisions about which replica to send a request to, considering factors like KV cache prefix match and LoRA availability. Load balancers make system-level decisions to distribute traffic evenly across replicas. In systems with prefix caching, naive round-robin load balancing misses cache opportunities; intelligent routing increases cache hit rates and reduces TTFT. \ No newline at end of file +Routers make request-level decisions about which replica to send a request to, considering factors like KV cache prefix match and LoRA availability. Load balancers make system-level decisions to distribute traffic evenly across replicas. In systems with prefix caching, naive round-robin load balancing misses cache opportunities; intelligent routing increases cache hit rates and reduces TTFT. + +Visit the following resources to learn more: + +- [@article@The Llama 3 Herd of Models](https://arxiv.org/abs/2407.21783) +- [@book@Designing Data-Intensive Applications](https://dataintensive.net/) +- [@book@Site Reliability Engineering](https://sre.google/books/) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/sglang@YtlAf6qloKMmaSr9Bkkl_.md b/roadmaps/inference-engineering/content/sglang@YtlAf6qloKMmaSr9Bkkl_.md index 9b16777c7..2bb16ac62 100644 --- a/roadmaps/inference-engineering/content/sglang@YtlAf6qloKMmaSr9Bkkl_.md +++ b/roadmaps/inference-engineering/content/sglang@YtlAf6qloKMmaSr9Bkkl_.md @@ -1,3 +1,7 @@ # SGLang -SGLang is a high-throughput inference engine with a flexible frontend language for customizing server behavior. It has strong support for MoE models like DeepSeek and Qwen and works closely with model labs to optimize new architectural features. SGLang Diffusion extends it to image and video generation models with pipeline abstractions and diffusion-specific parallelism. \ No newline at end of file +SGLang is a high-throughput inference engine with a flexible frontend language for customizing server behavior. It has strong support for MoE models like DeepSeek and Qwen and works closely with model labs to optimize new architectural features. SGLang Diffusion extends it to image and video generation models with pipeline abstractions and diffusion-specific parallelism. + +Visit the following resources to learn more: + +- [@opensource@sgl-project/sglang](https://github.com/sgl-project/sglang) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/sizing--procurement@da0HGfSDr6_DV4ZJcrcIB.md b/roadmaps/inference-engineering/content/sizing--procurement@da0HGfSDr6_DV4ZJcrcIB.md index 96efcfae6..74e7213cd 100644 --- a/roadmaps/inference-engineering/content/sizing--procurement@da0HGfSDr6_DV4ZJcrcIB.md +++ b/roadmaps/inference-engineering/content/sizing--procurement@da0HGfSDr6_DV4ZJcrcIB.md @@ -1,3 +1,9 @@ # Sizing and Procurement -Estimating GPU requirements starts with VRAM: roughly 1 GB per billion parameters at FP8 for weights, plus at least 50 percent headroom for the KV cache. This total is rounded up to the nearest available instance size. Cloud GPUs are available from hyperscalers like AWS and GCP, neoclouds like CoreWeave and Nebius, and spot markets; large deployments combine reserved instances for baseline load with on-demand and spot capacity for bursts. \ No newline at end of file +Estimating GPU requirements starts with VRAM: roughly 1 GB per billion parameters at FP8 for weights, plus at least 50 percent headroom for the KV cache. This total is rounded up to the nearest available instance size. Cloud GPUs are available from hyperscalers like AWS and GCP, neoclouds like CoreWeave and Nebius, and spot markets; large deployments combine reserved instances for baseline load with on-demand and spot capacity for bursts. + +Visit the following resources to learn more: + +- [@article@SemiAnalysis](https://semianalysis.com/) +- [@book@Designing Data-Intensive Applications](https://dataintensive.net/) +- [@book@Site Reliability Engineering](https://sre.google/books/) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/speculative-decoding@gMWgd10cxcQG_RG2oSAEH.md b/roadmaps/inference-engineering/content/speculative-decoding@gMWgd10cxcQG_RG2oSAEH.md index dd768cce6..0e185c989 100644 --- a/roadmaps/inference-engineering/content/speculative-decoding@gMWgd10cxcQG_RG2oSAEH.md +++ b/roadmaps/inference-engineering/content/speculative-decoding@gMWgd10cxcQG_RG2oSAEH.md @@ -1,3 +1,8 @@ # Speculative Decoding -Speculative decoding takes advantage of spare compute during memory-bound decode by generating multiple draft tokens and validating them in a single forward pass. If the target model accepts N draft tokens, the forward pass produces N+1 tokens instead of one. The performance gain depends on draft token cost, draft sequence length, and acceptance rate, which decreases at higher temperatures and larger batch sizes. \ No newline at end of file +Speculative decoding takes advantage of spare compute during memory-bound decode by generating multiple draft tokens and validating them in a single forward pass. If the target model accepts N draft tokens, the forward pass produces N+1 tokens instead of one. The performance gain depends on draft token cost, draft sequence length, and acceptance rate, which decreases at higher temperatures and larger batch sizes. + +Visit the following resources to learn more: + +- [@article@Fast Inference from Transformers via Speculative Decoding](https://arxiv.org/abs/2211.17192) +- [@article@Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads](https://arxiv.org/abs/2401.10774) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/tensor-parallelism@csghcNLDywO8oTIZNXeyP.md b/roadmaps/inference-engineering/content/tensor-parallelism@csghcNLDywO8oTIZNXeyP.md index 07d8e654e..01e480138 100644 --- a/roadmaps/inference-engineering/content/tensor-parallelism@csghcNLDywO8oTIZNXeyP.md +++ b/roadmaps/inference-engineering/content/tensor-parallelism@csghcNLDywO8oTIZNXeyP.md @@ -1,3 +1,8 @@ # Tensor Parallelism -Tensor Parallelism splits each layer's weight matrices across multiple GPUs and distributes the matmul computation. Each GPU handles a shard of each layer and the results are combined via an all-reduce operation before the next layer. It reduces per-user latency by sharing both VRAM and compute across GPUs, but requires fast NVLink interconnects and is not well-suited for multi-node deployments. \ No newline at end of file +Tensor Parallelism splits each layer's weight matrices across multiple GPUs and distributes the matmul computation. Each GPU handles a shard of each layer and the results are combined via an all-reduce operation before the next layer. It reduces per-user latency by sharing both VRAM and compute across GPUs, but requires fast NVLink interconnects and is not well-suited for multi-node deployments. + +Visit the following resources to learn more: + +- [@opensource@NVIDIA/Megatron-LM](https://github.com/NVIDIA/Megatron-LM) +- [@article@Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://arxiv.org/abs/1909.08053) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/tensorrt-llm@n2_zIGkQgyYCMYPypz3vb.md b/roadmaps/inference-engineering/content/tensorrt-llm@n2_zIGkQgyYCMYPypz3vb.md index 9d418284c..88c0bba93 100644 --- a/roadmaps/inference-engineering/content/tensorrt-llm@n2_zIGkQgyYCMYPypz3vb.md +++ b/roadmaps/inference-engineering/content/tensorrt-llm@n2_zIGkQgyYCMYPypz3vb.md @@ -1,3 +1,7 @@ # TensorRT-LLM -TensorRT-LLM is NVIDIA's inference engine combining PyTorch-based model execution with proprietary NVIDIA kernels unavailable in other engines. It achieves the highest performance among the three major engines, with excellent support for Hopper and Blackwell architectures and NVIDIA-specific formats like NVFP4. The tradeoff is a steeper learning curve and narrower model support compared to vLLM or SGLang. \ No newline at end of file +TensorRT-LLM is NVIDIA's inference engine combining PyTorch-based model execution with proprietary NVIDIA kernels unavailable in other engines. It achieves the highest performance among the three major engines, with excellent support for Hopper and Blackwell architectures and NVIDIA-specific formats like NVFP4. The tradeoff is a steeper learning curve and narrower model support compared to vLLM or SGLang. + +Visit the following resources to learn more: + +- [@opensource@NVIDIA/TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/tensorrt@M4qjJSEAnsjrmP3A_O8HC.md b/roadmaps/inference-engineering/content/tensorrt@M4qjJSEAnsjrmP3A_O8HC.md index 9d0e866ea..f8dcc59d4 100644 --- a/roadmaps/inference-engineering/content/tensorrt@M4qjJSEAnsjrmP3A_O8HC.md +++ b/roadmaps/inference-engineering/content/tensorrt@M4qjJSEAnsjrmP3A_O8HC.md @@ -1,3 +1,7 @@ # TensorRT -TensorRT is NVIDIA's high-performance inference runtime that compiles PyTorch or ONNX models into optimized engines using fused kernels, quantization, and hardware-specific tuning. It is the standard choice for image and video generation models and for any workload where maximum throughput on NVIDIA hardware is the primary goal. Compiled engines are cached to disk, eliminating recompilation overhead on subsequent cold starts. \ No newline at end of file +TensorRT is NVIDIA's high-performance inference runtime that compiles PyTorch or ONNX models into optimized engines using fused kernels, quantization, and hardware-specific tuning. It is the standard choice for image and video generation models and for any workload where maximum throughput on NVIDIA hardware is the primary goal. Compiled engines are cached to disk, eliminating recompilation overhead on subsequent cold starts. + +Visit the following resources to learn more: + +- [@official@TensorRT Documentation](https://docs.nvidia.com/deeplearning/tensorrt/) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/transformer-blocks@xHM7qjigMmXEwxeMEeMWl.md b/roadmaps/inference-engineering/content/transformer-blocks@xHM7qjigMmXEwxeMEeMWl.md index 98dc09529..097af156b 100644 --- a/roadmaps/inference-engineering/content/transformer-blocks@xHM7qjigMmXEwxeMEeMWl.md +++ b/roadmaps/inference-engineering/content/transformer-blocks@xHM7qjigMmXEwxeMEeMWl.md @@ -1,3 +1,7 @@ # Transformer Blocks -Transformer blocks are the repeated hidden layers of an LLM. Each block contains an attention sublayer, a feed-forward neural network sublayer, and normalization layers. The feed-forward network holds the majority of model parameters, while attention is the most computationally complex operation per token. \ No newline at end of file +Transformer blocks are the repeated hidden layers of an LLM. Each block contains an attention sublayer, a feed-forward neural network sublayer, and normalization layers. The feed-forward network holds the majority of model parameters, while attention is the most computationally complex operation per token. + +Visit the following resources to learn more: + +- [@article@Attention Is All You Need](https://arxiv.org/abs/1706.03762) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/transformers--diffusers@H_e3KZ7p7enUCYE5M5Lr7.md b/roadmaps/inference-engineering/content/transformers--diffusers@H_e3KZ7p7enUCYE5M5Lr7.md index cde7c03ab..56e1f2bb5 100644 --- a/roadmaps/inference-engineering/content/transformers--diffusers@H_e3KZ7p7enUCYE5M5Lr7.md +++ b/roadmaps/inference-engineering/content/transformers--diffusers@H_e3KZ7p7enUCYE5M5Lr7.md @@ -1,3 +1,8 @@ # Hugging Face Transformers and Diffusers -The transformers and diffusers libraries provide reference implementations of popular LLM and image/video generation architectures. They are not designed for production serving but are essential for understanding model configurations, downloading weights, and prototyping inference. Model cards on Hugging Face use transformers or diffusers sample code as the canonical usage examples. \ No newline at end of file +The transformers and diffusers libraries provide reference implementations of popular LLM and image/video generation architectures. They are not designed for production serving but are essential for understanding model configurations, downloading weights, and prototyping inference. Model cards on Hugging Face use transformers or diffusers sample code as the canonical usage examples. + +Visit the following resources to learn more: + +- [@official@Hugging Face Diffusers](https://huggingface.co/docs/diffusers/index) +- [@official@Hugging Face Transformers](https://huggingface.co/docs/transformers/index) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/video-generation-inference@PqEH1ez5dg5tEsspgfAK1.md b/roadmaps/inference-engineering/content/video-generation-inference@PqEH1ez5dg5tEsspgfAK1.md index d0427fbd3..e6b6bda00 100644 --- a/roadmaps/inference-engineering/content/video-generation-inference@PqEH1ez5dg5tEsspgfAK1.md +++ b/roadmaps/inference-engineering/content/video-generation-inference@PqEH1ez5dg5tEsspgfAK1.md @@ -1,3 +1,9 @@ # Video Generation Inference -Video generation models run on full nodes of eight GPUs with a batch size of one because the latent space is too large to batch effectively. Attention over the three-dimensional video latent space consumes 70 to 80 percent of compute time and is the primary optimization target. Attention caching techniques that reuse outputs from previous timesteps or transformer layers can improve speed by 30 to 40 percent. \ No newline at end of file +Video generation models run on full nodes of eight GPUs with a batch size of one because the latent space is too large to batch effectively. Attention over the three-dimensional video latent space consumes 70 to 80 percent of compute time and is the primary optimization target. Attention caching techniques that reuse outputs from previous timesteps or transformer layers can improve speed by 30 to 40 percent. + +Visit the following resources to learn more: + +- [@opensource@Cache-DiT](https://github.com/vipshop/cache-dit) +- [@opensource@TeaCache: Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model](https://github.com/ali-vilab/TeaCache) +- [@article@SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration](https://arxiv.org/abs/2410.02367) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/video-generation-models@yTSrQfJsE-Mf4ZU_hJwYT.md b/roadmaps/inference-engineering/content/video-generation-models@yTSrQfJsE-Mf4ZU_hJwYT.md index 6698c248d..bb589aa69 100644 --- a/roadmaps/inference-engineering/content/video-generation-models@yTSrQfJsE-Mf4ZU_hJwYT.md +++ b/roadmaps/inference-engineering/content/video-generation-models@yTSrQfJsE-Mf4ZU_hJwYT.md @@ -1,3 +1,8 @@ # Video Generation Models -Video generation models extend image generation to three-dimensional latent space, representing width, height, and time simultaneously. They run the same number of denoising steps as image models but process far more data per step. Attention over the full video latent space dominates compute time, making video generation the most resource-intensive common inference workload. \ No newline at end of file +Video generation models extend image generation to three-dimensional latent space, representing width, height, and time simultaneously. They run the same number of denoising steps as image models but process far more data per step. Attention over the full video latent space dominates compute time, making video generation the most resource-intensive common inference workload. + +Visit the following resources to learn more: + +- [@article@Video Diffusion Models](https://arxiv.org/abs/2204.03458) +- [@article@Imagen Video: High Definition Video Generation with Diffusion Models](https://arxiv.org/abs/2210.02303) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/vision-language-models@JsjmQrN1V_dM1Kw6UnYLc.md b/roadmaps/inference-engineering/content/vision-language-models@JsjmQrN1V_dM1Kw6UnYLc.md index 704121da9..e0c34469a 100644 --- a/roadmaps/inference-engineering/content/vision-language-models@JsjmQrN1V_dM1Kw6UnYLc.md +++ b/roadmaps/inference-engineering/content/vision-language-models@JsjmQrN1V_dM1Kw6UnYLc.md @@ -1,3 +1,10 @@ # Vision Language Models (VLMs) -VLMs combine a large language model with a small vision encoder that converts input images or video into a sequence of visual tokens. A high-resolution image typically adds about 1,000 tokens to the input sequence. All LLM inference optimization techniques apply to VLMs, but longer sequences make quantization, prefix caching, and disaggregation especially important. \ No newline at end of file +VLMs combine a large language model with a small vision encoder that converts input images or video into a sequence of visual tokens. A high-resolution image typically adds about 1,000 tokens to the input sequence. All LLM inference optimization techniques apply to VLMs, but longer sequences make quantization, prefix caching, and disaggregation especially important. + +Visit the following resources to learn more: + +- [@article@BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://arxiv.org/abs/2301.12597) +- [@article@Visual Instruction Tuning](https://arxiv.org/abs/2304.08485) +- [@article@Segment Anything](https://arxiv.org/abs/2304.02643) +- [@article@SpecVLM: Fast Speculative Decoding in Vision-Language Models](https://arxiv.org/abs/2509.11815) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/vllm@wcAtbab81IJdpFh6L-heP.md b/roadmaps/inference-engineering/content/vllm@wcAtbab81IJdpFh6L-heP.md index dd2d4680c..934a0d996 100644 --- a/roadmaps/inference-engineering/content/vllm@wcAtbab81IJdpFh6L-heP.md +++ b/roadmaps/inference-engineering/content/vllm@wcAtbab81IJdpFh6L-heP.md @@ -1,3 +1,7 @@ # vLLM -vLLM is the most widely adopted LLM inference engine, offering broad support for model architectures, hardware platforms (NVIDIA, AMD, Google TPUs), and optimization techniques including prefix caching, quantization, speculative decoding, and parallelism. Its primary advantage is ease of adoption and the fastest day-zero support for newly released open models. It also supports multimodal inference via vLLM Omni. \ No newline at end of file +vLLM is the most widely adopted LLM inference engine, offering broad support for model architectures, hardware platforms (NVIDIA, AMD, Google TPUs), and optimization techniques including prefix caching, quantization, speculative decoding, and parallelism. Its primary advantage is ease of adoption and the fastest day-zero support for newly released open models. It also supports multimodal inference via vLLM Omni. + +Visit the following resources to learn more: + +- [@opensource@vllm-project/vllm](https://github.com/vllm-project/vllm) \ No newline at end of file diff --git a/roadmaps/inference-engineering/content/writing-custom-kernels@1dWPVaSQ4TfVU3Obx3m19.md b/roadmaps/inference-engineering/content/writing-custom-kernels@1dWPVaSQ4TfVU3Obx3m19.md index 2901724f5..1a3694d14 100644 --- a/roadmaps/inference-engineering/content/writing-custom-kernels@1dWPVaSQ4TfVU3Obx3m19.md +++ b/roadmaps/inference-engineering/content/writing-custom-kernels@1dWPVaSQ4TfVU3Obx3m19.md @@ -1,3 +1,12 @@ # Writing Custom Kernels -Custom kernels are written when existing libraries do not fully utilize a GPU's capabilities for a specific operation or architecture. Most production kernels are written in C++ using CUTLASS and CuTe for GEMM-style operations and FlashInfer for attention variants; Triton offers a Python-based alternative with a lower barrier to entry. Writing an effective kernel requires understanding the target GPU's memory hierarchy, SM occupancy, and Tensor Core instruction set — skills that sit at the intersection of software and hardware. \ No newline at end of file +Custom kernels are written when existing libraries do not fully utilize a GPU's capabilities for a specific operation or architecture. Most production kernels are written in C++ using CUTLASS and CuTe for GEMM-style operations and FlashInfer for attention variants; Triton offers a Python-based alternative with a lower barrier to entry. Writing an effective kernel requires understanding the target GPU's memory hierarchy, SM occupancy, and Tensor Core instruction set — skills that sit at the intersection of software and hardware. + +Visit the following resources to learn more: + +- [@official@CUDA Programming Guide](https://docs.nvidia.com/cuda/cuda-programming-guide/) +- [@official@cuBLAS Documentation](https://docs.nvidia.com/cuda/cublas/) +- [@opensource@NVIDIA/cutlass](https://github.com/NVIDIA/cutlass) +- [@opensource@deepseek-ai/DeepGEMM](https://github.com/deepseek-ai/DeepGEMM) +- [@article@FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving](https://arxiv.org/abs/2501.01005) +- [@book@CUDA by Example: An Introduction to General-Purpose GPU Programming](https://developer.nvidia.com/cuda-example) \ No newline at end of file