opensource/developer-roadmap
3
0
Fork 0
mirror of https://github.com/kamranahmedse/developer-roadmap.git synced 2026-10-24 11:35:45 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
d7e10906113f703eadbc2f8d2889576f93050edf
developer-roadmap/roadmaps/inference-engineering/content
T
History
github-actions[bot]andnilbuild d7e1090611 chore: sync content to repo (#10398) 
Co-authored-by: nilbuild <4921183+nilbuild@users.noreply.github.com>
2026-09-30 17:28:38 +02:00
..
ai-native-applications@ZqFQmNizkU6ykTMCrmTWf.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
arithmetic-intensity@bPO6uwod30-ewwbo8i1eU.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
asynchronous-inference@59mMKSva9vkoMxHErNXOd.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
asynchronous-inference@LAD5cttwzdbMuWIv3XjG1.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
attention-mechanism@2KY3T5JVHQBR8pmAiuq0V.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
attention-optimization@KWM3V7_15_Aur0He9j2Va.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
attention-variants@6oTnNEsX_6Cu6lowEz9vg.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
auto-speech-recognition@6PRICRK6aRQCoThxnevB0.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
autoscaling@o_gotSHKIPtV5SgGN3Vja.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
benchmarking--profiling@TxK7B_FF38ms3eKPGug2w.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
bert-style-models@ef3EneMibOaDQkKcpipGY.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
blackwell@LihLABERuKBw30G_j48Fj.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
bottleneck-analysis@Gtd4g4MtB9zFfahRpbE2K.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
cache-aware-routing@3DR0Qvqu2qUTi9niU_63n.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
caching-approaches@2nGv7_8PTWYr7uqbudCcX.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
caching@PzEj5yq1lDQ324RWYpDdQ.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
capacity-management@xrr5Y1sbN2eyQyuA5LlAW.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
client-integration@3RkQpvRGWh1Hp_qHlTiXX.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
client-latency-overhead@gOWGIllIIxhic1NNRfugq.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
cold-starts@Q3zM9MN5lmHRAUWjdLdXR.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
compute@o_tknzA4tqcpbU6t5HO-T.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
consumer-vs-b2b@SFMvVdgTZv_POWFGWWM_s.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
containerization@--ohScNmF5LR04P8e95AP.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
continuous-batching@h_PDOMA-V-n7Pab3epuP3.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
cost-estimation@Gof4kW2LbkNCtH68c7Xvq.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
cuda@DwJJLFLPHGn9zdccTlLY2.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
dependency-management@wEaM8VyAo6MFsc3RXQ7Ar.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
desktop-inference@fQDl7_3nRGa8NFivOIgTP.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
diffusion-models@aagMwPPmi6ngfkIqZL-Mg.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
disaggregation@f6PWIqf4LubR7lQztaoB5.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
distillation@IdNkeL9pzy4lFv17ak1Vy.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
draft-target-decoding@m0EIHovHWGkWULN0a3C3v.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
eagle@0FZTg3_ZnTpMOQ078OOxE.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
embedding-model-inference@GmCkfSeVDJ8T8WLZ7DoOp.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
embedding-models@Tvt-vX1gWuJZJBHh5vfKu.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
encoder-decoder-arch@nORbxBOAtl9sNU4sIJgP3.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
end-to-end-metrics@tENGACXKSKVPmZy9a3D2z.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
expert-parallelism@62y78y-RXZT_QCUfd-YFE.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
few-step-image-generation@biVjpCA3bsKI7sa_aBIUE.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
fine-tuning@fWJplrAuUStEMNNLQc75G.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
flashattention@mQcj-H3R789o6nDh8tG9k.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
frameworks--libraries@D4uNOZgG0w-nFL3a89pDM.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
geo-aware-load-balancing@mzF2mHSj8cvr9mvr16R0N.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
gpu-architecture@00JLMvn8DUJBCw6-BxO0y.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
gpu-models@umz3jScWQViO9-lACUU4f.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
gpu-procurement@4mbGBgTu8AtrFHVYiffkB.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
grace--vera@kPT86kA0WwE_fWUAAKCTT.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
hopper@XQMbaW1tFdO8Os6mxcR2M.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
image--video-bottlenecks@0lRP9cmoS-_wxQSsIcRQg.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
image--video-generation@U0l_xCUiizbJfjfrlBaJ0.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
image--video-models@Mebx7HS-I-eNP5i3f6Clf.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
image-generation-arch@xsV4WevPzhfydyvxVzGcw.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
image-generation-inference@lnIHjq4nvCs3zvUln-jHR.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
indep-component-scaling@SKs0yuBEAyeTPQxLll3EM.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
inference-engines@01j1KkEB-nIXxP-rENcJ3.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
inference-vs-training@fVUfQYb8BP_hFMmd1S2-J.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
instances--interconnects@dAo4PMB_VnRFMSjyDrLj0.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
inter-token-latency@sre_s0dvpOP0DTM66a3hu.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
introduction@GL-9aJKTJT9s4maeT37Uf.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
kernel-selection--fusion@iKW1wqzd5EfRKvNIRshNN.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
kv-cache-storage@gkt2Gr34RJJf1NZgbmtXf.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
kv-cache@V678kX8FPChTZCe7Nqwhv.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
large-language-models@irgGvEnY1_WW1EPtuMKgC.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
latency-percentiles@8y280d5BPXrDZwtsHZCaX.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
latency-vs-throughput@kvNIi3aY5rn69_hYkDyUK.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
llm-based-models@M7K6d-Qw4BpM724LfM1u6.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
llm-bottlenecks@QVadKzBfLdegat-4wgOMX.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
local--edge-inference@FilQGz3geyyvHZGyKPW3G.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
locust@IQuJFfMP1dW0ELlip6LOd.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
long-context-handling@uL3Fn8XfS1Yv_PJd9R8wi.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
lora-aware-routing@KdceJEDBdy3TbzG5suE38.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
measuring-quality-impact@mK_4R8liXtl64iG7ONvgz.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
memory@5829d5dvXq9qlMSD1CG0N.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
mixture-of-experts-moe@7BddTfOkWmlL4HAkFBXtq.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
mobile-inference@kqzHHoiZ9y4tCS6MMlLHQ.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
model-evaluation@lVf5GoJF_PVTkTiQFTNVg.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
model-formats--runtimes@lTjWXB8YRwx11e8lxSgNm.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
model-parallelism@CiXsdHHL0e2DN4ICI5jeX.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
model-selection@jQAWZ1plw4_wPE1UvE_Ux.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
modelopt@zhiKpdZm8zOg4e3bm1OgN.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
multi-cloud@d78UiI_FZyLsU6O52btXg.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
multi-node-inference@FZx2xsqw_cmYLFS3GMAl-.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
n-gram@ZOclQMd7JASGXTRyctfW2.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
neural-networks@p2S0W3EDn768jbqzc7CE7.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
nims@KB7ExU9X49omOuk9D_Qal.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
nvidia-dynamo@ctdMz6rHa4Cwl1x19V_5k.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
nvidia-dynamo@KhTFecXEkHR4WX-_TYY3g.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
nvidia-genai-perf@zffK61pNUJDwkX7Ukql8y.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
nvidia@Uq3mE-0WIASHfcBDEnbLj.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
observability@w-dVv5dFNxD7Rqy16m4uL.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
omni-modal-models@plw2WiOoGxg64P-Kv-Xm7.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
online-vs-offline-inf@SkvPauxjmqok4TyNWSACL.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
onnx-runtime@Vmtbdre3SZzwq3FxOO-Bc.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
open-vs-closed-models@KAt5FzglOUtH7jLlzDn-I.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
opsbyte-ratio@uc2enkdtRqA5K7kbgrYJF.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
other-accelerators@am4Hh-14g6Awk0WcFmVN7.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
pagedattention@VcIRY4vSi7JkcV6g13vkN.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
performance-benchmarking@XXfjlqd0CYHwBR5-Kdmmi.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
pipeline-parallelism@9KLS3gWArBofLEULu3sQc.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
post-training-quantization@Zvvs8cB98IRee1okBILsS.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
prefill--decode-phases@hwKZmfYFiIu6plbiXtJRI.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
profiling-performance@S2GZw9etrrvEJzV9V-1C_.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
pytorch@AHVfxQRXmRoiTxC1eeMme.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
quantization-aware-training@QCXV4Utq0Ajs1krLM3fhg.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
quantization@DaSqcLHfnJ4_n9utVVTSk.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
real-time-factor@0-BRllrMiShLlytidGkST.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
reasoning-models@Y9YkcfblXHEvS-gdEWwaE.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
request-queueing@k-zXOGynI0B0yyT-n3raa.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
roofline-model@XF-wMQCqPZiWv1XmryPBP.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
routing--load-balancing@DSyTAtFC1TvC1P7yD10Nb.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
safetensors@jkbrf0_JDljWmVq_o9XE-.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
scale-to-zero@Pi87dWgu5NTIJRQUc169s.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
security-and-compliance@4x-4BFZH7bfNteD5ZAwTI.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
sglang-genai-bench@KHDnY7byKgmaPtC_0uDBM.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
sglang@YtlAf6qloKMmaSr9Bkkl_.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
shared-vs-dedicated-inf@2DsjfW9r4dOI3LDQU80qt.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
sizing--procurement@da0HGfSDr6_DV4ZJcrcIB.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
speculative-decoding@gMWgd10cxcQG_RG2oSAEH.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
speech-to-speech-models@moyS740HQoUUovilx1Yyt.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
speech@Lk8906Mpi3yxiSt8EC_BC.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
streaming--protocol-support@SGQCvuHAkg4TcJa_e2f-y.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
streaming-tts@3VbPNaSnzWZ-pC-lGFMrh.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
tensor-parallelism@csghcNLDywO8oTIZNXeyP.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
tensorrt-llm@n2_zIGkQgyYCMYPypz3vb.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
tensorrt@M4qjJSEAnsjrmP3A_O8HC.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
testing--deployment@ZlOtQtxJST10nuHjhbUx6.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
text-to-speech-tts@XnhhN1QVWzlp9DUwkii9-.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
the-inference-stack@uIhVVlMsykuxJCa02f-i-.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
time-to-first-token@xCgNhwbS_ELxEyRHD1enO.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
tokens-per-second@uxm_BIAyhfFmcE94k9i9J.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
transformer-blocks@xHM7qjigMmXEwxeMEeMWl.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
transformers--diffusers@H_e3KZ7p7enUCYE5M5Lr7.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
video-generation-inference@PqEH1ez5dg5tEsspgfAK1.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
video-generation-models@yTSrQfJsE-Mf4ZU_hJwYT.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
video-processing-for-vlms@7pRZu7xa2_43HVwKty9px.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
vision-language-models@JsjmQrN1V_dM1Kw6UnYLc.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
vllm@wcAtbab81IJdpFh6L-heP.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
voice-activity-detection@P0BA_p3xqpxkQsFHqM0AZ.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
writing-custom-kernels@1dWPVaSQ4TfVU3Obx3m19.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00
zero-downtime-deplo@le4sPQ0h9l-vlmCB4G8ZY.md
chore: sync content to repo (#10398)
2026-09-30 17:28:38 +02:00