mirror of
https://github.com/kamranahmedse/developer-roadmap.git
synced 2026-10-28 02:43:08 +08:00
1013 B
1013 B
Vision Language Models (VLMs)
VLMs combine a large language model with a small vision encoder that converts input images or video into a sequence of visual tokens. A high-resolution image typically adds about 1,000 tokens to the input sequence. All LLM inference optimization techniques apply to VLMs, but longer sequences make quantization, prefix caching, and disaggregation especially important.
Visit the following resources to learn more:
- @article@BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
- @article@Visual Instruction Tuning
- @article@Segment Anything
- @article@SpecVLM: Fast Speculative Decoding in Vision-Language Models
- @course@A Multimodal World - Hugging Face Computer Vision Course