Architecture timelines · vision-language

How we got to Qwen3‑VL.

Eleven years from attention over pixels to a natively multimodal frontier model. In vision-language the paper and the weights almost always land together. Every stop links to a plain-English explanation or a live interactive graph.

8 stops 6 papers 7 model graphs 2014 to 2025
Concept Model Paper and model in one release
Seeing becomes sequencing2014 to 2020

Attention learns to look across modalities, and images become token sequences.

ConceptSep 2014Cross-attentionQueries from one sequence, keys and values from another. The first wiring that let language models look at something other than text.other sequenceNeural Machine Translation by Jointly Learning to Align and TranslateBahdanau et al. · Sep 2014Read the glossary entry →
Paper + modelOct 2020google/vit-base-patch16-224An image becomes a sequence of patch tokens and the transformer takes over vision. The checkpoints ship with the paper.google/vit-base-patch16-224 architecture graphimage-classification · ↓ 4.7M · ♡ 991An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleDosovitskiy et al. · Oct 2020Patch embeddingOpen in visualizer →Read glossary entry →
Pixels meet words2021 to 2023

Contrastive encoders align images with text, then a small projector glues them to a language model.

Paper + modelJan 2021openai/clip-vit-base-patch32One embedding space for images and text. Its vision tower becomes the eyes of a whole generation of VLMs, and it is still the most downloaded vision model on the Hub.openai/clip-vit-base-patch32 architecture graphzero-shot-image-classification · ↓ 20.8M · ♡ 1kLearning Transferable Visual Models From Natural Language SupervisionRadford et al. · Mar 2021Open in visualizer →
Paper + modelMar 2023google/siglip-so400m-patch14-384Swap CLIP’s softmax for a sigmoid loss and the contrastive encoder trains better at scale. The so400m tower becomes the default eyes of open VLMs.google/siglip-so400m-patch14-384 architecture graphzero-shot-image-classification · ↓ 2.2M · ♡ 682Sigmoid Loss for Language Image Pre-TrainingZhai et al. · Mar 2023Open in visualizer →
Paper + modelApr 2023llava-hf/llava-1.5-7b-hfGlue a CLIP tower to an open LLM with a small projector and instruction-tune on images. The 1.5 checkpoint is the one everyone runs.llava-hf/llava-1.5-7b-hf architecture graphMLPimage-text-to-text · ↓ 2.9M · ♡ 370Visual Instruction TuningLiu et al. · Apr 2023Multimodal projectorOpen in visualizer →Read glossary entry →
Native multimodality2024 to 2025

Vision stops being a bolt-on: resolution goes dynamic and the tower trains with the language model.

Paper + modelSep 2024Qwen/Qwen2-VL-7B-InstructNative dynamic resolution and multimodal RoPE, with a patch merger shortening what the language model reads. Vision stops being a bolt-on.Qwen/Qwen2-VL-7B-Instruct architecture graphimage-text-to-text · ↓ 1.5M · ♡ 1kQwen2-VL: Enhancing Vision-Language Model’s Perception of the World at Any ResolutionWang et al. · Sep 2024Patch mergerOpen in visualizer →Read glossary entry →
ModelJan 2025Qwen/Qwen2.5-VL-7B-InstructThe open VLM workhorse: ten million downloads a month for document parsing, grounding and video understanding.Qwen/Qwen2.5-VL-7B-Instruct architecture graphimage-text-to-text · ↓ 9.1M · ♡ 2kOpen in visualizer →
Oct 2025 · where it all lands

Qwen/Qwen3-VL-8B-Instruct

Every idea above, alive in one graph: a ViT tower reading patches, a merger shortening them, and a language model treating vision as just more tokens. Open it and follow the pixels.

Qwen/Qwen3-VL-8B-Instruct architecture graph
Vision encoderPatch embeddingPatch mergerGQARoPEQK-Norm
Open Qwen3-VL in the visualizer →

Want the definitions behind each stop? The glossary explains all 40 concepts, and the catalog has interactive graphs for 3,500+ models.

AlsoHow we got to DeepSeek-V4-Pro · the LLM timeline →AlsoHow we got to FLUX.2 · the image generation timeline →