The Next-Token Flipbook

What the picture shows

Qwen3.5-4B as a box, seen close up and open: its numbers inside as 32 columns of points, one per layer, the full-attention layers drawn open, a wave of the signal colour passing through them (an impression, a few shown, labelled 4,205,751,296 numbers). The camera pulls back, and the lid comes down, printed like a carton with the model card's real figures: 32 layers, 2,560 wide, a vocabulary of 248,077, 4-bit, net contents 4,205,751,296 parameters. Then the question ‘What is the capital of France?’ goes in from one side, the box runs, and the answer's first tokens come out on the other, one by one: ‘The capital of France is’.

The next-token loop of mlx-community/Qwen3.5-4B-4bit on the text “The capital of France is”, 5 tokens. At each step, the three most likely next tokens: step 1: space, “Paris” 45%, space, “located” 7.8%, space, “known” 5.4%; chose space, “Paris”; step 2: “.” (punctuation or formatting) 79%, “,” (punctuation or formatting) 16%, space, “and” 1.3%; chose “.” (punctuation or formatting); step 3: line break 25%, 2 line breaks 13%, space, “The” 12%; chose line break; step 4: “A” 37%, “The” 18%, “I” 2.0%; chose “A”; step 5: “.” (punctuation or formatting) 89%, space, “city” 2.1%, “:” (punctuation or formatting) 0.6%; chose “.” (punctuation or formatting); step 6: space, “True” 31%, space 20%, space, “False” 4.1%; chose space, “True”. 5 samples at temperature 1, seeds 0 to 4: space, “not”, space, “Paris”, space, “but”, space, “blessed”, space, “with”, space, “neither”, space, “the”, space, “sun”; space, “Paris”, “.” (punctuation or formatting), space, “(” (punctuation or formatting), “T”, “/F”, “)” (punctuation or formatting), 2 line breaks, “M”; space, “the”, space, “__” (punctuation or formatting), “________” (punctuation or formatting), “base”, space, “of”, space, “our”, space, “national”, space, “monument”; space, “_____” (punctuation or formatting), space, “but”, space, “that”, space, “of”, space, “Greece”, space, “is”, space, “Athens”, “.” (punctuation or formatting); space, “Paris”, “.” (punctuation or formatting), line break, “A”, “.” (punctuation or formatting), space, “I”, “RR”, “ATIONAL”.

The next-token loop of mlx-community/Qwen3.5-4B-4bit on the question “What is the capital of France?” wrapped in the chat template, 19 tokens: “<|im_start|>” (special token), “user”, line break, “What”, space, “is”, space, “the”, space, “capital”, space, “of”, space, “France”, “?” (punctuation or formatting), “<|im_end|>” (special token), line break, “<|im_start|>” (special token), “assistant”, line break, “<think>” (special token), 2 line breaks, “</think>”, 2 line breaks. At each step, the three most likely next tokens: step 1: “The” 100%, “**” (punctuation or formatting) <0.1%, “Paris” <0.1%; chose “The”; step 2: space, “capital” 100%, space, “national” <0.1%, space, “**” (punctuation or formatting) <0.1%; chose space, “capital”; step 3: space, “of” 100%, space, “city” <0.1%, space, “and” <0.1%; chose space, “of”; step 4: space, “France” 100%, space, “**” (punctuation or formatting) <0.1%, space, “the” <0.1%; chose space, “France”; step 5: space, “is” 100%, space, “(” (punctuation or formatting) <0.1%, “,” (punctuation or formatting) <0.1%; chose space, “is”; step 6: space, “**” (punctuation or formatting) 100%, space, “Paris” 0.2%, space, “the” <0.1%; chose space, “**” (punctuation or formatting); step 7: “Paris” 100%, space, “Paris” <0.1%, “巴黎” (Chinese characters) <0.1%; chose “Paris”; step 8: “**.” (punctuation or formatting) 95%, “**” (punctuation or formatting) 4.7%, “**,” (punctuation or formatting) <0.1%; chose “**.” (punctuation or formatting).

Qwen3.5-4B: 32 layers, 2560 numbers wide, 248,077 tokens in its vocabulary, 4,205,751,296 parameters in 2,367,118,848 bytes, quantised to 4 bits in groups of 64.