How Does a Large Language Model Work?
Root Concept
Inside an LLM there are no rules and no stored facts — only billions of numbers tuned by training, plus a mechanism that lets every word weigh the words around it to settle what it means.
CodePLU Goal
Upgrading Human Mental Models
Learn how to think in Workflows
Concept Development By codeplu.com
One word, three meanings — the neighbouring words decide, and that is what an LLM computes
What Is Actually Happening Inside an LLM?
You already know an LLM writes text by continuing what came before. But how? There is no dictionary inside it, no grammar rulebook, no table of facts. Open one up and you find something almost disappointing: a colossal grid of numbers. Billions of them.
Those numbers are the dials that training tuned, and they hold everything the model 'knows' — not as sentences you could read, but as settings that make good continuations likely and bad ones unlikely. On top of that sits one genuinely clever mechanism, the thing that turned language models from clumsy into convincing: every word gets to look at the other words around it and adjust what it means in this particular sentence.
That is the idea you will practise above. The word 'spring' is in all three sentences, but a mattress, some seeds, and a mountain stream each drag it toward a completely different meaning. No dictionary required — the neighbours settle it. Once you see that, an LLM stops feeling like magic and starts looking like machinery.
What Is Inside the Model, and How Does It Decide?
What is actually inside the model?
Numbers, and almost nothing else. A large model holds billions of adjustable values — think of them as volume dials — that were set during training and are now frozen. There is no list of rules a person wrote, and no folder of facts it consults. This surprises most beginners, who reasonably expect that something so knowledgeable must store its knowledge somewhere readable. It does not. If you opened the file you would see endless numbers with no labels, and no single dial that means 'Paris is the capital of France'. The knowledge is spread across millions of them at once, which is exactly why nobody can point at a spot inside a model and read off what it believes.
How can numbers hold the meaning of a word?
Each chunk of text gets turned into a long list of numbers, and the trick is that similar meanings end up with similar lists. 'Cat' lands near 'kitten' and not far from 'dog', while 'Tuesday' lands somewhere else entirely, close to 'Wednesday'. Nobody sat down and assigned these positions — they emerged from training, because words that appear in similar company end up with similar numbers. This is why an LLM copes with a word you never taught it explicitly, and why it can answer a question phrased in a way it has never seen: what matters is where the meaning sits, not the exact letters you used.
How does the model know which word 'it' refers to?
Read this: 'The trophy would not fit into the suitcase because it was too big.' You instantly know 'it' means the trophy. Now swap one word: 'because it was too small' — and 'it' becomes the suitcase. Nothing else changed. No grammar rule gets you there; you had to weigh the words against each other. This weighing is precisely what an LLM does, and it is called attention: at every position, the model looks across all the other words and decides how much each one matters for understanding this one. That mechanism is the reason modern models handle long, tangled sentences that older systems mangled.
Why does the same word mean different things?
Because meaning is not stored in the word, it is assembled from the company the word keeps. 'Spring' beside 'mattress' is metal; beside 'planted seeds' it is a season; beside 'drank from' it is water. The model does not look up three dictionary entries and pick one — attention pulls the numbers for 'spring' toward whichever sense the neighbours support, and the result is a version of the word that only exists in this sentence. And this happens in layers, over and over, so each pass refines the picture: first the local clues, then the wider sentence, then the whole conversation.
So how does all this turn into an answer?
After all that weighing, the model has a rich internal picture of everything you wrote. It uses that picture for one final job: producing a score for every chunk in its vocabulary, saying how well each would continue the text. High score for the sensible continuations, low for the nonsense. That single step is the whole output of the model — one set of scores. Turning those scores into flowing paragraphs takes one more idea, which is choosing a chunk and going around again. That belongs to the concept of how LLMs generate responses, waiting next in this track.
Real World Example
How Does One Word Change What "It" Means?
This is the clearest way to feel attention working. Read both sentences slowly and notice what your own mind does — because the model does the same thing with numbers.
The first sentence
'She poured water from the jug into the glass until it was full.' Ask yourself what 'it' refers to. The glass, obviously — you did not hesitate. But notice that nothing in the word 'it' says so.
Change one word
'She poured water from the jug into the glass until it was empty.' Now 'it' is the jug. One adjective flipped, and the meaning of a completely different word moved with it. This is why simple word-by-word rules were never enough for language.
What you just did, the model does with numbers
You weighed 'full' and 'empty' against 'jug' and 'glass', using what you know about pouring. The model performs that same weighing as arithmetic: attention gives 'it' a strong link to whichever container the surrounding words support, and the numbers for 'it' shift accordingly.
It happens everywhere, not just for tricky pronouns
Every word gets this treatment at once, in layer after layer. 'Bank' checks whether rivers or cheques are nearby. 'Charge' checks for batteries, bulls, or credit cards. By the final layer, each word carries a meaning shaped by everything else in your message.
Why this is the whole game
Older systems that could not weigh context lost the thread after a few words, which is why they sounded robotic. Attention is what let models follow long sentences, track who did what to whom, and keep a conversation coherent — the single change that turned language models from a curiosity into something useful.
FAQs
Final Words
Inside a large language model there is no rulebook and no filing cabinet — just billions of numbers tuned by training, and a mechanism that lets every word weigh its neighbours to settle what it means. In the playground you did that weighing yourself: a mattress, some seeds, and a mountain stream each dragged the same word toward a different meaning.
Hold on to two ideas. Meaning lives in the company a word keeps, not in the word itself — which is why context is so powerful and why vague prompts leave the model guessing. And one pass of the model produces only scores for what might come next. How those scores become a flowing, streaming answer is the story of the next concept.
Continue This Track
This concept is part 2 of How Large Language Models Work.
What Are Large Language Models?
An LLM is not a search engine and not a database. Find out what it really is — a machine that learned the patterns of language from an enormous amount of text, and writes new text from them.
How Does a Large Language Model Work?
Inside an LLM there are no rules and no stored facts — only billions of tuned numbers and one clever trick: every word looks at the words around it to settle what it means.
How Does an LLM Turn Your Words Into Tokens?
An AI never sees your letters. Follow your sentence as it is chopped into tokens, turned into numbers, predicted over, and stitched back into text — and see why tokens explain AI's oddest failures.
How Do LLMs Generate a Response?
An AI does not write an answer — it grows one, adding a single chunk of text at a time and re-reading everything before each choice. That one fact explains streaming, randomness, rambling, and mistakes it cannot take back.
