Everyone talks about LLMs, but how many people actually know what's happening inside one? I went back to basics to break down what a large language model really is, the components that make it work, and why it matters more than the buzzwords suggest. If you've ever wanted to understand the "what" before the "how," thi...
For a long time I used the term "large language model" the way most people use it, loosely, to mean "the AI thing that generates text." It wasn't until I started building on top of OpenAI's API in early 2024, on a side project that eventually turned into an internal documentation assistant at work, that I felt like I actually needed to understand what was going on inside. Not just to sound smart in meetings, but because the model kept doing things I didn't expect, and I had no mental model for why. So I went back to basics. Real basics. Let's Start With What "Language Model" Even Means A language model is, at its simplest, a probability distribution over sequences of tokens. That's it. Given some input, it predicts what comes next. Not a word, exactly. A token, which is a chunk of text that might be a whole word, part of a word, or even a single character depending on the tokenizer. GPT-4o uses a byte-pair encoding (BPE) tokenizer. The word "unhappiness" might become three tokens: "un", "happi", "ness". "ChatGPT" might be one token or two depending on context. This stuff matters more than people think, because the model never actually sees raw text. It sees integers. Token IDs. A sentence like "The cat sat" becomes something like [464, 3797, 3332] before the model ever touches it. I didn't fully appreciate this until I was debugging a weird truncation issue in our doc assistant. The context window was 8,192 tokens (this was before GPT-4o's 128k window), and we were hitting the limit way sooner than expected. Turned out our average paragraph was running about 180 tokens, not the ~100 words I'd mentally estimated. Off by almost 2x. Not great. The "Large" Part The "large" in LLM refers to parameters. Lots of them. A parameter is just a number. A weight in the network. During training, the model adjusts billions of these weights to minimize prediction error across a massive text corpus. GPT-2 had 1.5 billion parameters. GPT-3 had 175 billion. GPT-4o's exact count is undisclosed, but current frontier models are estimated somewhere in the hundreds of billions to over a trillion (GPT-4o and Gemini 1.5 Pro are both widely believed to use mixture-of-experts architectures, but that's a whole other tangent). Meta's Llama 3.1 405B is the largest publicly disclosed open-weights model at 405 billion parameters, which gives you a sense of the scale we're talking about. What does a single parameter actually do? In isolation, almost nothing. But collectively, these weights encode something like compressed statistical knowledge about language, facts, reasoning patterns, and relationships between concepts. It's not magic. It's matrix multiplication at a scale that's hard to picture. // Conceptually, a single "layer" in a transformer is doing something like this. // A linear transformation: output = input * weights + bias public class LinearLayer { private final double[][] weights; // shape: (inputDim, outputDim) private final double[] bias; // shape: (outputDim) public LinearLayer(double[][] weights, double[] bias) { this.weights = weights; this.bias = bias; } // x: input token embedding vector, shape (inputDim) public double[] forward(double[] x) { int outputDim = weights[0].length; double[] result = new double[outputDim]; for (int j = 0; j < outputDim; j++) { result[j] = bias[j]; for (int i = 0; i < x.length; i++) { result[j] += x[i] * weights[i][j]; } } return result; } } Real transformers stack dozens of these layers, with attention mechanisms between them. But the core idea is the same: learned weights transforming input vectors into output predictions. The Transformer Architecture This is the actual engine. The "T" in GPT stands for Transformer, and every major LLM shipping in 2025 (GPT-4o, Claude 3.7 Sonnet, Llama 3.3, Gemini 2.0 Flash, Mistral Large 2, DeepSeek-V3) is built on this architecture, originally published by the Google Brain team in the 2017 paper " Attention Is All You Need. " The key insight is the attention mechanism. Rather than processing tokens one by one in sequence (which is what RNNs did, and why they were painful to scale), attention lets the model look at all tokens simultaneously and figure out which ones are most relevant to each other. Here's the thing that finally clicked for me: when the model is processing the word "it" in the sentence "The trophy didn't fit in the suitcase because it was too big," it needs to figure out that "it" refers to "trophy" and not "suitcase." Attention is what resolves this. The model learns to attend to the right tokens based on context. And it does this across every token in the sequence, in parallel. A transformer block, roughly, contains: Multi-head self-attention : multiple attention heads running in parallel, each learning to focus on different types of relationships (syntax, coreference, subject-verb agreement, etc.) Feed-forward network : a two-layer MLP applied to each token position independently. This is where a lot of the "knowledge" lives, actually Layer normalization : stabilizes training Residual connections : lets gradients flow back through deep networks without vanishing Stack 96 of these blocks (that's GPT-3's count; modern models like Llama 3.1 405B use 126 layers) and you get the full model. Well, plus the embedding layer at the start and the output head at the end. // Simplified pseudocode for a transformer forward pass in Java-style public class Transformer { private final EmbeddingTable embeddingTable; private final List<TransformerBlock> layers; private final OutputHead outputHead; public Transformer(EmbeddingTable embeddingTable, List<TransformerBlock> layers, OutputHead outputHead) { this.embeddingTable = embeddingTable; this.layers = layers; this.outputHead = outputHead; } // Returns a probability distribution over the vocabulary // for the next token at each position. public double[][] forward(int[] tokenIds) { // 1. Convert token IDs to embedding vectors double[][] x = embeddingTable.lookup(tokenIds); // shape: (seqLen, dModel) // 2. Add positional encodings x = addPositionalEncoding(x); // 3. Pass through each transformer block (attention + FFN + norm + residual) for (TransformerBlock layer : layers) { x = layer.forward(x); } // 4. Project to vocabulary size and apply softmax double[][] logits = outputHead.project(x); // shape: (seqLen, vocabSize) return softmax(logits); } } The output is a probability distribution over the entire vocabulary for the next token. The model picks (or samples) from that distribution, appends the new token, and repeats. This is called autoregressive generation. Pretraining vs. Fine-Tuning vs. RLHF When people say "the model was trained on the internet," they're talking about pretraining. The model sees hundreds of billions of tokens of text, and at each step it tries to predict the next token. That's the entire pretraining objective. Simple, but extraordinarily powerful at scale. But a pretrained base model is kind of weird to interact with. It's a text completion engine. Ask it a question and it might complete the question with more questions (because that's what appears in training data) rather than answering. This is why raw base models aren't what you interact with in ChatGPT. Fine-tuning on instruction-following examples, commonly called SFT (supervised fine-tuning), teaches the model to actually respond to instructions. And then RLHF (Reinforcement Learning from Human Feedback) pushes it further. Human raters compare model outputs and rank them. A separate "reward model" learns to predict those rankings. The main model is then optimized to score higher on the reward model using PPO (Proximal Poli...