Most explanations of LLMs either drown you in math or oversimplify to the point of being useless. I wanted to write something different. Here's my attempt at actually explaining what a Large Language Model is, in a way that finally makes it click
Let me try to explain it the way I wish someone had explained it to me when I first started digging into this stuff seriously. That was around early 2023, when we were evaluating whether to build a document Q&A feature on top of GPT-3.5 or fine-tune something ourselves. Spoiler: we didn't fine-tune. But that whole evaluation process forced me to actually understand what was going on under the hood, which I'm glad about in retrospect. The Name Is Technically Accurate, Which Is Rare LLM stands for Large Language Model. And unlike most tech acronyms, each word actually means something. Large : trained on enormous amounts of text data. We're talking hundreds of billions to trillions of tokens. GPT-4 hasn't had its training data fully disclosed, but estimates put it in the trillions-of-tokens range. Llama 3, Meta's open-source model, was trained on over 15 trillion tokens. That's not a rounding error; the scale is genuinely hard to wrap your head around. Language : the input and output are text (or tokens, technically, but text is the human-readable version of that). These models were built around language as the medium. Not images, not structured databases, not rule sets. Language. Model : it's a statistical model. A very, very complex one, but at its foundation it's a function that takes input and produces output based on patterns learned during training. That's it. Genuinely. The rest is implementation detail. What "Predicting the Next Token" Actually Means Here's where most explanations either stop too early or get too mathematical. I'll try to land somewhere in the middle. An LLM is trained to predict what comes next in a sequence of text. Given "The sky is," the model learns that "blue" is more likely than "purple" or "a sandwich." But the training corpus is so massive and varied that the model ends up learning an enormous amount of implicit structure: grammar, facts, reasoning patterns, code syntax, conversational norms, even some level of common sense. Tokens are not words, by the way. This confused me for longer than I'd like to admit. A token is roughly a word or a piece of a word, depending on the tokenizer. If you're working in a Java stack, you can call the OpenAI API directly to inspect how text gets tokenized, or use the jtokkit library, which is a Java port of OpenAI's tiktoken. Here's how you'd count tokens with jtokkit: import com.knuddels.jtokkit.Encodings; import com.knuddels.jtokkit.api.Encoding; import com.knuddels.jtokkit.api.EncodingRegistry; import com.knuddels.jtokkit.api.EncodingType; import java.util.List; public class TokenizerExample { public static void main(String[] args) { EncodingRegistry registry = Encodings.newDefaultEncodingRegistry(); // cl100k_base is the encoding used by GPT-4 and GPT-3.5-turbo Encoding enc = registry.getEncoding(EncodingType.CL100K_BASE); String text = "Large Language Models are surprisingly straightforward once you understand tokenization."; List<Integer> tokens = enc.encode(text); System.out.println("Text: " + text); System.out.println("Token count: " + tokens.size()); // Decode each token individually to see how the text was split for (Integer token : tokens) { System.out.print("[" + enc.decode(List.of(token)) + "] "); } System.out.println(); } } Run that and you'll see something like "surprisingly" gets split into two tokens, while common short words stay as one. This matters because context windows are measured in tokens, not words. When someone says GPT-4 has a 128k context window, they mean 128,000 tokens, which is somewhere around 90,000 to 100,000 English words depending on the text. The Architecture Behind It: Transformers (Without the PhD) LLMs are built on the transformer architecture, introduced in the 2017 paper "Attention Is All You Need" by Vaswani et al. at Google. That paper is genuinely readable if you have some math background, but the key idea is the attention mechanism. Attention lets the model look at every other token in the input when deciding what the current token means. So when processing the word "it" in "The cat sat on the mat because it was tired," the model can attend back to "cat" and figure out what "it" refers to. Previous architectures (RNNs, LSTMs) had to process tokens sequentially and often lost track of long-range dependencies. Transformers handle that much better. The "large" part comes in when you scale this architecture up: more layers, more attention heads, more parameters. A parameter is just a learned numerical weight. GPT-2 had 1.5 billion parameters. GPT-3 had 175 billion. Current frontier models are in the hundreds of billions to possibly trillions range, though exact counts aren't always published. More parameters, trained on more data, generally means better performance. Not always, and not infinitely, but that relationship held up well enough to drive the last several years of development. How Training Actually Works There are really two phases most production LLMs go through. Pre-training is the expensive part. The model reads through the training corpus and adjusts its weights to get better at predicting the next token. This is unsupervised; there are no labels, no one telling the model what's right or wrong. It just sees text and learns patterns. Training GPT-4 reportedly cost somewhere around $100 million in compute. That's not a typo. Fine-tuning and RLHF come after. RLHF stands for Reinforcement Learning from Human Feedback, and this is where you take the pre-trained model and shape its behavior to be more helpful, less harmful, and more aligned with what humans actually want. Human raters compare model outputs and indicate which is better, and that signal gets used to further train the model. This is largely what turns a raw language model into something like ChatGPT or Claude. There's also instruction fine-tuning, which is more straightforward: you show the model a bunch of examples of instructions paired with good responses, and it learns to follow that format. Llama 3 Instruct, for instance, is the base Llama 3 model after this kind of fine-tuning. The difference in behavior between the base model and the instruct version is dramatic. Night and day, honestly. What LLMs Are Good At (And Where They Fall Apart) Honestly, I think the hype around LLMs made it harder to understand what they're actually good for, because the claims got so big that everything seemed possible. Not great for calibrating expectations. What they're genuinely excellent at: Generating fluent, coherent text in almost any style or format Summarizing long documents. We used this in a pipeline to condense 40-page legal agreements into 5-bullet summaries during a contract review project last spring, and it worked surprisingly well Code generation and explanation, especially with GPT-4 or Claude 3.5 Sonnet Translating between languages Answering questions when the answer is already in the context (RAG setups, where you retrieve relevant documents and stuff them into the prompt) Following complex formatting instructions Where they fall apart: Precise arithmetic. They'll attempt it and often get it wrong. Use a calculator tool or code interpreter for anything beyond basic math. Real-time information. The model's knowledge has a training cutoff. GPT-4's is April 2023. Llama 3's is around December 2023. Anything after that, it doesn't know. Consistent factual accuracy on obscure topics. This is the hallucination problem. The model will sometimes generate plausible-sounding facts that are simply wrong. Confidently wrong. Long chains of precise logical reasoning. They've gotten better, but they're still unreliable for multi-step formal proofs or rigorous deductive chains without scaffolding. The hallucination thing is real and I don't think people take it seriously enough. On that Q&A project I mentioned earlier, we ran evals where we asked the model questions a...