AI from scratch

1. It's not a program

The first thing to unlearn is the most natural one.

You are a programmer. When something answers a question, you assume there is a program behind it. Some if statements. A search index. A bunch of special cases someone wrote down. If the answer is good, the program is clever. If the answer is weird, there is a bug.

A language model is not that.

There is no if user_asked_about_python: return python_help(). There is no database of facts it looks up, at least not inside the model itself. There is no inner narrator deciding what to say.

What there is: a giant pile of numbers, and a short, fixed procedure for using those numbers to turn the text so far into a guess about the next piece of text.

That is the whole thing.

The pile of numbers is called the weights. The procedure is called the architecture. The guess is a list of probabilities over possible next pieces of text.

When people say "the model," they usually mean the weights plus the architecture together. When they say "Qwen3.8," they mean one particular pile of numbers, produced by one particular training run, that fits one particular architecture.

This is closer to a huge spreadsheet than to a codebase. The spreadsheet has a few billion, or a few trillion, cells. Each cell is a number like 0.0137 or -1.82. Nobody wrote those numbers. A training process found them by trying, over and over, to get better at guessing the next piece of text.

You can write the procedure that uses the spreadsheet in a few thousand lines of Python. Hugging Face does. The interesting part is not the Python. The interesting part is that the numbers, arranged that way, do something that looks like thinking.

It is not thinking in the way you think. It is also not a trick with Mad Libs. It sits in a third category that does not have a comfortable everyday name, which is why people keep arguing about what to call it.

For this book we will stay concrete. A model is a function:

next_piece = model(text_so_far)

The function is deterministic if you force it to be. The same input, with the same settings, produces the same pile of probabilities. What feels like personality, or mood, or "the model decided to be careful" is almost always either the randomness used when picking from those probabilities, or the text that was already in the prompt.

This matters because it changes what questions are worth asking.

"Does it know Python?" is a fuzzy question. A better one is: after seeing a lot of Python, did the weights end up in a shape that makes def a likely continuation of import os, and IndentationError a likely continuation of a bad indent?

"Is it conscious?" is not a question this book can answer. "What computation runs when you hit enter?" is. That one we can walk through all the way down.

One more thing, because it will keep coming back.

The model does not have a separate "knowledge module" and "personality module" and "coding module." It is one function. Everything it can do — write a function, refuse a request, pretend to be a pirate, get a fact wrong with total confidence — comes out of the same next-piece-of-text machinery.

If that sounds too simple to produce Qwen3.8, good. That feeling is the right place to start. The rest of the book is about how something that simple gets this far.