Do LLMs understand the world?

It's quite incredible to see how far AI has come through LLMs. Although earlier versions made very obvious blunders (we all remember the glue on pizza recommendation from Gemini) and you had to really second guess every statement made, it's now at the point where, for most topics, I trust it.

The part that's amazing to me is that, in most cases, it doesn't seem to be just regurgitating web articles or random blog posts. Instead, it seems to show some level of undertanding of the topic we're discussing, in a way that is hard to believe comes from minimizing a loss function on next token prediction and then doing some RL to fine tune it.

Are LLMs world models?

This has been a debate by prominent researchers for a while now, with Yann LeCun probably being the most vocal and authoritative researcher arguing that LLMs are not world models and that we need world models for AGI.

But to understand this, first let's look at what the definition of a world model is. Essentially, a world model explicitly learns to predict the next state of the world given the current state and some action by some agent. Whereas LLMs train on predicting the next token in a text extract (for the most part), world models learn to predict the next state of the world they're in, which is usually much richer than generating a single token at a time. By doing this, they learn cause and effect explictly.

The reason I've made the word explicitly bold, is because some argue that LLMs are partial world models that have been trained implicitly. Ilya Sutskever would likely fall into this camp, if I understand his stance correctly (see his lecture: An Observation on Generalization) and that's what we'll be focusing on here.

How can LLMs learn to be world models?

I'll not focus on whether they are or are not world models in this post, as that's been done extensively, and I think for most people the anser is that they're limited partial world models. I'll try to present why they might gain this capability, even if not trained for it explicitly.

Algorithmic information theory and compression

To best undertand this its worth at least superficially understanding what the Kolmogorov_complexity is. In very basic terms, this is the length of a shortest computer program that could produce some output (and equivalently, some output given some input).

A canonical example is something like, given some string: "hello hello hello hello hello hello hello hello hello hello hello hello hello hello hello", write some program that outputs it. One way to do so would be:

def print_hello():
  return "hello hello hello hello hello hello hello hello hello hello hello hello hello hello hello"

but that's wasteful, as it uses too many characters and it would even worse the more times "hello" were repeated. We could instead write:

def print_hello():
  return " ".join(["hello"]*15)

which grows logarithmically with the number of repetitions (as the integer grows).

What are LLMs trying to learn?

LLMs are effectively trying to learn a massive joint probability distribution over all tokens in a text. Effectively, they want to learn the joint distribution:

P(t_1, t_2, t_3, ..., t_T) = p(t_1)p(t_2|t_1)p(t_3|t_2, t_1)...p(t_T|t_1, ..., t_{T-1})

where $t_i$ is token $i$ from a vocabulary of $V$ and the sequence length is $T$ . One way to model this is to create lookup tables for each probability term on the right hand side. For instance, $p(t_1)$ is a table of size $V$ . $p(t_2|t_1)$ is a table of $V\times V$ . $p(t_3|t_2, t_1)$ is a 3 dimensional table with $V^3$ elements, etc. Now suppose that the tokens come from a vocabulary of size 100k tokens (as is the case for many modern LLMs). This would mean that for the first term we'd have 100k elements, for the second we'd have $10^{10}$ elements, for the third $10^{15}$ , and so on. We'd run out of RAM even for the third probability distribution ( $10^{15}$ floating point numbers at a minimum), never mind context lengths running into the thousands of tokens, as is typical for modern day LLMs.

But with LLMs we seem to learn this whole distribution well enough, with much less memory ( $10^9$ to $10^{12}$ params). In this way, we can clearly see that LLMs somehow compress the data and we can think of them trying to find the shortest computer program (in terms of model architecture and number of parameters) to be able to produce all the data in the training set i.e. what we saw in the Kolmogorov Complexity.

What would be the shortest program?

When you read the phrase: "If a bear and a bumblebee bat were to get in a fight, the winner would be the ..." you can fill in the blank with bear. You've likely never come across a sentence like this before. But you're able to answer it, because you're able to deduce that a bumblebee bat is a very small animal and a bear is a big animal. In most physical fights, bigger means more likely to win. Therefore, the bear will win. That's not all, though. By knowing that the bear is larger, you also likely can infer that it is heavier, requires more calories, has a larger shadow, etc.

In other words, by understanding some rules about the world, you don't need to store any conditional probabilities for every combination of animals. Similarly, if an LLM could learn about the world, then it's very likely that this would be the most efficient way to maximize its objective during training.

Let's continue with the example above. Let's assume that we want to naively learn the conditional probabilities (taking a value of either 0 or 1 for simplicity) for each of the relationships mentioned above between a set of $N$ animals. Doing so would require $O(N^2)$ conditional probabilities per relationship. If there are $R$ relationships, we'd get $O(RN^2)$ parameters (conditional probabilities). If we instead save the following rules though:

Bigger animals win in fights
Bigger animals are heavier
Bigger animals need more calories
Bigger animals cast larger shadows

We can then just save the sizes of each of the animals and evaluate relative size on the fly ( $O(N)$ ) and just pick out the rule for the relationship we care about ( $O(R)$ ). In other words, the total number of parameters becomes $O(N + R)$ ! Even with this toy example, which isn't optimal at all in its representation, we can see how much shorter the code would need to be to create such a model.

Of course, this doesn't prove that it does do that. It just presents a sufficient condition of some sorts. In the absence of infinite model capacity, any model will need to learn some sort of compressed representation to perform well. Given that LLMs do seem to have some level of understanding, it doesn't seem too farfetched to believe that this is what's happening (to some extent).

Imperfection

Of course, there are limitations. They're seeing a much lower dimensional representation of the world (just text), so they won't be able to understand some things perfectly and there is a plethora of material showing cases where LLMs truly do fail. But people also have limited understandings of many topics that we can't physically experience, and yet we're still able to make progress in those topics e.g. sub-atomic physics, astronomy, cellular biology, etc.

As the quote goes: "All models are wrong, but some are useful". In my opinition, LLMs are no exception.

Theo Pavlakou

Are LLMs world models