ChatGPT Is Not an LLM: Models, Agents and Systems, Precisely Defined

I want to be precise about something that has become sloppy, because the sloppiness is now producing bad strategy.

People say “the LLM decided not to cite us.” They say “the model has a knowledge cutoff so it cannot know about our product.” They say “we need to optimise for the LLM.” Each of those sentences attributes to a neural network a behaviour that was produced by a piece of infrastructure sitting several layers above it — and if you misattribute the cause, you will fix the wrong thing.

The existing explanations of this distinction stop at the level of an analogy: the model is the engine, ChatGPT is the car. That is true and it is not useful. Nobody changes what they do on Monday because of an analogy. What follows is the mechanism, the places it fails, and the specific consequences for anyone trying to be visible inside these systems.

What a language model actually is

Strip everything away and a transformer language model is a pure function. You hand it a sequence of tokens. It returns a probability distribution over the next token. That is the entire contract.

f(token_sequence, weights) → probability distribution over vocabulary

Properties of f:
  - deterministic given identical input and identical weights
  - no memory of any previous call
  - no access to anything outside the token sequence
  - no concept of "true"; only "likely given training distribution"

Everything you experience as conversation is that function called repeatedly, with the previous output appended to the input each time. The model does not know it is having a conversation. It is being handed a longer and longer document and asked what comes next.

Statelessness, stated correctly

Here is where most explanations, including the ones currently ranking for this question, become imprecise in a way that matters.

“The model is stateless” is repeated as though it were absolute. It is not. There is no persistence between requests, which is the sense people mean. But within a single forward pass there is a great deal of state: the attention mechanism computes a key-value representation for every token processed, and in production that KV cache is retained across the generation of a response and frequently across turns in a session for efficiency. The model is not remembering in any meaningful sense, and it is also not the blank slate the phrasing implies.

Why does this matter to anyone outside an inference team? Because prompt caching — which is now standard commercially — means the cost and latency of a request depend on what came before it in ways that a naive “stateless” model does not predict. And because the sloppy version of statelessness leads people to conclude that context is free and interchangeable, which it is not.

The system around the model

When you type into ChatGPT, the model is one component among many, and it is not the first thing your query touches.

Layer	What it does	Why it matters for visibility
Request routing	Decides which model serves you, based on tier, load, query type	Two users asking the same question may be answered by different models
Safety classification	Screens input and output against policy	Entire query categories are answered from a template, not a model
Tool selection	Decides whether to search, run code, browse, or answer directly	This is the single biggest determinant of whether you can be cited at all
Retrieval	Issues queries, fetches documents, chunks and ranks them	Your page competes here, not in the model
Context assembly	Builds the final prompt: system instructions, memory, retrieved chunks, history	Position and budget in this assembly decide what the model actually sees
The model	Generates tokens from that assembled context	By this point most of the selection has already happened
Post-processing	Citation formatting, link resolution, rendering	Determines whether your URL appears or is silently dropped

Read that table with a visibility question in mind and the implication is uncomfortable. The layer people obsess over — the model, its training data, its “opinion” of your brand — sits second from the bottom of a stack where almost every consequential decision has already been made.

Memory, implemented rather than described

“It remembers” is the most misleading two-word description in this space. Nothing about the weights changed. What happened is that text was written to a store and later reinserted into your prompt.

There are three distinct mechanisms wearing the same word, and conflating them produces incorrect expectations about all three.

• Conversation history. Prior turns appended to the context window. Bounded by the window; older turns are truncated or summarised. Not memory, just a longer document.
• Extracted persistent memory. A separate model reads your conversation, decides what is worth keeping, writes short natural-language facts to a store, and those get injected into future system prompts. The extraction step is itself a model call and it is lossy, opinionated and frequently wrong about what mattered.
• Retrieval over prior conversations. Embedding past exchanges and pulling semantically similar ones back. This is a vector search, with all the failure modes of a vector search.

The consequence I would highlight: memory is a retrieval problem, not a knowledge problem, and retrieval problems fail in characteristic ways — semantically similar but irrelevant material surfaces, and precisely relevant material with different vocabulary does not. If a system “forgot” something you told it, it did not forget. The retrieval step failed to select it, or the extraction step never wrote it down.

Where the system fails, and how to tell which layer broke

This is the section absent from every explanation of this topic I have read, and it is the one with operational value. When output is wrong, the layer that failed determines the fix. Attributing everything to “hallucination” is a diagnostic dead end.

What you observe	Layer that failed	What is actually wrong
Confident, fluent, entirely fabricated	Model	Generation from training distribution with no grounding. The classic case, and rarer than blamed
Answered without searching when it should have searched	Tool selection	The router judged the query answerable from parameters. You were never in the running
Searched, but your page was not among sources	Retrieval	Query formulation or ranking. Your content may be fine and simply not selected
Your page was retrieved but not cited	Context assembly or generation	Chunk ranked low, got truncated, or the model preferred another source
Cited but the claim is not in your page	Chunk boundary	A fragment was lifted without the qualifying sentence around it
Correct answer, no link rendered	Post-processing	Citation resolution failed. Invisible in most monitoring tools
Refused or gave a generic response	Safety classification	Never reached a model in any meaningful sense

Why “the model doesn’t know about us” is usually the wrong worry

I hear this constantly and it reflects the same category error. The concern is that a brand is absent from training data, and the proposed fix is usually to publish more content in the hope of appearing in the next training run.

That reasoning has three problems.

1. Most commercially relevant answers are retrieved, not recalled. Anything current — pricing, availability, comparison, recent events — triggers a search. Training data is not the pathway for those queries.
2. You cannot meaningfully target a training run. Composition, cutoffs and filtering are undisclosed, the cycle is long, and the relationship between publishing volume and parametric representation is not one you can measure or control.
3. Parametric knowledge is the least reliable path anyway. A model recalling your brand from weights will produce plausible-sounding detail with no grounding. Being retrieved is better than being remembered, because retrieval carries a citation and recall carries an invented fact.

The strategic conclusion, which I hold firmly: optimise to be retrievable, not memorable. Retrieval is a system you can influence with structure, clarity and freshness. Training data is a lottery you cannot enter deliberately.

What follows for practice

• Stop saying “the LLM.” Say which layer you mean. If you cannot, you have not diagnosed the problem yet.
• Determine whether your query class triggers retrieval at all. Ask the same question in a way that forces browsing and compare. If the answer differs substantially, you are dealing with tool selection, and that is a different game.
• Treat non-determinism as a measurement problem. Same prompt, same day, different answers, because sampling is stochastic and routing varies. One observation is not a result.
• Check whether your content survives being cut into pieces. Retrieval operates on chunks. A paragraph that only makes sense with the two above it will be quoted incorrectly or skipped.
• Expect the ground to move for reasons unrelated to you. A routing change, a new safety rule, a modified retrieval threshold. None of it appears in a changelog.

The harder truth

Traditional search was adversarial but legible. Google published guidelines, announced updates, and operated a system whose behaviour you could infer from observation across many sites. It was possible to be wrong, and it was possible to find out.

Assistant systems are neither. There is no changelog, no guideline document that describes retrieval behaviour, no stable surface to measure against, and the vendors have no obligation or commercial incentive to provide any. The system is modified continuously by teams optimising for engagement and cost, and visibility is a side effect of decisions made for other reasons entirely.

I do not think that is a temporary condition that better tooling resolves. It is the shape of the thing. The rational response is not to chase the surface — it is to understand the mechanism well enough to know which movements are signal and which are somebody else’s deployment, and to build the kind of content that retrieval systems select for structurally rather than the kind that games a particular configuration.

Starting with the vocabulary. ChatGPT is not an LLM. Once you hold that distinction properly, most of the confused advice in this field becomes easy to identify.

The retrieval layer is where the real contest happens, and I have set out its mechanics in how retrieval actually selects your page. The measurement problem this creates is covered in why AI visibility tools disagree with each other.

Frequently asked questions