What is a context window in a large language model?

A context window is the maximum amount of text — measured in tokens — that a large language model can "see" at once, including everything you've said in the conversation so far. Once a conversation grows past that limit, the oldest parts fall out of view and the model can no longer refer to them.

AI & ML — INTRODUCTION

What Is a Context Window? An LLM’s Working Memory, Explained

Ever had a long chat with an AI assistant, mentioned your name near the start, and then had it ask you your name again twenty messages later? That’s not a bug — it’s the context window.

The Quick Answer

A context window is the maximum amount of text — measured in tokens, the small chunks from the “How LLMs Work” article — that a model can “see” at once, including everything said so far in the conversation. Once that fills up, the oldest parts fall out of view, and the model genuinely can’t refer back to them anymore.

What the Experts Say

Like humans, who have limited working memory capacity, LLMs have an “attention budget” that they draw on when parsing large volumes of context.
— Anthropic, “Effective Context Engineering for AI Agents”

How It Actually Works

1
It’s a fixed size. Every model has a maximum context window, measured in tokens — a few thousand for older models, over a million for the newest ones.
2
Everything shares the same window. Your messages, the model’s replies, and any instructions behind the scenes all count against the same limit — it’s one shared space, not separate buckets.
3
Full window means something falls out. Once the conversation grows past the limit, the oldest messages are dropped so the newest ones fit — like a queue where the front keeps getting pushed off.
4
It resets between conversations. Start a brand-new chat and the window is empty again — this isn’t the same as the model’s permanent, trained-in knowledge, which it always keeps.
5
Working memory, not a hard drive. Think of it as what a person can actively hold in mind during a conversation, not a filing cabinet where everything ever said is saved forever.

Watch a Conversation Fill Up

Below is a tiny context window that holds only 6 messages at a time — far smaller than any real model, just so you can fill it in a few clicks. Keep sending messages and watch what happens once it’s full.

Now Try Asking About Something It Forgot

This time, the very first message contains a fact — a name. Send enough follow-up messages to push it out of the window, then ask the model about it and see what it says.

👉 Ask right away and it knows. Fill the window first, then ask — and it genuinely doesn’t know anymore.

Fun Fact

The original GPT-3, released in 2020, had a context window of just 2,048 tokens — about 1,500 words, or roughly three pages of text. By 2026, flagship models routinely ship with windows of 1 million tokens or more — one model even reaching 10 million — enough to hold an entire novel, several times over, in a single conversation.

Test Yourself

What is a context window?

What happens when a conversation grows past the context window limit?

Ready for the engineering details — why context windows are expensive to grow, truncation strategies, and why a bigger window doesn’t always mean better recall? Continue to the Intermediate level →