Law 03 · Context & Reliability
Position Is Power
Models read the edges; the middle gets lost.
The principle
Given a long input, a model attends most reliably to the very beginning and the very end. Critical facts buried in the middle quietly lose their grip — present but functionally ignored. The information was technically 'in context' and still got missed, which is the worst kind of bug because nothing looks wrong.
Why it happens
The U-shaped attention curve is not a quirk of one benchmark: it falls out of how positional encoding and softmax attention distribute weight, so tokens at the extremes stay salient while middle tokens get diluted across a long sequence. The effect compounds badly as context grows. The NoLiMa benchmark (Modarressi et al., 2025) showed that once you remove literal keyword overlap and force the model to follow an association, 11 models fell below half their short-context score at 32K tokens, and even GPT-4o dropped from a 99.3% baseline to 69.7%. The lesson is that present in the window and actually used are different states: a fact buried mid-context with no lexical hook to the query is the most likely thing to be silently ignored, which is why it produces no error, just a wrong answer.
Watch for
- The agent misses a fact you can confirm is sitting in the middle of a long input.
- Accuracy on the same task degrades sharply as you lengthen the context.
- Reordering the input so the key fact is near the top or bottom suddenly fixes the answer.
In practice
You paste a 12-page contract into context and ask the agent to flag the termination clause, but it confidently misses the 90-day notice buried on page 7 because that clause sat dead-center in the input. Nothing errored; the fact was technically in context and still ignored. Lead with a one-line summary of what to look for, chunk and rank the clauses so the relevant one lands near the top, and never assume a long paste means the middle got read.
Apply it
- Lead with a short summary of what to find, and restate the critical instruction at the very end.
- Rank and place the most relevant retrieved passages at the edges of the context, not the middle.
- Test long-context retrieval with questions that have no keyword overlap, not just literal needle matches.
The takeaway
Put the most important instructions and findings at the top or the bottom. Lead with a summary, structure with explicit headers, and never assume that 'in the context' means 'actually used'.