Long context windows buy retrieval. They do not buy correlation.

Posted this on 2025-02-06, when the industry was celebrating million-token windows as if stuffing more text into the prompt solved reasoning across it.

I had used those windows. They would retrieve the right passage and still fail to hold a relationship across passages. Retrieval is a stuffing problem. Correlation is not.

I did not have a mechanical derivation. I had the failure mode in my hands, and I named it before the paper did.