The Query Cannot See the Question: A Short Convolution's Reach Decides Which Part of a Query Conditions Retrieval, and What Falls Outside Becomes a Confident Wrong Answer
This paper demonstrates that the short causal depthwise convolution in linear-attention and state-space models imposes a hard, non-degradable limit on which parts of a query can condition retrieval, causing the model to confidently generate wrong answers when critical information falls outside this narrow window, a failure that persists in shallow layers but can be mitigated by deeper architectures or larger kernels.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Modern artificial intelligence systems that remember information often work like a librarian who keeps a vast, persistent archive of facts. When asked a question, the system must first locate the right page in that archive and then read the answer. But there is a second, harder task: the system must also know when the answer simply does not exist. If it cannot find the information, a smart system should admit it does not know, rather than making up a confident but false answer. This ability to distinguish between "I don't know" and "I know this" is what separates a reliable memory from a confabulator. For years, researchers have assumed that if a model fails to retrieve the correct fact, it is because the information was too complex or the model was not trained well enough. However, a new study suggests the problem is often much simpler and more mechanical: the part of the question that matters is physically invisible to the part of the system doing the searching.
The research, conducted by independent researcher Maximiliano Speranza, investigates a specific type of artificial intelligence architecture that uses a short, local window to form its questions. In these systems, the "query" used to search the memory is built by looking only at the current word and the few words immediately preceding it. The standard setting for this window is so small that it can only see three words back. The study reveals that this limited view is not just a minor detail of how the system smooths out language; it is a hard barrier. If a crucial piece of information in a question falls outside this tiny window, the system does not just ignore it or treat it as weak; it effectively ceases to exist for the search mechanism. The system then proceeds to answer the question with total confidence, using only the fragment of the question it can actually see, often leading to a wrong answer that sounds perfectly certain.
To prove this, the researcher built a small language model trained on a synthetic language where facts were stored in a persistent archive. The model was tested with questions like "What is the [relation] of [entity]?" where the entity was one word away from the search point, but the relation was three words away. In the standard setup, the window could only reach two words back. This meant the relation was always outside the window. The results were stark: the system's sensitivity to the relation dropped to absolute zero the moment it passed the window's edge. When the relation was changed, the search results did not budge. The model was not ignoring the missing word; it was literally blind to it. Consequently, when asked about a relation that had never been stated for a specific entity, the model did not say "I don't know." Instead, it confidently retrieved a different, unrelated fact that happened to be nearby, creating a hallucination that looked like a confident error.
The study then tested whether simply widening the window could fix this blindness. By increasing the window size by just one word on each side—a change that added only a tiny fraction of parameters to the model—the system's ability to admit ignorance skyrocketed. In the difficult cases where the model previously hallucinated, the wider window allowed it to correctly abstain from answering almost every time, with accuracy rates jumping from roughly 60 percent to nearly 100 percent. This demonstrated that the failure was not due to a lack of intelligence or training, but a simple geometric limitation: the search tool was too short to reach the necessary information.
The researchers did not stop at their own small model; they checked if this phenomenon existed in larger, real-world systems. They examined a public, pre-trained model with 130 million parameters and found the same mechanical disconnect. In this larger model, the internal state of the system could "see" the entire sequence of words, but the specific query mechanism used to read that state was still limited by its short window. When a word was changed far outside the window, the query output remained exactly zero, even though the rest of the system registered the change. This confirmed that the "blindness" was a structural feature of the architecture, not a quirk of a specific training run.
However, the story has a second layer that complicates the initial finding. While the short window acts as a hard block in the very first layer of processing, deeper models have many subsequent layers where information can be passed along. The study found that in these deeper models, the signal from the "invisible" word is not blocked forever; it is merely weakened as it travels through the layers. It arrives at a much lower strength, but it is still there. When the researchers fine-tuned a deep model on the same task, they found that the model could eventually learn to use the information that was initially outside the window, but it took much longer to learn. The short window did not make the information unusable; it just made it expensive to learn. In a shallow model with no extra layers to help, the window sets a hard ceiling on performance. In a deep model, it sets a toll that must be paid over time.
Finally, the study looked at whether this configuration was a rare edge case or a common occurrence in real life. The researchers analyzed tens of thousands of real-world questions from various sources, including search engine queries and trivia databases. They found that in between 90 and 100 percent of real questions, the critical parts of the question were separated by more than the reach of the standard short window. The problem was not a corner case; it was the ordinary case. The geometry of human language naturally places the most important parts of a question far enough apart that a standard, short-window search mechanism cannot see them all at once.
The paper concludes with a simple, cost-free way to diagnose this issue in any system that uses a local window to search memory. One does not need to train a new model or run complex tests. One simply needs to take a question, replace a word that is far back in the sentence with a different word of the same type, and see if the system's search changes. If the search does not move, that word was invisible to the system, and any confidence the system expresses about it is unearned. The study reveals that while these systems can eventually learn to overcome their own architectural blindness if given enough depth and time, the initial failure is a mechanical certainty. The system answers with full confidence based on a partial view, and until the architecture changes or the model is given enough layers to pay the toll, that confidence is often misplaced.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.