Your best questions aren't retrieval questions

In short

Retrieval assumes answers are stored somewhere and just need finding. That holds for a lot of questions and fails for an important class of them: the ones whose answer exists across records rather than in any one of them. No embedding model reaches those. Neither does a bigger context window.

The short version

• Retrieval assumes the answer is written down somewhere. A second class of question, structural questions, has answers that are properties of the relationships between records, and no record contains them.
• The beneficial ownership loop is the clean case: three correct filings, a chain that closes on itself, and a finding that lives in no single row.
• Similarity search cannot rank the record that matters, because what makes it matter is not written in its text. Whether it arrives depends on the corpus, and nothing announces its absence.
• Handing the model the whole subgraph does not save it: cycles, aggregated percentages, and threshold breaches are computations to perform and verify rather than things to notice in prose.
• What closes the gap is an ontology: what the entities are, what the relations mean, and what rules hold over them, written down once where both the analyst's knowledge and the engine's query can come from it.

01 A question with no answer in the file

There is a class of question that enterprises most want answered and that the current generation of agents cannot reach. This is the first post in a series about such questions. The examples that follow come from banking, where the rules are written down, the stakes are legible, and the failure modes already have names, but very little of what follows is specific to banks. The same pattern appears wherever an answer depends on how records relate rather than on what any one of them says: a manufacturer tracing a component back through four tiers of suppliers, an insurer checking whether the same adjuster sits on both sides of a claim, a hospital reconciling one patient across systems that never agreed on an identifier. Swap the vocabulary, and the argument still holds.

Every bank has to name the human beings behind a corporate customer. That is the beneficial ownership requirement: follow ownership up through whatever holding structures exist until the chain ends in natural persons, then screen those names against sanctions and politically exposed person (PEP) lists. It runs thousands of times a day and it is almost always mechanical.

Then an analyst files a note that reads: we can never get to a real person on this one.

Nothing in the data looks wrong. Three columns do the work, customer_id, parent_entity_id and ownership_pct, and the filings are present and correct.

FILING 1 ACM-4471 Acme Corp is 100% owned by KES-2210 Kestrel Holdings

FILING 2 KES-2210 Kestrel Holdings is 100% owned by ALD-8853 Alder Group

FILING 3 ALD-8853 Alder Group is 100% owned by ACM-4471 Acme Corp

Read them one at a time and there is nothing special to see. Companies own companies all the time. Every record is clean and every field populated.

And the requirement still cannot be satisfied, because the chain has no bottom. Follow the ownership and you arrive back where you started, without ever reaching a person.

No single filing carries that fact. It belongs to the three of them together, and it is written down nowhere.

'Something' or 'someone' has to know that a chain of parent_entity_id pointers is an ownership structure, that walking it is what the regulation means by tracing, and that a chain closing back on itself means the ownership never reaches a person, rather than meaning the data is broken.

None of that knowledge is in the schema. None of it is in the embeddings. It lives in the analyst's head, and it gets re-derived, imperfectly, on every single query.

02 The model underneath

Retrieval is incomplete rather than wrong: it describes half the problem, and the field has been treating it as the whole one.

Beneficial ownership is exactly the kind of work agents are now pointed at. Sit an agent on the bank's data, let an analyst ask in plain language, get an answer back without a ticket and a three-day wait.

The standard advice for making that agent answer better tends to be the same. Better embeddings. Better chunking. Better reranking. Hybrid search. A bigger context window. Every item on it improves fetching. The assumption underneath, mostly unexamined, is that answering a question means going and getting the thing that contains the answer.

That assumption is good. It is just partial. It holds perfectly for questions whose answers were written down somewhere, and those questions are common enough that the edge of it rarely comes up. What's this customer's balance? What does our refund policy say? The answer is sitting in a place. Retrieval is the right tool there.

The ownership question is the edge. Fetch all three records perfectly, with a perfect ranker, and the answer still isn't among them, because none of them contains it. The retrieval ran correctly, against a question that was never a retrieval question.

That is the missing half. Alongside the questions with an answer somewhere sits a second class, whose answers are properties of the relationships between things rather than properties of any thing. Let's call them structural questions.

03 Recognizing them in the wild

Once the missing half is visible, structural questions turn out to be most of the ones worth asking.

Structuring detection

Nine cash deposits between $6,400 and $9,100, spread over thirty-one days across four branches, totaling $71,800. The reporting threshold is $10,000 and not one deposit reaches it. The account's largest single deposit in the preceding twelve months was $2,300. Every deposit is legal, unremarkable, and correctly recorded.

SIGNAL a property of the set, and of its distance from that account's own baseline

Cross-border flow mapping

2.1m euro leaves a Frankfurt account held by Meridian GmbH, lands in Nicosia, moves to Dubai eleven days later, and returns to a second Frankfurt account held by Meridian Logistik GmbH. Different legal entity, same beneficial owner. Each hop is a legitimate transfer between legitimate accounts and each clears review on its own. Nothing in hop three records that hop one ever happened.

SIGNAL the route, and the fact that it closes on the party it started from

Four-eyes violations

Every alert is supposed to be raised by one analyst and cleared by a different one. Over eighteen months, how many times did the same person appear on both ends, whether directly or through a colleague they also supervise? No memo records a violation, because no single memo contains both ends of the pair.

SIGNAL a count of paths through people and documents, computed, not stored

Shared-attribute clustering

Nine companies file as independent entities. Three list the same address in Slough, four name the same director, two give the same Berlin phone number. Plenty of companies legitimately share a registered office, so no single filing is irregular. The signal only exists once the nine are placed side by side.

SIGNAL the collision, invisible until the records are co-located

There is a practical filter in here. Lookup questions already have good answers; a database has handled them well for decades, and wrapping natural language around them is a real convenience but rarely the reason anyone funds an agent. The questions above are the ones with no explicitly written answer, and they are where an agent earns its place.

It makes a useful check against our roadmaps: how many of the target questions are structural, and does the current stack have a way to reach them?

04 Where each way of fetching breaks

Walk the two standard retrieval methods against the ownership question. Each fails in a specific, nameable place, and neither failure is fixed by turning a dial harder.

FAILURE 1 · SIMILARITY SEARCH

The ranking is flat, and nothing in it tells you whether the chain is complete.

Embed who ultimately owns Acme Corp and pull the k nearest records. Acme's own filing comes back first, because it contains the string. After that the ranking goes flat.

Every record in the corpus is a corporate filing, and they are all written the same way: an entity, a status, a parent. To an embedder they are near-identical. Kestrel Holdings, the record the chain runs through, scores 0.67. Brightwater Holdings, which has nothing to do with this customer, scores 0.70. Kestrel Marine Ltd, which shares half a name and no relationship at all, scores 0.71.

That spread is noise. The property that makes Kestrel's filing the one you cannot do without, that it is the middle link between Acme and Alder, is not written in its text, so no score can reflect it. What the scores reflect is that all of these documents look like corporate filings.

So the retrieval is a maybe. At k=10 today Kestrel lands just inside the window and the agent finds the loop. Onboard forty thousand entities next quarter and Kestrel is at rank 31, the retrieval window closes above it, and the agent reports that Acme is owned by Alder Group. Same pipeline, same prompt, same confident tone, opposite answer, and nothing in either output distinguishes the two runs.

Reranking does not help, because a reranker scores the same text against the same query. Raising k does not help either; it widens the window and adds more filings that look exactly alike, which lowers the odds that a reader spots the one that mattered. There is no threshold or confidence band, and no error to catch, because as far as the retriever is concerned, nothing went wrong.

Even with all three records in hand, similarity has no way to represent these three close into a loop, because the loop is a property of no single record. Similarity ranks records against a query. It cannot rank a relationship that isn't written in any of them.

FAILURE 2 · QUERY RETRIEVAL

The database can compute it, but only if someone already knew to ask.

A recursive CTE (common table expression) with cycle detection finds the loop in milliseconds, and it comes with the thing similarity search cannot offer: a definite answer, and a path you can read back to check it.

To write it, though, you have to already suspect a cycle, already know that a cycle is what no beneficial owner reduces to, and already know which of the forty ownership-shaped columns is the one to walk.

A query is a question made precise. The analyst holding the question cannot express it that way, and the engine will not volunteer it. SQL doesn't surface findings; it confirms hypotheses you already have.

The data supports the query and the question gets answered are not the same statement.

So similarity retrieval cannot reliably assemble the relevant records, and cannot tell you when it failed. Query retrieval can do both, and cannot be written by the person who needs it. The tempting fix is to be generous: skip the retrieval problem entirely, hand the model the whole neighborhood, and let it work the answer out.

05 Handing over the subgraph doesn't save it

Even given the exact right records, retrieve-then-generate is the wrong shape, because the finding now depends on the model noticing it, every time, with nothing to check.

Suppose you solve retrieval by brute force. Pull Acme's entire ownership neighborhood, every entity and every edge, and drop the whole subgraph into the context window. The model now has three companies and three ownership links sitting in front of it. Ask again: who ultimately owns Acme?

This is RAG (retrieval-augmented generation) in its most generous form, and it is still the wrong architecture. Having the loop in the context is not the same as knowing there is a loop. The model has to notice that Acme → Kestrel → Alder returns to Acme, understand that a returning chain means the ownership never grounds out in a person, and do that reliably, in prose, buried among whatever else got swept into the window.

On three clean nodes, it might. That might is the entire problem, and it is the same might as the last section, moved one stage downstream.

Because the real question isn't three nodes. It is a customer with forty holding entities across six jurisdictions, where control has to be aggregated along every branch and compared against the 25% threshold. A path of 40% × 70% × 90% comes to 25.2% and has to be reported. A path of 40% × 60% × 90% comes to 21.6% and does not. Those two branches look identical in prose, and there are two hundred of them.

Read all of it and reason it out degrades exactly where it matters, and it degrades invisibly. The model returns a confident paragraph either way, and nothing in the output tells you whether it multiplied the percentages down each branch or pattern-matched something plausible.

The data was all there. Retrieve-then-generate fails because the finding is a computation over structure, and generation was asked to notice it rather than perform it.

It's worth separating two things here. A cycle. An aggregated ownership percentage. A shortest path between two parties. A set that breaches a threshold. These are things you compute, deterministically, and then verify by reading back the path that produced them. They are not things you hope a model spotted in a wall of retrieved text.

A better retriever puts more into the window. It does nothing about the fact that the answer has to be reasoned out of the structure, reliably and checkably.

06 The architecture is wrong, not the components

Both stages of the standard pattern miss, and the thing that would close the gap was never written down.

Put the two halves together and the standard pattern collapses. Retrieval ranks by similarity, so whether the record that matters arrives is a matter of luck, and nothing announces its absence. Generation is handed whatever did arrive and asked to find the structure by reading, which it does unreliably and unverifiably. Fetch, then generate: both stages miss, and both miss quietly.

That pattern has a name, and it is the default way agents get built today. RAG is superb when the answer is sitting in a document and the job is to find the document. It is the wrong architecture the moment the answer is a property of how things connect, because neither of its two moves is the move the question needs. Retrieval doesn't traverse, and generation narrates rather than computes.

The fair objection at this point is that all of this is solved. Graph databases traverse relationships natively, recursive SQL finds the loop, any competent engineer writes the query in an afternoon. All true, and all beside the point, because a query is an answer to a question you already knew how to ask, and the ownership loop only gets caught by someone who already suspected it was there.

So the problem splits in two. The analyst staring at the alert knows what to look for but cannot express it in SQL. The engineer who can write the SQL doesn't know that a loop is what no beneficial owner reduces to. The person with the question and the person who can pose it to the machine are never the same person.

Which points back at the first section. What the analyst knows, that these pointers are an ownership structure, that walking them is what tracing means, that a chain closing on itself is a finding rather than a data error, has never been written anywhere a machine can read. The schema certainly doesn't systematically hold it. A schema is enough to store the column correctly and says nothing whatsoever about what the column means.

Write those five statements down in a form a machine can evaluate and the question changes shape. No beneficial owner stops being a matter of phrasing and gains a definition: walk the ownership relation out from this entity, return the natural persons at the ends, and if the walk closes on itself without reaching one, that is the finding, and here is the path it took.

The analyst no longer needs to know what a cycle is. The engineer no longer needs to know what a beneficial owner is. Neither has to become the other, because both halves are written down once, in the same place.

That account of what the entities are, what the relations mean, and what rules hold over them is an ontology. The schema describes storage, and the knowledge graph is the data arranged in that shape. The ontology is the layer that makes the question answerable at all, and it is the one most stacks never needed until an agent turned up to ask.

None of this helps the analyst from the first section, not yet. The alert is still open and the chain still doesn't reach a person. What would close it is somewhere for the domain knowledge to live, rather than a bigger model or a faster engine: ownership defined once, tracing defined once, and an agent that can walk those definitions and hand back the path it took. Part two of this series picks up on what it takes to build this layer.