Rerankers: the step that decides which passages get cited

Retrieval and citation are usually described as one event. They are not. Retrieval produces a candidate list, typically far larger than anything an answer can use, and a second model then decides which few candidates survive. That second model is a reranker. It is the stage where most retrieved passages are discarded, and it is the stage the usual advice skips.

The two-stage pattern

The architecture is documented in the retrieval literature. Passage Re-ranking with BERT, published by Rodrigo Nogueira and Kyunghyun Cho in 2019, describes it directly: first, a large number of possibly relevant documents, for example a thousand, are retrieved from a corpus by a standard mechanism such as BM25; in the second stage, each of those documents is scored and re-ranked by a more computationally intensive method. The paper is a description of the technique, not of any commercial answer engine.

The division of labor is a cost decision. Stage one has to touch a very large index, so it must be cheap per item, which means approximate methods: keyword matching, or nearest-neighbor search over embeddings. Cheap methods buy recall. They are good at making sure the right passage is somewhere in the top thousand and poor at knowing which of the thousand is right. Stage two only has to touch the shortlist, so it can afford a model that reads properly.

Why embedding similarity is not enough

An embedding retriever encodes the query and each passage separately, then compares the two vectors. The passage vector is computed before the query exists, which is what makes the index fast, and also what makes it coarse. The passage is compressed into a fixed representation with no knowledge of what will be asked of it.

A cross-encoder does the opposite. The retrieve and re-rank documentation from the Sentence-Transformers project states that the query and a possible document are passed simultaneously to the transformer network, which outputs a single score. Reading both together lets the model check whether this passage answers this query, rather than whether the two are broadly about the same topic. The same documentation explains why it cannot be the only stage: scoring thousands or millions of query and document pairs would be slow, so a retriever produces a candidate set of around a hundred, which the cross-encoder then reranks.

In production reranking is offered as a standalone step. Cohere's rerank documentation describes the operation plainly: given a query and a list of documents, it orders the documents from most to least semantically relevant to the query and returns a relevance score for each. Anthropic's write-up on contextual retrieval calls reranking a commonly used filtering technique for ensuring that only the most relevant chunks are passed to the model, and reports that adding it to its own retrieval setup reduced failed retrievals further than retrieval improvements alone. Both statements describe developer retrieval products.

What a reranker in an answer system is judging

Relevance is the base criterion, and in a system that has to write a sourced answer it is not the only one. A passage can be on topic and still useless: it can restate the question, gesture at the answer, or discuss the subject without asserting anything. What the generation step needs is support. It needs a passage that makes the claim the answer is about to make, in words that can be attributed.

That distinction has a practical shape. A passage that names the subject and states the answer as a sentence is directly usable. A passage that merely contains the right keywords, distributed across three paragraphs of setup, is not. This is the mechanism behind advice that otherwise sounds like style preference. Answer-first structure outperforms keyword coverage because the selection stage is looking for a supporting statement, not a topical match. What makes a statement worth quoting once it is selected is covered in how to get cited by AI engines.

What this changes for a publisher

Being retrievable is necessary and not sufficient. The entry conditions described in how answer engines choose their sources get a page into the candidate pool. They do not get it through the filter that follows.

Two consequences follow. First, the unit under evaluation is the passage, not the page, so a strong page with no plainly stated answer loses to a weaker page that has one. How that unit is created is covered in chunking and how retrieval systems split a page. Second, the passage has to answer the query in its own words. A reranker reading query and passage together is not counting term overlap. A passage written to include a phrase, without asserting anything about it, has nothing for the scorer to reward.

How this compounds with query fan-out

Google's AI features guide describes query fan-out as a set of concurrent, related queries generated by the model to fetch additional relevant results for a single user question. Each of those sub-queries runs its own retrieval, and each retrieved set is scored on its own.

That multiplies the chances of selection. A page does not need to win the user's literal question. It needs to win one sub-query, and a page that covers the adjacent questions of a topic in separate, plainly answered sections presents a separate candidate to each pass. It also means a page can be cited for a question nobody typed. The practical instruction is the same in both directions: cover the neighboring questions properly, each under its own heading, each answered in its own words.

The honest limits

No major answer engine documents its reranking stack. The models, the size of the candidate set, the scoring thresholds and the diversity rules that stop one domain filling an answer are all private, and none of them can be inferred from the outside. Published sources describe the technique, and their numbers belong to their own benchmarks.

Treat any claim about how to rank inside a reranker as unverifiable. No third party has visibility into these systems, and the output of an answer engine is too unstable to reverse-engineer a scorer from, for the sampling reasons set out in measuring AI search visibility honestly. What is defensible is the mechanism: the selection stage reads query and passage together, and it rewards passages that answer.