# reranker Also called: cross-encoder reranker, cross-encoder, rerank model, second-stage ranker, neural reranker A reranker is a model that takes a query and a short list of candidate passages and returns them scored by relevance to the query. It runs after a first-stage retriever, usually vector search or BM25, and reorders that retriever's top results before they reach the language model. The reranker scores each query and passage together in one forward pass, which is why its ordering beats the bi-encoder similarity that produced the candidates. ## What is a reranker and where does it sit in the pipeline? A retrieval pipeline usually has two stages. The first stage sweeps a large corpus and returns the top-k candidates fast, using a vector index over embeddings or a keyword index like BM25. That stage is cheap per document, and it scores the query and each document in isolation. The reranker is the second stage: it takes the query and each candidate together, and returns a relevance score per pair. Providers expose this as a single call. Cohere's Rerank endpoint takes a query and a list of documents and returns them with a relevance_score per document; Voyage AI's rerank endpoint accepts up to 1000 documents and 8000 query tokens and returns results sorted by descending score. Architecturally most rerankers are cross-encoders. The model concatenates the query and one candidate, runs the pair through a transformer, and emits a single relevance score from a regression head instead of a next-token distribution. That is more expensive per pair than a dot product between two precomputed vectors, which is why nobody uses a cross-encoder as a first stage. Rerankers are called on 20 to 200 candidates, not 20 million documents. Open cross-encoder rerankers exist too. BAAI publishes bge-reranker-base and bge-reranker-large in the FlagEmbedding repository, and the project's own documentation calls the reranker more accurate but less efficient than the bi-encoder embedding models that feed it. That is the trade in one line: precision at the top of the list, in exchange for a per-candidate model call at query time. ## Why does a reranker beat the vector search that fed it? An embedding model runs once on the query and once per document, produces two vectors, and reduces the match to a dot product. The two encodings never meet. A cross-encoder reranker runs one forward pass over the concatenated pair, so every attention layer sees both sides at once. That interaction picks up matches vectors miss: rare terms, negations, entity constraints, and the way a phrase modifies the noun beside it. It also lets the model weigh the query and the passage against each other rather than against the whole training distribution, which is how a bi-encoder scores. The cost is that you cannot precompute anything. A vector index answers a query in milliseconds because the document vectors are already on disk. A reranker has to run the model at query time for every candidate, which is why the pipeline hands it a short list rather than the whole corpus. Common shapes are 100 candidates in and 5 out, or 50 in and 10. ## When is a reranker worth the extra call? A reranker earns its place when the first stage returns roughly the right neighbourhood but the order inside that neighbourhood is wrong. If recall at 50 is high and precision at 5 is low, the right passage is in the list but ranked below a near-duplicate; that is what a reranker fixes. If recall at 50 is already low, the reranker cannot recover documents the retriever never fetched. Fix chunking, the embedding model, or the query rewrite step first. The other factor is latency and cost. A rerank call adds a network round trip and a per-candidate charge that scales with query and document length. Passing 200 candidates to a hosted reranker costs meaningfully more time and dollars than passing 25, and the marginal precision gain flattens fast. If the answer is user-facing and the LLM call is already the slow step, an extra rerank hop usually pays. If the pipeline runs over millions of queries offline, the reranker can cost more than the retriever it sits behind. ## Rerank narrows 25 vector-search hits to the top 3 the LLM sees import cohere co = cohere.ClientV2() query = "how do I rotate the signing key without downtime?" candidates = [ "Rotate the JWT signing key by publishing the new key first", "The signing key is stored in the KMS keyring named auth", "Deployment downtime windows are scheduled monthly on Sunday", # ... 22 more chunks returned by the vector index ] resp = co.rerank( model="rerank-v3.5", query=query, documents=candidates, top_n=3, ) # resp.results, sorted by relevance_score descending: # index= 0 relevance_score=0.94 "Rotate the JWT signing key by publishing..." # index= 7 relevance_score=0.71 "Serve both keys during the overlap window..." # index= 1 relevance_score=0.42 "The signing key is stored in the KMS keyring..." The retriever placed chunk 7 fifth in its list. The reranker moves it to second because the query is about doing the rotation without downtime, which the phrase overlap window matches and cosine similarity between two isolated embeddings did not. ## Frequently asked Q: Is a reranker just a smaller LLM? A: No, though they share the transformer core. A reranker is trained to output one number per query and document pair, and its output layer is a scalar regression head rather than a next-token distribution. That specialisation is why a few-hundred-million-parameter cross-encoder can outrank a general 7B model on relevance judgement while costing far less per call. Some newer rerankers do wrap an LLM and ask it to score the pair in text; those are called LLM-as-reranker and cost roughly the same per candidate as a normal LLM call. Q: Do I need a reranker if my embeddings are already good? A: Test both on your own data. Measure recall at 50 and precision at 5 on a labelled set from your corpus. If recall is high and precision is low, a reranker will help. If recall is already low, the reranker has nothing to work with because the retriever did not surface the right passage. Fix chunk size, the embedding model, or query rewriting before adding a rerank stage. Q: How many candidates should I pass to the reranker? A: Enough that the relevant passage is almost certainly in the list, and no more. The right number depends on your recall curve, but 25 to 100 is the usual range for hosted APIs. Passing 1000 candidates rarely pays for itself: the reranker cost scales linearly with the number of candidates, while the incremental precision from candidate 500 down to 1000 is close to zero on most corpora. Voyage caps the request at 1000 documents, which is a ceiling rather than a target. Q: Is reranking the same as reciprocal rank fusion? A: No. Reciprocal rank fusion combines the ranked lists from two or more retrievers by summing 1 divided by (k plus rank) across lists, and never runs a model. A reranker replaces those ranks with fresh scores from a cross-encoder over the query and document text. The two compose: a common pipeline runs BM25 and dense retrieval, fuses the lists with RRF, and reranks the fused top-k with a cross-encoder before the LLM sees it. ## Sources - Cohere docs: Rerank overview: https://docs.cohere.com/docs/rerank-overview the hosted Rerank endpoint, its inputs and outputs, and the relevance_score field - Voyage AI docs: Reranker: https://docs.voyageai.com/docs/reranker the 1000-document and 8000-token limits, the RerankingResult schema and score semantics - FlagEmbedding (BAAI): BGE reranker: https://github.com/FlagOpen/FlagEmbedding open cross-encoder rerankers (bge-reranker-base, bge-reranker-large) and their distinction from bi-encoder embedding models ## Related terms - vector database: https://ai.developertools.io/vector-database/index.md - embedding: https://ai.developertools.io/embedding/index.md - RAG: https://ai.developertools.io/rag/index.md - chunking: https://ai.developertools.io/chunking/index.md - latency: https://ai.developertools.io/latency/index.md Canonical page: https://ai.developertools.io/reranker/ Edited by Derek Wonk. Last updated 2026-09-23.