What matters in AI.

Subscribe

Learn / AI basics

Definition · AI basics

Reranker

A reranker is a model that reads a query and the text of each candidate result together, at query time. It gives each candidate a relevance score. The search system then sorts the candidates again and keeps the best. A reranker is the second step of a search: a fast first step finds the candidates.

Last reviewed

Key points

  • A reranker is the second step of a search. A fast first step, such as BM25 or an embedding search, finds the candidates.
  • A cross-encoder reranker reads the query and one text together and gives a score. It is more accurate than embeddings, but too slow for a full collection.
  • In an Anthropic test (September 2024), retrieval with a reranker had 67% fewer failures. With no reranker, the decrease was 49%.
  • A reranker reads the text of each document. Thus, a document can attack it. Prompt injection in one candidate can change the result of an LLM reranker.

How a reranker works

A reranker is the second of two steps in a search.

  1. Find candidates. BM25 keyword search or an embedding model search finds a short list, for example the top 100.
  2. Score each candidate. The reranker reads the query and the text of one candidate together. It gives a relevance score.
  3. Sort and cut. The system sorts by score and keeps the top results. In a RAG system, these go into the prompt of the model.
A reranker scores the short list that a fast first step finds, and keeps only the best.
A reranker scores the short list that a fast first step finds, and keeps only the best.

The numbers in the diagram come from the Anthropic test of September 2024: 150 candidates in, 20 out.

Why a reranker matters: speed vs accuracy

No single model is both fast and accurate on a full collection. Thus, search uses two steps.

First step: bi-encoder Reranker: cross-encoder
Input one text at a time query and text together
Output one embedding for each text one score for each pair
Speed fast slow
Accuracy lower higher

The Sentence Transformers documentation gives a scale. For 10,000 sentences, a cross-encoder needs about 50 million pair scores, about 65 hours. A bi-encoder makes the 10,000 embeddings in about 5 seconds.

Security risks of a reranker

A reranker reads the full text of each candidate. An attacker who controls one document can thus write text that the reranker reads.

Yin and colleagues (February 2026) put prompt injection text into candidate documents for LLM rerankers. They found:

  • Simple injected text can change the result of an LLM reranker a lot.
  • “Larger, more capable LLMs are generally more vulnerable.”
  • Encoder-decoder models (Flan-T5) are “substantially more robust than decoder-only LLMs”.

A second study by the same group (May 2026) sent attacks through a full pipeline: retriever, LLM reranker, then generator. Gradient-based and instruction override attacks usually did not go through to the generator. LLM-driven prompt injections got through to the generator.

Three types of reranker

Each type makes a different trade off.

  • Cross-encoder. A small transformer trained to score a pair. Nogueira and Cho (January 2019) made a BERT reranker of this type. On MS MARCO, it was 27% (relative) better in MRR@10 than the previous best.
  • LLM reranker. A language model gets the query and the candidates in a prompt and gives a sequence. Sun and colleagues (EMNLP 2023) found that ChatGPT and GPT-4 can equal or do better than supervised methods. A distilled 440M model did better than a 3B supervised model on BEIR.
  • Hosted reranker API. A vendor operates the model. Cohere Rerank takes a query and a list of documents. It gives back a relevance_score for each. Its models include rerank-v3.5 and rerank-v5.0-pro.

Questions and answers

What is the difference between a reranker and an embedding model?

An embedding model turns each text into a vector alone. A system can then compare vectors fast across a full collection. A cross-encoder reranker reads the query and one text together and gives a score. It is more accurate, but it is too slow for large data sets. Most systems use the two in sequence.

Can a reranker stop prompt injection or RAG poisoning?

No. A reranker scores relevance, not safety. Yin and colleagues (2026) found that injected text in a candidate can change the result of an LLM reranker. In a full pipeline of retriever, LLM reranker and generator, many attacks did not get to the generator. LLM-driven prompt injections went through to the generator.

Does a reranker make a search slower?

Yes, by a small quantity. Anthropic (September 2024) says that a reranker adds latency, also when it scores all chunks in parallel. More candidates give better results but more latency and cost.

Sources

  1. Cross-Encoders, Sentence Transformers documentationSentence Transformers (sbert.net)
  2. Passage Re-ranking with BERTarXiv (Nogueira and Cho), 13 Jan 2019
  3. Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking AgentsarXiv (Sun et al., EMNLP 2023), 19 Apr 2023
  4. Introducing Contextual RetrievalAnthropic, 19 Sep 2024
  5. Rerank overviewCohere
  6. The Vulnerability of LLM Rankers to Prompt Injection AttacksarXiv (Yin, Wang, Koopman and Zuccon), 18 Feb 2026
  7. Can It Reach the Generator? Investigating the Survival of Prompt-Injection Attacks in Realistic RAG SettingsarXiv (Yin, Wang, Koopman and Zuccon), 27 May 2026

In the news