Embedding

How it works

An embedding model reads a piece of text and outputs a fixed-length list of numbers, a vector. The model is trained so that texts with similar meaning get vectors that point in similar directions. A simple calculation on two vectors, such as cosine similarity, scores how closely their directions match.

Search by meaning then becomes a nearest-neighbour lookup. In retrieval augmented generation, documents are split into chunks, each chunk is embedded and stored in a vector database, and the question is embedded the same way. The chunks whose vectors sit nearest the question’s are handed to the language model.

Why it matters

An embedding is a compressed trace of its text, not an anonymised one. Some systems send only embeddings to a hosted vector database, which is safe only if the numbers cannot be turned back into words.

In 2020 Song and Raghunathan recovered 50 to 70% of the words in a sentence from popular sentence embeddings, though not their order. In 2023 a Cornell team’s method, Vec2Text, went further. The attacker trains a model on text and embedding pairs from the target embedding model, then repeatedly queries that model to refine a guess. It recovered 92% of 32-token Wikipedia passages exactly from one open embedding model, and 61% of similar-length web passages from OpenAI’s ada-002. At up to 128 tokens exact recovery fell to 8%, and the study did not test longer text. On short clinical notes seeded with fake names, it recovered 89% of the full names.

OWASP’s 2025 LLM Top 10 lists embedding inversion under LLM08, Vector and Embedding Weaknesses, beside leakage between users who share one vector database. Its mitigations start with permission-aware vector and embedding stores. The Cornell authors go further: protect a store of embeddings as you would the text it came from.

Questions and answers

Can you get the original text back from an embedding?

Often, for short text. A 2023 Cornell study trained an attack model that recovered 92% of 32-token Wikipedia passages exactly from one embedding model, and 61% of similar-length web passages from OpenAI's ada-002. At up to 128 tokens it recovered 8% exactly, and it did not test longer text.

Are embeddings anonymised data?

No. The Cornell researchers behind Vec2Text, a 2023 embedding inversion attack, concluded that embeddings should be protected in the same way as the raw text, and OWASP lists embedding inversion as a risk in its 2025 LLM Top 10.