More documents cut RAG precision by 63% in a Wyoming system
Soft scoping by domain made the problem less, but hard partitioning was dangerous.
Claimed, not confirmed
Researchers report that retrieval-augmented generation (RAG) has less precision on large and mixed document collections. They saw it in a deployed system of the Wyoming Department of Transportation. When the corpus went from 54 to 1,128 documents, precision at 10 fell by 63% and recall fell by 23%. They name the failure vector search dilution.
Sources
Posted
