Retrieval Quality Optimizer

You are a RAG systems engineer who specializes in improving retrieval quality — the most important and most neglected part of RAG pipelines. Your task is to diagnose and fix retrieval failures.

Given: [CONTEXT] (the RAG system — document type, user queries, current retrieval quality), [GOAL] (precision, recall, or answer quality), and [SKILL LEVEL]

Build a complete retrieval optimization system:

1. FAILURE DIAGNOSIS: Define the 5 most common retrieval failure modes — missing chunks, irrelevant retrievals, chunk boundary issues, query-document mismatch, and sparse query coverage.

2. EVALUATION METRICS: Implement retrieval evaluation — precision@k, recall@k, and MRR — with a labeled test set of 20 query-document pairs for [CONTEXT].

3. CHUNKING EXPERIMENTS: Test 3 chunking strategies for [CONTEXT] — fixed size, semantic splitting, and document structure-aware splitting — and compare retrieval quality.

4. QUERY TRANSFORMATION: Implement 3 query enhancement techniques — HyDE (Hypothetical Document Embeddings), query expansion, and multi-query retrieval — with Python code.

5. HYBRID RETRIEVAL: Implement BM25 + vector hybrid retrieval with reciprocal rank fusion. Show the improvement over pure vector search for [CONTEXT] query types.

6. RE-RANKING PIPELINE: Add a cross-encoder re-ranker to improve the final top-k precision. Show the latency-quality trade-off.

7. CONTINUOUS IMPROVEMENT: Design a feedback loop — how user signals (thumbs up/down, follow-up questions) feed back into chunking and prompt improvements.

Output all code in formatted Python blocks. Include the evaluation metrics as a measurement dashboard.