RAG architecture audit on an AI solution already in production
Full audit of a production RAG pipeline, from chunking to source attribution.
The challenge
A document AI solution in production returned plausible but sometimes poorly referenced answers, without the team being able to identify whether the problem came from the data, the retrieval or the model.
Our response
We audited the full pipeline, from document ingestion through to citation rendering. The analysis isolated two structural flaws independent of the model: document chunking carried out without preserving the hierarchical attachment of passages, and a source list displayed without post-generation filtering, therefore including references not used in the answer. We delivered the diagnosis, the prioritisation of fixes and the target architecture.
Key points
End-to-end audit: ingestion, segmentation, indexing, retrieval, generation, citation
Identification of structural flaws independent of the model choice
Recommendations prioritised by impact and cost of remediation
Documented and transferable target architecture
Technical stack
- Ingestion and segmentation pipeline analysis
- Retrieval relevance evaluation
- Source attribution diagnosis
- Benchmark of embeddings and reranking strategies
- Reproducible evaluation protocol
Sector
Technical advisory & auditLet's talk about your project
Tell us about your business challenge and we'll explore together how a tailored AI solution could support it.