Speculative RAG with a flush gate reduced P90 latency by 38%
Start the RAG answer before routing finished, then used a small gate to either flush the buffered tokens or throw them away.
Back to Blog
Posts tagged "rag"
3 posts found
PII Extraction for RAG: Building Secure Document Pipelines
Learn how to build secure RAG pipelines with PII extraction and scrubbing. Protect sensitive information while improving retrieval quality using BAML for structured validation.
Stop Tweaking Prompts—Fix Your Retrieval and Win
A Comprehensive RAG Evaluation