Skip to main content
Back to the Library
Coding

Design a Retrieval-Augmented Generation Architecture

Create an implementable retrieval-augmented generation architecture covering governed ingestion, real citations, tool boundaries, and an acceptance plan.

How to use this prompt

Use this prompt to design a robust, production-ready Retrieval-Augmented Generation (RAG) system. Fill in your data constraints, latency requirements, and infrastructure preferences. The model returns a comprehensive architectural blueprint complete with ingestion pipelines, citation mechanisms, and failure modes.

The prompt

## Role & objective
You are a principal systems architect specializing in production-grade LLM applications. Your objective is to design a robust, implementable Retrieval-Augmented Generation (RAG) architecture that addresses data governance, accurate citation tracking, strict tool boundaries, and verification.

## Inputs
- Use case and domain: [e.g., internal HR policy bot, external customer technical support]
- Source data types and volumes: [e.g., 500k PDF manuals, dynamic SQL databases, Confluence pages]
- Target infrastructure constraints: [e.g., AWS, on-premise Kubernetes, latency limits under 2 seconds]
- Security and compliance requirements: [e.g., SOC2, PII redaction, role-based access control]

## Instructions
1. Before writing the design, briefly outline your core architectural assumptions regarding document chunking strategies, embedding models, and vector storage.
2. Design the ingestion and governance pipeline: detail chunking, metadata enrichment, embedding generation, and access-control filtering.
3. Design the retrieval and generation loop: specify query transformation, hybrid search (keyword + semantic), re-ranking, context window assembly, and prompt construction.
4. Establish citation mechanics: explain how retrieved chunks map to explicit inline citations in the final output without hallucinating source IDs.
5. Define tool boundaries: outline what queries must trigger fallback behaviors or external tool calls rather than relying solely on vector search.
6. Create an acceptance and evaluation plan: define specific metrics (e.g., retrieval relevance, hallucination rate) and test datasets.

## Constraints
- If any critical input is missing or ambiguous, ask 1 to 2 clarifying questions before producing the final output.
- Avoid hand-wavy solutions; specify concrete toolchains, database types, and algorithmic approaches.
- Quality bar: An engineer should be able to hand this design to an implementation team to begin sprint planning immediately.

## Output format
Provide the design using clear Markdown headings matching the instruction steps, including a ASCII data-flow diagram of the ingestion and query pipeline.