RAG (Retrieval-Augmented Generation) Overview

RAG lets you upload your own documents and use them to ground LLM responses in facts from your knowledge base.

How It Works

User Question
     │
     ▼
POST /api/rag/search  ──► Qdrant vector DB  ──► Top-N relevant chunks
     │
     ▼
Inject chunks as system context
     │
     ▼
POST /api/v1/chat/completions  ──► LLM  ──► Grounded answer

1. Ingest — You upload a document. The platform extracts its text, splits it into overlapping chunks (~1 800 characters each), embeds each chunk using nomic-embed-text (via Ollama), and stores the vectors in Qdrant.
2. Retrieve — At query time, the platform embeds your question and finds the most similar chunks using cosine similarity.
3. Generate — You pass the retrieved chunks as a system message alongside the user’s question when calling the chat completions API.

Prerequisites

• USE_DATABASE=true — RAG requires PostgreSQL for document metadata storage
• A running Qdrant instance (configured via QDRANT_URL, default http://qdrant:6333)
• The nomic-embed-text model pulled into Ollama (or a custom model set via EMBEDDING_MODEL)

RAG is scoped to projects — each project gets its own isolated vector collection. Documents from one project are never visible to another.

Supported File Types

Format	MIME type
Plain text	text/plain and any text/*
PDF	application/pdf

Chunking Details

Parameter	Value
Chunk size	~1 800 characters
Overlap	200 characters
Embedding model	nomic-embed-text (configurable)
Similarity metric	Cosine

Next Steps

• Managing Documents — upload, list, and delete documents
• RAG-Enhanced Queries — retrieve context and call the LLM