Build a RAG Chatbot with Next.js in 2026

RAG (Retrieval-Augmented Generation) is the technique behind every custom AI chatbot that answers questions about YOUR data - your documents, your website, your knowledge base.

This tutorial walks you through building a RAG chatbot with Next.js, OpenAI, and a vector database from scratch.

What is RAG?

Regular ChatGPT only knows what it was trained on. RAG fixes this by:

1. Storing your data as vector embeddings in a database
2. When a user asks a question, finding the most relevant data chunks
3. Sending those chunks + the question to an AI model
4. The AI generates an answer based on YOUR data, not just its training

Use cases:

• Customer support bot trained on your help docs
• Internal knowledge base chatbot for employees
• Study assistant trained on your course materials
• Documentation chatbot for your product
• Legal assistant trained on case documents

Architecture Overview

User Question
     ↓
[1. Embed Question] → Convert to vector using OpenAI
     ↓
[2. Vector Search] → Find similar document chunks in database
     ↓
[3. Build Prompt] → Combine question + relevant chunks
     ↓
[4. Generate Answer] → Send to GPT model
     ↓
AI Response (grounded in your data)

Prerequisites

• Node.js 18+
• Basic React/Next.js knowledge
• OpenAI API key
• Neon or Supabase account (for PostgreSQL + pgvector)

Step 1: Project Setup

Create a new Next.js app:

npx create-next-app@latest rag-chatbot --[typescript](/blog/typescript-for-beginners-2026-complete-guide) --tailwind --app
cd rag-chatbot

Install dependencies:

npm install openai @neondatabase/serverless ai drizzle-orm
npm install -D drizzle-kit

Key packages:

• openai - OpenAI SDK for embeddings and chat
• @neondatabase/serverless - PostgreSQL with pgvector support
• ai - Vercel AI SDK for streaming responses
• drizzle-orm - type-safe database queries

Step 2: Database Schema

Create a table to store document chunks and their vector embeddings.

Enable pgvector in your Neon database:

CREATE EXTENSION IF NOT EXISTS vector;

Create the documents table:

CREATE TABLE documents (
  id SERIAL PRIMARY KEY,
  content TEXT NOT NULL,
  metadata JSONB DEFAULT '{}',
  embedding vector(1536)
);

CREATE INDEX ON documents USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100);

What each column does:

• content - the actual text chunk from your document
• metadata - source file, page number, section title (JSON)
• embedding - 1536-dimensional vector from OpenAI's embedding model

Step 3: Document Ingestion

Before your chatbot can answer questions, you need to ingest your documents into the vector database.

The ingestion process:

1. Split documents into chunks (500-1000 tokens each)
2. Generate embeddings for each chunk using OpenAI
3. Store chunks and embeddings in the database

Why chunking matters:

• AI models have context limits (cannot send entire documents)
• Smaller chunks = more precise retrieval
• Overlap between chunks prevents missing context at boundaries
• Optimal chunk size: 500-1000 tokens with 100-token overlap

Chunking strategies:

• Fixed size: Split every N characters (simplest)
• Semantic: Split at paragraph/section boundaries (better quality)
• Recursive: Try paragraph, then sentence, then character splits (best)

Step 4: Embedding Generation

OpenAI's text-embedding-3-small model converts text to 1536-dimensional vectors.

How embeddings work:

• Similar text produces similar vectors
• "How to deploy React" and "React deployment guide" produce very close vectors
• "Best pizza recipe" produces a very different vector
• Vector similarity (cosine distance) measures how related texts are

Batch processing tips:

• Process in batches of 100-500 chunks
• Add rate limiting to avoid API errors
• Store embeddings immediately after generation
• Cost: ~$0.02 per million tokens (very cheap)

Step 5: Building the Chat API

Create an API route that handles the RAG pipeline:

The RAG pipeline in code:

1. Receive user question
2. Generate embedding for the question
3. Query vector database for similar chunks
4. Build prompt with context + question
5. Stream response from GPT model

Important considerations:

• Top-K retrieval: Retrieve 3-5 most similar chunks (not too many)
• Similarity threshold: Ignore chunks below 0.7 similarity score
• Context window: Ensure total prompt fits within model's context limit
• System prompt: Instruct the model to only answer based on provided context
• Streaming: Use Server-Sent Events for real-time response display

System prompt example:

You are a helpful assistant. Answer questions based ONLY on the provided context.
If the context does not contain enough information to answer, say "I don't have
enough information to answer that question."
Do not make up information that is not in the context.

Step 6: Frontend Chat Interface

Build a chat UI with message history, loading states, and streaming responses:

UI components needed:

• Message list (user and AI messages)
• Input field with send button
• Loading indicator while AI responds
• Scroll to bottom on new messages
• Source citations (which documents were used)

Using Vercel AI SDK: The AI SDK provides the useChat hook that handles:

• Message state management
• Streaming response display
• Loading states
• Error handling
• Message history

Step 7: Adding Source Citations

A good RAG chatbot shows WHERE the answer came from:

Implementation:

1. Return source metadata alongside the AI response
2. Display source links/references below the answer
3. Let users click to see the original document
4. Show confidence score for transparency

This builds trust - users can verify answers by checking the original source.

Step 8: Optimization

Improving Answer Quality

1. Better chunking - semantic chunking > fixed-size chunking
2. Hybrid search - combine vector search with keyword search
3. Re-ranking - use a cross-encoder to re-rank retrieved chunks
4. Conversation history - include previous Q&A in the prompt
5. Metadata filtering - filter by document type, date, or category

Improving Performance

1. Cache embeddings - do not regenerate for unchanged documents
2. Cache frequent queries - store responses for common questions
3. Use streaming - show response as it generates
4. Optimize vector index - IVFFlat or HNSW index for faster search
5. Limit context size - only send the most relevant chunks

Cost Optimization

• Use text-embedding-3-small ($0.02/M tokens) instead of ada-002
• Cache embeddings for unchanged documents
• Use GPT-4o-mini for simple questions, GPT-4o for complex ones
• Implement response caching for frequently asked questions

Deployment

Deploy to Vercel:

vercel deploy

Environment variables needed:

• OPENAI_API_KEY - your OpenAI API key
• DATABASE_URL - your Neon PostgreSQL connection string

Production checklist:

• Rate limiting on the API route
• Error handling for OpenAI API failures
• Loading and error states in the UI
• Input sanitization and validation
• Usage monitoring and cost alerts
• Document update pipeline (re-ingest when docs change)

Real-World Examples of RAG

• Notion AI - answers questions about your workspace
• GitHub Copilot Chat - understands your codebase
• Customer support bots - trained on help documentation
• Legal research tools - search case law and statutes
• Medical assistants - answer questions from medical literature

Common RAG Mistakes

1. Chunks too large - AI cannot find specific answers in 5000-token chunks
2. Chunks too small - lose context, answers are fragmented
3. No overlap - important information at chunk boundaries gets lost
4. Wrong embedding model - use text-embedding-3-small for best price/performance
5. No system prompt - AI will make up answers if not instructed to stay grounded
6. Ignoring metadata - source, date, and category help with retrieval quality
7. Not evaluating quality - test with real questions and measure accuracy

Related ByteVerse guides

Next, read Docker for Beginners 2026, Python AI Agent Tutorial 2026, Next.js 16 Deployment Guide 2026, React 19 Best Practices 2026, and Best AI Coding Assistants 2026 to build a stronger workflow around this topic.