Vetradocs

AI Configuration

Customize the RAG pipeline, system prompts, search settings, and UI

AI Configuration

This page covers how to customize the AI behavior in VectraDocs. The AI is built on LangChain and uses Orama for client-side vector search.

Which Version Are You Using?

This page is for the main VectraDocs Next.js starter.

If you're using VitePress, Docusaurus, or Scalar plugins with an external backend, see Backend Setup instead.


How It Works

VectraDocs uses Retrieval-Augmented Generation (RAG):

  1. User asks a question → The frontend sends the message to /api/chat
  2. Search relevant docs → Orama searches the documentation index
  3. Build context → The top results are combined into a context string
  4. Generate response → LangChain sends the question + context to the LLM
  5. Stream response → The AI's answer is streamed back to the user

1. Customizing the System Prompt

The AI's "personality" is defined in app/api/chat/route.ts.

Find the ChatPromptTemplate:

const prompt = ChatPromptTemplate.fromMessages([
    ["system", `You are a helpful documentation assistant for "VectraDocs".
    Answer questions based ONLY on the provided documentation context.
    
    Context:
    {context}
    
    IMPORTANT RULES:
    1. Only answer based on the provided context
    2. If the answer is not in the context, apologize and say you don't have information
    3. Be concise but helpful
    4. Include code examples when relevant
    5. At the END of your response, ALWAYS include a "📚 References" section
    6. Use the EXACT URLs from the context (relative paths like /docs/installation)
    7. Format references as markdown links: [Page Title](/docs/page-path)
    
    Example reference section:
    
    📚 **References:**
    - [Installation Guide](/docs/installation)
    - [Backend Setup](/docs/backend-setup)`],
    new MessagesPlaceholder("history"),
    ["human", "{input}"],
]);

Automatic References

The AI automatically includes documentation references at the end of each response. URLs are relative paths (like /docs/installation) so they work on any domain.

Customization Ideas

GoalPrompt Addition
Brand voice"You represent the ExampleCo brand. Be friendly and professional."
Code language"Always provide code examples in TypeScript."
Restrict topics"Only answer questions about our product. Decline unrelated questions."
Response length"Keep responses under 200 words unless code is needed."
Include sources"At the end, list the documentation pages you referenced."

2. Adjusting Search Settings

The RAG system uses Orama to find relevant documentation. You can tune this in app/api/chat/route.ts:

// Search for relevant documents
const searchResult = await search(db, { 
  term: lastMessage, 
  limit: 3  // Number of results to include
});

Settings

ParameterDefaultDescription
limit3Number of doc sections to include in context
tolerance0.5Fuzzy matching tolerance (0-1)
threshold0Minimum score to include a result

Recommendations

  • Simple docs: limit: 3 is usually enough
  • Complex docs: Increase to limit: 5-10
  • Token budget concerns: Keep limit low, use summaries
  • Accuracy issues: Increase limit and add threshold: 0.3

3. Switching LLM Providers

VectraDocs uses environment variables to configure the LLM:

LLM_BASE_URL="https://api.openai.com/v1"
LLM_API_KEY="sk-..."
LLM_MODEL="gpt-4o"

Supported Providers

ProviderBase URLModels
OpenAIhttps://api.openai.com/v1gpt-4o, gpt-4o-mini, gpt-3.5-turbo
Anthropichttps://api.anthropic.com/v1claude-3-sonnet, claude-3-opus
Groqhttps://api.groq.com/openai/v1llama-3.1-70b-versatile
Together AIhttps://api.together.xyz/v1meta-llama/Llama-3-70b-chat
Ollamahttp://localhost:11434/v1llama3, mistral, etc.

Using Anthropic (Claude)

For deeper Claude integration:

npm install @langchain/anthropic

Update app/api/chat/route.ts:

import { ChatAnthropic } from "@langchain/anthropic";

const model = new ChatAnthropic({
  model: "claude-3-sonnet-20240229",
  apiKey: process.env.ANTHROPIC_API_KEY,
});

4. UI Customization

The chat UI lives in components/ai-chat.tsx.

Accent Color

Change the primary color throughout the component:

// Find and replace occurrences of:
border-orange-500  →  border-purple-500
bg-orange-500      →  bg-purple-500
text-orange-500    →  text-purple-500

Floating Bar Position

Modify the position in the floating bar styles:

// Bottom center (default)
className="fixed bottom-8 left-1/2 -translate-x-1/2"

// Bottom right
className="fixed bottom-8 right-8"

// Bottom left
className="fixed bottom-8 left-8"

Markdown Rendering

Customize how AI responses render:

<ReactMarkdown
  components={{
    h1: ({ children }) => <h1 className="text-2xl font-bold mt-4">{children}</h1>,
    h2: ({ children }) => <h2 className="text-xl font-semibold mt-3">{children}</h2>,
    code: ({ children }) => (
      <code className="bg-zinc-800 px-1.5 py-0.5 rounded text-orange-400">
        {children}
      </code>
    ),
    a: ({ href, children }) => (
      <a href={href} className="text-orange-500 hover:underline">
        {children}
      </a>
    ),
  }}
>
  {message.content}
</ReactMarkdown>

5. Adding Hidden AI Context

You can add hidden metadata in your MDX files that the AI can search but users won't see:

---
title: My Page
description: A page about something
---

# My Page

{/* AI Context: This page covers X, Y, and Z. Related topics include A and B. Common questions: How do I...? What is...? */}

Visible content here...

This is powered by MDX comments ({/* ... */}) which are:

  • ✅ Indexed by the search system
  • ✅ Available to the AI as context
  • ❌ Not rendered to users

6. Using Vector Databases

For larger documentation sites, you might want to use a cloud vector database:

Pinecone

npm install @pinecone-database/pinecone @langchain/pinecone
import { Pinecone } from "@pinecone-database/pinecone";
import { PineconeStore } from "@langchain/pinecone";

const pinecone = new Pinecone({ apiKey: process.env.PINECONE_API_KEY });
const index = pinecone.Index("your-index");

const vectorStore = await PineconeStore.fromExistingIndex(embeddings, {
  pineconeIndex: index,
});

const retriever = vectorStore.asRetriever({ k: 5 });

Supabase

npm install @supabase/supabase-js @langchain/community

Check LangChain docs for Supabase vector store integration.


Troubleshooting

AI gives generic or wrong answers

  1. Check that npm run build:index was run after editing docs
  2. Increase limit in search settings
  3. Add more specific content to your documentation

AI takes too long to respond

  1. Use a faster model (gpt-4o-mini vs gpt-4o)
  2. Reduce limit in search settings
  3. Use a local model with Ollama

AI refuses to answer valid questions

  1. The answer might not be in your docs - add it!
  2. Adjust the system prompt to be less restrictive
  3. Check if the question matches indexed content

Next Steps

Ctrl+I
Assistant

How can I help?

Ask me about configuration, installation, or specific features.