LlamaIndex

LlamaIndex is a popular framework for working with AI, Vectors, and embeddings. LlamaIndex supports using Neon as a vector store, using the pgvector extension. Initialize Postgres Vector Store. LlamaI...

LlamaIndex is a popular framework for working with AI, Vectors, and embeddings. LlamaIndex supports using Neon as a vector store, using the pgvector extension.

Initialize Postgres Vector Store

LlamaIndex simplifies the complexity of managing document insertion and embeddings generation using vector stores by providing streamlined methods for these tasks.

Here's how you can initialize Postgres Vector with LlamaIndex. OpenAIEmbedding works against either OpenAI directly or the Neon AI Gateway, which serves embedding models on an OpenAI-compatible endpoint using the same Neon credential as the rest of your project:

TSX

// File: vectorStore.ts

import { OpenAIEmbedding, Settings } from 'llamaindex';
import { PGVectorStore } from 'llamaindex/storage/vectorStore/PGVectorStore';

Settings.embedModel = new OpenAIEmbedding({
  model: 'qwen3-embedding-0-6b', // 1024-dimensional; supports the `dimensions` parameter
  dimensions: 512,
  apiKey: process.env.NEON_AI_GATEWAY_TOKEN,
  additionalSessionOptions: {
    baseURL: `${process.env.NEON_AI_GATEWAY_BASE_URL}/v1`,
  },
});

const vectorStore = new PGVectorStore({
  dimensions: 512,
  connectionString: process.env.POSTGRES_URL,
});

export default vectorStore;

// Use in your code (say, in API routes)
const index = await VectorStoreIndex.fromVectorStore(vectorStore);

TSX

// File: vectorStore.ts

import { OpenAIEmbedding, Settings } from 'llamaindex';
import { PGVectorStore } from 'llamaindex/storage/vectorStore/PGVectorStore';

Settings.embedModel = new OpenAIEmbedding({
  dimensions: 512,
  model: 'text-embedding-3-small',
});

const vectorStore = new PGVectorStore({
  dimensions: 512,
  connectionString: process.env.POSTGRES_URL,
});

export default vectorStore;

// Use in your code (say, in API routes)
const index = await VectorStoreIndex.fromVectorStore(vectorStore);

Note: Set NEON_AI_GATEWAY_TOKEN and NEON_AI_GATEWAY_BASE_URL from your branch (neon env pull, or the Neon Console's Connect panel). Keep the PGVectorStore dimensions in sync with the embedding model's output size. See Embeddings for model IDs and dimensions.

Generate Embeddings with OpenAI

LlamaIndex handles embedding generation internally while adding vectors to the Postgres database, simplifying the process for users. For more detailed control over embeddings, refer to the respective JavaScript and Python documentation.

Stream Chat Completions with OpenAI

LlamaIndex can find similar documents to the user's latest query and invoke the OpenAI API to power chat completion responses, providing a seamless integration for creating dynamic interactions.

Here's how you can power chat completions in an API route:

TSX

import vectorStore from './vectorStore';

import { ContextChatEngine, VectorStoreIndex } from 'llamaindex';

interface Message {
  role: 'user' | 'assistant' | 'system' | 'memory';
  content: string;
}

export async function POST(request: Request) {
  const encoder = new TextEncoder();
  const { messages = [] } = (await request.json()) as { messages: Message[] };
  const userMessages = messages.filter((i) => i.role === 'user');
  const query = userMessages[userMessages.length - 1].content;
  const index = await VectorStoreIndex.fromVectorStore(vectorStore);
  const retriever = index.asRetriever();
  const chatEngine = new ContextChatEngine({ retriever });
  const customReadable = new ReadableStream({
    async start(controller) {
      const stream = await chatEngine.chat({ message: query, chatHistory: messages, stream: true });
      for await (const chunk of stream) {
        controller.enqueue(encoder.encode(chunk.response));
      }
      controller.close();
    },
  });
  return new Response(customReadable, {
    headers: {
      Connection: 'keep-alive',
      'Content-Encoding': 'none',
      'Cache-Control': 'no-cache, no-transform',
      'Content-Type': 'text/plain; charset=utf-8',
    },
  });
}

Starter apps

Hackable, fully-featured, pre-built starter apps to get you up and running with LlamaIndex and Postgres.

AI chatbot (OpenAI + LllamIndex)

A Next.js AI chatbot starter app built with OpenAI and LlamaIndex

RAG chatbot (OpenAI + LlamaIndex)

A Next.js RAG chatbot starter app built with OpenAI and LlamaIndex

Semantic search chatbot (OpenAI + LlamaIndex)

A Next.js Semantic Search chatbot starter app built with OpenAI and LlamaIndex

Reverse image search (OpenAI + LlamaIndex)

A Next.js Reverse Image Search Engine starter app built with OpenAI and LlamaIndex

Chat with PDF (OpenAI + LlamaIndex)

A Next.js Chat with PDF chatbot starter app built with OpenAI and LlamaIndex