Build a "Talk to Your PDFs" RAG App with Embeddings

Large Language Models (LLMs) excel at reasoning and synthesis, but they don't know the specifics of your private business documents, legal contracts, or technical manuals. Feeding entire 100-page PDF documents into every prompt consumes massive context windows and inflates token costs dramatically.

The solution is Retrieval-Augmented Generation (RAG). By converting your documents into numerical vector representations (embeddings), you can search for only the most relevant text chunks and pass them to the LLM to get accurate, cited answers.

In this guide, we will build a complete "Talk to Your PDFs" Python application using ChromaDB, PyPDF, and Velona's unified REST API endpoints for vector embeddings and chat completions.

Architecture of a RAG Pipeline

A standard RAG pipeline consists of two distinct phases:

Document Indexing Phase:
- Parse the PDF document into text blocks.
- Split text into overlapping chunks (e.g., 500 tokens each).
- Generate vector embeddings for each chunk via Velona's /gateway/v1/inference/embed endpoint.
- Store the vectors and text chunks in a vector database like ChromaDB.
Query & Generation Phase:
- Convert the user's natural language question into a vector embedding.
- Perform vector similarity search to retrieve the top-3 matching document chunks.
- Pass the retrieved chunks as context to an inference model via /gateway/v1/inference/run to generate the final answer.

Step 1: Set Up Your Velona Account

To access both vector embedding and chat completion models:

Sign up on the Velona Registration Page.
Go to your Prepaid Wallet Dashboard and top up your account balance starting at ₹10 using UPI (GPay, PhonePe) or RuPay.
Navigate to API Key Management and create an API key for your RAG pipeline.

Step 2: Install Python Dependencies

Open your terminal and install the required Python libraries:

pip install pypdf chromadb requests

Step 3: Build the PDF Indexer and RAG Query Engine

Create a file named rag_pdf.py and add the following Python code:

import os
import requests
from pypdf import PdfReader
import chromadb

VELONA_API_KEY = "YOUR_VELONA_API_KEY"
EMBED_URL = "https://velona.in/gateway/v1/inference/embed"
INFERENCE_URL = "https://velona.in/gateway/v1/inference/run"

headers = {
    "Authorization": f"Bearer {VELONA_API_KEY}",
    "Content-Type": "application/json"
}

# Helper: Helper function to generate vector embeddings
def get_embedding(text):
    payload = {
        "model": "text-embedding-3-small",
        "input": text
    }
    res = requests.post(EMBED_URL, json=payload, headers=headers)
    return res.json()["data"][0]["embedding"]

# Step A: Parse PDF and Chunk Text
reader = PdfReader("sample_contract.pdf")
full_text = ""
for page in reader.pages:
    full_text += page.extract_text() + "\n"

# Simple chunking by 1000 character windows
chunks = [full_text[i:i+1000] for i in range(0, len(full_text), 800)]

# Step B: Initialize Local ChromaDB Vector Store
client = chromadb.Client()
collection = client.create_collection(name="pdf_documents")

print(f"Indexing {len(chunks)} document chunks...")
for idx, chunk in enumerate(chunks):
    embedding = get_embedding(chunk)
    collection.add(
        documents=[chunk],
        embeddings=[embedding],
        ids=[f"doc_chunk_{idx}"]
    )

print("PDF Indexing Complete!")

# Step C: Query the Document Database
def ask_pdf(question):
    # 1. Embed the user question
    query_vector = get_embedding(question)
    
    # 2. Retrieve top matching chunks
    results = collection.query(
        query_embeddings=[query_vector],
        n_results=2
    )
    retrieved_context = "\n---\n".join(results["documents"][0])
    
    # 3. Formulate RAG Prompt
    prompt = f"""Answer the question based ONLY on the provided document context below.
If the answer is not contained in the context, respond with 'I cannot find the answer in the document.'

Document Context:
{retrieved_context}

Question: {question}"""

    # 4. Generate Answer via Velona Gateway
    payload = {
        "model": "google/gemini-flash-1.5",
        "turns": [
            {"role": "user", "content": prompt}
        ]
    }
    response = requests.post(INFERENCE_URL, json=payload, headers=headers)
    return response.json()["choices"][0]["message"]["content"]

# Example Execution
answer = ask_pdf("What is the termination clause and notice period specified in the agreement?")
print("\nAI Response:")
print(answer)

Choosing Models for Embedding vs. Generation

A major advantage of using Velona's unified gateway is that you can pair lightweight embedding models with high-speed generation models under a single subscription:

Embedding Models: Use models like text-embedding-3-small or bge-large-en to turn text chunks into dense 1536-dimensional vectors for a fraction of a paise.
RAG Synthesis Models: Use low-latency models like google/gemini-flash-1.5 or meta/llama-3.1-70b-instruct for fast, factual context processing.

You can check real-time per-token prices in INR on the Velona Pricing Calculator and track your daily usage on the Usage Analytics Dashboard.

Developer Utilities & Testing Playground

When preparing JSON payloads or verifying character lengths for chunking boundaries, utilize Velona's 49 Life Time Free Developer Tools. Tools like the Character Counter, JSON Formatter, and Base64 Encoder run completely free without deducting credits from your wallet.

If you want to test prompt templates before writing code, try out the Free Public Chat Playground or evaluate output accuracy across multiple models simultaneously using the Model Evaluator.

Conclusion

Building a RAG application allows you to transform static documents into interactive knowledge bases. By leveraging Velona's unified embeddings and chat completion endpoints, developers in India can deploy secure, production-grade document intelligence apps with simple UPI top-ups and zero international card hassles.

Ready to build your first RAG app? Create a free Velona account and start querying your documents today!

"Talk to Your PDFs" RAG App using Custom Vector Embeddings