Build a "Talk to Your PDFs" RAG App with Embeddings

Large Language Models (LLMs) excel at reasoning and synthesis, but they don't know the specifics of your private business documents, legal contracts, or technical manuals. Feeding entire 100-page PDF documents into every prompt consumes massive context windows and inflates token costs dramatically.

The solution is Retrieval-Augmented Generation (RAG). By converting your documents into numerical vector representations (embeddings), you can search for only the most relevant text chunks and pass them to the LLM to get accurate, grounded answers.

In this guide, we will build a complete "Talk to Your PDFs" Python application using ChromaDB, PyPDF, and Velona's API for both embeddings and answers. Indexing a 100-page PDF costs about ten paise, and each answer costs a few paise.

Architecture of a RAG Pipeline

A standard RAG pipeline consists of two distinct phases:

Document Indexing Phase:
- Parse the PDF document into text.
- Split the text into overlapping chunks (about 1,000 characters each).
- Generate vector embeddings for the chunks via Velona's /gateway/v1/inference/embed endpoint.
- Store the vectors and text chunks in a vector database like ChromaDB.
Query and Generation Phase:
- Convert the user's question into a vector embedding.
- Run a similarity search to retrieve the best matching chunks.
- Pass the retrieved chunks as context to a chat model via /gateway/v1/inference/run to generate the final answer.

Step 1: Set Up Your Velona Account

One API key gives you both embedding and chat models:

Sign up on the Velona Registration Page.
Go to your Prepaid Wallet Dashboard and top up starting at ₹10 using UPI (GPay, PhonePe) or RuPay.
Navigate to API Key Management and create an API key for your RAG pipeline, then export it: export VELONA_API_KEY=your_key.

Step 2: Install Python Dependencies

Open your terminal and install the required Python libraries:

pip install pypdf chromadb requests

Step 3: Build the PDF Indexer and RAG Query Engine

Create a file named rag_pdf.py and add the following Python code:

import os
import requests
import chromadb
from pypdf import PdfReader

VELONA_API_KEY = os.environ["VELONA_API_KEY"]
EMBED_URL = "https://velona.in/gateway/v1/inference/embed"
INFERENCE_URL = "https://velona.in/gateway/v1/inference/run"
EMBED_MODEL = "openai/text-embedding-3-small"   # 1536 dimensions
CHAT_MODEL = "openai/gpt-4o-mini"

headers = {
    "Authorization": f"Bearer {VELONA_API_KEY}",
    "Content-Type": "application/json",
}


def get_embeddings(texts):
    """Embed a list of strings in one request. Returns a list of vectors."""
    res = requests.post(
        EMBED_URL,
        json={"model": EMBED_MODEL, "inputs": texts},
        headers=headers,
        timeout=60,
    )
    res.raise_for_status()
    vectors = res.json()["data"]["vectors"]
    vectors.sort(key=lambda v: v["index"])
    return [v["embedding"] for v in vectors]


# Step A: Parse the PDF and chunk the text
reader = PdfReader("sample_contract.pdf")
full_text = "\n".join((page.extract_text() or "") for page in reader.pages)

# 1,000 character windows with 200 characters of overlap
chunks = [full_text[i:i + 1000] for i in range(0, len(full_text), 800)]

# Step B: Initialise a local ChromaDB vector store
client = chromadb.Client()
collection = client.get_or_create_collection(name="pdf_documents")

print(f"Indexing {len(chunks)} document chunks...")
BATCH = 64
for start in range(0, len(chunks), BATCH):
    batch = chunks[start:start + BATCH]
    collection.add(
        documents=batch,
        embeddings=get_embeddings(batch),
        ids=[f"doc_chunk_{start + i}" for i in range(len(batch))],
    )
print("PDF indexing complete!")


# Step C: Query the document
def ask_pdf(question):
    query_vector = get_embeddings([question])[0]

    results = collection.query(query_embeddings=[query_vector], n_results=3)
    retrieved_context = "\n---\n".join(results["documents"][0])

    prompt = f"""Answer the question based ONLY on the document context below.
If the answer is not in the context, reply: 'I cannot find the answer in the document.'

Document Context:
{retrieved_context}

Question: {question}"""

    response = requests.post(
        INFERENCE_URL,
        json={"model": CHAT_MODEL, "turns": [{"role": "user", "content": prompt}]},
        headers=headers,
        timeout=60,
    )
    response.raise_for_status()
    # The native endpoint returns the reply text in data.output
    return response.json()["data"]["output"]


answer = ask_pdf("What is the termination clause and notice period in the agreement?")
print("\nAI Response:")
print(answer)

The embed endpoint accepts a single string or a list in inputs, and returns one entry per input in data.vectors. Sending chunks in batches is faster and makes far fewer HTTP calls than one request per chunk.

Choosing Models for Embedding vs. Generation

With Velona you pay per token from one prepaid rupee wallet, so you can pair a cheap embedding model with whichever chat model suits your documents:

Role	Model ID	₹ per 1M tokens (input / output)	Notes
Embeddings	`openai/text-embedding-3-small`	about ₹2 (input only)	1536-dimension vectors
Answers, lowest cost	`deepseek/deepseek-v4-flash`	₹2.90 / ₹5.80	1M context, good for long chunks
Answers, default	`openai/gpt-4o-mini`	₹15.54 / ₹62.17	Reliable, follows "only use the context" well
Answers, scanned PDFs	`google/gemini-2.5-flash`	₹31.08 / ₹259.02	Multimodal, can read page images directly

Check current rupee prices on the Velona pricing page.

What This Costs in Rupees

Indexing: a 100-page contract is roughly 50,000 tokens, so embedding it costs about ₹0.10. You only pay this once per document.
Each question: about 1,000 input tokens (three chunks plus the prompt) and 200 output tokens. That is about ₹0.004 on DeepSeek V4 Flash, ₹0.03 on GPT-4o Mini, and ₹0.08 on Gemini 2.5 Flash.
₹10 covers indexing dozens of documents and a few hundred questions on GPT-4o Mini.

Every request and its exact rupee cost appears on your usage dashboard.

Developer Utilities and Testing

When preparing JSON payloads or checking character lengths for chunk boundaries, use Velona's free developer tools. The Character Counter, JSON Formatter, and Base64 Encoder run in your browser and never touch your wallet.

If you want to test prompt templates before writing code, compare answers across models side by side in the Model Evaluator.

Conclusion

A RAG application turns static documents into an interactive knowledge base. With Velona's embeddings and chat endpoints behind one key, developers in India can build document Q&A apps and pay for them with a UPI top-up instead of an international card.

Ready to build your first RAG app? Create a free Velona account, add ₹10, and start querying your documents today.

"Talk to Your PDFs" RAG App using Custom Vector Embeddings