LlamaIndex

Readers, nodes, indexes, retrievers

The core package is small. Embeddings and vector databases are separate integrations. You can retrieve with a local embedding model and no API key, then attach a chat model when you want prose. Bad PDF extraction makes every later score meaningless. A clean markdown help centre does not need a fancy parser. A scanned invoice often does.

Starter

print("Document -> node parser -> VectorStoreIndex -> as_query_engine")
print("response.source_nodes  # the pages the answer used")

The core package is small. Embeddings, OpenAI, and a specific vector database are separate integration packages. You can run retrieval with a local model such as all-MiniLM-L6-v2 and no API key, then attach a real chat model when you want generated answers.

A shop help centre

Build the index once, query it many times. Re-index a file when it changes. You do not retrain a model to update a price. One index per client, with a client_id filter on every query, is how you avoid a cross-client leak.

Starter

docs = [
    {"text": "Returns: 30 days if unused.", "audience": "customer"},
    {"text": "Staff can override a return up to $25.", "audience": "staff"},
]
visible = [d for d in docs if d["audience"] == "customer"]
print(visible[0]["text"])

from llama_index.core import Document, VectorStoreIndex, Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
# Settings.llm = ...  # attach a chat model when you want generated prose

docs = [
    Document(
        text="Returns: 30 days if unused, with receipt. Opened earbuds are final sale.",
        metadata={"page": "returns", "audience": "customer"},
    ),
    Document(
        text="Staff can override a return up to $25 with a manager code.",
        metadata={"page": "returns-staff", "audience": "staff"},
    ),
    Document(
        text="Shipping is free over $50 and $6 otherwise. Delivery takes 3 to 5 business days.",
        metadata={"page": "shipping", "audience": "customer"},
    ),
]
index = VectorStoreIndex.from_documents(docs)

Metadata is not decoration. The staff override must not be retrieved for a customer. Filter before you generate.

Ask, and show the page you used

Print the prompt once. If the node text is missing, or the instruction never says to refuse, no model choice will save you. `similarity_top_k` of 2 is a start. Raise it only when the answer is split across pages.

Starter

nodes = [
    {"page": "returns", "score": 0.82, "text": "30 days if unused."},
    {"page": "shipping", "score": 0.21, "text": "Free over $50."},
]
best = max(nodes, key=lambda n: n["score"])
print(best["page"], best["text"])

from llama_index.core.vector_stores import MetadataFilters, ExactMatchFilter

filters = MetadataFilters(filters=[ExactMatchFilter(key="audience", value="customer")])
engine = index.as_query_engine(similarity_top_k=2, filters=filters)
response = engine.query("How long do I have to return earbuds?")

print(response)
for node in response.source_nodes:
    print(node.metadata.get("page"), round(node.score or 0, 3))

A good help bot shows the page id next to the answer. Users trust “returns” more than a fluent paragraph, and you can debug a bad answer by reading the node that was actually retrieved.

Refuse when the pages do not know

Put the refusal sentence in the prompt and demand that exact string, so your UI can branch on it. A chat engine can keep turns, but still embed only the new question. Do not embed the whole transcript as the query or you will retrieve whatever you talked about last Tuesday.

Starter

context = "Shipping is free over $50."
question = "Do you price-match?"
print("I don't know" if "price" not in context.lower() else context)

from llama_index.core import PromptTemplate

qa = PromptTemplate(
    "Context:\n{context_str}\n\n"
    "Answer the question using only the context. "
    "If the context does not contain the answer, reply exactly: I don't know.\n"
    "Question: {query_str}\nAnswer:"
)
engine = index.as_query_engine(text_qa_template=qa, similarity_top_k=2, filters=filters)
print(engine.query("Do you price-match other stores?"))

LlamaIndex, LangChain, and a vector database

You do not have to adopt the whole framework. A vector index is a convenience over embed, store, and search. If you already have Postgres with pgvector, keep it and use LlamaIndex only for readers. Hosted parsers help on messy PDFs. They are overhead on markdown.

Starter

def tool_retrieve(question: str) -> str:
    return "Returns: 30 days if unused."  # the cabinet

print("agent may call tool_retrieve, then answer")

LlamaIndex is strongest when the job is “documents in, grounded answer out”: loaders, parsing, indexes, citations. LangChain is strongest when the job is “compose a model, tools, and an agent”. Many teams use LlamaIndex (or a plain vector store) for retrieval and LangGraph for the agent that decides when to retrieve.

You do not have to adopt the whole framework. VectorStoreIndex is a convenience over embed → store → search. If you already have Postgres with pgvector, you can keep it and only use LlamaIndex’s readers.

Hosted add-ons exist for hard documents: LlamaParse for messy PDFs, LlamaCloud for a managed index. A clean markdown help centre does not need them. A scanned invoice PDF often does, because bad extraction makes every later score meaningless.

Intermediate patterns

A router sends shipping questions to the shipping index, not the company dump. Sentence-window or parent chunks fix the “30 days without the exception” bug. Re-index changed files. Do not rebuild the world for one edited paragraph.

Starter

hit = "30 days"
parent = "Returns: 30 days if unused. Opened earbuds are final sale."
print("send the parent, not only:", hit)

• Sentence window or parent chunks: retrieve a small chunk for precision, then pass the surrounding section to the model so it sees the exception clause.
• Router: a question about shipping should hit the shipping index, not the entire company dump.
• Chat engine: index.as_chat_engine() keeps turns. Still send only the new question through retrieval; do not embed the whole transcript as the query.

Practice

• Add a gift-card page and ask a question that should hit it.
• Remove the audience filter and show how a customer can see the staff override.
• Ask a question the docs do not answer and confirm the refusal string.
• Print source_nodes and decide if top-k should be 2 or 4.