Embeddings Basics
Turn text into vectors so similarity search powers RAG, recommendations, and clustering.
Interview tip Explain: same embedding model for index + query; cosine similarity; dimension size affects cost.
① What you must know (30 sec)
An embedding is a dense vector (list of floats) representing text semantics. Similar meanings map to nearby vectors, enabling similarity search, clustering, deduplication, and RAG retrieval. Use the same model when indexing documents and embedding queries; compare with cosine similarity (or dot product on normalized vectors).
Analogy: Embeddings are GPS coordinates for meaning — "puppy" and "dog" sit close; "dog" and "airplane" are far apart.
② How it works
Similarity in vector space
"refund policy"→Embed→[0.02, -0.15, ...]→Nearest neighbors→Top matches
Cosine similarity measures angle between vectors (0–1). Euclidean distance measures straight-line distance — cosine is standard for text because magnitude matters less than direction.
Batch embed documents offline (cheap, parallel); embed queries online (low latency, one string at a time).
③ Step-by-step (hands-on)
Step 1 — Choose an embedding model
OpenAI text-embedding-3-small/large, Cohere embed-v3, BGE, Voyage — pick one and stick with it for the index.
Step 2 — Preprocess text
Lowercase optional; strip HTML; keep meaningful headers. Very short strings embed poorly — add context prefix.
Step 3 — Batch embed documents
Send 100–500 chunks per API call. Store vector + metadata (doc_id, chunk_index, text snippet).
Step 4 — Index in vector store
Pinecone, pgvector, Chroma — create index with correct dimension count from model spec.
Step 5 — Embed query at search time
Same model, same dimensions. Normalize if using dot product as cosine proxy.
Step 6 — Evaluate retrieval
Labeled question → relevant doc pairs. Measure recall@k and MRR before tuning chunking or model.
④ Code / config patterns
| Model tier | Dims (approx) | Trade-off |
|---|---|---|
| Small / fast | 384–768 | Lower cost, good for POC |
| Standard | 1024–1536 | Best balance for RAG |
| Large | 3072+ | Higher quality, storage cost |
| Multimodal | Varies | Image + text same space (CLIP-style) |
from openai import OpenAI
client = OpenAI()
# Index time (batch)
resp = client.embeddings.create(
model="text-embedding-3-small",
input=["chunk one text", "chunk two text"]
)
vectors = [d.embedding for d in resp.data]
# Query time — same model
q_vec = client.embeddings.create(
model="text-embedding-3-small", input=[user_query]
).data[0].embedding
scores = cosine_similarity(q_vec, indexed_vectors)⑤ Production & pitfalls
| Pitfall | Why it hurts | Fix |
|---|---|---|
| Mixed models in one index | Nonsense similarity scores | Re-embed entire corpus on model change |
| Not normalizing vectors | Wrong ranking with dot product | L2-normalize or use cosine metric in DB |
| Embedding tiny chunks | Weak semantic signal | Minimum ~50 tokens or prepend title/context |
| Ignoring multilingual needs | Poor cross-language retrieval | Use multilingual model (e.g. multilingual-e5) |
| No retrieval eval | Blind chunk/model tweaks | recall@k on 50+ labeled pairs |
| Storing only vectors | Cannot show user source text | Keep text or doc_id in metadata |
Production tips:
- Cache query embeddings for frequent searches
- Quantization (PQ) for large indexes to cut memory
- Monitor embedding API latency and error rate
- Version index when model changes — blue/green re-index
⑥ Interview / on-the-job Q&A
| Question | Answer |
|---|---|
| What is an embedding? | Fixed-size vector capturing semantic meaning of text for similarity comparison. |
| Cosine vs Euclidean? | Cosine compares direction (standard for text); Euclidean compares magnitude and direction. |
| Why same model for index and query? | Different models use incompatible vector spaces — similarity would be meaningless. |
| What affects embedding quality? | Model choice, text preprocessing, chunk size, and domain match. |
| Embeddings vs keywords? | Embeddings capture semantic similarity; keywords need exact term overlap (BM25). |
| How many dimensions? | Set by model — trade storage/cost vs quality; cannot mix dims in one index. |
⑦ Tools & ecosystem
- APIs: OpenAI embeddings, Cohere, Voyage AI
- Open models: sentence-transformers, BGE, E5
- Stores: Pinecone, pgvector, Weaviate, FAISS (local)
- Eval: custom recall@k scripts, RAGAS
⑧ Revision checklist
- Single embedding model chosen and documented
- Dimension matches vector DB index config
- Documents batch-embedded with metadata
- Queries embedded with identical model
- Cosine similarity or normalized dot product used
- Chunk size tested — not too small
- recall@k measured on labeled set
- Re-index plan documented for model upgrades
- Query embedding cache considered for hot queries