ModelRefs / Embeddings & Semantic Search — Tutorial
Embeddings & Semantic Search — Tutorial
Turn text into vectors that capture meaning, then search by similarity. Covers Numbers that capture meaning, Cosine similarity — measuring semantic closeness.
Overview
Turn text into vectors that capture meaning, then search by similarity
Level: Advanced. Estimated reading time: 35 minutes.
Numbers that capture meaning
An embedding is a dense vector of floats (typically 512–4096 dimensions) that represents the semantic content of text. Similar concepts land near each other in the embedding space: "cat" and "kitten" are closer than "cat" and "database."
Embeddings are produced by a dedicated encoder model (not a generative LLM). Popular choices: OpenAI text-embedding-3-small/large, Anthropic's claude-3 embedding capability (via voyage-3), sentence-transformers (open-source, runs locally). The output is a fixed-size vector regardless of input length.
Cosine similarity — measuring semantic closeness
Cosine similarity = (a · b) / (|a| |b|). It measures the angle between two vectors: 1.0 means identical direction (same meaning), 0 means perpendicular (unrelated), −1 means opposite direction.
Use cosine similarity, not Euclidean distance, for comparing embeddings. Embedding models are trained to encode semantics in direction, not magnitude — two embeddings can have very different magnitudes but nearly identical meaning.
From embeddings to a search engine
A semantic search engine: (1) Embed all your documents at index time. (2) For a query, embed the query. (3) Find the top-K documents by cosine similarity.
For small corpora (<100k documents), NumPy matrix multiplication is sufficient. For large corpora, use a vector database (Pinecone, Chroma, Weaviate, pgvector) which builds an approximate nearest-neighbour (ANN) index for millisecond retrieval at billions of vectors.
Hybrid search — combining semantic (dense) retrieval with keyword (BM25, sparse) retrieval — often outperforms either alone. Most production RAG stacks use hybrid.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Embeddings & Semantic Search — Tutorial.