Understanding Vector Databases and Embeddings

Introduction

Vector databases have become a cornerstone of modern AI infrastructure. They power semantic search, retrieval-augmented generation (RAG), recommendation systems, and anomaly detection. At their core, they solve a deceptively simple problem: finding the most similar items in a high-dimensional space, fast.

In this article, we explore what embeddings are, how vector databases store and query them, and how to build a practical semantic search pipeline.

What Are Embeddings?

An embedding is a dense numerical representation of data — text, images, audio, or any structured input — in a continuous vector space. The key property is that semantically similar items are mapped to nearby points in this space.

from openai import OpenAI

client = OpenAI()

response = client.embeddings.create(
    model="text-embedding-3-small",
    input=["Reinforcement learning optimizes policies via rewards",
           "RL agents learn through trial and error",
           "The stock market closed higher today"]
)

embeddings = [item.embedding for item in response.data]
# embeddings[0] and embeddings[1] will be close (similar topics)
# embeddings[2] will be far from both (different topic)

Modern embedding models like OpenAI's text-embedding-3-small (1536 dimensions), Cohere's embed-v3, or open-source models like BAAI/bge-large-en map text into vectors where cosine similarity correlates with semantic relatedness.

Similarity Metrics

Vector databases use distance or similarity functions to rank results. The three most common are:

| Metric | Formula | Best For | |--------|---------|----------| | Cosine Similarity | A dot B / (norm(A) * norm(B)) | Normalized text embeddings | | Euclidean (L2) | sqrt(sum((a_i - b_i)^2)) | Spatial/geographic data | | Dot Product | sum(a_i * b_i) | When magnitude matters |

import numpy as np

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

def euclidean_distance(a, b):
    return np.linalg.norm(np.array(a) - np.array(b))

# Example
sim = cosine_similarity(embeddings[0], embeddings[1])
print(f"Semantic similarity: {sim:.4f}")  # ~0.85 (high)

How Vector Databases Work

Traditional databases use B-trees or hash indexes — optimized for exact matches. Vector databases solve the approximate nearest neighbor (ANN) problem using specialized index structures:

• HNSW (Hierarchical Navigable Small World): A graph-based approach that builds a multi-layer proximity graph. It offers excellent recall and query speed and is used by Pinecone, Qdrant, and pgvector.
• IVF (Inverted File Index): Partitions the vector space into clusters using k-means. Queries search only the nearest clusters, trading some recall for speed.
• Product Quantization (PQ): Compresses vectors by splitting them into sub-vectors and quantizing each independently. Dramatically reduces memory at the cost of some accuracy.

Building a RAG Pipeline with pgvector

Let's build a practical retrieval-augmented generation pipeline using PostgreSQL with the pgvector extension — a natural choice if you already use Postgres.

-- Enable the vector extension
CREATE EXTENSION IF NOT EXISTS vector;

-- Create a documents table with vector column
CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    title TEXT NOT NULL,
    content TEXT NOT NULL,
    embedding vector(1536)
);

-- Create an HNSW index for fast similarity search
CREATE INDEX ON documents
USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);

-- Query: find the 5 most similar documents
SELECT id, title,
       1 - (embedding <=> $1::vector) AS similarity
FROM documents
ORDER BY embedding <=> $1::vector
LIMIT 5;

In Python, the full pipeline looks like this:

import psycopg2
from openai import OpenAI

client = OpenAI()
conn = psycopg2.connect(DATABASE_URL)

def embed(text: str) -> list[float]:
    resp = client.embeddings.create(
        model="text-embedding-3-small", input=[text]
    )
    return resp.data[0].embedding

def search(query: str, top_k: int = 5):
    query_vec = embed(query)
    cur = conn.cursor()
    cur.execute(
        "SELECT title, content, "
        "1 - (embedding <=> %s::vector) AS similarity "
        "FROM documents "
        "ORDER BY embedding <=> %s::vector "
        "LIMIT %s",
        (query_vec, query_vec, top_k)
    )
    return cur.fetchall()

# Retrieve context and generate answer
results = search("How does backpropagation work?")
context = "\n".join([r[1] for r in results])

Choosing a Vector Database

| Database | Type | Strengths | |----------|------|-----------| | pgvector | Postgres extension | No new infra, great for hybrid queries | | Pinecone | Managed cloud | Zero-ops, high scale | | Qdrant | Open source | Rich filtering, Rust performance | | Weaviate | Open source | Built-in vectorizers, GraphQL API | | ChromaDB | Embedded | Simple local dev, Python-native |

Conclusion

Vector databases are no longer a niche technology — they are essential infrastructure for AI-powered applications. Whether you are building a semantic search engine, a RAG pipeline, or a recommendation system, understanding embeddings and vector similarity is a foundational skill for modern ML engineers. Start with pgvector if you already use Postgres, or explore managed solutions like Pinecone for production scale.