Vector Databases Explained for Beginners

Something changed in how software stores and retrieves information when large language models went mainstream. The old model was straightforward. Data goes into a database as structured records. Queries find records by matching exact values or ranges. A user whose name is Sarah gets found by looking for rows where the name column equals Sarah. A product priced between fifty and a hundred dollars gets found by filtering the price column within that range.

This model works perfectly for the kind of data it was designed for. It does not work well at all for a different kind of retrieval problem that has become central to modern AI applications. How do you find documents that are semantically similar to a query even when they share no words in common? How do you find images that look like a given image? How do you find songs that sound like other songs a user enjoys? How do you build a search system that understands that a query about automobile repair is relevant to a document about fixing cars even though neither of those exact words appears in both?

These are similarity search problems and they require a fundamentally different approach to storage and retrieval. Vector databases are the infrastructure built specifically to solve them.

This guide explains what vector databases are, how they work under the hood, what their key components and tradeoffs are, and how they fit into the AI application architectures that have made them one of the fastest-growing categories in data infrastructure.

What a Vector Is and Why It Matters

Before understanding vector databases you need to understand what a vector is in this context and why representing data as vectors is useful.

A vector is simply an ordered list of numbers. A vector in three dimensions might look like [0.2, -0.8, 0.5]. A vector in a thousand dimensions looks like the same thing but longer, a list of a thousand floating-point numbers.

What makes vectors useful for representing data is that they can encode meaning in their numerical values. A machine learning model called an embedding model takes a piece of content, a sentence, an image, a piece of audio, a product description, and converts it into a vector where the numerical values capture the semantic content of the input. Two pieces of content that are similar in meaning produce vectors that are numerically close to each other. Two pieces of content that are unrelated produce vectors that are far apart.

This is the key insight. Semantic similarity becomes geometric proximity. Finding content that is similar to a query becomes finding vectors that are close to the query vector in a high-dimensional space. And that is a well-defined mathematical problem that can be solved efficiently with the right data structures.

Consider a concrete example. The sentences “the dog ran across the park” and “a puppy sprinted through the playground” are semantically similar even though they share almost no words. An embedding model converts both into vectors where the numerical values reflect their meaning, not their exact words. The resulting vectors are geometrically close to each other. A vector database can find that the second sentence is similar to the first by measuring the distance between their vectors, even without any word overlap.

What an Embedding Model Does

Embedding models are the component that converts raw content into vectors. Understanding them at a high level is necessary for understanding why vector databases work.

An embedding model is a neural network trained to produce vector representations where semantic similarity corresponds to geometric proximity. Different embedding models exist for different types of content. Text embedding models convert sentences, paragraphs, or documents into vectors. Image embedding models convert images into vectors. Multimodal embedding models convert both text and images into the same vector space, which enables searching images with text queries.

Popular text embedding models include OpenAI’s text-embedding-3-small and text-embedding-3-large, Cohere’s embedding models, and open source models like sentence-transformers from Hugging Face. These models vary in the dimensionality of the vectors they produce, typically ranging from 384 dimensions to 3072 dimensions, and in their performance on semantic similarity benchmarks.

The dimensionality of the vectors matters for vector database performance. Higher-dimensional vectors capture more nuanced semantic information but require more storage and more computation to search. Lower-dimensional vectors are cheaper to store and search but may lose some semantic precision. Choosing the right embedding model involves balancing these tradeoffs for your specific use case.

The crucial point for understanding vector databases is that the embedding model lives outside the database. You use an embedding model to convert your content into vectors, then store those vectors in the vector database. At query time, you embed the query using the same model and search for similar vectors in the database. The database itself does not do the embedding. It stores vectors and finds similar ones efficiently.

How Similarity Search Works

Once data is represented as vectors, similarity search is the core operation that makes vector databases useful. Similarity search answers the question: given a query vector, which stored vectors are closest to it?

Distance is measured using one of several mathematical metrics depending on the application and the embedding model used.

Euclidean distance measures the straight-line distance between two points in the vector space. If two vectors are [1, 2, 3] and [1, 2, 4], the Euclidean distance is the length of the line connecting those two points. Smaller distance means more similar.

Cosine similarity measures the angle between two vectors rather than the distance between their endpoints. It ranges from -1 to 1 where 1 means the vectors point in exactly the same direction, meaning they are maximally similar, and -1 means they point in opposite directions. Cosine similarity is preferred when the magnitude of the vector is not meaningful and only the direction matters, which is common in text embeddings.

Dot product is mathematically related to cosine similarity but also accounts for vector magnitude. Some modern embedding models are optimized for dot product similarity and perform better with it than with cosine similarity.

The exact similarity metric to use depends on which embedding model you are using. Most embedding model documentation specifies which metric is appropriate for that model’s output.

The Challenge of Searching High-Dimensional Vectors

If you have a thousand vectors, finding the most similar one to a query is trivial. Compare the query vector to all thousand stored vectors, compute the distance for each, and return the closest one. This is called exact nearest neighbor search and it always returns the correct answer.

The problem is that this approach does not scale. If you have ten million vectors and each is a thousand dimensions, comparing the query to every stored vector requires ten billion floating-point operations for a single query. At the query volumes a production application handles, this is far too slow.

This is the core technical challenge that vector databases are built to solve. How do you find the most similar vectors in a large collection without comparing the query to every single stored vector?

The answer is approximate nearest neighbor search, usually abbreviated ANN. ANN algorithms find vectors that are very likely to be among the most similar without guaranteeing that they are the absolute closest. The tradeoff is a small reduction in accuracy in exchange for a dramatic reduction in computation. In practice, ANN algorithms return results that are so close to exact that the difference is imperceptible in most applications, while being orders of magnitude faster.

ANN Indexing Algorithms

The indexing algorithms that make ANN search fast are the technical heart of vector databases. Different algorithms make different tradeoffs between speed, accuracy, memory usage, and build time. Understanding the major ones helps you make sense of vector database documentation and configuration options.

HNSW (Hierarchical Navigable Small World)

HNSW is the most widely adopted ANN algorithm and is the default in most vector databases. It builds a multi-layer graph structure where each node is a vector and edges connect nearby vectors. The graph has multiple layers with different densities. The top layers are sparse and allow large jumps across the space. Lower layers are denser and allow precise local navigation.

To search, HNSW starts at an entry point in the top layer and greedily moves toward the query vector, always stepping to the neighbor that is closest to the query. When no closer neighbor exists at the current layer, it descends to the next layer and repeats. By the bottom layer, it has navigated to the region of the space near the query and returns the nearest neighbors found there.

HNSW is fast at query time and produces high recall (the fraction of true nearest neighbors returned). Its main drawback is memory usage. The graph structure requires storing all vectors and their connections in memory for fast access. For very large collections, the memory requirement can become expensive.

Key tuning parameters:

• M: the number of connections each node maintains. Higher M improves recall but increases memory and build time.
• ef_construction: controls the size of the candidate set during index construction. Higher values improve recall at the cost of longer build times.
• ef_search: controls the size of the candidate set during search. Higher values improve recall at the cost of slower queries.

IVF (Inverted File Index)

IVF partitions the vector space into clusters using a clustering algorithm like k-means. Each cluster has a centroid vector. To search, IVF first finds the clusters whose centroids are closest to the query, then searches only the vectors within those clusters.

The advantage of IVF is that it can work with vectors stored on disk rather than requiring everything in memory, which makes it more scalable to very large collections. The disadvantage is that vectors near cluster boundaries may be assigned to a different cluster than they naturally belong to, causing them to be missed in searches that only examine nearby clusters. This is why IVF typically needs to search multiple clusters (a parameter called nprobe) to achieve good recall.

IVF is often combined with quantization techniques that compress vectors to reduce memory usage. The combination of IVF and product quantization (IVF-PQ) is a popular configuration for very large collections where memory efficiency matters more than maximum recall.

LSH (Locality Sensitive Hashing)

LSH uses hash functions designed so that similar vectors are more likely to hash to the same bucket than dissimilar vectors. At search time, you hash the query vector and examine only vectors in the same or nearby buckets.

LSH is simple to implement and works well for certain distance metrics. Its recall is generally lower than HNSW for the same query speed, which is why it has been largely superseded by HNSW in production vector database implementations. It remains useful for specific use cases and as an educational foundation for understanding ANN concepts.

ScaNN (Scalable Nearest Neighbors)

ScaNN was developed at Google and uses a combination of partitioning and quantization optimized for modern hardware. It achieves excellent query throughput by taking advantage of SIMD instructions and hardware-specific optimizations. It is particularly strong when query throughput matters more than latency, such as in batch similarity search scenarios.

The Anatomy of a Vector Database

A production vector database is more than just an ANN index. Understanding its full component set clarifies what you are getting with different tools.

Vector storage holds the raw vectors associated with each record. Some databases store vectors entirely in memory for maximum speed. Others store vectors on disk and load them into memory as needed, trading latency for the ability to handle larger collections at lower cost.

The ANN index is the data structure that enables fast approximate similarity search. Most databases let you choose the indexing algorithm and configure its parameters to tune the recall-speed tradeoff for your use case.

Metadata storage holds non-vector attributes associated with each record. A vector representing a product also has a name, price, category, and availability status. A vector representing a document chunk also has its source document, page number, and creation date. The metadata storage keeps these attributes queryable alongside the vector search results.

Filtering lets you combine vector similarity search with metadata filters. Find the ten most similar products to this query, but only among products in the Electronics category that are currently in stock. Pre-filtering applies the metadata filter before the vector search, reducing the candidate set. Post-filtering applies it after, which is faster but may return fewer results than requested if many top results are filtered out. Different databases handle this tradeoff differently and it significantly affects result quality for filtered queries.

The query layer handles incoming search requests, embedding optional (some databases accept raw vectors, others accept raw content and embed it internally), and returns ranked results with their distances and metadata.

Persistence and replication ensure that indexed data survives restarts and that the database can operate reliably at production scale with multiple replicas.

Leading Vector Databases

Pinecone

Pinecone is a fully managed vector database offered exclusively as a cloud service. There is no self-hosted option. The value proposition is operational simplicity: you do not manage any infrastructure. Create an index, upsert vectors, query. Pinecone handles scaling, replication, and availability automatically.

Pinecone supports HNSW and its own proprietary indexing approach, offers namespace-based data isolation within a single index, and provides a straightforward SDK for Python, JavaScript, and other languages. Its serverless tier allows storage of vectors with costs based on usage rather than reserved capacity.

The limitation of Pinecone is cost at scale and the absence of self-hosting. For organizations with data residency requirements or that need to minimize cloud spend at very large scale, the managed-only model is a constraint.

Best for: teams that want to ship quickly without managing vector database infrastructure and are comfortable with a fully managed cloud service.

Weaviate

Weaviate is an open source vector database with both self-hosted and managed cloud deployment options. It uses HNSW for indexing, supports multiple embedding model integrations directly within the database (meaning Weaviate can call embedding APIs for you), and provides a GraphQL and REST API for queries.

Weaviate’s module system is distinctive. Modules integrate with specific embedding models and enable the database to handle the embedding step, accepting raw text or images and converting them to vectors internally. For teams that want a simpler integration where the database handles more of the pipeline, this is appealing.

Weaviate also supports hybrid search, combining vector similarity search with keyword-based BM25 search, which can improve results for queries where exact keyword matching is also relevant.

Best for: teams that want open source flexibility, hybrid search capability, and the option to have the database manage embedding calls.

Qdrant

Qdrant is a high-performance open source vector database written in Rust. It offers self-hosted deployment and a managed cloud option. Qdrant has a reputation for strong performance benchmarks and memory efficiency, with features like scalar quantization and product quantization to reduce memory usage for large collections.

Qdrant’s filtering implementation is particularly strong. It handles filtered vector search efficiently using a combination of pre-filtering and index-level filter awareness, which produces better results than simple post-filtering for selective filters.

Payload (metadata) in Qdrant is flexible and supports rich filter expressions with nested conditions. The REST and gRPC APIs are clean and the Python client is well-maintained.

Best for: teams that prioritize performance and memory efficiency, need robust filtered search, and want an open source option with strong engineering quality.

Chroma

Chroma is a lightweight open source vector database designed specifically for use in AI and LLM applications. It prioritizes simplicity and developer experience over operational sophistication. Getting started with Chroma requires almost no configuration and the Python API is straightforward.

Chroma runs in-memory by default and can persist to disk. It is not designed for distributed production deployments at very large scale. Its value is in rapid prototyping, development, and smaller production use cases where operational simplicity matters more than maximum performance.

For building a RAG prototype or developing an AI application locally, Chroma is often the fastest path to a working vector search component. For production at scale, most teams migrate to a more operationally mature option.

Best for: prototyping, development, and smaller production deployments where simplicity and ease of use matter more than scale.

FAISS (Facebook AI Similarity Search)

FAISS is not a database in the traditional sense. It is a library of vector indexing and search algorithms developed by Meta. It does not provide persistence, a query API, metadata storage, or filtering. It provides fast implementations of IVF, HNSW, and other ANN algorithms that you can use to build vector search into your own application.

FAISS is what many production vector databases use under the hood. Understanding FAISS gives you insight into the algorithms that power vector search broadly. For teams that need to embed vector search into a custom application without the overhead of a full database system, FAISS is a viable approach. For teams that need a complete database with persistence and metadata, FAISS is a building block rather than a solution.

Best for: building custom vector search implementations, research and experimentation with ANN algorithms, and situations where you need fine-grained control over the indexing and search stack.

pgvector

pgvector is a PostgreSQL extension that adds vector storage and similarity search to PostgreSQL. It supports HNSW and IVF indexes for approximate search as well as exact nearest neighbor search for smaller collections.

The appeal of pgvector is that it adds vector search to a database your team already knows how to operate. If your application data lives in PostgreSQL, storing vectors there too keeps everything in one system. Metadata filtering uses standard SQL WHERE clauses and joins, which is both familiar and powerful.

The limitation is that PostgreSQL was designed for transactional workloads and pgvector inherits some of its constraints. For very large vector collections with high query throughput requirements, purpose-built vector databases outperform pgvector. For moderate scale where operational simplicity and SQL familiarity matter, pgvector is a very practical choice.

Best for: teams already using PostgreSQL who want to add vector search without adopting a new database system, and applications where moderate scale and SQL-native querying matter more than maximum vector search performance.

Vector Databases and RAG

The use case that has driven most of the growth in vector database adoption is retrieval-augmented generation, commonly called RAG. Understanding how vector databases fit into RAG architectures explains why they have become so central to AI application development.

Large language models have two fundamental limitations for enterprise applications. Their knowledge is frozen at their training cutoff date, so they do not know about events, documents, or data created after training. And they cannot access private organizational data, internal documents, product databases, or customer records that were never part of their training data.

RAG addresses both limitations by retrieving relevant context from an external knowledge base and including it in the prompt sent to the language model. The model generates its response based on both its trained knowledge and the retrieved context.

The retrieval step is where vector databases come in. The knowledge base, whether it is company documentation, a product catalog, a codebase, or a collection of research papers, is chunked into pieces, embedded into vectors, and stored in the vector database. When a user asks a question, the question is embedded and the vector database finds the most semantically similar chunks. Those chunks are included in the prompt as context. The language model generates an answer grounded in the retrieved information.

A basic RAG pipeline looks like this:

python

# Indexing time (done once, updated as knowledge base changes)
from openai import OpenAI
import chromadb

client = OpenAI()
chroma = chromadb.Client()
collection = chroma.create_collection("knowledge_base")

documents = [
    "Our return policy allows returns within 30 days of purchase.",
    "Shipping typically takes 3 to 5 business days for standard orders.",
    "Premium members receive free two-day shipping on all orders."
]

# Embed each document
embeddings = [
    client.embeddings.create(
        input=doc,
        model="text-embedding-3-small"
    ).data[0].embedding
    for doc in documents
]

# Store in vector database
collection.add(
    documents=documents,
    embeddings=embeddings,
    ids=[f"doc_{i}" for i in range(len(documents))]
)

# Query time (happens on each user question)
user_question = "How long do I have to return something?"

query_embedding = client.embeddings.create(
    input=user_question,
    model="text-embedding-3-small"
).data[0].embedding

results = collection.query(
    query_embeddings=[query_embedding],
    n_results=2
)

retrieved_context = "\n".join(results["documents"][0])

prompt = f"""Answer the user's question based on the context below.

Context:
{retrieved_context}

Question: {user_question}
"""

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": prompt}]
)

print(response.choices[0].message.content)

The quality of a RAG system depends heavily on the quality of the retrieval step and that depends on the embedding model, the chunking strategy, the vector database configuration, and whether hybrid search or metadata filtering is used to refine results.

Vector Database vs Relational Database vs Search Engine

Understanding how vector databases compare to tools you likely already use clarifies when each is the right choice.

A relational database like PostgreSQL or MySQL stores structured data and retrieves it through exact matching and range queries. It is the right tool when you know exactly what you are looking for and can express that as a structured query. Find all orders where the status is pending and the amount is greater than one hundred dollars. Relational databases are not designed for semantic similarity search and perform poorly at it without extensions like pgvector.

A search engine like Elasticsearch or Solr indexes text for full-text keyword search. It finds documents containing specific words or phrases, handles synonyms and stemming, and ranks results by relevance using statistical methods like BM25. It is the right tool when users are searching with keywords and you want to find documents that contain those keywords. Traditional search engines do not understand semantic similarity the way embedding-based vector search does.

A vector database stores embedding vectors and retrieves them by semantic similarity. It is the right tool when the query and the stored content may not share any words but are semantically related, when you want to find content similar to other content rather than matching specific terms, or when you are building AI applications that need to retrieve context for language models.

In practice these tools are complementary rather than competing. Hybrid search, combining vector similarity with keyword search, often outperforms either approach alone. Many teams run both a traditional search index and a vector index on the same content and merge the results.

Vector Database Cheat Sheet

Feature	Pinecone	Weaviate	Qdrant	Chroma	pgvector
Open source	No	Yes	Yes	Yes	Yes
Self-hosted	No	Yes	Yes	Yes	Yes (PostgreSQL)
Managed cloud	Yes	Yes	Yes	Yes	Via cloud Postgres
Default index	Proprietary	HNSW	HNSW	HNSW	HNSW / IVF
Hybrid search	Yes	Yes	Yes	No	With pg extensions
Built-in embedding	No	Yes (modules)	No	No	No
Best for	Managed simplicity	Open source + hybrid	Performance + filtering	Prototyping	PostgreSQL teams
Scale	Very large	Large	Large	Small-medium	Medium

Common Misconceptions

A vector database does not replace your regular database. It solves a specific problem, similarity search on embedding vectors, that relational databases were not designed for. Most applications that use a vector database also use a relational database for structured operational data. They are complementary systems.

Storing vectors in a vector database does not automatically make search good. The quality of search results depends primarily on the quality of the embedding model and how well the chunking strategy matches the retrieval use case. A great vector database with a poor embedding model produces poor results. Choosing and evaluating embedding models is as important as choosing a vector database.

More dimensions are not always better. Higher-dimensional embeddings capture more information but cost more in storage and compute. Modern embedding models at 1536 or 3072 dimensions provide strong performance, but for many applications, smaller models at 384 or 768 dimensions perform comparably at significantly lower cost.

ANN search returning approximate results does not mean search is inaccurate in a damaging way. The approximation is typically extremely close to exact. In benchmarks, well-configured HNSW indexes routinely achieve over 99 percent recall, meaning they return essentially all the true nearest neighbors. The small accuracy loss from approximation is imperceptible in practice for most applications.

FAQs