What Is a Vector Database? How Vector Search Works and When You Need One

A vector database is a database designed to store and search vector embeddings, which are lists of numbers that capture the meaning of text, images, audio, or other data. Instead of matching exact keywords, it returns the items whose vectors are closest to the query's vector, so a search for "laptop won't turn on" can find a document titled "troubleshooting power failures." Vector databases are a core building block of modern AI applications.

How vector search works

1. Embed: an embedding model converts each item, such as a paragraph or a product description, into a vector of hundreds or thousands of dimensions. See our guide to AI embeddings.
2. Index: the database organizes vectors so that similar ones can be found without comparing every item.
3. Query: the user's query is embedded with the same model.
4. Retrieve: the database returns the nearest vectors using a similarity measure such as cosine similarity, dot product, or Euclidean distance, often combined with metadata filters such as "language = English" or "customer = Acme."

Why approximate nearest neighbor indexes matter

Comparing a query with every stored vector (exact search) becomes slow at millions of items. Approximate nearest neighbor (ANN) algorithms trade a tiny amount of accuracy for large speed gains. The most widely used is HNSW (Hierarchical Navigable Small World graphs), described by Malkov and Yashunin. Others include IVF (inverted file) indexes and product quantization, popularized by Meta's FAISS library. Key tuning trade-offs are recall, latency, memory use, and index build time.

Vector database vs. traditional database

Common use cases

• Retrieval-augmented generation (RAG): fetching relevant passages so an LLM answers from your data. See RAG explained.
• Semantic and hybrid search across documentation, tickets, and knowledge bases.
• Recommendations: similar products, articles, or candidates.
• Deduplication and clustering of near-identical records.
• Image and multimodal search.
• Anomaly detection by finding items far from normal clusters.

Do you need a dedicated vector database?

• Maybe not if you already run PostgreSQL and have up to a few million vectors. The open-source pgvector extension adds HNSW and IVFFlat indexes and keeps vectors next to your relational data.
• Consider a dedicated engine when you need very large scale, high query throughput, advanced filtering, hybrid search, multi-tenancy, or managed operations.

We compare the main options in Pinecone vs. Weaviate vs. Qdrant vs. ChromaDB.

Embeddings explained with a simple example

An embedding is a list of numbers that represents the meaning of a piece of content. An embedding model is trained so that content with similar meaning produces vectors that are close together. Imagine three support articles: "How to reset your password", "I forgot my login details", and "Updating your billing address". The first two use different words but describe the same problem, so their embeddings sit close together. The third is about something else, so its embedding is further away.

When a user searches for "can't sign in", the search system converts that question into an embedding with the same model and finds the stored vectors closest to it. The password articles come back first, even though neither contains the words "can't" or "sign in". This is the core advantage of vector search over keyword search: it matches meaning rather than exact terms. Our guide to embedding models explains how to choose the model that produces these vectors.

Distance metrics

To decide which vectors are "close", a vector database uses a distance or similarity metric:

• Cosine similarity compares the angle between two vectors, ignoring their length. It is the most common choice for text embeddings.
• Dot product (inner product) considers both angle and length. For normalized vectors, it gives the same ranking as cosine similarity and can be faster.
• Euclidean distance (L2) measures the straight-line distance between vectors and is common for some image and scientific embeddings.

Always use the metric your embedding model was designed for. The model's documentation usually states this, and using the wrong metric can quietly reduce search quality.

Common index types

Comparing a query against every stored vector is accurate but slow at scale. Vector databases use approximate nearest neighbor (ANN) indexes to search far fewer vectors while still finding most of the true nearest neighbors.

Most databases expose tuning parameters, such as how many graph neighbors to explore or how many clusters to probe, that let you trade speed for recall.

Metadata filtering

Real applications rarely search the whole collection. A support assistant may need only articles for a specific product, a legal search only documents a user is allowed to see, and a store only items in stock in the shopper's region. Vector databases store metadata alongside each vector and apply filters during search. How filtering is implemented matters: filtering after the search can return too few results, while filtering integrated with the index keeps results accurate and fast. When comparing databases, test with the filters you will actually use. See our vector database comparison for how popular options handle this.

Hybrid search

Vector search is excellent at meaning but can miss exact terms such as product codes, names, and error messages. Keyword search is the opposite. Hybrid search runs both and combines the results, often with a reranking model that orders the final list. For many business search and RAG systems, hybrid search gives the best overall relevance. Our hybrid search guide explains how to implement it.

Measuring vector search quality

Build a test set of real queries with the documents that should be returned for each, then measure:

• Recall at k: how often the correct documents appear in the top k results.
• Mean reciprocal rank: how high the first correct result appears on average.
• Latency: typical and worst-case response times at expected traffic.
• Index recall: how closely the approximate index matches an exact search, which shows whether index tuning is costing you accuracy.

Rerun the test whenever you change the embedding model, chunking, index settings, or database.

Scaling and operations

Memory is usually the main cost driver. As a rough guide, a 1,536-dimension vector stored as 32-bit floats needs about 6 KB, so ten million vectors need around 60 GB before index overhead. Quantization and disk-based indexes can reduce this substantially. Plan also for:

• Updates and deletions: how quickly changes become searchable and whether frequent updates degrade the index.
• Backups and restores: tested regularly, with the embedding model version recorded so indexes can be rebuilt.
• Replication and availability: for production search, a single node is a single point of failure.
• Multi-tenancy: separating customers' data through namespaces, collections, or filters, with access checked server-side.
• Re-embedding: switching embedding models means re-embedding every document, so plan for it as a migration.

Security considerations

Vector databases often hold embeddings of sensitive documents. Embeddings are not a form of encryption, and research has shown that information can sometimes be recovered from them, so protect them like the source data. Use authentication and TLS, restrict network access, encrypt data at rest, and enforce document-level permissions with metadata filters applied on the server, never on the client. Log queries for audit where required. Our RAG guide covers how these controls fit into AI assistants.

Choosing a vector database: a short checklist

1. How many vectors will you store now and in two years?
2. Which filters must every query support?
3. Do you need hybrid keyword and vector search?
4. What latency and throughput do you need?
5. Do you want a managed service or to run it yourself?
6. Does it fit your security, residency, and compliance requirements?
7. Could your existing database, such as PostgreSQL with pgvector, meet these needs?

If you want to go deeper, read our guides to Pinecone, Weaviate, Qdrant, and ChromaDB.

Example: building semantic search for a help center

A software company has around 2,000 help articles and finds that customers often cannot find answers with the existing keyword search. The team splits each article into sections of a few hundred words, embeds each section with a text embedding model, and stores the vectors in PostgreSQL with the pgvector extension, alongside metadata for product, plan, and language. Because the collection is modest and the company already runs PostgreSQL, a dedicated vector database is not needed at this stage.

Search requests are embedded, filtered by the user's product and language, and matched against the stored sections. Results are combined with the existing keyword search using reciprocal rank fusion, so exact error codes still match. The team builds a test set of 150 real customer queries from support tickets and measures recall before and after the change. Customers now find relevant articles more often, and the same index later powers an AI assistant that answers questions with citations. If the collection grows to tens of millions of vectors or needs very high query volume, the team can migrate to a dedicated database without changing the application's search API.

Common misconceptions

• "Vector databases replace traditional databases." They complement them. Transactions, reporting, and exact lookups still belong in relational or document databases.
• "More dimensions always means better search." Larger embeddings can capture more nuance but cost more to store and search. Test whether a smaller model performs as well on your data.
• "Vector search understands everything." Quality depends on the embedding model, how content is chunked, and whether the query and documents use the same model.
• "Approximate means unreliable." Well-tuned ANN indexes typically return nearly all of the true nearest neighbors while being far faster than exact search.
• "You need a vector database for any AI project." Small collections can work well with pgvector, a search engine with vector support, or even an in-memory index.

If you are planning a production system, our vector database development guide describes how we approach design, migration, and integration.

Where to start

Begin with a small, well-defined collection and a test set of real queries. Measure keyword search as a baseline, then add vector search and compare. If results improve, decide whether your existing database can host the vectors or whether a dedicated vector database is justified by scale, filtering needs, or latency targets. This incremental approach avoids adopting new infrastructure before you know it pays off, and it gives you the evaluation data you need for every later decision.

Vector database development with Vertex Cyber Tech

Our AI/ML team designs and integrates vector search with Pinecone, Weaviate, Qdrant, pgvector, and cloud-native services. See our AI backend and vector database capabilities, or book a consultation.

Related guides

• Vector vs. keyword vs. hybrid search
• Vector database development and integration

Sources and further reading

• Malkov and Yashunin, "Efficient and robust approximate nearest neighbor search using HNSW graphs"
• Meta FAISS library
• pgvector: open-source vector similarity search for Postgres