AI Embeddings Explained: How to Choose an Embedding Model for Search and RAG

AI embeddings are numerical vector representations of data produced by a neural network, called an embedding model, so that items with similar meaning end up close together in vector space. They let computers compare meaning mathematically, which powers semantic search, recommendations, clustering, and retrieval-augmented generation. Embeddings are stored and searched in a vector database.

How embedding models work

Modern text embedding models are typically transformer networks trained so that related pairs, such as a question and its answer, produce similar vectors, while unrelated pairs produce distant ones. Sentence-level embedding approaches were popularized by Sentence-BERT. Multimodal models extend the idea to images, so a text query can retrieve matching pictures.

How to compare embedding models

The Massive Text Embedding Benchmark (MTEB) evaluates models across tasks such as retrieval, classification, clustering, and reranking, with a public leaderboard hosted on Hugging Face. Look at the retrieval scores for languages and domains like yours, but treat the leaderboard as a shortlist, not a final answer.

Selection criteria

API models vs. open models

• API models from providers such as OpenAI, Google (via Vertex AI), Cohere, and others are quick to adopt and maintained for you. See Google Vertex AI.
• Open models available on Hugging Face can run on your own GPUs or CPUs, keeping data in-house and avoiding per-call fees, at the cost of running inference infrastructure.

A practical evaluation process

1. Build a set of 50 to 200 real queries with known relevant documents.
2. Shortlist three to five models from MTEB that fit your language, length, and hosting needs.
3. Embed the same corpus with each model and measure recall at k and mean reciprocal rank.
4. Test with hybrid search and reranking enabled, as in production.
5. Compare quality against cost and latency, then pick the simplest model that meets your target.

The full pipeline is in our production RAG guide.

Common mistakes

• Mixing vectors from different models in one index. They are not comparable.
• Embedding queries and documents with different settings when the model expects specific task prefixes or input types.
• Forgetting to normalize vectors when using dot product as a substitute for cosine similarity.
• Embedding text that lacks context, such as a table row without its headers.

Understanding embedding dimensions

An embedding's dimension is the number of values in each vector, commonly somewhere between a few hundred and a few thousand. More dimensions can capture finer distinctions in meaning, but they increase storage, memory, and search cost roughly in proportion. A collection of ten million 3,072-dimension vectors needs about four times the memory of the same collection at 768 dimensions. Some modern models are trained so that embeddings can be shortened, a technique often called Matryoshka representation learning, letting you keep only the first part of each vector with a modest loss in quality. Test shorter dimensions on your evaluation set; the savings can be significant.

Domain and language considerations

General-purpose embedding models perform well on everyday text but can struggle with specialized vocabularies such as legal clauses, medical terms, chemical names, internal product codes, or source code. If your content is specialized, include those examples in your evaluation set and compare general models with domain-specific ones. For multilingual content, choose a model trained on the languages you need and test cross-language retrieval, for example a Hindi question that should find an English document. Do not assume that strong performance on public benchmarks will transfer to your domain.

Building an evaluation set for embeddings

A good embedding evaluation set contains real queries paired with the documents or passages that should be retrieved. Practical ways to build one:

1. Collect questions from support tickets, search logs, chat transcripts, or interviews with users.
2. For each question, have a subject-matter expert mark the passages that answer it.
3. Include difficult cases: synonyms, abbreviations, misspellings, specialized terms, and questions with no answer.
4. Aim for at least 100 queries for a meaningful comparison, more if your domain is varied.
5. Keep the set fixed so you can compare models and settings fairly over time.

Measure recall at k and mean reciprocal rank for each candidate model, using the same chunking and index settings.

Normalization and similarity

Many embedding models are designed for cosine similarity. Normalizing vectors to unit length lets you use the faster dot product with identical ranking. Check the model's documentation: some models expect specific prefixes or task types for queries and documents, for example labelling text as a search query or a document, and using them incorrectly can reduce quality noticeably. Use the same model, version, and preprocessing for queries and documents; mixing models produces vectors that cannot be compared meaningfully.

Cost and throughput planning

Embedding costs have two parts. The initial indexing cost is proportional to the total text you embed, so a large document collection can be expensive to embed once, and again every time you change models. The ongoing cost depends on query volume and on how often documents change. API-based models charge per token; self-hosted open models cost compute time, often on GPUs. For large collections, batch embedding requests, embed only changed documents, cache query embeddings for repeated questions, and consider smaller or shortened embeddings where quality allows.

Versioning and migration

Treat the embedding model as part of your data schema. Record the model name, version, dimension, and preprocessing with every index. When you switch models, plan a migration: create a new index, re-embed all content, run your evaluation set against both indexes, then switch traffic when the new index is proven. Keep the old index until you are confident. Because embedding providers occasionally retire models, plan for re-embedding as a normal maintenance task rather than an emergency.

Self-hosting open embedding models

Open embedding models can be run on your own infrastructure using libraries such as Sentence Transformers or dedicated inference servers. Self-hosting gives you full control over data, consistent behavior because the model never changes unexpectedly, and predictable costs at high volume. The trade-offs are operational: you need to provision CPU or GPU capacity, handle batching for throughput, monitor latency, and update the model yourself. Smaller embedding models often run acceptably on CPUs for moderate query volumes, while bulk indexing of large collections usually benefits from GPUs. For sensitive data that must not leave your environment, self-hosting is often the deciding factor.

Fine-tuning embedding models

If general models underperform on your domain, you can fine-tune an open embedding model on pairs of queries and relevant passages from your own data. Even a few thousand good training pairs can noticeably improve retrieval for specialized vocabularies. Training pairs can come from search logs where users clicked results, from support tickets matched to the articles that resolved them, or from questions generated by a language model for each passage and then reviewed. Always evaluate the fine-tuned model on a held-out test set that was not used in training, and compare it against the best general model to confirm the improvement is real.

Embeddings beyond text search

The same techniques power several other applications:

• Recommendations: find products, articles, or courses similar to ones a user has viewed.
• Clustering and topic discovery: group support tickets, reviews, or survey responses to find common themes.
• Duplicate detection: identify near-duplicate documents, listings, or tickets.
• Classification: train a lightweight classifier on embeddings to route requests or tag content, often cheaper than calling a large model each time.
• Multimodal search: models that embed images and text into the same space allow searching photos with text queries.

Example: choosing an embedding model for a legal research tool

A legal services firm wants to search tens of thousands of contracts and case summaries. The team builds an evaluation set of 200 real research questions, each with passages marked by a lawyer. It compares two API-based models and two open models using the same chunking strategy. One general API model performs best overall, but an open model performs nearly as well and can be hosted inside the firm's own cloud account, which satisfies the client confidentiality policy. The team chooses the open model, shortens its vectors to reduce memory with only a small drop in recall, and adds hybrid keyword search to catch exact clause references. The decision was driven by measured quality on real questions combined with the firm's data requirements.

Embedding model selection checklist

• Does the model support your languages and domain vocabulary?
• How does it score on your own evaluation set, not only public benchmarks?
• What are its dimensions, and can they be shortened?
• Which similarity metric and input prefixes does it expect?
• What is the cost to embed your full collection and your expected query volume?
• Can it run where your data must stay?
• How will you handle a future model change?

For how embeddings are stored and searched, see our guides to vector databases and hybrid search.

How embeddings fit into a RAG system

In a retrieval-augmented generation system, the embedding model decides which passages the language model sees. If retrieval misses the right passage, even the best language model cannot produce a correct, grounded answer. That is why improving the embedding model, chunking, or hybrid search often raises answer quality more than switching to a larger generation model. When diagnosing a weak RAG system, measure retrieval recall first. If the correct passage is rarely in the top results, focus on embeddings, chunking, and reranking before changing prompts or generation models.

Embeddings are also only as good as the text they represent. Chunks that include the document title and section heading, that keep tables intact, and that avoid boilerplate consistently produce better retrieval than raw extracted text. Our production RAG guide covers chunking and parsing in detail, and our guide to RAG explained shows how retrieval fits into the whole pipeline.

Quick recommendations

• Starting a prototype: use a well-regarded general API embedding model, measure recall, and move on to retrieval and prompt quality.
• Strict data residency: evaluate strong open models that you can host in your own environment.
• Multilingual content: choose a model explicitly trained for your languages and test cross-language queries.
• Very large collections: prioritize models that support shortened dimensions, and consider quantization in the vector database.
• Specialized domains: compare domain models and consider fine-tuning once you have enough query-passage pairs.

Whatever you choose, record the decision and the evaluation results, so the next review starts from evidence rather than assumptions.

Re-run your comparison whenever a promising new embedding model is released. The field moves quickly, and a short evaluation on your own data will tell you whether switching is worth the cost of re-embedding your collection.

Get expert help

Our NLP and AI engineers select, benchmark, and deploy embedding models for search and RAG, including self-hosted options. Book a consultation.

Sources and further reading

• Muennighoff et al., "MTEB: Massive Text Embedding Benchmark"
• MTEB leaderboard on Hugging Face
• Reimers and Gurevych, "Sentence-BERT"
• Kusupati et al., "Matryoshka Representation Learning"