Key takeaway: An embedding is only meaningful relative to the model that produced it. Upgrading the model means recomputing the entire corpus, which makes the initial choice more consequential than it appears.
Why Vectors Do Not Transfer
An embedding model maps text into a coordinate space it learned during training. The dimensions have no external meaning — position 47 in one model’s output has no relationship to position 47 in another’s.
Comparing a vector from model A against a vector from model B therefore produces a number that is arithmetically valid and semantically meaningless. Cosine similarity happily returns 0.83 for two vectors that share no interpretive frame at all.
Nothing errors. The index accepts both, the query returns results, and the results are wrong in a way that looks plausible. This is among the more insidious failure modes in retrieval systems because there is no signal that anything broke.
What Actually Differentiates Models
Dimension count is the specification everyone compares and rarely the deciding factor.
| Property | Practical impact |
|---|---|
| Dimensions | Storage and query cost, not quality directly |
| Max input length | Whether your chunks fit without truncation |
| Training domain | Performance on your specific vocabulary |
| Multilingual support | Whether non-English content retrieves at all |
| Asymmetric support | Whether queries and documents embed differently |
| Self-hosted or API | Cost model and data residency |
Input length limits cause a quiet failure worth naming. A model accepting 512 tokens silently truncates a 900-token chunk, so half the content is simply absent from the vector. Retrieval then fails for anything in the discarded portion, and the cause is invisible.
Asymmetric embedding matters for search quality. A short question and a long document are different kinds of text, and models trained to embed them with separate prefixes or separate encoders outperform models that treat both identically. Using an asymmetric model without applying the prefixes discards most of that advantage.
Planning for the Migration You Will Need
Assume you will change models, because you will — a better one appears, pricing changes, or a provider deprecates yours.
Store the model name and version alongside every vector. Without that, a partially migrated corpus is unrecoverable because you cannot tell which rows need recomputation.
Design the index so two model versions can coexist. Adding a second vector column, or a version field in the index metadata, converts migration from a flag-day rebuild into a background backfill with a cutover at the end.
Keep the source text. Recomputing embeddings requires the original content, and a system that stored only vectors cannot migrate at all.
Deciding Whether to Upgrade
A newer model with better benchmark scores is not automatically better for your corpus. General benchmarks measure general text, and domain-specific retrieval frequently disagrees.
Build an evaluation set of real queries with known correct documents, then measure recall at k for both models over the same corpus. That number is the only basis for the decision, and the migration cost means it should be a clear improvement rather than a marginal one.
Also weigh dimension count against storage. Moving from 768 to 3072 dimensions quadruples index size and memory, which for a large corpus is a real operating cost that a small accuracy gain may not justify.
The Bottom Line
Record the model name and version with every vector, keep the source text so recomputation is possible, and design the index to hold two versions during migration. Choose based on measured recall against your own queries rather than on dimension count or general benchmarks.
