pgvector "expected 1536 dimensions, not 1024" after switching embedding models

Our marketplace has semantic search over about 3.2 million product listings, stored in PostgreSQL with pgvector. For two years we embedded listings with a hosted model that produces 1,536-dimension vectors. In March we evaluated a newer multilingual model that outputs 1,024 dimensions, costs less per token and scored better on our relevance set for non-English queries. We changed the model name in the shared embedding client and deployed. Within a minute, the indexing workers and the search API both started failing:

ERROR:  expected 1536 dimensions, not 1024
STATEMENT:  INSERT INTO listing_embeddings (listing_id, embedding) VALUES ($1, $2)
  ON CONFLICT (listing_id) DO UPDATE SET embedding = EXCLUDED.embedding

ERROR:  different vector dimensions 1536 and 1024
STATEMENT:  SELECT listing_id FROM listing_embeddings ORDER BY embedding <=> $1 LIMIT 50

Search returned errors for 6 minutes until we rolled the client back. Keyword fallback kept the site usable, but conversion on search pages dropped noticeably in that window.

Reproducing it

Trivially, in any database with pgvector:

CREATE TABLE t (id int PRIMARY KEY, embedding vector(3));
INSERT INTO t VALUES (1, '[1,2]');
-- ERROR:  expected 3 dimensions, not 2
SELECT '[1,2,3]'::vector <=> '[1,2]'::vector;
-- ERROR:  different vector dimensions 3 and 2

What we ruled out

• A driver or serialisation problem. We logged the length of the arrays leaving the client: 1,024, exactly as the new model documents.
• A partial deploy. Both services ran the new client; the errors were consistent, not intermittent.
• Truncating or padding to fit. Someone suggested zero-padding the new vectors to 1,536 so the column would accept them. That would have stopped the errors and silently broken search, which is worse.

Root cause

The immediate cause is mechanical: the column is typed vector(1536) and pgvector enforces the declared dimension on insert, and distance operators require both operands to have the same dimension. The real cause is a design assumption: we treated the embedding model as a configuration detail of a shared client, when it actually defines the coordinate space of every stored vector. Vectors from two different models are not comparable even when their dimensions happen to match. Changing the model is a data migration, not a config change.

What is happening under the hood

An embedding model maps text to a point in its own learned space. Distances are only meaningful between points produced by the same model (and the same preprocessing). A 1,024-dimension vector from model B compared with a 1,536-dimension vector from model A is meaningless, and pgvector refuses to do it. Even if model B had also produced 1,536 dimensions, cosine similarity between A and B vectors would be noise: the query would run and return confident-looking but random results. Padding with zeros has the same effect.

Index constraints matter too. pgvector's HNSW and IVFFlat indexes on the vector type support up to 2,000 dimensions, and an index is built for one column with one dimension. So a new model always means a new column (or table) and a new index.

Environment

Minimal reproduction

The single line that caused the outage:

- EMBEDDING_MODEL = "vendor-embed-v2"        # 1536 dims
+ EMBEDDING_MODEL = "vendor-embed-multi-v3"  # 1024 dims

Nothing tied the model name to the column it writes into or the column the query reads from.

Who is affected

Anyone changing embedding models, providers or even model versions behind the same name, and anyone using a provider option that shortens embeddings (some APIs accept a dimensions parameter). Teams with several services sharing an embedding client are especially exposed, because one change hits writers and readers at different times.

The fix

We treated the migration like any other schema change with a backfill, using expand, backfill, switch, contract.

1. Make the model version explicit in the schema. A new column, not an altered one, so the old index keeps serving traffic:

ALTER TABLE listing_embeddings
  ADD COLUMN embedding_v3 vector(1024),
  ADD COLUMN embedding_v3_at timestamptz;

Adding a nullable column without a default is a metadata-only change in PostgreSQL, so it did not rewrite the table.

2. Dual write. The indexing workers now compute both embeddings for every new or changed listing. The embedding client was changed to return vectors tagged with the model they came from, and each writer declares which model and column it targets:

EMBEDDING_TARGETS = {
    "embedding":    {"model": "vendor-embed-v2",       "dims": 1536},
    "embedding_v3": {"model": "vendor-embed-multi-v3", "dims": 1024},
}

def embed_for(column: str, texts: list[str]) -> list[list[float]]:
    target = EMBEDDING_TARGETS[column]
    vectors = client.embed(model=target["model"], input=texts)
    for v in vectors:
        if len(v) != target["dims"]:
            raise ValueError(f"{target['model']} returned {len(v)} dims, column {column} expects {target['dims']}")
    return vectors

3. Backfill in batches. A job walks listings by primary key in batches of 2,000, embeds them with the new model (through the provider's batch endpoint, at a lower price) and updates embedding_v3. It is resumable from the last processed ID and throttled to keep replica lag under 5 seconds. The 3.2 million listings took about 19 hours.

4. Build the new index without blocking writes.

SET maintenance_work_mem = '16GB';
SET max_parallel_maintenance_workers = 7;
CREATE INDEX CONCURRENTLY listing_embeddings_v3_hnsw
  ON listing_embeddings USING hnsw (embedding_v3 vector_cosine_ops);

The build took 52 minutes.

5. Switch reads behind a flag, then compare. The search API reads the column named by a feature flag. We ran 10% of traffic on the new column for a week and compared click-through and add-to-cart rates with the control group before going to 100%.

6. Contract. Two weeks after full rollout we stopped dual-writing, dropped the old index and finally the old column.

Alternatives and trade-offs

• A separate table per model (listing_embeddings_v3). Cleaner isolation and easier to drop, at the cost of a join or a second lookup. Good choice if you expect to test several models in parallel.
• Blue-green index in a separate database or vector store. Makes sense at much larger scale, where a backfill on the primary would be too heavy.
• Generic vector column without a dimension. pgvector allows it, but you cannot build an HNSW or IVFFlat index on a column without a fixed dimension, except through an expression index with a cast, and you lose the check that saved us here.

How to test the fix

Unit tests cover the dimension guard and the model-to-column mapping. For relevance we used our labelled query set (1,800 queries with graded results in eight languages) against both columns before switching:

python -m search_eval --column embedding --k 20 --out out/v2.json
python -m search_eval --column embedding_v3 --k 20 --out out/v3.json
python -m search_eval.compare out/v2.json out/v3.json

We also added a check that runs on worker startup: it embeds a fixed sentence and verifies the returned length against every target column, so a misconfiguration fails the deploy instead of the first insert.

Confirming in production

• The 10% experiment showed nDCG@20 on the labelled set rising from 0.612 to 0.668 overall and from 0.48 to 0.61 for non-English queries, and search click-through up 3.1% relative in the treatment group.
• After full rollout, p95 vector query latency fell from 38 ms to 29 ms, mostly because the smaller vectors make the index smaller (from about 25 GB to about 17 GB).
• No dimension errors in logs since the switch, and embedding spend fell by about 35%.

Regression guard

• The schema now records the model in a table comment and the config maps every vector column to exactly one model; a CI test fails if a model name appears in code without a column mapping.
• The startup dimension check blocks deploys with mismatched configuration.
• Changing an embedding model is listed in our migration runbook as a data migration requiring the expand, backfill, switch, contract steps and a relevance comparison.

Security implications

None beyond the availability impact of the original outage. The backfill sent all listing titles and descriptions to the embedding provider again; these are public listings, but for private data the same migration would need the same data-processing review as the original integration. Embeddings can leak information about their source text, so the old column was dropped rather than left behind, and the backups that still contain it follow the normal retention policy.

Prevention checklist

• Treat the embedding model as part of the schema; store vectors from different models in different columns or tables.
• Assert vector length against the target column before writing.
• Migrate with dual writes, a resumable backfill and CREATE INDEX CONCURRENTLY.
• Switch reads behind a flag and compare relevance before cutting over.
• Never pad, truncate or mix vectors from different models to make types line up.

References

• pgvector README: vector types, HNSW, indexing limits — https://github.com/pgvector/pgvector
• pgvector CHANGELOG — https://github.com/pgvector/pgvector/blob/master/CHANGELOG.md
• PostgreSQL documentation: CREATE INDEX (CONCURRENTLY) — https://www.postgresql.org/docs/current/sql-createindex.html
• PostgreSQL documentation: ALTER TABLE — https://www.postgresql.org/docs/current/sql-altertable.html