Vector Databases & Indexes

A commonly overlooked detail is that a vector database does not store only vectors. Every record is really a bundle of three things: the vector itself, used purely for nearest-neighbor search; the original text, which is what you actually want returned; and metadata, used for filtering on fields like tenant, date, document type, or permissions. Search only ever compares against the vector field, but what comes back is the whole record, so 'the chunk' is really just reading the text field off that returned record. This matters because it clears up the most common confusion about retrieval: asking for the top-k chunks returns k full records, and the text is pulled off each one and assembled into the prompt afterward. Since the text has to be stored somewhere, some systems, Pinecone among them, have no separate text column and expect the text to be placed directly inside the metadata, while others, pgvector and Chroma, provide an explicit text column; the underlying pattern is identical, only the syntax differs between them.

Three widely used vector databases all implement the same three concepts, vector plus text plus metadata, filtering on metadata, searching on the vector, in slightly different API dialects. pgvector is a Postgres extension, which means a team already running Postgres gets vector search without introducing a second database system to keep in sync; its distance operator has to be chosen to match how the embedding model was trained, and metadata filters applied in the same query run before the vector comparison, which is exactly what makes multi-tenant isolation safe. Pinecone is fully managed with no server to run, storing the text directly inside the metadata dictionary since it has no separate text column, making the 'record bundles vector, text, and metadata' idea completely literal in its API. Chroma runs in-process with essentially zero setup, which makes it well suited to prototyping rather than large-scale production use. Learning any one of the three transfers almost entirely to the others, since the underlying concepts never change, only the vocabulary does.

Metadata filtering is what turns a vector database from a toy into something safe to run in production, because it is the actual security and correctness boundary in any multi-tenant system. The underlying problem is that embedding similarity has no concept of ownership at all: a search for how to reset a password can return a different customer's internal runbook purely because the two are semantically similar, unless a filter restricts the search to the right customer's documents in the first place. Two filtering patterns exist, each with a real tradeoff. Pre-filtering narrows the candidate set before running the vector search, which is safe but can starve an approximate index of enough neighbors to search when the filter is very selective, hurting recall. Post-filtering searches broadly first and drops unauthorized results afterward, preserving recall but sometimes returning fewer than the requested number of results once unauthorized hits are removed. The rule that matters more than which pattern is chosen is that the permission filter has to run as part of the query itself, never afterward in application code, because retrieving broadly and checking permissions only after the fact means an unauthorized document has already entered the context sent to the model, and the leak has already happened by the time the check runs. The test worth writing is a negative one: proving directly that a user scoped to one tenant cannot retrieve a document belonging to another.