Vector Database Selection Guide for RAG Systems
How to choose the right vector database for retrieval-augmented generation. Compares Pinecone, Weaviate, Qdrant, Milvus, pgvector, and ChromaDB across performance, cost, and operational complexity.
Every RAG system needs a vector database, and the choice you make here echoes through your entire architecture. Pick wrong and you’ll spend months migrating. Pick right and the infrastructure disappears into the background where it belongs. The market has exploded with options — each optimized for different scale, cost, and operational profiles.
The honest truth: for most teams starting out, the right answer is “whatever you can operate reliably.” A well-tuned pgvector instance outperforms a poorly configured Pinecone cluster. But as you scale beyond 10M vectors and need sub-50ms P99 latency, the architectural differences between these systems start to matter enormously.
The Decision Matrix
| Database | Best For | Scale Ceiling | Operational Complexity | Cost Model |
|---|---|---|---|---|
| Pinecone | Managed simplicity, fast start | 1B+ vectors | Very Low | Per-vector + query |
| Weaviate | Hybrid search, multi-modal | 100M+ vectors | Medium | Self-hosted or cloud |
| Qdrant | Performance-critical workloads | 100M+ vectors | Medium | Self-hosted or cloud |
| Milvus | Massive scale, GPU acceleration | 10B+ vectors | High | Self-hosted |
| pgvector | PostgreSQL shops, small-medium scale | 10M vectors | Very Low | Part of Postgres |
| ChromaDB | Prototyping, local development | 1M vectors | Very Low | Free / self-hosted |
Pinecone: The Managed Standard
Pinecone dominates the managed vector database market for good reason: zero operational overhead, consistent performance, and a generous free tier for development.
When to choose Pinecone:
- Your team doesn’t want to operate infrastructure
- You need sub-100ms query latency consistently
- Your vector count is under 100M (cost-effective range)
- You want integrated embedding and reranking
When to avoid Pinecone:
- You need to keep data on-premise (compliance requirements)
- Your vector count exceeds 100M (costs escalate)
- You need complex filtering with vector search (improving but limited)
Architecture note: Pinecone’s Serverless offering changed the economics significantly. You pay for storage and queries, not for always-on pods. For bursty workloads, this can reduce costs by 80%+.
pgvector: The PostgreSQL Extension
If you’re already running PostgreSQL, pgvector is the pragmatic choice for small to medium vector workloads. No new infrastructure, no new operational runbook, no new vendor relationship.
When to choose pgvector:
- You already run PostgreSQL
- Your vector count is under 5-10M
- You need transactional consistency with relational data
- You want a single database for vectors and metadata
When to avoid pgvector:
- You need sub-20ms P99 query latency at scale
- Your vector count exceeds 10M
- You need distributed vector search across regions
Performance tuning: The default pgvector configuration is slow. Critical settings:
-- Use HNSW index (much faster than IVFFlat for most workloads)
CREATE INDEX ON embeddings USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 200);
-- Set search parameters
SET hnsw.ef_search = 100;
-- Increase maintenance_work_mem for faster index builds
SET maintenance_work_mem = '2GB';With proper tuning, pgvector handles 5M vectors with sub-100ms latency on a single good server.
Evaluation Framework
When evaluating vector databases, run these benchmarks with YOUR data, not vendor benchmarks:
1. Ingestion Performance
- Bulk load 1M vectors: measure wall-clock time
- Incremental insert: measure single-vector insert latency
- Index build time: critical for large datasets
2. Query Performance
- P50/P95/P99 latency at your expected QPS
- Accuracy (recall@10) compared to brute-force search
- Performance under concurrent load (10, 100, 1000 concurrent queries)
3. Filtering Performance
- Vector search with metadata filters (the real-world use case)
- Complex filters (AND/OR combinations, range queries)
- Filter selectivity impact on latency
4. Operational Metrics
- Memory consumption per million vectors
- Backup and restore time
- Scaling operations (adding nodes, rebalancing)
The Migration Question
Starting with ChromaDB for prototyping and migrating to Pinecone for production is a common and valid pattern. But plan the migration interface from day one:
from abc import ABC, abstractmethod
class VectorStore(ABC):
@abstractmethod
async def upsert(self, id: str, vector: list, metadata: dict): ...
@abstractmethod
async def search(self, vector: list, top_k: int, filter: dict) -> list: ...
@abstractmethod
async def delete(self, id: str): ...Wrap your vector database behind an abstraction from the start. The 30 minutes you spend writing this interface saves weeks of migration pain later.
Recommendation by Stage
| Stage | Recommendation | Reasoning |
|---|---|---|
| Prototype | ChromaDB or pgvector | Zero setup, free, fast iteration |
| MVP (< 1M vectors) | pgvector or Pinecone Serverless | Minimal ops, good enough performance |
| Production (1-50M) | Pinecone or Qdrant Cloud | Managed, reliable, cost-effective |
| Scale (50M+) | Qdrant self-hosted or Milvus | Control over infrastructure, cost optimization |
| Enterprise (regulated) | Qdrant or Weaviate self-hosted | On-premise compliance, data sovereignty |
The best vector database is the one your team can operate reliably at your current scale. Premature optimization in vector database selection is just as dangerous as in code — solve today’s problem, plan for tomorrow’s.
