Location
rag/ -> Retrieval-Augmented Generation pipeline
Issue
Vector similarity search operates directly on high-dimensional embeddings in memory, causing OOM errors as the knowledge base scales.
Proposed Fix
Migrate the vector index to a dedicated vector database (e.g., Pinecone, Milvus, Qdrant) or pgvector, and implement a hybrid search (sparse + dense) mechanism.
Why This Matters
In-memory FAISS/Annoy indices are not horizontally scalable and risk bringing down the entire API server.
Difficulty
Hard( Data migration, schema design for vectors, and tuning retrieval metrics (MRR, NDCG) across a distributed database.)
I would like to work on this issue.
@Eshajha19 /assign
Location
rag/-> Retrieval-Augmented Generation pipelineIssue
Vector similarity search operates directly on high-dimensional embeddings in memory, causing OOM errors as the knowledge base scales.
Proposed Fix
Migrate the vector index to a dedicated vector database (e.g., Pinecone, Milvus, Qdrant) or pgvector, and implement a hybrid search (sparse + dense) mechanism.
Why This Matters
In-memory FAISS/Annoy indices are not horizontally scalable and risk bringing down the entire API server.
Difficulty
Hard( Data migration, schema design for vectors, and tuning retrieval metrics (MRR, NDCG) across a distributed database.)
I would like to work on this issue.
@Eshajha19 /assign