Popular repositories Loading

1. turboquant turboquant Public

   First open-source TurboQuant KV cache compression for LLM inference. Drop-in for HuggingFace. pip install turboquant.

   Python 31 5

2. turboquant-vectors turboquant-vectors Public

   Compress embeddings 6x instantly with TurboQuant. First pip package using Google's TurboQuant (ICLR 2026) for vector search. 71.9% recall vs FAISS PQ 13.3%.

   Python 1 1

3. back2matching back2matching Public

4. kvcache-bench kvcache-bench Public

   Benchmark every KV cache compression method on your GPU. One command, real numbers. Supports Ollama + llama.cpp.

   Python

5. quant-sim quant-sim Public

   Which quantization should I use? One command benchmarks every quant level on YOUR GPU.

   Python