Skip to content

fix(storage): stop ChromaDB from crashing when reopening an existing …#1262

Open
Legion345 wants to merge 1 commit intoMemPalace:developfrom
Legion345:fix/stop-hook-crash
Open

fix(storage): stop ChromaDB from crashing when reopening an existing …#1262
Legion345 wants to merge 1 commit intoMemPalace:developfrom
Legion345:fix/stop-hook-crash

Conversation

@Legion345
Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes #1089.

When opening an existing palace, get_or_create_collection was called with hnsw metadata on every open. In ChromaDB 1.5.x, if that metadata differs from what's already stored, the Rust bindings segfault — no traceback, just a core dump. This is what causes the stop hook to crash at session end.

Fix: try get_collection first, fall back to create_collection only when the collection doesn't exist yet. Existing palaces open without touching their metadata; new ones are created with the full settings as before.

Credit to @jphein who described this exact approach in #1089 and has been running it on their fork since April 10 with zero crashes across 400+ starts.

How to test

Open a palace twice in the same session:

from mempalace.backends.chroma import ChromaBackend
import tempfile, os

with tempfile.TemporaryDirectory() as d:
    palace = os.path.join(d, "palace")
    backend = ChromaBackend()
    backend.get_collection(palace, collection_name="mempalace_drawers", create=True)
    backend.get_collection(palace, collection_name="mempalace_drawers", create=True)
    print("no crash") 

Or run the new regression tests directly:
python -m pytest tests/test_backends.py -v -k "idempotent or preserves_existing"

Checklist

  • Tests pass (python -m pytest tests/ -v)
  • No hardcoded paths
  • Linter passes (ruff check .)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SIGSEGV on get_or_create_collection when call-site metadata differs from stored collection metadata (chromadb 1.5.x)

1 participant