v0.0.4 - Crash harness & failpoint hooks

What's new

BeachDB now has a structured crash testing harness and an internal failpoint framework for exercising durability under process crashes and injected engine faults.

Crash harness (cmd/crash)

Replaces the previous kill-loop orchestrator with a controller/worker architecture:

• Deterministic workloads — pre-generated from a seed, fully reproducible
• NDJSON protocol — binary-safe controller↔worker communication over stdin/stdout
• Oracle verification — after every crash/reopen cycle, recovered state is checked against all acknowledged operations
• Replayable artifacts — each run writes a JSON artifact containing workload, events, cycle metadata, and any verification failure
• Subcommands: crash run, crash replay, crash worker

Failpoint framework (internal/crashhook)

Environment-variable-armed crash and fault injection hooks wired into the engine:

• Crash points: wal_after_append, wal_after_sync, flush_after_file_sync, flush_after_publish
• Fault points: wal_sync_error, sst_write_error, sst_publish_error

All hook sites are marked with // FAILPOINT: <name> for grep-ability.

Other changes

• make crash-check runs a short deterministic smoke run
• docs/testing.md updated with harness and fault injection documentation
• Engine-level subprocess tests for crash hooks (engine/crashhook_test.go)

v0.0.3 - SSTables v1

What's new

BeachDB can now persist data to disk. This release ships the SSTable format, the engine flush path, and the sst_dump inspection tool.

SSTable format (v1)

• Blocked layout: data blocks + index block + fixed-size footer
• Per-block CRC32C checksums on all data, index, and footer
• Full internal keys (user_key + seqno + kind) stored uncompressed
• Self-describing files — readable by sst_dump without running the engine

Writer

• Produces valid SSTables from a sorted stream of key-value entries
• Automatic block rotation at configurable target block size
• fsync on close, followed by fsync of the parent directory

Reader

• Validates footer magic, version, and checksum on open
• Eagerly loads index, lazily loads data blocks
• Point lookup with binary search on the index and cross-block key continuation
• Tombstone detection (ErrKeyDeleted)
• Safe for concurrent reads

Iterator

• Forward scan in internal-key order across all data blocks
• Lazy block loading

Engine flush path

• DB.flushMemtable() writes the active memtable to a new SSTable
• DB.Get() checks active memtable → immutable memtable → SSTables newest-first
• DB.Open() discovers and loads existing SSTables from the database directory

Auto-flush (background)

• WithMemtableFlushSize(n) option enables automatic background flushing
• Background goroutine using sync.Cond — no spinning, no channels
• Writer stall when an immutable memtable is already being flushed
• DB.Close() drains the flush goroutine before returning

sst_dump tool

• sst_dump <path> — prints footer metadata and block index
• sst_dump -entries <path> — prints all key-value entries

What's not in this release

• Merge iterator (cross-source iteration)
• Manifest/versioning (directory scanning is the current stopgap)
• Bloom filters, compaction, block cache, compression

v0.0.2 - Memtable

v0.0.2 - Memtable

The in-memory sorted structure that makes writes findable.

What's new

Memtable (skip list)

• Skip list implementation with O(log n) insert/lookup
• Internal key format: (user_key, seqno, kind) with MVCC semantics
• Key ordering: user_key ASC, seqno DESC (newest version first)
• Tombstone support for deletes
• Forward iterator over all entries
• Thread-safe with sync.RWMutex
• 98.9% test coverage

New packages

• internal/keys — InternalKey type with Compare, Encode, Decode
• internal/memtable — Memtable interface, SkipList, Iterator

Bug fixes

• Directory fsync: Fixed durability bug where WAL file could be orphaned after crash if directory wasn't synced

Other

• Upgraded to Go 1.26
• Reference model now supports MVCC with PutAt/DeleteAt/GetAt
• Documentation: docs/memtable.md, updated docs/scope.md

What's next

Milestone 4: SSTables — flush memtable to immutable sorted files on disk.

v0.0.1 - WAL v1 Milestone

Milestone 1: Durability and Recovery

BeachDB's first milestone: a write-ahead log that doesn't lie about durability.

What's New

Core Engine:

• WAL with record framing and CRC32C checksums
• Deterministic crash recovery on startup
• Atomic batch writes (future protocol boundary)
• Basic engine API: Open/Close, Put/Get/Delete, Write

Testing and Verification:

• Crash-loop proof tests with reference model
• Randomized corruption/truncation stress tests

Tools:

• wal_dump - Inspect WAL files and detect corruption
• crash - SIGKILL durability testing utility

Documentation:

• Complete WAL format specification
• Batch encoding format specification
• Runnable examples (basic usage, batches, recovery, options)

Blog Post

Read more: https://aalhour.com/posts/beachdb-wal-v1-milestone/

What's Next

Milestone 3: Memtable with clear semantics and reference-model randomized tests.

---

Note: BeachDB is a toy database built for learning. Not for production use.