Skip to content

Deterministic test infrastructure with injectable clocks and project reorganization#119

Merged
wallstop merged 5 commits intomainfrom
dev/wallstop/fix-z3
Mar 21, 2026
Merged

Deterministic test infrastructure with injectable clocks and project reorganization#119
wallstop merged 5 commits intomainfrom
dev/wallstop/fix-z3

Conversation

@wallstop
Copy link
Copy Markdown
Owner

Summary

Introduce an injectable ClockFn abstraction and in-memory ChannelSocket transport to enable fully deterministic, fast, and platform-independent integration tests — replacing real UDP sockets, thread::sleep(), and #[serial] test ordering throughout the test suite. Also reorganizes scripts and LLM skills into domain-based subdirectories.

Clock injection & deterministic time control

  • Add ClockFn type alias (Arc<dyn Fn() -> Instant + Send + Sync>) and ProtocolConfig::clock field for injectable time sources
  • Add ChaosSocket::with_clock() builder method for deterministic chaos/network-condition testing
  • Replace all Instant::now() calls in Protocol and ChaosSocket with the injected clock (falling back to system time)
  • Breaking: ProtocolConfig no longer implements Copy (due to Arc-based clock field)

New test infrastructure

  • ChannelSocket: in-memory NonBlockingSocket using mpsc channels — eliminates all real UDP I/O from tests
  • TestClock: manually-advanceable virtual clock with advance(duration), as_protocol_clock(), and as_chaos_clock() helpers
  • Deterministic test utilities: synchronize_sessions_deterministic(), poll_with_advance(), run_p2p_frame_advancement_test_deterministic()

Test suite migration

  • Converted all resilience, p2p, p2p_enum, and spectator tests from real sockets + thread::sleep + #[serial] to ChannelSocket + TestClock
  • Removed serial_test dependency and all #[serial] attributes from migrated tests
  • Removed #[cfg_attr(miri, ignore)] from timing-dependent chaos socket tests (now virtual-time-based)

Scripts & skills reorganization

  • Reorganized scripts/ into build/, ci/, docs/, verification/ subdirectories
  • Reorganized .llm/skills/ into 8 category subdirectories (rust-language, testing, formal-verification, etc.)
  • Added 8 new workflow skills (code-review, adversarial-review, dev-pipeline, investigation, etc.)
  • Added .llm/design-history/ and .llm/templates/ask-user-question.md

Documentation & CI

  • Updated user guide with custom clock documentation, new config presets, and expanded API reference
  • Updated all wiki pages with corrected examples, API signatures, and verification status
  • Updated all CI workflows and pre-commit hooks for new script paths
  • Expanded devcontainer with additional VS Code extensions and editor settings
  • Added logo banner SVG

Test plan

  • cargo nextest run --no-capture — all tests pass with deterministic infrastructure
  • cargo clippy --all-targets --features tokio,json — no warnings
  • CI workflows resolve scripts at new paths
  • Pre-commit hooks work with reorganized script layout
  • Verify no #[serial] tests remain in migrated test files

🤖 Generated with Claude Code

@github-actions github-actions Bot added the dependencies Pull requests that update a dependency file label Mar 21, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces deterministic time control and in-memory networking primitives to make integration tests fast and reproducible, while also reorganizing repository scripts/LLM skills and updating docs/CI paths accordingly.

Changes:

  • Add injectable clock support (ClockFn + ProtocolConfig::clock) and propagate it through session/protocol setup to enable virtual-time testing.
  • Introduce deterministic integration-test infrastructure (ChannelSocket, TestClock, deterministic polling/synchronization helpers) and migrate selected tests away from UDP + sleeps + serial ordering.
  • Reorganize scripts/ and .llm/skills/ into domain subdirectories and update docs/workflows/pre-commit hooks to the new paths.

Reviewed changes

Copilot reviewed 96 out of 140 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
wiki/assets/logo-banner.svg Adds new wiki banner asset.
wiki/Spec-Divergences.md Updates spec↔code links/paths and verification command path.
wiki/Migration.md Updates migration examples (builder ?, desync handling, Session trait).
wiki/Home.md Small homepage snippet updates (test count).
wiki/Fortress-vs-GGRS.md Updates examples/claims to match new APIs and stats.
wiki/Formal-Specification.md Updates formal spec text and message-shape snippets.
wiki/Determinism-Model.md Updates determinism doc examples to match new APIs.
wiki/Contributing.md Updates script paths after scripts reorg.
tests/verification/property.rs Adjusts checksum exchange polling logic (removed sleep).
tests/sessions/p2p_enum.rs Migrates enum-input P2P test to deterministic helper; removes serial.
tests/network/multi_process.rs Updates internal skill link path after skills reorg.
tests/config.rs Adjusts Copy/Clone expectations; adds ProtocolConfig Clone test.
tests/common/mod.rs Exposes/re-exports ChannelSocket + TestClock + deterministic helpers.
src/sessions/builder.rs Clones protocol_config where Copy is no longer available.
src/lib.rs Re-exports ClockFn from sessions config.
specs/tla/README.md Updates verification script paths and skill links.
scripts/verification/verify-tla.sh Relocates script and fixes project root resolution.
scripts/verification/verify-kani.sh Relocates script and fixes project root resolution.
scripts/verification/verify-all.sh Relocates script and fixes project root resolution.
scripts/verification/check-kani-coverage.sh Updates paths to relocated Kani scripts.
scripts/verification/check-kani-coverage.py Updates project root and verify-kani.sh path.
scripts/tests/test_validate_wiki_output.py Updates import path for moved docs scripts.
scripts/tests/test_sync_wiki.py Updates sys.path/import path for moved docs scripts.
scripts/tests/test_check_wiki_consistency.py Updates sys.path/import path for moved docs scripts.
scripts/tests/test_check_llm_skills.sh Updates path to moved check-llm-skills.sh.
scripts/tests/test_check_links.py Updates sys.path/import path for moved docs scripts.
scripts/tests/test_cargo_linker.py Updates sys.path to import from scripts/build.
scripts/hooks/regenerate-skills-index.py Recurses skills subdirs and uses relative paths in index links.
scripts/docs/verify-markdown-code.sh Updates usage header and project root derivation.
scripts/docs/validate-wiki-output.py Updates usage header and project root derivation.
scripts/docs/sync-wiki.py Updates usage header examples to new script path.
scripts/docs/check-wiki-consistency.py Updates usage header and sync script path.
scripts/docs/check-rustdoc-links.py Imports shared cargo linker helper from scripts/build.
scripts/docs/check-llm-skills.sh Updates repo-root derivation for new location.
scripts/docs/check-links.sh Updates usage header and repo-root derivation.
scripts/docs/check-links.py Updates docstring and project-root derivation.
scripts/docs/check-code-fence-syntax.sh Updates usage header examples to new script path.
scripts/docs/init.py Marks scripts/docs as a Python package.
scripts/ci/verify-sccache.sh Updates usage header to new path.
scripts/ci/test-verify-sccache.sh Fixes workflow-dir path after scripts reorg.
scripts/ci/docker-network-tests.sh Updates usage header and project root derivation.
scripts/ci/check-tools.sh New devcontainer tool verification script.
scripts/ci/check-shell-portability.sh Updates usage header and project root derivation.
scripts/build/run-cargo-fmt.py New pre-commit wrapper to run fmt and stage only formatted Rust files.
scripts/build/run-cargo-clippy.py New pre-commit wrapper to run clippy with linker fallback env.
scripts/build/check-cargo-hack.py New optional feature-powerset check wrapper for pre-commit.
scripts/build/cargo_linker.py New shared helper to disable custom linker flags when lld isn’t available.
scripts/build/init.py Marks scripts/build as a Python package.
docs/specs/spec-divergences.md Updates verification script path.
docs/contributing.md Updates script paths after scripts reorg.
docker/README.md Updates docker test script path after scripts reorg.
Cargo.toml Bumps crate version to 0.7.0.
Cargo.lock Updates crate version to 0.7.0.
CLAUDE.md Adds clarifying-question template reference and test-output rule.
CHANGELOG.md Adds 0.7.0 entry for clock injection / ProtocolConfig Copy break; updates script paths.
AGENTS.md Adds clarifying-question template reference and test-output rule.
.typos.toml Updates reference path after skills reorg.
.pre-commit-config.yaml Updates hook entries and file globs for reorganized scripts dirs.
.llm/templates/ask-user-question.md New template for concise clarifying questions.
.llm/skills/workflows/user-facing-docs.md New workflow skill for public-doc updates.
.llm/skills/workflows/review-readiness.md New workflow skill for pre-review gating.
.llm/skills/workflows/design-decisions.md New workflow skill for design decision logging.
.llm/skills/workflows/code-review.md New structured review workflow guidance.
.llm/skills/workflows/adversarial-review.md New adversarial review guidance.
.llm/skills/workflows/adversarial-handoff.md New escalation/handoff workflow.
.llm/skills/testing/testing.md Adds “no tail/head on test output” guidance.
.llm/skills/testing/network-chaos-testing.md New chaos testing skill (moved/expanded).
.llm/skills/testing/mutation-testing.md New mutation testing skill.
.llm/skills/testing/fuzzing.md New fuzzing skill.
.llm/skills/rust-language/zero-copy.md New Rust zero-copy skill.
.llm/skills/rust-language/type-driven-design.md New type-driven design skill.
.llm/skills/rust-language/text-parsing.md New text parsing skill.
.llm/skills/rust-language/rust-pitfalls.md New Rust pitfalls skill.
.llm/skills/rust-language/rust-idioms.md New Rust idioms skill.
.llm/skills/rust-language/rust-design-patterns.md New Rust design patterns skill.
.llm/skills/rust-language/refactoring.md New refactoring skill.
.llm/skills/rust-language/no-std.md New no_std skill.
.llm/skills/rust-language/ffi.md New FFI skill.
.llm/skills/rust-language/defensive-programming.md New defensive programming (zero-panic) skill.
.llm/skills/rust-language/concurrency.md New concurrency skill.
.llm/skills/rust-language/async-rust.md New async Rust skill.
.llm/skills/rust-language/api-design.md New public API design skill.
.llm/skills/publishing-organization/changelog.md New changelog practices skill.
.llm/skills/performance-quality/performance.md Updates reference to moved cargo_linker helper path.
.llm/skills/performance-quality/binary-size.md New binary size optimization skill.
.llm/skills/formal-verification/verification.md Updates verification script path reference.
.llm/skills/formal-verification/kani.md Updates verify-kani/check-kani-coverage path references.
.llm/skills/determinism-rollback/dst.md New deterministic simulation testing (DST) skill.
.llm/skills/ci-cd-tooling/wiki-sync.md Updates wiki sync script paths and relative-link rules after skills reorg.
.llm/skills/ci-cd-tooling/markdown.md Updates docs script paths and relative-link rules after skills reorg.
.llm/skills/ci-cd-tooling/github-actions.md New GitHub Actions best practices skill.
.llm/skills/ci-cd-tooling/doc-code-sync.md New doc↔code consistency skill.
.llm/skills/ci-cd-tooling/ci-debugging.md Updates Kani coverage script path reference.
.llm/design-history/sync-layer.txt Adds sync-layer decision-log entry.
.llm/design-history/network.txt Adds network decision-log entries.
.llm/design-history/README.md Documents design-history format.
.llm/context.md Updates core repo guidance (test output, moved skill links, new workflow links).
.github/workflows/wiki-sync.yml Updates docs script paths in wiki sync workflow.
.github/workflows/publish.yml Updates sccache verify script path.
.github/workflows/ci-verification.yml Updates verification script paths and sccache script path.
.github/workflows/ci-security.yml Updates sccache verify script path.
.github/workflows/ci-rust.yml Updates verification script paths and sccache script path.
.github/workflows/ci-network.yml Updates docker test script path and sccache script path.
.github/workflows/ci-llm-lint.yml Updates check-llm-skills script path.
.github/workflows/ci-docs.yml Updates docs script paths (wiki/links/code-fence/markdown verify).
.github/workflows/ci-benchmarks.yml Updates sccache verify script path.
.github/copilot-instructions.md Adds clarifying-question template reference.
.devcontainer/welcome.sh Updates verification script paths shown to users.
.devcontainer/devcontainer.json Updates postCreate tool-check command and adds editor extensions/settings.
.devcontainer/README.md Updates verification/tool-check script paths.
.config/nextest.toml Removes serial test-group config now that port conflicts are reduced.

Comment thread wiki/Spec-Divergences.md Outdated
Comment thread tests/verification/property.rs Outdated
Comment thread wiki/Formal-Specification.md Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 114 out of 158 changed files in this pull request and generated 1 comment.

Comment on lines +1457 to 1461
use crate::common::test_utils::{
bind_socket_with_retry, synchronize_sessions, SyncConfig, POLL_INTERVAL_DETERMINISTIC,
};

/// Property: sync_health returns Pending when desync detection is off
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test module still relies on real UDP sockets (bind_socket_with_retry) and #[serial] ordering, which conflicts with the PR description’s claim of eliminating real UDP I/O / serial_test via ChannelSocket + virtual time. If the intent is full migration, consider switching this test to ChannelSocket (and dropping #[serial]), or update the PR description/scope to reflect that some P2P checksum tests still use OS sockets (including Windows retry sleeps inside bind_socket_with_retry).

Copilot uses AI. Check for mistakes.
@wallstop wallstop merged commit 9b8b6bd into main Mar 21, 2026
82 checks passed
@wallstop wallstop deleted the dev/wallstop/fix-z3 branch March 21, 2026 01:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants