Add analysis rule engine foundation (Phase 1-2: ADR, resolver, decisi…#129
Add analysis rule engine foundation (Phase 1-2: ADR, resolver, decisi…#129
Conversation
…ons, helpers) Phase 1-2 of the Hy-based declarative rule engine for hypothesis evaluation. All pure Python — no Hy dependency needed yet. - ADR-0054: documents all architectural decisions (advisor pattern, two-engine coexistence, evidence resolver, audit-first, AI-60 ceiling as predicate, priority first-match, dirty-tree refusal) - EvidenceResolver: resolves STIX IDs to source_platform and TRUST_LEVEL via WorkspaceStore bulk lookup with per-eval cache - Decision types: SetStatusDecision, AnnotateDecision, NoOpDecision with transition-slot semantics (frozen dataclasses) - RuleContext: evaluation-scoped immutable context (resolver, policy) - RuleEvaluationResult: collects firings + errors, primary_decision - RuleEnginePolicy: 7-field config from INI [rules] section with GNAT_ALLOW_DIRTY_RULES env var override, feature flag default OFF - 26 helper functions across 6 modules (evidence, confidence, temporal, status, policy, source) — all plain Python - WorkspaceStore: get_source_platform + get_source_platforms_bulk - IS_AI_CONNECTOR attribute added to ChatGPT and Copilot connectors - 82 tests passing, ruff clean https://claude.ai/code/session_01H5UbjsuiiGya5n1eUCxoaR
There was a problem hiding this comment.
Pull request overview
Lays the groundwork for a declarative hypothesis-evaluation rule engine in gnat.analysis.rules, including resolver/policy/decision primitives, helper predicates, and initial unit tests, plus an ADR documenting the architecture.
Changes:
- Added core rule-engine primitives (policy, context, decisions, evaluation result) and helper predicate modules (evidence/confidence/temporal/status/source/policy).
- Implemented
EvidenceResolverbacked by newWorkspaceStoresource-platform lookup methods (single + bulk) and addedIS_AI_CONNECTORmarkers to AI connectors. - Added ADR-0054 and unit tests covering resolver/policy/decisions and helper functions.
Reviewed changes
Copilot reviewed 28 out of 30 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/analysis/rules/test_resolver.py | Adds unit tests for EvidenceResolver and trust_at_least. |
| tests/unit/analysis/rules/test_policy.py | Adds unit tests for RuleEnginePolicy INI/env parsing. |
| tests/unit/analysis/rules/test_decisions.py | Adds unit tests for decision dataclasses/factories and RuleEvaluationResult. |
| tests/unit/analysis/rules/helpers/test_temporal.py | Tests temporal helper predicates. |
| tests/unit/analysis/rules/helpers/test_status.py | Tests status helper predicates. |
| tests/unit/analysis/rules/helpers/test_source.py | Tests source/trust helpers (AI-only, trust aggregation, etc.). |
| tests/unit/analysis/rules/helpers/test_policy.py | Tests AI ceiling policy helper predicate. |
| tests/unit/analysis/rules/helpers/test_evidence.py | Tests evidence counting/ratio helpers. |
| tests/unit/analysis/rules/helpers/test_confidence.py | Tests confidence/reliability/credibility helpers. |
| tests/unit/analysis/rules/helpers/init.py | Test package marker for helper tests. |
| tests/unit/analysis/rules/conftest.py | Adds shared hypothesis/context builders for rule-engine tests. |
| tests/unit/analysis/rules/init.py | Test package marker for rule-engine tests. |
| gnat/context/store.py | Adds get_source_platform and get_source_platforms_bulk queries to WorkspaceStore. |
| gnat/connectors/copilot/client.py | Marks Copilot connector as AI via IS_AI_CONNECTOR. |
| gnat/connectors/chatgpt/client.py | Marks ChatGPT connector as AI via IS_AI_CONNECTOR. |
| gnat/analysis/rules/result.py | Introduces RuleEvaluationResult and RuleFiring with primary_decision. |
| gnat/analysis/rules/resolver.py | Introduces EvidenceResolver, ResolvedEvidence, and trust_at_least. |
| gnat/analysis/rules/policy.py | Introduces RuleEnginePolicy loaded from INI with env override. |
| gnat/analysis/rules/helpers/temporal.py | Adds temporal helper predicates (age/stale/fresh). |
| gnat/analysis/rules/helpers/status.py | Adds status helper predicates. |
| gnat/analysis/rules/helpers/source.py | Adds source/trust helpers including AI-only detection. |
| gnat/analysis/rules/helpers/policy.py | Adds AI confidence ceiling predicate. |
| gnat/analysis/rules/helpers/evidence.py | Adds evidence count/ratio helpers. |
| gnat/analysis/rules/helpers/confidence.py | Adds confidence/reliability/credibility helpers. |
| gnat/analysis/rules/helpers/init.py | Package marker for rule predicate helpers. |
| gnat/analysis/rules/decisions.py | Adds decision dataclasses and factory functions. |
| gnat/analysis/rules/context.py | Adds immutable per-evaluation RuleContext. |
| gnat/analysis/rules/init.py | Adds package docstring describing rule engine and install note. |
| docs/explanation/architecture/adrs/README.md | Registers ADR-0054 in the ADR index. |
| docs/explanation/architecture/adrs/0054-ADR-analysis-rule-engine.md | Adds ADR documenting architecture and key rule-engine decisions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| .filter( | ||
| WorkspaceObjectModel.workspace_id == workspace_id, | ||
| WorkspaceObjectModel.stix_id.in_(stix_ids), | ||
| WorkspaceObjectModel.is_deleted == False, # noqa: E712 |
There was a problem hiding this comment.
In SQLAlchemy, comparing boolean columns with == False is discouraged and can behave differently than expected depending on dialect; it also forces a # noqa: E712. Prefer WorkspaceObjectModel.is_deleted.is_(False) (or switch back to .filter_by(..., is_deleted=False) if compatible) to avoid linter suppression and ensure correct boolean semantics.
| WorkspaceObjectModel.is_deleted == False, # noqa: E712 | |
| WorkspaceObjectModel.is_deleted.is_(False), |
| ctx = make_context(platform_map={"ind-1": "crowdstrike"}) | ||
| from gnat.clients import CLIENT_REGISTRY | ||
| cls = CLIENT_REGISTRY.get("crowdstrike") | ||
| if cls and getattr(cls, "TRUST_LEVEL", "") == "trusted_internal": | ||
| assert has_trusted_evidence(h, ctx) is True |
There was a problem hiding this comment.
This test is effectively a no-op on current registry values: crowdstrike is registered but its TRUST_LEVEL is semi_trusted, so the if ... == "trusted_internal" branch never runs and nothing is asserted. Make the test deterministic by using a connector that is actually trusted_internal (e.g. splunk) or by monkeypatching CLIENT_REGISTRY to include a temporary fake connector class with TRUST_LEVEL="trusted_internal".
| ctx = make_context(platform_map={"ind-1": "crowdstrike"}) | |
| from gnat.clients import CLIENT_REGISTRY | |
| cls = CLIENT_REGISTRY.get("crowdstrike") | |
| if cls and getattr(cls, "TRUST_LEVEL", "") == "trusted_internal": | |
| assert has_trusted_evidence(h, ctx) is True | |
| ctx = make_context(platform_map={"ind-1": "splunk"}) | |
| assert has_trusted_evidence(h, ctx) is True |
| as ``.hy`` files, loaded dynamically, and return status transition | ||
| decisions without mutating state directly. | ||
|
|
||
| Install Hy dependency with ``pip install "gnat[rules]"``. |
There was a problem hiding this comment.
The module docstring suggests installing Hy via pip install "gnat[rules]", but there is currently no rules optional-dependency extra defined in pyproject.toml (only extras like analysis, nlp, telemetry, etc.). Either add a rules = ["hy>=…"] extra (and any other needed deps) or adjust this text to match the actual packaging plan.
| Install Hy dependency with ``pip install "gnat[rules]"``. | |
| This module requires Hy support to author or load ``.hy`` rule files. |
…ons, helpers)
Phase 1-2 of the Hy-based declarative rule engine for hypothesis evaluation. All pure Python — no Hy dependency needed yet.
https://claude.ai/code/session_01H5UbjsuiiGya5n1eUCxoaR