Add cross-platform investigation example and fix three module bugs by wrhalpin · Pull Request #72 · wrhalpin/GNAT

wrhalpin · 2026-04-07T07:11:45Z

examples/investigation_xsoar_tq_gm_powerbi.py:

End-to-end script: XSOAR + ThreatQ + GreyMatter → EvidenceGraph → GNAT workspace → Power BI xlsx export
Exercises all three expand paths (alerts/tasks/timeline, observables/tasks, indicators/adversaries) plus seed expansion across all three platforms
--mock flag runs entirely in-process with no live credentials required
Completeness check verifies 14 investigation methods across 3 platforms, confirms incident nodes from every platform, and counts cross-platform correlation edges

Bugs found and fixed by the example:

builder.py (CASE_ID seed expansion):

_safe_call wraps a single-dict get_object response in a list; the branch was passing the list directly to normalize() instead of iterating it

normalizer.py (ThreatQ dispatch):

Missing ("threatq", "incident") alias — builder calls normalize(platform, "incident", raw) for all CASE_ID seeds, but ThreatQ only had "event" and "observed-data" entries; ThreatQ events are the investigation container so the alias routes to _tq_event

workspace.py (STIXBase attribute assignment):

_node_to_stix_base and Relationship tagging used obj["key"] = value item assignment; STIXBase exposes _properties via setattr not setitem; changed to obj.key = value attribute access — fixes "0 nodes materialised" regression and allows the workspace to reach 14 nodes + 17 edges

https://claude.ai/code/session_01BDoue9HxB83ijLzFARAugq

examples/investigation_xsoar_tq_gm_powerbi.py: - End-to-end script: XSOAR + ThreatQ + GreyMatter → EvidenceGraph → GNAT workspace → Power BI xlsx export - Exercises all three expand paths (alerts/tasks/timeline, observables/tasks, indicators/adversaries) plus seed expansion across all three platforms - --mock flag runs entirely in-process with no live credentials required - Completeness check verifies 14 investigation methods across 3 platforms, confirms incident nodes from every platform, and counts cross-platform correlation edges Bugs found and fixed by the example: builder.py (CASE_ID seed expansion): - _safe_call wraps a single-dict get_object response in a list; the branch was passing the list directly to normalize() instead of iterating it normalizer.py (ThreatQ dispatch): - Missing ("threatq", "incident") alias — builder calls normalize(platform, "incident", raw) for all CASE_ID seeds, but ThreatQ only had "event" and "observed-data" entries; ThreatQ events are the investigation container so the alias routes to _tq_event workspace.py (STIXBase attribute assignment): - _node_to_stix_base and Relationship tagging used obj["key"] = value item assignment; STIXBase exposes _properties via __setattr__ not __setitem__; changed to obj.key = value attribute access — fixes "0 nodes materialised" regression and allows the workspace to reach 14 nodes + 17 edges https://claude.ai/code/session_01BDoue9HxB83ijLzFARAugq

Copilot

Pull request overview

Adds a cross-platform investigations example (XSOAR + ThreatQ + GreyMatter → GNAT workspace → Power BI export) and fixes several investigations-module regressions uncovered by the example.

Changes:

Fix seed expansion for CASE_ID/ALERT_ID/TICKET_REF to iterate _safe_call(get_object(...)) results correctly.
Add ThreatQ ("threatq", "incident") normalizer alias to route CASE_ID seed normalization to the ThreatQ event handler.
Fix workspace materialization to use attribute assignment for STIXBase/Relationship custom properties, and add a new end-to-end example script.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
`gnat/investigations/workspace.py`	Adjusts metadata tagging on STIX objects/relationships during materialization.
`gnat/investigations/normalizer.py`	Adds ThreatQ incident→event dispatch alias for seed normalization.
`gnat/investigations/builder.py`	Fixes CASE_ID seed expansion to iterate list-wrapped `get_object()` results.
`examples/investigation_xsoar_tq_gm_powerbi.py`	Introduces an end-to-end investigation + Power BI export example with mock mode and completeness checks.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-07T07:17:06Z

 def _node_to_stix_base(node: EvidenceNode) -> STIXBase:
    """Wrap a normalised node's STIX dict as a :class:`~gnat.orm.base.STIXBase`."""
    stix_type = node.stix.get("type", "x-evidence-node")
    obj = STIXBase(stix_type=stix_type, **{
        k: v for k, v in node.stix.items() if k != "type"
    })
    # Tag with investigation metadata not already in the STIX dict
-    obj["x_evidence_node_id"]   = node.node_id
-    obj["x_evidence_node_type"] = node.node_type
-    obj["x_source_platform"]    = node.platform
-    obj["x_source_id"]          = node.source_id
+    obj.x_evidence_node_id   = node.node_id
+    obj.x_evidence_node_type = node.node_type
+    obj.x_source_platform    = node.platform


_node_to_stix_base() attempts to set the STIX type by passing stix_type=... into STIXBase(...), but STIXBase.__init__ does not consume that kwarg and STIXBase.to_dict() uses the stix_type attribute (defaulting to "stix-object") for the type field. As a result, materialized nodes will serialize with an incorrect STIX type. Set obj.stix_type = stix_type (instance attribute) after construction, or construct the correct ORM subclass based on node.stix['type'] so to_dict() emits the right type.

Copilot · 2026-04-07T07:17:07Z

+
+def _mock_xsoar() -> MagicMock:
+    """Return a MagicMock that mimics the XSOAR investigation sub-API."""
+    c = MagicMock(name="XSOARClient")


In --mock mode, these mocks are created as unrestricted MagicMocks. That makes hasattr(mock, "some_method") always return True and can also cause _safe_call(list(result)) to silently return empty lists for unconfigured methods, so the “completeness check” and expansion behavior can become misleading. Consider using MagicMock(spec_set=...) / create_autospec(...) against the real client classes (or explicitly setting only the required methods) so missing methods are detected reliably.

Suggested change

c = MagicMock(name="XSOARClient")

c = MagicMock(

name="XSOARClient",

spec_set=("get_object", "list_objects", "search_indicators_by_value"),

)

Copilot · 2026-04-07T07:17:07Z

+            failures.append(f"ThreatQ missing method: {method}")
+
+    # ── At least one node per platform ───────────────────────────────────────
+    from gnat.investigations.model import EvidenceGraph


EvidenceGraph is imported inside _verify_investigations_module() but never used, which will fail Ruff's F401 (unused import) when CI runs ruff check .. Remove this import (or use it for a real runtime/type check if intended).

Suggested change

from gnat.investigations.model import EvidenceGraph

Copilot · 2026-04-07T07:17:07Z

+    Assert that the investigations module is complete for all three platforms.
+
+    Checks:
+    - Each expand method was found on its connector (hasattr)
+    - At least one node was collected per platform
+    - Cross-platform correlation found at least one shared IOC (IP / domain
+      appears in all three platforms' data)


The _verify_investigations_module() docstring says it asserts cross-platform correlation and that a shared IOC appears in all three platforms, but the implementation only warns when no cross-platform edges are found and never checks for shared IOC values specifically. Update the docstring to match behavior, or add an explicit check (and failure) for shared IOCs when running as a completeness gate.

Suggested change

Assert that the investigations module is complete for all three platforms.

Checks:

- Each expand method was found on its connector (hasattr)

- At least one node was collected per platform

- Cross-platform correlation found at least one shared IOC (IP / domain

appears in all three platforms' data)

Verify that the investigations module wiring is present for all three platforms.

Checks:

- Each expected expand method is present on its connector (via ``hasattr``)

- At least one node was collected per platform

- Whether any cross-platform correlation edges were created in the graph

Note:

This helper does not currently assert that a specific IOC value appears in

all three platforms' data; it only validates the available connector

surface and observed graph coverage/correlation.

Copilot · 2026-04-07T07:17:08Z

+    if failures:
+        print(f"  FAILURES ({len(failures)}):")
+        for f in failures:
+            print(f"    [FAIL] {f}")
+        print("=" * 70 + "\n")
+        logger.error("Investigations module completeness check FAILED: %s", failures)
+    else:
+        print("  All checks passed:")
+        total_methods = len(xsoar_methods) + len(gm_methods) + len(tq_methods)
+        print(f"    [OK] {total_methods} investigation methods verified across 3 platforms")
+        print(f"    [OK] Nodes collected from: {sorted(nodes_by_platform.keys())}")
+        print(f"    [OK] Incident nodes present for: {sorted(incident_platforms)}")
+        if cross_edges:
+            print(f"    [OK] {len(cross_edges)} cross-platform correlation edges")
+        print("=" * 70 + "\n")
+        logger.info("Investigations module completeness check PASSED.")


Even when failures are detected, the script only logs an error and still returns exit code 0 from main(), which makes it hard to use as an automated completeness check. Consider exiting non-zero (e.g., raise an exception or return 1) when failures is non-empty so CI/users can reliably detect a failed run.

Copilot AI review requested due to automatic review settings April 7, 2026 07:11

wrhalpin merged commit a6c216b into main Apr 7, 2026
11 of 19 checks passed

Copilot started reviewing on behalf of wrhalpin April 7, 2026 07:12 View session

Copilot AI reviewed Apr 7, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add cross-platform investigation example and fix three module bugs#72

Add cross-platform investigation example and fix three module bugs#72
wrhalpin merged 1 commit intomainfrom
claude/add-claude-documentation-k8vvJ

wrhalpin commented Apr 7, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

Copilot AI Apr 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

wrhalpin commented Apr 7, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants