chore: harden codebase for AI coding agents (Tier 1 sensors)

[harrymunro]

Summary

• Install Ruff with AI-targeted thresholds in a new pyproject.toml so every lint message becomes self-correction feedback for an agent.
• Add .pre-commit-config.yaml (Gitleaks + Ruff + standard hygiene) and a new CI job (python-lint + secret-scan) so the same checks fire locally, on commit, and in CI.
• Document the sensors in CLAUDE.md and AGENTS.md — including the suppress-with-reason convention and the brownfield refactor backlog — so future agent invocations know to run them and how to interpret violations.

Based on the sensors-as-prompts and suppressions-as-review-anchors ideas from Birgitta Böckeler, Maintainability sensors for coding agents (martinfowler.com), via the harden-for-agents skill.

Note on scope

This PR also lands the merge of PR #120 (nelson_data_patterns.py learned-standing-orders pipeline, ~2,400 LOC). That feature is not part of the Tier 1 sensors work — it merged into this branch via 9113af9 before the hardening commit and rides along to main. The Tier 1 sensors changes themselves are confined to pyproject.toml, .pre-commit-config.yaml, .github/workflows/ci.yml, CLAUDE.md, AGENTS.md, docs/project_structure.md, plus the auto-fixes and reasoned noqas the new ruff config required.

Adversarial-review remediation (commits 361a506..82ebd74)

A five-agent adversarial review surfaced three CRITICAL issues plus a long tail of HIGH/MEDIUM follow-ups. Applied in five CI-safe commits:

1. fix: restore LICENSE and README ownership — Revert LICENSE copyright to "2025 Harry Munro" (matches main); fix the stale aspegio/nelson marketplace path in README quick-start to harrymunro/nelson.
2. chore: tighten lint config and triage surfaced violations — Drop the bogus PLR0913 global ignore (the previous "covered by PLR0917" comment was wrong — PLR0917 is preview-only). The rule now fires; test/conftest helpers covered via per-file ignore, four production sites get reasoned per-function noqas pointing at the nelson-e6j refactor backlog. Also: anchor test glob to **/test_*.py, drop S108 from test ignore (replaced with one per-line noqa for a single placeholder), tighten the S603 rationale, delete the decorative [tool.mypy] block (roll-out tracked under nelson-8q8), and delete the test_regex*.py scratch files at the repo root (nelson-6bw).
3. chore: suppression hygiene + safer subprocess test pattern — Add a reason to the bare # type: ignore on hooks/nelson_hooks.py:653 (matches the documented shape on line 625). Refactor the _write_json crash/cleanup tests in test_nelson_data_memory.py to pass paths via env= injection rather than interpolating fixture values into a python -c string — the S603 suppression rationale becomes minimal and accurate.
4. ci: SHA-pin actions and pre-commit, add parity job, parallel tests — SHA-pin every third-party uses: (tag preserved as trailing comment); pre-commit autoupdate --freeze for the three hook repos; new pre-commit CI job using pre-commit/action@<sha> to close local↔CI parity for trailing-whitespace, end-of-file-fixer, check-{yaml,toml,json}, check-added-large-files, check-merge-conflict, ruff, and gitleaks; pin GITLEAKS_VERSION: 8.30.0 in the secret-scan job to match the pre-commit engine; cache: pip + cache-dependency-path; concurrency.cancel-in-progress scoped to PRs only; test job becomes a matrix of [skills/nelson/scripts, hooks, scripts] with fail-fast: false.
5. docs: soften brownfield wording, trim AGENTS.md, document sensors — Rewrite the Brownfield complexity backlog paragraph in CLAUDE.md so it no longer creates a perverse incentive to avoid flagged functions; collapse the duplicated Maintainability sensors section in AGENTS.md to a single pointer at CLAUDE.md; add pyproject.toml, .pre-commit-config.yaml, and .github/workflows/ci.yml to docs/project_structure.md; add a Local development section to CONTRIBUTING.md.

What's in the lint config

Setting	Value	Why
max-args	5	Agents over-stuff signatures; dataclasses/typed dicts are usually the answer
max-branches	10	Caps inflation of cmd_* / main() handlers
max-statements	50	Same
max-complexity	10	Mccabe gate
max-returns	6
line-length	120	Brownfield baseline; codebase comfortable at 110-120, p75 confirms
Per-dir ignores	scripts/, hooks/, **/test_*.py, conftest.py	Each layer has legitimate uses of subprocess, asserts, late imports, print()

Triage outcome (brownfield)

Started at 183 violations after enabling the rules; ended at 0:

• Auto-fixed (mechanical): import order, datetime UTC, unused imports.
• Real bugs fixed: missing check= and strict= kwargs, unused variables, redefined loop variables, three lazy import json calls moved to module top.
• Suppressed with reason on the largest functions (# noqa: C901, PLR0912, PLR0915 -- ... refactor tracked in nelson-e6j). Brownfield-safe — the rules still fire if anything gets worse, and the suppressions point at a refactor backlog rather than disappearing into noise.
• Disabled globally with rationale: SIM108 (if/else→ternary often degrades readability), RUF002 (Unicode in docstrings is intentional prose).
• Format: one-time ruff format baseline applied across 22 files.

What's deferred (filed as follow-up issues)

• mypy strict mode (nelson-8q8) — ≈82% of non-test functions annotated (178/217 across 12 files; scripts/count-tokens.py has 0/6), but strict mode would drown out the linter signal. Roll out incrementally.
• Tier 2 (nelson-7rm) — coverage gate, import-linter contracts, Semgrep SAST.
• Tier 3 — mutation testing, scheduled inferential AI reviews.
• Unify pytest config (nelson-y2l) — each test dir has its own conftest.py with overlapping fixture names; CI runs them as a matrix for now.
• Narrow PLR0915/PLR0912 blanket ignore in test files — surface and triage ~5-10 large-test refactor candidates (new follow-up to file post-merge).
• Replace gitleaks-action@v2 with the direct gitleaks binary — closes the fork-PR fail-open scenario where GITLEAKS_LICENSE is missing (new follow-up to file post-merge).

Test plan

• ruff check — All checks passed
• ruff format --check — 25 files already formatted
• pytest skills/nelson/scripts/ — 353 passed
• pytest hooks/ — 64 passed
• pytest scripts/ — 21 passed
• pre-commit run --all-files — all hooks pass
• ruff check --select PLR0913 . — exits 0 (sensor actually fires now)
• CI green on push (new python-lint, pre-commit, and matrix test jobs alongside existing secret-scan, markdown, yaml, links, spelling, references)
• After merge: pre-commit install once locally so hooks fire on every commit