* test(stats): Monte Carlo validation of brain.prove() trend test (closes #8) Issue #8 asked for multi-user validation of a claimed paired t-test. Correction: brain.prove() uses Mann-Kendall (_stats.trend_analysis) — a non-parametric rank test that is more robust than a paired t-test for integer correction counts (non-normal, heteroskedastic, heavy-tied). Rather than blocking on real multi-user data, validate statistically via Monte Carlo simulation where ground truth is known by construction. Tests (400 trials each): - Type I error ≈ α on normal, integer-count, heavy-tailed, skewed noise - Power >80% under real −0.3/session downtrend + gaussian noise - Power non-decreasing as n grows (10 → 30) - Robustness: n<2 returns no-signal, all-ties → p=1.0, heavy ties don't inflate false-positive rate Seeded for determinism, ~0.5s runtime. Closes #8. Co-Authored-By: Gradata <noreply@gradata.ai> * test(stats): address ultrareview blockers (tighten bounds, add integration) - Fix flaky power-growth test: require p_long >= p_short + 0.10 (bound well above one SE) instead of bare >= which could flip on a single trial. - Tighten tie-heavy Type I bound from <= 0.12 to [0.02, 0.10] to match other Type I tests — actually validates tie correction rather than letting 12% false-positives through. - Fix misleading '95% CI' comment (bounds are ~99.9% CI). - Add end-to-end guard: brain_prove correctly reads p_value (not slope) for the 'strong' gate — catches regressions in the consumer of trend_analysis. - Add 50-element cap regression test (trend_analysis silently truncates). - Docstring: clarify methodology rationale vs real-data study (Laplace(0,1) not "Laplace-ish", acknowledge external-validity follow-up). Co-Authored-By: Gradata <noreply@gradata.ai> * test(stats): pytest.approx for float equality (CR nit) Co-Authored-By: Gradata <noreply@gradata.ai> * test(stats): pytest.approx on remaining float eq + S311 noqa (CR nits) Co-Authored-By: Gradata <noreply@gradata.ai> * test(stats): ASCII-only docstrings + decouple prove test from brain_efficiency (CR nits) - Replace ambiguous Unicode (alpha, sigma, minus, arrows, em-dashes) with ASCII equivalents to satisfy Ruff RUF001/RUF002. - Patch gradata._core.brain_efficiency in the end-to-end prove test so the assertion is deterministic regardless of the real efficiency path. Co-Authored-By: Gradata <noreply@gradata.ai> * test(stats): strengthen cap test + return type annotations (CR nits) - Replace zeros prefix with down-trend prefix in test_fifty_element_cap_is_applied so the test actually distinguishes last-50 (correct) from first-50 (broken). Previous version would have passed either way. - Add return type annotations on gen/factory helpers (ANN202). Co-Authored-By: Gradata <noreply@gradata.ai> --------- Co-authored-by: Gradata <noreply@gradata.ai>

brain.prove() needs validation with real multi-user data #8

Description

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions