update report h6

2026-07-20 23:47:09 +07:00
parent f462079435
commit 4b6819f578
38 changed files with 1244 additions and 62 deletions
@@ -197,6 +197,26 @@ docker volume rm casan-local-prod_minio-data
 This is destructive and cannot be undone. Do not use it if the MinIO bucket has
 evidence you need to retain.

+### Python used by the local auth bridge
+
+`local-full.sh` validates the Python interpreter before starting the host-side
+provider auth bridge. On macOS it prefers `/usr/bin/python3`, avoiding an old
+Intel-only framework Python that may be first in `PATH` but terminated by macOS.
+To select another working Python explicitly, set `CASAN_PYTHON_BIN`:
+
+```bash
+CASAN_PYTHON_BIN=/opt/homebrew/bin/python3 \
+  bash packages/casan-harness/scripts/bash/local-full.sh start
+```
+
+The launcher also validates Docker's configured credential helper. If macOS
+terminates that helper, CASAN uses an isolated, temporary anonymous Docker
+configuration for this local stack's public images while preserving the active
+Docker Desktop socket and Compose plugin. The same fallback uses Docker's classic
+builder if the bundled BuildKit metadata helper is also terminated. It does not
+edit `~/.docker/config.json` or delete stored registry credentials. Reinstall or
+update Docker Desktop later to repair the helper globally.
+
 ## Current scope and next phase

 This local environment includes every currently implemented CASAN component.
@@ -0,0 +1,90 @@
+# CASAN Plan-19 — Harness Reporting Contract
+
+## Purpose
+
+Turn harness evidence into reviewable, portable reports. H6 is the first implemented
+report; H1–H7 share one contract so later reports do not invent incompatible envelopes,
+filter semantics or verdict fields.
+
+## Source-of-truth boundary
+
+H6 is a read-only projection over three canonical files. The report API and the existing
+Control Panel resolve these paths through the same `app-root.ts` contract; harness writers
+resolve them through `casan-paths.sh`.
+
+| Source | Canonical environment variable | Default path | Required |
+|---|---|---|---|
+| Runtime metrics | `CASAN_TELEMETRY_METRICS_LOG` | `.specify/logs/cost/metrics.jsonl` | yes |
+| Provider usage | `CASAN_TELEMETRY_PROVIDER_LOG` | `.specify/logs/level5/provider-usage.jsonl` | no |
+| Alerts | `CASAN_TELEMETRY_ALERTS_LOG` | `.specify/agentops/alerts.log` | no |
+
+Legacy names (`CASAN_DASHBOARD_METRICS`, `CASAN_CP_PROVIDER_USAGE`,
+`CASAN_PROVIDER_LOG`, `CASAN_DASHBOARD_ALERTS`, and chat-specific metrics aliases) are
+compatibility fallbacks only. New deployments should set the canonical variables.
+
+Freshness is source-specific and based on filesystem modification time. A source becomes
+stale when its age exceeds `CASAN_DASHBOARD_STALE_S` (default `3600` seconds). Missing
+required metrics produce a `missing` freshness status and an insufficient-data verdict.
+
+## Common H1–H7 contract
+
+`packages/casan-control-panel/backend/src/reports/report.contract.ts` defines schema
+version `1`. Every harness report has:
+
+- identity: `report_id`, `harness`, `title`, `generated_at`;
+- scope: `project`, `from`, `to`, `run`;
+- decision: `verdict`, `verdict_reasons`, and explicit `thresholds`;
+- evidence: source-level freshness, record counts and findings;
+- quality: `complete`, `partial` or `insufficient` with warnings;
+- presentation-neutral `summary` and `details` data.
+
+The catalog endpoint marks H6 as `implemented`; H1–H5 and H7 are `contract_ready`. This
+is deliberate: report presence must not be confused with measured evidence or a passing
+score.
+
+## H6 API and UI
+
+| Capability | Route |
+|---|---|
+| Report catalog | `GET /api/v1/reports` |
+| H6 JSON API | `GET /api/v1/reports/h6` |
+| Independent export | `GET /api/v1/reports/h6/export?format=json|html` |
+| Browser report | `/reports/h6` |
+
+Optional filters are `project`, `from`, `to`, and `run`; `limit` only controls recent-row
+detail and does not change summary aggregation. Project/run identifiers are allowlisted to
+safe identifier characters, dates must parse as ISO-compatible values, and inverted ranges
+fail with `400`.
+
+The H6 summary includes runtime counts and outcomes, failure rate, retries, latency
+distribution, local/provider tokens, estimated/provider-reported cost, provider calls and
+alert signals. Verdicts are evidence-derived:
+
+- `no_data`: no runtime or provider record matches the scope;
+- `fail`: required metrics are missing or failure rate exceeds its threshold;
+- `attention`: stale telemetry, failures, degraded runs, high P95 latency or alerts exist;
+- `pass`: data is present and no finding crosses a configured threshold.
+
+Defaults are `5%` maximum failure rate and `5000 ms` maximum P95 latency, configurable
+through `CASAN_H6_FAILURE_RATE_THRESHOLD_PCT` and
+`CASAN_H6_P95_LATENCY_THRESHOLD_MS`.
+
+## Export integrity
+
+HTML and JSON exports are generated from the exact report object returned by the H6 service.
+The HTML artifact is standalone and print-ready; JSON keeps schema version `1` for automated
+consumers. Neither export reads a second data source or contains a hard-coded maturity
+score. Source paths outside the app root are represented as `[external]/<filename>` so an
+export does not disclose host directory structure.
+
+## Verification
+
+```bash
+npm run build -w @casan/control-panel-backend
+npm run build -w @casan/control-panel-frontend
+cd packages/casan-control-panel/backend
+node --import tsx --test test/reports.test.ts
+```
+
+The report tests cover the common catalog, filtered aggregation, exact-run/no-data behavior,
+query validation, HTML escaping and absence of hard-coded scoring.