feat(plan-01): Phase 1 — relocate harness code to packages/casan-harness (symlink facade)

Physically move the pure-code subtrees out of .specify into the package, leaving compat symlinks at the old .specify/<dir> paths so every existing reference (internal CASAN_HARNESS_ROOT + external CI/docker/mjs) keeps resolving. Runtime state stays put. Moved (git mv): scripts/ tests/ security/ templates/ config/ governance/ memory/ .specify/<dir> -> packages/casan-harness/<dir> (+ .specify/<dir> symlink) Stays in .specify (state/governance/domain, handled later): logs/ agentops/ level5/ init-options.json traceability-map.json Python `.resolve()` self-location followed the compat symlink into packages and lost the app root; generate-casan-demo-context.py, generate-agentops-dashboard.py and dashboard-server.py now walk UP for the `.specify` state marker instead of a fixed parent depth (fixes "missing trace files" in run-casan4). Full gate: PASS=64 FAIL=0 SKIP=3 (CASAN_CI_STEP_TIMEOUT_SEC=1200 — track-a ~450s runs close to the 600s default and can tip over under load; this is timing variance, not a regression — it passed cleanly with headroom). Runtime log/audit artifacts kept unstaged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 00:06:00 +09:00
parent 2c765c9a45
commit 664bd1f00c
229 changed files with 268 additions and 3 deletions
@@ -0,0 +1,68 @@
+#!/usr/bin/env bash
+set -uo pipefail
+
+# CASAN H6 — local-vs-provider telemetry reconciliation (D2).
+# Provider-API usage records are the billing ground truth; local metrics must
+# not under-report tokens (the cost-hiding attack: trim local metrics so a
+# runaway/exfil step looks cheap). Per step, local claimed tokens must cover
+# provider-reported tokens within a tolerance; a provider step entirely absent
+# from local metrics is also a discrepancy (hidden run).
+#
+# Usage: telemetry-reconcile.sh <local-metrics.jsonl> <provider-usage.jsonl> [tolerance_pct]
+#
+# Greppable outputs: TELEMETRY_RECONCILED | TELEMETRY_DISCREPANCY
+
+LOCAL_LOG="${1:-}"
+PROVIDER_LOG="${2:-}"
+TOLERANCE_PCT="${3:-10}"
+
+if [[ -z "$LOCAL_LOG" || -z "$PROVIDER_LOG" || ! -f "$PROVIDER_LOG" ]]; then
+  echo "Usage: telemetry-reconcile.sh <local-metrics.jsonl> <provider-usage.jsonl> [tolerance_pct]" >&2
+  exit 64
+fi
+
+python - "$LOCAL_LOG" "$PROVIDER_LOG" "$TOLERANCE_PCT" <<'PY'
+import json
+import sys
+
+local_path, provider_path, tol_pct = sys.argv[1], sys.argv[2], float(sys.argv[3])
+
+def sums_by_step(path):
+    totals = {}
+    try:
+        with open(path, encoding="utf-8") as fh:
+            for line in fh:
+                line = line.strip()
+                if not line:
+                    continue
+                rec = json.loads(line)
+                step = rec.get("step")
+                if step is None:
+                    continue
+                totals[step] = totals.get(step, 0) + int(rec.get("total_tokens", 0))
+    except OSError:
+        pass
+    return totals
+
+local = sums_by_step(local_path)
+provider = sums_by_step(provider_path)
+if not provider:
+    raise SystemExit("TELEMETRY_DISCREPANCY provider log empty — nothing to reconcile against")
+
+issues = []
+for step, prov_tokens in sorted(provider.items()):
+    loc_tokens = local.get(step)
+    if loc_tokens is None:
+        issues.append(f"step={step} local=MISSING provider={prov_tokens}")
+        continue
+    floor = prov_tokens * (1 - tol_pct / 100.0)
+    if loc_tokens < floor:
+        issues.append(f"step={step} local={loc_tokens} provider={prov_tokens} (under-reported beyond {tol_pct}%)")
+
+if issues:
+    for issue in issues:
+        print(f"TELEMETRY_DISCREPANCY {issue}", file=sys.stderr)
+    raise SystemExit(1)
+print(f"TELEMETRY_RECONCILED steps={len(provider)} tolerance_pct={tol_pct}")
+PY
+exit $?