thanhnvandClaude Fable 5
da66a36f97
Close the three gaps the scoring report itself flagged for H6 plus V15, each as a real MVP + fail-able adversarial test (same pattern that lifted H5): - D1 alert-dispatch.sh: alerts POST to a real HTTP webhook (severity routing, dedup window, retry) + dead-letter queue with redelivery; fail-loud in strict. Wired into agent-metrics.sh so a failing step pages live end-to-end. - D2 provider-usage-fetch.sh + telemetry-reconcile.sh: pull usage from a provider usage HTTP API (all-or-nothing schema gate, fail-loud) + reconcile local vs provider ground truth — token under-reporting/hidden runs => TELEMETRY_DISCREPANCY. - D3 dashboard-serve.sh + dashboard-server.py: serve the dashboard over HTTP with a stale-aware /healthz probe (fresh=200 ok, telemetry silent-death=503 stale). - D4 circuit-breaker-check.sh: sliding-window failure-rate breaker (V15) — interleaved successes no longer evade the consecutive-failure breaker (CIRCUIT_OPEN_WINDOW). New suite phase-h6-agentops-tests.sh: 20/20, all live against local HTTP endpoints (webhook sink, mock provider API, dashboard server) — deterministic, no model needed. Also fix sign-policy-bundle.sh key-sync invariant: the local-fallback branch only exported policy-public.pem when generating a NEW key, so a Vault-DOWN run after a Vault-signed run verified a local-key signature against the Vault pubkey (RSA padding error, run-casan4 died mid-suite). Now always re-exports the pubkey before signing — same fix class as tool-audit-lib.sh / governance-check.sh. Full battery re-run sequentially: 175/175 PASS, 0 FAIL across 8 suites (KMS SKIP this run — Vault down; validated live 2026-07-04). Docs synced: scoring-run-report (H6 79→80, no harness below 80, 155→175), CASAN_HARDENING_STATUS (Phase 5 D1–D4), Plan-07, submission README, and run-hardening.sh (H6+ scenes HO1–HO4). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
79 lines
2.9 KiB
Python
79 lines
2.9 KiB
Python
#!/usr/bin/env python3
|
|
"""CASAN H6 — hosted AgentOps dashboard server (D3).
|
|
|
|
Serves the generated dashboard over HTTP with a stale-aware /healthz probe so
|
|
an external monitor (uptime check / load-balancer) can page when telemetry
|
|
stops flowing — not just when the process dies.
|
|
|
|
Routes:
|
|
GET / -> dashboard HTML (also /dashboard)
|
|
GET /healthz -> 200 {"status":"ok",...} while metrics are fresh,
|
|
503 {"status":"stale",...} when metrics are older than
|
|
CASAN_DASHBOARD_STALE_S (default 3600s) or missing.
|
|
|
|
Usage: dashboard-server.py <port>
|
|
"""
|
|
import json
|
|
import os
|
|
import pathlib
|
|
import sys
|
|
import time
|
|
from http.server import BaseHTTPRequestHandler, HTTPServer
|
|
|
|
ROOT = pathlib.Path(__file__).resolve().parents[3]
|
|
PORT = int(sys.argv[1]) if len(sys.argv) > 1 else 8787
|
|
DASH = pathlib.Path(os.environ.get(
|
|
"CASAN_DASHBOARD_HTML", ROOT / "docs" / "output" / "casan" / "central-agentops-dashboard.html"))
|
|
METRICS = pathlib.Path(os.environ.get(
|
|
"CASAN_DASHBOARD_METRICS", ROOT / ".specify" / "logs" / "cost" / "metrics.jsonl"))
|
|
ALERTS = pathlib.Path(os.environ.get(
|
|
"CASAN_DASHBOARD_ALERTS", ROOT / ".specify" / "agentops" / "alerts.log"))
|
|
STALE_S = int(os.environ.get("CASAN_DASHBOARD_STALE_S", "3600"))
|
|
|
|
|
|
def count_lines(path: pathlib.Path) -> int:
|
|
if not path.exists():
|
|
return 0
|
|
return sum(1 for line in path.read_text(encoding="utf-8").splitlines() if line.strip())
|
|
|
|
|
|
class Handler(BaseHTTPRequestHandler):
|
|
def _send(self, code: int, ctype: str, body: str) -> None:
|
|
data = body.encode("utf-8")
|
|
self.send_response(code)
|
|
self.send_header("Content-Type", ctype)
|
|
self.send_header("Content-Length", str(len(data)))
|
|
self.end_headers()
|
|
self.wfile.write(data)
|
|
|
|
def do_GET(self): # noqa: N802 (http.server API)
|
|
if self.path == "/healthz":
|
|
if METRICS.exists():
|
|
age = int(time.time() - METRICS.stat().st_mtime)
|
|
stale = age > STALE_S
|
|
else:
|
|
age = -1
|
|
stale = True
|
|
body = json.dumps({
|
|
"status": "stale" if stale else "ok",
|
|
"metrics_age_s": age,
|
|
"stale_after_s": STALE_S,
|
|
"runs": count_lines(METRICS),
|
|
"alerts": count_lines(ALERTS),
|
|
})
|
|
self._send(503 if stale else 200, "application/json", body)
|
|
elif self.path in ("/", "/dashboard"):
|
|
if DASH.exists():
|
|
self._send(200, "text/html; charset=utf-8", DASH.read_text(encoding="utf-8"))
|
|
else:
|
|
self._send(404, "text/plain", "dashboard not generated")
|
|
else:
|
|
self._send(404, "text/plain", "not found")
|
|
|
|
def log_message(self, *args): # silence per-request stderr noise
|
|
pass
|
|
|
|
|
|
if __name__ == "__main__":
|
|
HTTPServer(("127.0.0.1", PORT), Handler).serve_forever()
|