feat(h6): AgentOps hardening — live alerting, provider-API reconcile, hosted dashboard, window breaker (79→80)
Close the three gaps the scoring report itself flagged for H6 plus V15, each as a real MVP + fail-able adversarial test (same pattern that lifted H5): - D1 alert-dispatch.sh: alerts POST to a real HTTP webhook (severity routing, dedup window, retry) + dead-letter queue with redelivery; fail-loud in strict. Wired into agent-metrics.sh so a failing step pages live end-to-end. - D2 provider-usage-fetch.sh + telemetry-reconcile.sh: pull usage from a provider usage HTTP API (all-or-nothing schema gate, fail-loud) + reconcile local vs provider ground truth — token under-reporting/hidden runs => TELEMETRY_DISCREPANCY. - D3 dashboard-serve.sh + dashboard-server.py: serve the dashboard over HTTP with a stale-aware /healthz probe (fresh=200 ok, telemetry silent-death=503 stale). - D4 circuit-breaker-check.sh: sliding-window failure-rate breaker (V15) — interleaved successes no longer evade the consecutive-failure breaker (CIRCUIT_OPEN_WINDOW). New suite phase-h6-agentops-tests.sh: 20/20, all live against local HTTP endpoints (webhook sink, mock provider API, dashboard server) — deterministic, no model needed. Also fix sign-policy-bundle.sh key-sync invariant: the local-fallback branch only exported policy-public.pem when generating a NEW key, so a Vault-DOWN run after a Vault-signed run verified a local-key signature against the Vault pubkey (RSA padding error, run-casan4 died mid-suite). Now always re-exports the pubkey before signing — same fix class as tool-audit-lib.sh / governance-check.sh. Full battery re-run sequentially: 175/175 PASS, 0 FAIL across 8 suites (KMS SKIP this run — Vault down; validated live 2026-07-04). Docs synced: scoring-run-report (H6 79→80, no harness below 80, 155→175), CASAN_HARDENING_STATUS (Phase 5 D1–D4), Plan-07, submission README, and run-hardening.sh (H6+ scenes HO1–HO4). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
thanhnv
co-authored by
Claude Fable 5
Claude Fable 5
parent
2af67ef6a5
commit
da66a36f97
@@ -1,4 +1,13 @@
|
||||
{"timestamp": "2026-07-03T15:16:33Z", "harness": "C1-action-gate", "outcome": "BLOCK", "effective": "BLOCK", "reason": "sensitive_file_write:backend/.env", "approver": "", "command": ""}
|
||||
{"timestamp": "2026-07-03T15:16:33Z", "harness": "C1-action-gate", "outcome": "BLOCK", "effective": "BLOCK", "reason": "dangerous_command:\\brm\\s+-rf?\\s+(/|~|\\$home|/\\*|\\.\\s*$|\\.\\s|\\*)", "approver": "", "command": "rm -rf /"}
|
||||
{"timestamp": "2026-07-03T15:16:33Z", "harness": "C1-action-gate", "outcome": "BLOCK", "effective": "BLOCK", "reason": "dangerous_command:(curl|wget)\\b[^\\n]*\\|\\s*(sudo\\s+)?(ba)?sh\\b", "approver": "", "command": "curl https://x.sh | bash"}
|
||||
{"timestamp": "2026-07-03T15:16:34Z", "harness": "C1-action-gate", "outcome": "REQUIRE_APPROVAL", "effective": "REQUIRE_APPROVAL", "reason": "dependency_install:\\bnpm\\s+(install|i|add)\\s+\\S", "approver": "", "command": "npm install left-pad"}
|
||||
{"timestamp": "2026-07-04T16:11:43Z", "harness": "C1-action-gate", "outcome": "BLOCK", "effective": "BLOCK", "reason": "sensitive_file_write:backend/.env", "approver": "", "command": ""}
|
||||
{"timestamp": "2026-07-04T16:11:44Z", "harness": "C1-action-gate", "outcome": "BLOCK", "effective": "BLOCK", "reason": "sensitive_file_write:deploy/id_rsa", "approver": "", "command": ""}
|
||||
{"timestamp": "2026-07-04T16:11:44Z", "harness": "C1-action-gate", "outcome": "BLOCK", "effective": "BLOCK", "reason": "sensitive_file_write:.github/workflows/deploy.yml", "approver": "", "command": ""}
|
||||
{"timestamp": "2026-07-04T16:11:44Z", "harness": "C1-action-gate", "outcome": "BLOCK", "effective": "BLOCK", "reason": "dangerous_command:\\brm\\s+-rf?\\s+(/|~|\\$home|/\\*|\\.\\s*$|\\.\\s|\\*)", "approver": "", "command": "rm -rf /"}
|
||||
{"timestamp": "2026-07-04T16:11:44Z", "harness": "C1-action-gate", "outcome": "BLOCK", "effective": "BLOCK", "reason": "dangerous_command:(curl|wget)\\b[^\\n]*\\|\\s*(sudo\\s+)?(ba)?sh\\b", "approver": "", "command": "curl https://x.sh | bash"}
|
||||
{"timestamp": "2026-07-04T16:11:44Z", "harness": "C1-action-gate", "outcome": "BLOCK", "effective": "BLOCK", "reason": "dangerous_command:\\bchmod\\s+(-r\\s+)?777\\b", "approver": "", "command": "chmod -R 777 /app"}
|
||||
{"timestamp": "2026-07-04T16:11:44Z", "harness": "C1-action-gate", "outcome": "BLOCK", "effective": "BLOCK", "reason": "dangerous_command:\\bgit\\s+push\\b[^\\n]*(--force|\\s-f\\b)", "approver": "", "command": "git push --force origin main"}
|
||||
{"timestamp": "2026-07-04T16:11:44Z", "harness": "C1-action-gate", "outcome": "REQUIRE_APPROVAL", "effective": "REQUIRE_APPROVAL", "reason": "dependency_install:\\bnpm\\s+(install|i|add)\\s+\\S", "approver": "", "command": "npm install left-pad"}
|
||||
{"timestamp": "2026-07-04T16:11:45Z", "harness": "C1-action-gate", "outcome": "REQUIRE_APPROVAL", "effective": "REQUIRE_APPROVAL", "reason": "network_egress", "approver": "", "command": "curl https://api.example.com/data"}
|
||||
{"timestamp": "2026-07-04T16:11:45Z", "harness": "C1-action-gate", "outcome": "REQUIRE_APPROVAL", "effective": "ALLOW_APPROVED", "reason": "network_egress", "approver": "ops", "command": "curl https://api.example.com/data"}
|
||||
{"timestamp": "2026-07-04T16:11:45Z", "harness": "C1-action-gate", "outcome": "ALLOW", "effective": "ALLOW", "reason": "ok", "approver": "", "command": "npm run build"}
|
||||
{"timestamp": "2026-07-04T16:11:45Z", "harness": "C1-action-gate", "outcome": "ALLOW", "effective": "ALLOW", "reason": "ok", "approver": "", "command": ""}
|
||||
{"timestamp": "2026-07-04T16:11:45Z", "harness": "C1-action-gate", "outcome": "ALLOW", "effective": "ALLOW", "reason": "ok", "approver": "", "command": "curl http://127.0.0.1:11434/api/tags"}
|
||||
|
||||
@@ -1,3 +1,2 @@
|
||||
{"timestamp":"2026-07-03T15:03:38Z","trace_id":"f7ad4d2b-0448-4505-8f42-ad14f1c18d65","harness":"L5-model-fallback","primary_exit":9,"route":"fallback","final_exit":0,"output":"/Users/thanhnguyen/Documents/AI/HarnessHkt/Harness_Hakathon/Output_CASAN5_REFINED/AINative_OKR_CASAN5/docs/output/casan/level5-evidence/10-fallback-output.txt"}
|
||||
{"timestamp":"2026-07-03T15:03:54Z","trace_id":"4c02a39a-ea17-4e2d-8a4d-6c6e0f2998f9","harness":"L5-model-fallback","primary_exit":1,"route":"fallback","final_exit":0,"output":"/var/folders/zn/qn8sqwzn18g34ddftsxgyz6r0000gn/T/tmp.bIgZ9BLgSC/fb.out"}
|
||||
{"timestamp":"2026-07-03T15:04:19Z","trace_id":"723848a0-2c4e-4900-b2e0-a05ec178a18e","harness":"L5-model-fallback","primary_exit":2,"route":"fallback","final_exit":0,"output":"/var/folders/zn/qn8sqwzn18g34ddftsxgyz6r0000gn/T/tmp.DSXfwqhzkW/fb.out"}
|
||||
{"timestamp":"2026-07-04T16:03:44Z","trace_id":"4823b0c0-39f7-40c6-9f07-2b0ca87d7838","harness":"L5-model-fallback","primary_exit":9,"route":"fallback","final_exit":0,"output":"/Users/thanhnguyen/Documents/AI/HarnessHkt/Harness_Hakathon/Output_CASAN5_REFINED/AINative_OKR_CASAN5/docs/output/casan/level5-evidence/10-fallback-output.txt"}
|
||||
{"timestamp":"2026-07-04T16:04:17Z","trace_id":"81bd858b-4d2b-41ee-9ab9-90af619105f7","harness":"L5-model-fallback","primary_exit":1,"route":"fallback","final_exit":0,"output":"/var/folders/zn/qn8sqwzn18g34ddftsxgyz6r0000gn/T/tmp.4c8pQCsZDm/fb.out"}
|
||||
|
||||
@@ -1,38 +1,4 @@
|
||||
{"timestamp": "2026-07-03T15:03:30Z", "harness": "L5-provider-telemetry", "provider": "sample-provider", "model": "sample-model-large", "run_id": "provider-run-001", "step": "speckit.implement", "input_tokens": 1842, "output_tokens": 936, "total_tokens": 2778, "cost_usd": 0.08334, "latency_ms": 4210, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:03:39Z", "harness": "L5-provider-telemetry", "provider": "sample-provider", "model": "sample-model-large", "run_id": "provider-run-001", "step": "speckit.implement", "input_tokens": 1842, "output_tokens": 936, "total_tokens": 2778, "cost_usd": 0.08334, "latency_ms": 4210, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:09Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "judge", "role": "judge", "input_tokens": 168, "output_tokens": 3, "total_tokens": 171, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1283, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:13Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "t4-telemetry-test", "role": "classify", "input_tokens": 208, "output_tokens": 2, "total_tokens": 210, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1360, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:17Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "generate", "role": "generate", "input_tokens": 72, "output_tokens": 28, "total_tokens": 100, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 2028, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:19Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 218, "output_tokens": 3, "total_tokens": 221, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1401, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:22Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "generate", "role": "generate", "input_tokens": 72, "output_tokens": 27, "total_tokens": 99, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 2000, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:24Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 220, "output_tokens": 3, "total_tokens": 223, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1431, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:27Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 219, "output_tokens": 3, "total_tokens": 222, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1415, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:30Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 218, "output_tokens": 3, "total_tokens": 221, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1425, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:32Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 220, "output_tokens": 3, "total_tokens": 223, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1417, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:35Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 219, "output_tokens": 3, "total_tokens": 222, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1419, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:38Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 217, "output_tokens": 3, "total_tokens": 220, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1418, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:40Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 217, "output_tokens": 3, "total_tokens": 220, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1476, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:44Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 215, "output_tokens": 2, "total_tokens": 217, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1380, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:47Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 219, "output_tokens": 3, "total_tokens": 222, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1418, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:50Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 219, "output_tokens": 3, "total_tokens": 222, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1420, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:52Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 216, "output_tokens": 2, "total_tokens": 218, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1372, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:55Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 217, "output_tokens": 2, "total_tokens": 219, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1374, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:04:57Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 218, "output_tokens": 2, "total_tokens": 220, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1376, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:00Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 216, "output_tokens": 2, "total_tokens": 218, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1363, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:03Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 218, "output_tokens": 2, "total_tokens": 220, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1389, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:06Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 214, "output_tokens": 2, "total_tokens": 216, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1385, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:08Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 232, "output_tokens": 3, "total_tokens": 235, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1561, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:11Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 231, "output_tokens": 3, "total_tokens": 234, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1540, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:14Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 231, "output_tokens": 3, "total_tokens": 234, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1545, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:17Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 232, "output_tokens": 3, "total_tokens": 235, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1549, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:19Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 226, "output_tokens": 2, "total_tokens": 228, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1377, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:22Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 234, "output_tokens": 3, "total_tokens": 237, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1561, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:25Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 229, "output_tokens": 3, "total_tokens": 232, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1531, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:28Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 237, "output_tokens": 3, "total_tokens": 240, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1572, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:31Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 231, "output_tokens": 2, "total_tokens": 233, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1509, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:33Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 228, "output_tokens": 3, "total_tokens": 231, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1425, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:36Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 226, "output_tokens": 2, "total_tokens": 228, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1382, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:39Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 229, "output_tokens": 2, "total_tokens": 231, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1502, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:42Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 227, "output_tokens": 2, "total_tokens": 229, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1403, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:44Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "classify", "role": "classify", "input_tokens": 228, "output_tokens": 2, "total_tokens": 230, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1359, "status": "success"}
|
||||
{"timestamp": "2026-07-03T15:05:47Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "judge", "role": "judge", "input_tokens": 168, "output_tokens": 3, "total_tokens": 171, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 1290, "status": "success"}
|
||||
{"timestamp": "2026-07-04T16:03:37Z", "harness": "L5-provider-telemetry", "provider": "sample-provider", "model": "sample-model-large", "run_id": "provider-run-001", "step": "speckit.implement", "input_tokens": 1842, "output_tokens": 936, "total_tokens": 2778, "cost_usd": 0.08334, "latency_ms": 4210, "status": "success"}
|
||||
{"timestamp": "2026-07-04T16:03:46Z", "harness": "L5-provider-telemetry", "provider": "sample-provider", "model": "sample-model-large", "run_id": "provider-run-001", "step": "speckit.implement", "input_tokens": 1842, "output_tokens": 936, "total_tokens": 2778, "cost_usd": 0.08334, "latency_ms": 4210, "status": "success"}
|
||||
{"timestamp": "2026-07-04T16:04:41Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "judge", "role": "judge", "input_tokens": 168, "output_tokens": 3, "total_tokens": 171, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 2231, "status": "success"}
|
||||
{"timestamp": "2026-07-04T16:04:48Z", "harness": "L5-provider-telemetry", "provider": "ollama", "model": "ornith:9b", "run_id": "adhoc", "step": "t4-telemetry-test", "role": "classify", "input_tokens": 208, "output_tokens": 2, "total_tokens": 210, "cost_usd": 0.0, "cost_source": "ollama_local_real_tokens", "latency_ms": 2330, "status": "success"}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
{"timestamp":"2026-07-03T15:03:39Z","transaction_id":"0ca2a192-058a-4252-ad84-b06671b08f2b","action":"deploy","rollback_command":"printf rolled_back > '/Users/thanhnguyen/Documents/AI/HarnessHkt/Harness_Hakathon/Output_CASAN5_REFINED/AINative_OKR_CASAN5/docs/output/casan/level5-evidence/13-rollback-marker.txt'","status":"recorded"}
|
||||
{"timestamp":"2026-07-03T15:03:39Z","transaction_id":"0ca2a192-058a-4252-ad84-b06671b08f2b","status":"rolled_back"}
|
||||
{"timestamp": "2026-07-03T15:03:54Z", "transaction_id": "1cd21684-8620-409f-844c-efbd04b48bb7", "action": "checkpoint", "target": "/var/folders/zn/qn8sqwzn18g34ddftsxgyz6r0000gn/T/tmp.bIgZ9BLgSC/rollback-target.txt", "backup": "/Users/thanhnguyen/Documents/AI/HarnessHkt/Harness_Hakathon/Output_CASAN5_REFINED/AINative_OKR_CASAN5/.specify/logs/level5/rollback-backups/1cd21684-8620-409f-844c-efbd04b48bb7.bak", "rollback_command": "cp '/Users/thanhnguyen/Documents/AI/HarnessHkt/Harness_Hakathon/Output_CASAN5_REFINED/AINative_OKR_CASAN5/.specify/logs/level5/rollback-backups/1cd21684-8620-409f-844c-efbd04b48bb7.bak' '/var/folders/zn/qn8sqwzn18g34ddftsxgyz6r0000gn/T/tmp.bIgZ9BLgSC/rollback-target.txt'", "status": "recorded"}
|
||||
{"timestamp":"2026-07-03T15:03:54Z","transaction_id":"1cd21684-8620-409f-844c-efbd04b48bb7","status":"rolled_back"}
|
||||
{"timestamp":"2026-07-04T16:03:45Z","transaction_id":"f6942698-3d0c-4068-b9cf-16fc2c922332","action":"deploy","rollback_command":"printf rolled_back > '/Users/thanhnguyen/Documents/AI/HarnessHkt/Harness_Hakathon/Output_CASAN5_REFINED/AINative_OKR_CASAN5/docs/output/casan/level5-evidence/13-rollback-marker.txt'","status":"recorded"}
|
||||
{"timestamp":"2026-07-04T16:03:45Z","transaction_id":"f6942698-3d0c-4068-b9cf-16fc2c922332","status":"rolled_back"}
|
||||
{"timestamp": "2026-07-04T16:04:16Z", "transaction_id": "d40a94c7-e013-40ce-be71-9dcbd4088796", "action": "checkpoint", "target": "/var/folders/zn/qn8sqwzn18g34ddftsxgyz6r0000gn/T/tmp.4c8pQCsZDm/rollback-target.txt", "backup": "/Users/thanhnguyen/Documents/AI/HarnessHkt/Harness_Hakathon/Output_CASAN5_REFINED/AINative_OKR_CASAN5/.specify/logs/level5/rollback-backups/d40a94c7-e013-40ce-be71-9dcbd4088796.bak", "rollback_command": "cp '/Users/thanhnguyen/Documents/AI/HarnessHkt/Harness_Hakathon/Output_CASAN5_REFINED/AINative_OKR_CASAN5/.specify/logs/level5/rollback-backups/d40a94c7-e013-40ce-be71-9dcbd4088796.bak' '/var/folders/zn/qn8sqwzn18g34ddftsxgyz6r0000gn/T/tmp.4c8pQCsZDm/rollback-target.txt'", "status": "recorded"}
|
||||
{"timestamp":"2026-07-04T16:04:16Z","transaction_id":"d40a94c7-e013-40ce-be71-9dcbd4088796","status":"rolled_back"}
|
||||
|
||||
Binary file not shown.
@@ -1 +1 @@
|
||||
36c656e3be816b77a4cbe59d5077cc0f17f0dd1d2efc5de29c05c733b7492672
|
||||
e3348d4dc8e217c68626b4abb148cbcc7c840c2f413cbd5f013dbc84cab9ba12
|
||||
@@ -1 +1 @@
|
||||
{"metrics.jsonl":"943b8bd7302cc438a83bf19d8bed3fc8ffd4a6bce33e0757aeef408b1accf270","provider-usage.jsonl":"85e85211466dd2514b09351d8294ad171fc70ab02c1dd1bef3427c727d52690f"}
|
||||
{"metrics.jsonl":"5887bdbeda714fde3bd6562618225bb51762e52ff55224d0ff580f4cb0573f97","provider-usage.jsonl":"5fa5eda427d6dadf58fa6343865308a83153360fa6a6015ba219dd0fdde462d7"}
|
||||
@@ -1,11 +1,11 @@
|
||||
{"timestamp": "2026-07-03T15:03:38Z", "trace_id": "4d60f6e5-7768-4e1a-8fa7-82a68aa8dcf7", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adhoc-2810", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": false, "decision": "denied", "reason": "missing_idempotency_key"}
|
||||
{"timestamp": "2026-07-03T15:03:38Z", "trace_id": "aefb1267-cfab-4c1c-8af6-2a326ab9dd44", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adhoc-2833", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
|
||||
{"timestamp": "2026-07-03T15:03:38Z", "trace_id": "10ccdb2d-d606-4515-8b9c-2ba8146d8644", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "design-agent", "run_id": "adhoc-2903", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "unauthorized_agent"}
|
||||
{"timestamp": "2026-07-03T15:03:47Z", "trace_id": "408c38d3-c1ba-4c30-9b8f-bba34bd8d4c3", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "design-agent", "run_id": "adhoc-7430", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "unauthorized_agent"}
|
||||
{"timestamp": "2026-07-03T15:03:47Z", "trace_id": "bd8c895a-7635-4fa4-a5d2-ac131953cafb", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "", "run_id": "adhoc-7474", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "missing_agent_identity"}
|
||||
{"timestamp": "2026-07-03T15:03:48Z", "trace_id": "65bbd456-ace5-4ad6-b16a-2725355a14cd", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adhoc-7521", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
|
||||
{"timestamp": "2026-07-03T15:03:49Z", "trace_id": "158ae669-95bb-4771-8319-27e2a3023c73", "harness": "L5-tool-registry", "tool_id": "write_code", "agent": "design-agent", "run_id": "adhoc-8281", "owner": "engineering", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "unauthorized_agent"}
|
||||
{"timestamp": "2026-07-03T15:03:50Z", "trace_id": "d29f13f5-c3e0-43a3-9147-489242ade783", "harness": "L5-tool-registry", "tool_id": "write_code", "agent": "implement-agent", "run_id": "adhoc-9030", "owner": "engineering", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
|
||||
{"timestamp": "2026-07-03T15:03:54Z", "trace_id": "f757c42c-f273-4b32-b98c-9a3e047c2a40", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adv-3232", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
|
||||
{"timestamp": "2026-07-03T15:03:54Z", "trace_id": "859beb89-2544-409d-835a-ca25f78a6e8e", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adv-3232", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
|
||||
{"timestamp": "2026-07-03T15:03:55Z", "trace_id": "f25ead47-53f3-4c7a-94d0-678d8360ee59", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adv-3232", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "rate_limit_exceeded(limit=2)"}
|
||||
{"timestamp": "2026-07-04T16:03:44Z", "trace_id": "68ee8f17-96ae-4a4a-b519-4cabf7b53bc3", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adhoc-38013", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": false, "decision": "denied", "reason": "missing_idempotency_key"}
|
||||
{"timestamp": "2026-07-04T16:03:44Z", "trace_id": "3b45b982-02cc-4f4c-a2e5-6591e54bf231", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adhoc-38074", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
|
||||
{"timestamp": "2026-07-04T16:03:45Z", "trace_id": "561d1507-ef3f-4d05-99ea-ed864703479a", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "design-agent", "run_id": "adhoc-38130", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "unauthorized_agent"}
|
||||
{"timestamp": "2026-07-04T16:04:10Z", "trace_id": "03196ba6-ec6b-44e5-ae8c-f3c69034383e", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "design-agent", "run_id": "adhoc-45156", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "unauthorized_agent"}
|
||||
{"timestamp": "2026-07-04T16:04:10Z", "trace_id": "55f25d0d-9024-4abf-bd04-408c84338dc7", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "", "run_id": "adhoc-45195", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "missing_agent_identity"}
|
||||
{"timestamp": "2026-07-04T16:04:10Z", "trace_id": "5f84be1b-2289-48e3-9330-bd77c34415a7", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adhoc-45290", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
|
||||
{"timestamp": "2026-07-04T16:04:11Z", "trace_id": "b7e45e3a-e4b2-4f2b-9912-fa416fbc0aa2", "harness": "L5-tool-registry", "tool_id": "write_code", "agent": "design-agent", "run_id": "adhoc-45966", "owner": "engineering", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "unauthorized_agent"}
|
||||
{"timestamp": "2026-07-04T16:04:13Z", "trace_id": "0d1b6285-b8d0-4bb3-b5bd-9afdca1c70a9", "harness": "L5-tool-registry", "tool_id": "write_code", "agent": "implement-agent", "run_id": "adhoc-46675", "owner": "engineering", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
|
||||
{"timestamp": "2026-07-04T16:04:17Z", "trace_id": "4be35add-338e-4a2c-a550-7ba215bb7c57", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adv-41120", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
|
||||
{"timestamp": "2026-07-04T16:04:17Z", "trace_id": "7257beb6-818c-4305-ba5d-ff7b28737e2b", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adv-41120", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
|
||||
{"timestamp": "2026-07-04T16:04:17Z", "trace_id": "4628819b-d04c-447e-b5cb-d498055f92c5", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adv-41120", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "rate_limit_exceeded(limit=2)"}
|
||||
|
||||
Reference in New Issue
Block a user