gitea-admin/CASAN
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

scorecard: chấm live 2 mục hardcode; pipeline: fallback real + honest scoring doc

Browse Source 
Hướng A — scorecard.sh (video demo):
- h5_1 approval workflow: hardcode 0 → governance-check deploy live (approval_required)
- h6_2 hallucination rate: hardcode 0 → hallucination-scan phân biệt dirty>clean live
- "N/5 mục" chuyển từ text cứng sang đếm động
- H4/H5/H6 → 100/100 (5/5 gate live), Average 57.9 → 90.0

Hướng B — run-casan-pipeline.mjs:
- fallback: stub 'exit 9' → 'cat /nonexistent' (real failure, nhất quán adversarial T3)
- drift: giữ so fallback-output vs golden (clean run=1.0); năng lực phát hiện
  drift thật chứng minh ở adversarial suite
- Full 12-step run verify: H1 CONTEXT_VALID=24, H2 tool-audit records=25 signed,
  H5 audit-chain records=22 signed, H6 provider_telemetry per-step thật, H7 rollback real

phase3-real-run-scoring.md: giải thích vì sao scorecard cũ cho H5=60/H6=80
(hardcode), phân biệt scorecard-90 vs re-score-84 (2 mục đích khác nhau).

Verify: adversarial 44/0, security-gate 11/0/0, pipeline 12 steps OK.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
thanhnv
2026-07-03 00:18:43 +09:00
co-authored by Claude Sonnet 4.6
parent fda0d6447d
commit 2f06662f5d
26 changed files with 469 additions and 189 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AINative_OKR_CASAN5/.specify/logs/level5/tool-registry.jsonl
+11 -11
View File
@@ -1,11 +1,11 @@
{"timestamp": "2026-07-01T08:22:36Z", "trace_id": "28f618f2-2187-4f95-9c41-a5fa3cdc8bfa", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adhoc-4053", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": false, "decision": "denied", "reason": "missing_idempotency_key"}
{"timestamp": "2026-07-01T08:22:36Z", "trace_id": "db4c5806-1365-43e9-a7c6-7c20bec08cd5", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adhoc-4085", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
{"timestamp": "2026-07-01T08:22:36Z", "trace_id": "05556471-81ba-45fd-ae51-636ece8bda45", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "design-agent", "run_id": "adhoc-4110", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "unauthorized_agent"}
{"timestamp": "2026-07-01T08:22:46Z", "trace_id": "8b024ab7-faf4-4030-b32c-02a67b4ed2da", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "design-agent", "run_id": "adhoc-6461", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "unauthorized_agent"}
{"timestamp": "2026-07-01T08:22:46Z", "trace_id": "df1127cb-401a-47f0-91c7-0ae102d761c8", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "", "run_id": "adhoc-6491", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "missing_agent_identity"}
{"timestamp": "2026-07-01T08:22:46Z", "trace_id": "a09af213-5ab5-497e-b35c-2ca0ba1ee71d", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adhoc-6510", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
{"timestamp": "2026-07-01T08:22:57Z", "trace_id": "0f10a69a-043f-4965-9419-6fc26f623a46", "harness": "L5-tool-registry", "tool_id": "write_code", "agent": "design-agent", "run_id": "adhoc-7086", "owner": "engineering", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "unauthorized_agent"}
{"timestamp": "2026-07-01T08:22:58Z", "trace_id": "76bb296c-ac6f-43c2-ae09-d17377b1361e", "harness": "L5-tool-registry", "tool_id": "write_code", "agent": "implement-agent", "run_id": "adhoc-7474", "owner": "engineering", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
{"timestamp": "2026-07-01T08:23:02Z", "trace_id": "0f6f5be7-04d4-43c2-968f-22a9832f984f", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adv-4331", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
{"timestamp": "2026-07-01T08:23:02Z", "trace_id": "4ed85702-36f5-453a-bf0a-3b041da5cd62", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adv-4331", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
{"timestamp": "2026-07-01T08:23:03Z", "trace_id": "b8baf09c-30d2-41df-92a5-a9da8d517bcc", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adv-4331", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "rate_limit_exceeded(limit=2)"}
{"timestamp": "2026-07-02T15:15:46Z", "trace_id": "3438ed09-602f-453f-9a99-6b2f3664b646", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adhoc-40617", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": false, "decision": "denied", "reason": "missing_idempotency_key"}
{"timestamp": "2026-07-02T15:15:46Z", "trace_id": "8377db5f-e282-42e5-a045-5e842edb1e8f", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adhoc-40639", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
{"timestamp": "2026-07-02T15:15:46Z", "trace_id": "d23c175c-e864-45ab-b2cc-1bbf86075e24", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "design-agent", "run_id": "adhoc-40688", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "unauthorized_agent"}
{"timestamp": "2026-07-02T15:15:53Z", "trace_id": "c63052ab-0cce-490a-a814-d18fbd49df5e", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "design-agent", "run_id": "adhoc-43498", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "unauthorized_agent"}
{"timestamp": "2026-07-02T15:15:53Z", "trace_id": "14fa1288-2d35-49c7-9614-0d2bcb202487", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "", "run_id": "adhoc-43564", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "missing_agent_identity"}
{"timestamp": "2026-07-02T15:15:53Z", "trace_id": "c670ece0-312c-432c-b35d-69e17f46c820", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adhoc-43584", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
{"timestamp": "2026-07-02T15:15:54Z", "trace_id": "cd43f413-dbbf-4519-8eba-7a26dd9a037e", "harness": "L5-tool-registry", "tool_id": "write_code", "agent": "design-agent", "run_id": "adhoc-44020", "owner": "engineering", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "unauthorized_agent"}
{"timestamp": "2026-07-02T15:15:55Z", "trace_id": "829faee4-8065-4c52-aa7b-e45fec893085", "harness": "L5-tool-registry", "tool_id": "write_code", "agent": "implement-agent", "run_id": "adhoc-44526", "owner": "engineering", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
{"timestamp": "2026-07-02T15:15:58Z", "trace_id": "1933731e-9fb0-474c-a80f-e1e35d909510", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adv-40962", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
{"timestamp": "2026-07-02T15:15:58Z", "trace_id": "4dab312c-eb11-4a23-acfc-026b7a71ba53", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adv-40962", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "approved", "reason": "registered"}
{"timestamp": "2026-07-02T15:15:58Z", "trace_id": "04ab4e1e-0b69-47a8-9389-ce8078035743", "harness": "L5-tool-registry", "tool_id": "deploy", "agent": "release-manager", "run_id": "adv-40962", "owner": "release-manager", "risk_level": "high", "side_effect": true, "idempotency_required": true, "idempotency_key_present": true, "decision": "denied", "reason": "rate_limit_exceeded(limit=2)"}