gitea-admin/CASAN
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix T1+T4: wire rollback into pipeline; real Ollama telemetry in metrics

Browse Source 
T1 (H7): casan-step.mjs now calls rollback-manager.sh checkpoint before
overwriting plan.md at attempt-2, writes tx-id to plan.checkpoint.txid
sidecar, and executes rollback on REJECTED verdict. rollback-transactions.jsonl
records a real cp restore command. Adversarial test: checkpoint exists,
real cp command recorded, plan hash matches pre-overwrite content.

T4 (H6): casan-harness.sh exports CASAN_STEP_NAME=$ACTION_NAME before
agent-metrics.sh so nested model calls (model-call.py) and the provider-
cost-lookup.py query share the same step label. metrics.jsonl now writes
cost_source=provider_telemetry instead of word_count_estimate when a real
Ollama call is made within the same step. Adversarial test: verified with
CASAN_STEP_NAME=t4-telemetry-test end-to-end.

adversarial-harness-tests.sh: 40 → 44 PASS / 0 FAIL (+3 T1, +1 T4)
security-gate.sh: PASS=10 FAIL=0 SKIP=0 (verified, local ornith:9b)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
thanhnv
2026-07-01 11:25:50 +09:00
co-authored by Claude Sonnet 4.6
parent f74a5b6e42
commit a8edbea534
18 changed files with 284 additions and 159 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AINative_OKR_CASAN5/.specify/logs/audit/tool-calls.jsonl
+19 -18
View File
@@ -1,18 +1,19 @@
{"timestamp": "2026-07-01T02:01:42Z", "trace_id": "50a5bd13-1bc7-45f0-828d-42027d08c86e", "agent": "demo.agent", "step": "step-1-srs", "tool": "Bash", "command": "bash -c cp \"$CASAN_INPUT\" \"$CASAN_OUTPUT\"", "exit_code": 0, "status": "success", "previous_record_hash": "", "record_hash": "aec3d6b27b81a43d8f5f56149ddd970a4d6e67632259d6b7d3eb111253671a2d"}
{"timestamp": "2026-07-01T02:01:43Z", "trace_id": "2c0bc2f4-b444-45a9-b485-5c53093ea174", "agent": "demo.agent", "step": "failing-step", "tool": "Bash", "command": "bash -c exit 7", "exit_code": 7, "status": "failed", "previous_record_hash": "aec3d6b27b81a43d8f5f56149ddd970a4d6e67632259d6b7d3eb111253671a2d", "record_hash": "8f4314bb205459a3c640c5c46a071f9250a4c74c999a9a146223846379cc362d"}
{"timestamp": "2026-07-01T02:01:46Z", "trace_id": "c68fdf2f-d0aa-48df-be14-e2ed9748c6a5", "agent": "wrapper.demo", "step": "wrapper-step", "tool": "Bash", "command": "bash -c cp \"$1\" \"$CASAN_OUTPUT\" _ /Users/thanhnguyen/Documents/AI/HarnessHkt/Harness_Hakathon/Output_CASAN5_REFINED/AINative_OKR_CASAN5/.specify/logs/idempotency/a695c82f9d29aba2adace11fc766fd1b0ffae9ef27514df5907bdf66a69e0bba.output", "exit_code": 0, "status": "success", "previous_record_hash": "8f4314bb205459a3c640c5c46a071f9250a4c74c999a9a146223846379cc362d", "record_hash": "785a6028c8bc7fc4aebb28b84b0143ec3dd85822022d3c7c3f9ba9ef77b8dc5e"}
{"timestamp": "2026-07-01T02:01:47Z", "trace_id": "70570443-dcbb-4ab9-b8bb-a289a8ec8159", "tool": "deploy", "agent": "release-manager", "idempotency_key": "", "decision": "denied", "reason": "missing_idempotency_key", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "785a6028c8bc7fc4aebb28b84b0143ec3dd85822022d3c7c3f9ba9ef77b8dc5e", "record_hash": "4351258e953c55b369bdcfe1795aec487d41684e098476a5cb77103d957d0a32"}
{"timestamp": "2026-07-01T02:01:47Z", "trace_id": "7f14dc67-a9cb-4319-9426-9bcdbd2a2b9c", "tool": "deploy", "agent": "release-manager", "idempotency_key": "deploy-demo-001", "decision": "approved", "reason": "registered", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "4351258e953c55b369bdcfe1795aec487d41684e098476a5cb77103d957d0a32", "record_hash": "14a0eb6d3ad90a80ecae3164f6b87f3f5d7bc4b5eef9000f7a2c343ee5ee7c34"}
{"timestamp": "2026-07-01T02:01:47Z", "trace_id": "57100327-59f7-446f-a816-37c30e937927", "tool": "deploy", "agent": "design-agent", "idempotency_key": "deploy-demo-002", "decision": "denied", "reason": "unauthorized_agent", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "14a0eb6d3ad90a80ecae3164f6b87f3f5d7bc4b5eef9000f7a2c343ee5ee7c34", "record_hash": "d209765dbee16cb2bfff7f1dd719a54beca46ba7c2f5ec4661d77b90feb8947b"}
{"timestamp": "2026-07-01T02:01:53Z", "trace_id": "46af3713-8749-420b-ab91-fcfe62525621", "tool": "deploy", "agent": "design-agent", "idempotency_key": "k1", "decision": "denied", "reason": "unauthorized_agent", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "d209765dbee16cb2bfff7f1dd719a54beca46ba7c2f5ec4661d77b90feb8947b", "record_hash": "41f8ef2293014f25a22a0a83c2c1563490c34d34341152d99cc7e548fcf9fef0"}
{"timestamp": "2026-07-01T02:01:53Z", "trace_id": "8ed3296a-09fa-4fff-88f7-4826776d64cb", "tool": "deploy", "agent": "", "idempotency_key": "k2", "decision": "denied", "reason": "missing_agent_identity", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "41f8ef2293014f25a22a0a83c2c1563490c34d34341152d99cc7e548fcf9fef0", "record_hash": "412535fd887e4ea883e1370c4af5462add8d2f3b9db0c8014b83409e9fd9122c"}
{"timestamp": "2026-07-01T02:01:53Z", "trace_id": "64a32c86-02b7-4cf6-bd94-5a8470091b77", "tool": "deploy", "agent": "release-manager", "idempotency_key": "k3", "decision": "approved", "reason": "registered", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "412535fd887e4ea883e1370c4af5462add8d2f3b9db0c8014b83409e9fd9122c", "record_hash": "e4abf5c0d93c128a44bf01b0161939cff1fca1e909fef0e8d27cdca0e3798945"}
{"timestamp": "2026-07-01T02:01:54Z", "trace_id": "fcae68a3-7bda-4ac5-b6ed-71042dd0dafb", "tool": "write_code", "agent": "design-agent", "idempotency_key": "d6d750d0ee93e4b9f9b917a42e3f7f6e8725c63bf412594778810bd91c175084", "decision": "denied", "reason": "unauthorized_agent", "risk_level": "high", "owner": "engineering", "previous_record_hash": "e4abf5c0d93c128a44bf01b0161939cff1fca1e909fef0e8d27cdca0e3798945", "record_hash": "996b5448194716b1ce9efd956e5d489b634e669f54ca5ddc3601b80aad72b1f0"}
{"timestamp": "2026-07-01T02:01:55Z", "trace_id": "f601e745-2e16-4291-803d-946578e9f3bc", "tool": "write_code", "agent": "implement-agent", "idempotency_key": "d6d750d0ee93e4b9f9b917a42e3f7f6e8725c63bf412594778810bd91c175084", "decision": "approved", "reason": "registered", "risk_level": "high", "owner": "engineering", "previous_record_hash": "996b5448194716b1ce9efd956e5d489b634e669f54ca5ddc3601b80aad72b1f0", "record_hash": "1bb8fb429d01d43776156a956b3794766dd20f5949b3d003a1d77194d886824f"}
{"timestamp": "2026-07-01T02:01:55Z", "trace_id": "50ce3010-3720-4ad5-9be9-40bc32ac177d", "agent": "unknown-agent", "step": "unknown-step", "tool": "Bash", "command": "/Users/thanhnguyen/Documents/AI/HarnessHkt/Harness_Hakathon/Output_CASAN5_REFINED/AINative_OKR_CASAN5/.specify/scripts/bash/tool-exec.sh 30 -- bash -c echo wrote", "exit_code": 0, "status": "success", "previous_record_hash": "1bb8fb429d01d43776156a956b3794766dd20f5949b3d003a1d77194d886824f", "record_hash": "2f87215a3544efdb703697baaabbdd785b013136b2f5f0bbc136b3b7949f9fec"}
{"timestamp": "2026-07-01T02:01:56Z", "trace_id": "7e0039e4-57db-4b3a-893f-653382dadbc5", "agent": "adv", "step": "step-1-srs", "tool": "Bash", "command": "bash -c cp \"$CASAN_INPUT\" \"$CASAN_OUTPUT\"", "exit_code": 0, "status": "success", "previous_record_hash": "2f87215a3544efdb703697baaabbdd785b013136b2f5f0bbc136b3b7949f9fec", "record_hash": "469f8cd1827a8cf8915b3fc534332c2bd9afcf3a2da53fe15f4deb3df04ad849"}
{"timestamp": "2026-07-01T02:01:57Z", "trace_id": "46dc7318-ab4b-4a11-960b-a0f9c883fc81", "agent": "adv", "step": "step-1-srs", "tool": "Bash", "command": "bash -c cp \"$CASAN_INPUT\" \"$CASAN_OUTPUT\"", "exit_code": 0, "status": "success", "previous_record_hash": "469f8cd1827a8cf8915b3fc534332c2bd9afcf3a2da53fe15f4deb3df04ad849", "record_hash": "34cef15b8d4d552b36a503bba7ccfc13820b5dc43b9a1577b0104def00e128f3"}
{"timestamp": "2026-07-01T02:01:57Z", "trace_id": "2c06a7a3-e6b2-4796-99e1-183cdbc8ee91", "tool": "deploy", "agent": "release-manager", "idempotency_key": "k1", "decision": "approved", "reason": "registered", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "34cef15b8d4d552b36a503bba7ccfc13820b5dc43b9a1577b0104def00e128f3", "record_hash": "ced8e1be7f3c93e0b8ccc9bd6e18b41c0246661b5d321dee8596e9bcaf068b62"}
{"timestamp": "2026-07-01T02:01:58Z", "trace_id": "652ea22a-4b46-4cf9-81aa-1588854a7569", "tool": "deploy", "agent": "release-manager", "idempotency_key": "k2", "decision": "approved", "reason": "registered", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "ced8e1be7f3c93e0b8ccc9bd6e18b41c0246661b5d321dee8596e9bcaf068b62", "record_hash": "d61a3b79d8e1275ab4e0b7d5a2716c061a23ed26e5e9d864ed2feb6ac41f014a"}
{"timestamp": "2026-07-01T02:01:58Z", "trace_id": "eb00313e-7c40-46ba-ae48-41d75587aac4", "tool": "deploy", "agent": "release-manager", "idempotency_key": "k3", "decision": "denied", "reason": "rate_limit_exceeded(limit=2)", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "d61a3b79d8e1275ab4e0b7d5a2716c061a23ed26e5e9d864ed2feb6ac41f014a", "record_hash": "a931f14be4d9e165dc8df8ad0508c78fa2297ef4f9d8dbdc27a10b8b2e7a4839"}
{"timestamp": "2026-07-01T02:02:02Z", "trace_id": "5d94660f-b81c-463a-8495-2b1bacd18888", "agent": "unknown-agent", "step": "unknown-step", "tool": "Bash", "command": "/Users/thanhnguyen/Documents/AI/HarnessHkt/Harness_Hakathon/Output_CASAN5_REFINED/AINative_OKR_CASAN5/.specify/scripts/bash/tool-exec.sh 2 -- sleep 60", "exit_code": 124, "status": "failed", "previous_record_hash": "a931f14be4d9e165dc8df8ad0508c78fa2297ef4f9d8dbdc27a10b8b2e7a4839", "record_hash": "26397d1bdb6235d7892de12ec091c8a397d91a832eaa3195b959c5d00e8fa433"}
{"timestamp": "2026-07-01T02:23:35Z", "trace_id": "01675f76-25f1-4f79-a55c-2571ecbc7314", "agent": "demo.agent", "step": "step-1-srs", "tool": "Bash", "command": "bash -c cp \"$CASAN_INPUT\" \"$CASAN_OUTPUT\"", "exit_code": 0, "status": "success", "previous_record_hash": "", "record_hash": "7fb362308b029c3f310d3b6011673ce6485d8121580cef101acce73bec6dbc97"}
{"timestamp": "2026-07-01T02:23:35Z", "trace_id": "47204bd3-200b-42d6-b724-9560d737ad6c", "agent": "demo.agent", "step": "failing-step", "tool": "Bash", "command": "bash -c exit 7", "exit_code": 7, "status": "failed", "previous_record_hash": "7fb362308b029c3f310d3b6011673ce6485d8121580cef101acce73bec6dbc97", "record_hash": "0f82bc60dc3bc7160c3287590412517389b1e4b7391b272f7843428b73d3cfbd"}
{"timestamp": "2026-07-01T02:23:40Z", "trace_id": "46fc9bca-1d15-4982-b8f0-1316afd939ca", "agent": "wrapper.demo", "step": "wrapper-step", "tool": "Bash", "command": "bash -c cp \"$1\" \"$CASAN_OUTPUT\" _ /Users/thanhnguyen/Documents/AI/HarnessHkt/Harness_Hakathon/Output_CASAN5_REFINED/AINative_OKR_CASAN5/.specify/logs/idempotency/a695c82f9d29aba2adace11fc766fd1b0ffae9ef27514df5907bdf66a69e0bba.output", "exit_code": 0, "status": "success", "previous_record_hash": "0f82bc60dc3bc7160c3287590412517389b1e4b7391b272f7843428b73d3cfbd", "record_hash": "5ae101c0e2de1d940bf92a7a07cf9e30a9665220415c0c6b4f2f755cc2a7fbda"}
{"timestamp": "2026-07-01T02:23:41Z", "trace_id": "78b8f3e2-9c34-40c9-815f-cc6d667b8962", "tool": "deploy", "agent": "release-manager", "idempotency_key": "", "decision": "denied", "reason": "missing_idempotency_key", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "5ae101c0e2de1d940bf92a7a07cf9e30a9665220415c0c6b4f2f755cc2a7fbda", "record_hash": "c53fc25975bdd12d6b8bf3c50fd0654e13431f1d46b461999bc4469280ebc51f"}
{"timestamp": "2026-07-01T02:23:41Z", "trace_id": "cb9c879f-3b54-4308-b7a9-6e9e17df393f", "tool": "deploy", "agent": "release-manager", "idempotency_key": "deploy-demo-001", "decision": "approved", "reason": "registered", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "c53fc25975bdd12d6b8bf3c50fd0654e13431f1d46b461999bc4469280ebc51f", "record_hash": "755046ec2380993db400d5899871c7d9f5dcc5b74825faa1346c5146e13a1a5b"}
{"timestamp": "2026-07-01T02:23:41Z", "trace_id": "62b6e28f-4f45-4520-a869-083943c4c21f", "tool": "deploy", "agent": "design-agent", "idempotency_key": "deploy-demo-002", "decision": "denied", "reason": "unauthorized_agent", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "755046ec2380993db400d5899871c7d9f5dcc5b74825faa1346c5146e13a1a5b", "record_hash": "93df34fcf0b306eb4332327e52f4f6adae1179c67d08d0d13926b8fb82259699"}
{"timestamp": "2026-07-01T02:23:47Z", "trace_id": "93e105ef-6af4-4170-874a-5bffb0f86e85", "tool": "deploy", "agent": "design-agent", "idempotency_key": "k1", "decision": "denied", "reason": "unauthorized_agent", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "93df34fcf0b306eb4332327e52f4f6adae1179c67d08d0d13926b8fb82259699", "record_hash": "c190894d7fdba2c40ec9f3cd6875a97c8e46c3c3a9fe4debba8479eb0a215f91"}
{"timestamp": "2026-07-01T02:23:47Z", "trace_id": "b2be71d6-8032-473d-95c0-f2ef944d6212", "tool": "deploy", "agent": "", "idempotency_key": "k2", "decision": "denied", "reason": "missing_agent_identity", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "c190894d7fdba2c40ec9f3cd6875a97c8e46c3c3a9fe4debba8479eb0a215f91", "record_hash": "f1212cf892abc15e09990820b027d8ee42e799d4b9bc5fc1ad28aad42e2b6a08"}
{"timestamp": "2026-07-01T02:23:47Z", "trace_id": "16a35597-ccd8-49bd-8e48-c47ef68d2d3a", "tool": "deploy", "agent": "release-manager", "idempotency_key": "k3", "decision": "approved", "reason": "registered", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "f1212cf892abc15e09990820b027d8ee42e799d4b9bc5fc1ad28aad42e2b6a08", "record_hash": "b0c1c53649a5da70bbde1a88a5fdb5b10a4627b0e8f6336583204ecfb7a732f1"}
{"timestamp": "2026-07-01T02:23:48Z", "trace_id": "850822a9-d189-4d4b-ab70-3db194568797", "tool": "write_code", "agent": "design-agent", "idempotency_key": "d6d750d0ee93e4b9f9b917a42e3f7f6e8725c63bf412594778810bd91c175084", "decision": "denied", "reason": "unauthorized_agent", "risk_level": "high", "owner": "engineering", "previous_record_hash": "b0c1c53649a5da70bbde1a88a5fdb5b10a4627b0e8f6336583204ecfb7a732f1", "record_hash": "2f4defc69eb4b825e122e46973db98155b42a92b6f6c48a45efd4c5193379a1e"}
{"timestamp": "2026-07-01T02:23:49Z", "trace_id": "0461ed55-ec5f-458e-b43f-707cb246f2d5", "tool": "write_code", "agent": "implement-agent", "idempotency_key": "d6d750d0ee93e4b9f9b917a42e3f7f6e8725c63bf412594778810bd91c175084", "decision": "approved", "reason": "registered", "risk_level": "high", "owner": "engineering", "previous_record_hash": "2f4defc69eb4b825e122e46973db98155b42a92b6f6c48a45efd4c5193379a1e", "record_hash": "183ebe889ddb00e98ecc0e3e4fa0968db50e147c9f59506feb4503f4c167de64"}
{"timestamp": "2026-07-01T02:23:49Z", "trace_id": "0a850cae-8333-436b-a056-a7845d23fc64", "agent": "unknown-agent", "step": "write_code", "tool": "Bash", "command": "/Users/thanhnguyen/Documents/AI/HarnessHkt/Harness_Hakathon/Output_CASAN5_REFINED/AINative_OKR_CASAN5/.specify/scripts/bash/tool-exec.sh 30 -- bash -c echo wrote", "exit_code": 0, "status": "success", "previous_record_hash": "183ebe889ddb00e98ecc0e3e4fa0968db50e147c9f59506feb4503f4c167de64", "record_hash": "d4c20a4f4f0bef73cb642dd9e684b9af0415fb677d713680251ba059d43abca6"}
{"timestamp": "2026-07-01T02:23:50Z", "trace_id": "c356f439-77bc-4815-b352-1e0e0187e150", "agent": "adv", "step": "step-1-srs", "tool": "Bash", "command": "bash -c cp \"$CASAN_INPUT\" \"$CASAN_OUTPUT\"", "exit_code": 0, "status": "success", "previous_record_hash": "d4c20a4f4f0bef73cb642dd9e684b9af0415fb677d713680251ba059d43abca6", "record_hash": "1d47e2fde26a57d65f820e564a0f172c932f86f74b16fecfa0ec011d4eb7a1f8"}
{"timestamp": "2026-07-01T02:23:51Z", "trace_id": "bf633728-b454-4a4d-949f-b52eba6e0c99", "agent": "adv", "step": "step-1-srs", "tool": "Bash", "command": "bash -c cp \"$CASAN_INPUT\" \"$CASAN_OUTPUT\"", "exit_code": 0, "status": "success", "previous_record_hash": "1d47e2fde26a57d65f820e564a0f172c932f86f74b16fecfa0ec011d4eb7a1f8", "record_hash": "63d9867e8bc33627fc355728fbf18d0c96a5aab7c76df21ce4c12fbc489a9575"}
{"timestamp": "2026-07-01T02:23:51Z", "trace_id": "00a374d2-dab2-43cc-bdfa-71a73b8e920d", "tool": "deploy", "agent": "release-manager", "idempotency_key": "k1", "decision": "approved", "reason": "registered", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "63d9867e8bc33627fc355728fbf18d0c96a5aab7c76df21ce4c12fbc489a9575", "record_hash": "3cb9a6f71ba0530d85c439aadee3b5fb7f4e65ce0a93bae5c5f683e035464d1e"}
{"timestamp": "2026-07-01T02:23:52Z", "trace_id": "a775ba64-e5f0-459c-9b36-5fd7d698e7f6", "tool": "deploy", "agent": "release-manager", "idempotency_key": "k2", "decision": "approved", "reason": "registered", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "3cb9a6f71ba0530d85c439aadee3b5fb7f4e65ce0a93bae5c5f683e035464d1e", "record_hash": "315bfcde9672eea13f2da6b17f846267869aab9ece04663a19d871027f582022"}
{"timestamp": "2026-07-01T02:23:52Z", "trace_id": "5d71fa2d-ce61-42f2-8b46-f8a27874b371", "tool": "deploy", "agent": "release-manager", "idempotency_key": "k3", "decision": "denied", "reason": "rate_limit_exceeded(limit=2)", "risk_level": "high", "owner": "release-manager", "previous_record_hash": "315bfcde9672eea13f2da6b17f846267869aab9ece04663a19d871027f582022", "record_hash": "98519945e6e1728bdb4cb9d2da63a9861c9152447b84f3584a5dbe8946985485"}
{"timestamp": "2026-07-01T02:23:56Z", "trace_id": "bf9ae0e4-61cc-48e2-b6b6-a6add126c35c", "agent": "unknown-agent", "step": "test_timeout", "tool": "Bash", "command": "/Users/thanhnguyen/Documents/AI/HarnessHkt/Harness_Hakathon/Output_CASAN5_REFINED/AINative_OKR_CASAN5/.specify/scripts/bash/tool-exec.sh 2 -- sleep 60", "exit_code": 124, "status": "failed", "previous_record_hash": "98519945e6e1728bdb4cb9d2da63a9861c9152447b84f3584a5dbe8946985485", "record_hash": "0683a95c0f4ce9ba96c78c45acb33df702473f3a8c0c8cd0f58bf5862de0618a"}
{"timestamp": "2026-07-01T02:24:05Z", "trace_id": "22168ecf-ed3a-47fe-b28b-262f589fd412", "agent": "unknown-agent", "step": "t4-telemetry-test", "tool": "Bash", "command": "bash -c cp \"$CASAN_INPUT\" \"$CASAN_OUTPUT\"", "exit_code": 0, "status": "success", "previous_record_hash": "0683a95c0f4ce9ba96c78c45acb33df702473f3a8c0c8cd0f58bf5862de0618a", "record_hash": "2a9f0d89d2aa02a2f11c787eca9a319ca9024fb1dc9f69a9cc32c45b84e7d6ba"}