gitea-admin/CASAN
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix T1+T4: wire rollback into pipeline; real Ollama telemetry in metrics

Browse Source 
T1 (H7): casan-step.mjs now calls rollback-manager.sh checkpoint before
overwriting plan.md at attempt-2, writes tx-id to plan.checkpoint.txid
sidecar, and executes rollback on REJECTED verdict. rollback-transactions.jsonl
records a real cp restore command. Adversarial test: checkpoint exists,
real cp command recorded, plan hash matches pre-overwrite content.

T4 (H6): casan-harness.sh exports CASAN_STEP_NAME=$ACTION_NAME before
agent-metrics.sh so nested model calls (model-call.py) and the provider-
cost-lookup.py query share the same step label. metrics.jsonl now writes
cost_source=provider_telemetry instead of word_count_estimate when a real
Ollama call is made within the same step. Adversarial test: verified with
CASAN_STEP_NAME=t4-telemetry-test end-to-end.

adversarial-harness-tests.sh: 40 → 44 PASS / 0 FAIL (+3 T1, +1 T4)
security-gate.sh: PASS=10 FAIL=0 SKIP=0 (verified, local ornith:9b)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
thanhnv
2026-07-01 11:25:50 +09:00
co-authored by Claude Sonnet 4.6
parent f74a5b6e42
commit a8edbea534
18 changed files with 284 additions and 159 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AINative_OKR_CASAN5/.specify/agentops/alerts.log
+5 -5
View File
@@ -1,5 +1,5 @@
{"timestamp":"2026-07-01T02:01:42Z","trace_id":"50a5bd13-1bc7-45f0-828d-42027d08c86e","severity":"WARN","resource":{"service.name":"demo.agent","service.version":"1.0.0"},"body":{"message":"Alert triggered: hallucination-suspected","alert.type":"hallucination-suspected","step.name":"step-1-srs"},"attributes":{"latency_ms":212,"status":"success"}}
{"timestamp":"2026-07-01T02:01:43Z","trace_id":"2c0bc2f4-b444-45a9-b485-5c53093ea174","severity":"WARN","resource":{"service.name":"demo.agent","service.version":"1.0.0"},"body":{"message":"Alert triggered: execution-failed","alert.type":"execution-failed","step.name":"failing-step"},"attributes":{"latency_ms":192,"status":"failed"}}
{"timestamp":"2026-07-01T02:01:55Z","trace_id":"50ce3010-3720-4ad5-9be9-40bc32ac177d","severity":"WARN","resource":{"service.name":"unknown-agent","service.version":"1.0.0"},"body":{"message":"Alert triggered: execution-failed","alert.type":"execution-failed","step.name":"unknown-step"},"attributes":{"latency_ms":208,"status":"failed"}}
{"timestamp":"2026-07-01T02:01:56Z","trace_id":"7e0039e4-57db-4b3a-893f-653382dadbc5","severity":"WARN","resource":{"service.name":"adv","service.version":"1.0.0"},"body":{"message":"Alert triggered: hallucination-suspected","alert.type":"hallucination-suspected","step.name":"step-1-srs"},"attributes":{"latency_ms":194,"status":"success"}}
{"timestamp":"2026-07-01T02:02:02Z","trace_id":"5d94660f-b81c-463a-8495-2b1bacd18888","severity":"WARN","resource":{"service.name":"unknown-agent","service.version":"1.0.0"},"body":{"message":"Alert triggered: execution-failed","alert.type":"execution-failed","step.name":"unknown-step"},"attributes":{"latency_ms":2236,"status":"failed"}}
{"timestamp":"2026-07-01T02:23:35Z","trace_id":"01675f76-25f1-4f79-a55c-2571ecbc7314","severity":"WARN","resource":{"service.name":"demo.agent","service.version":"1.0.0"},"body":{"message":"Alert triggered: hallucination-suspected","alert.type":"hallucination-suspected","step.name":"step-1-srs"},"attributes":{"latency_ms":193,"status":"success"}}
{"timestamp":"2026-07-01T02:23:35Z","trace_id":"47204bd3-200b-42d6-b724-9560d737ad6c","severity":"WARN","resource":{"service.name":"demo.agent","service.version":"1.0.0"},"body":{"message":"Alert triggered: execution-failed","alert.type":"execution-failed","step.name":"failing-step"},"attributes":{"latency_ms":175,"status":"failed"}}
{"timestamp":"2026-07-01T02:23:49Z","trace_id":"0a850cae-8333-436b-a056-a7845d23fc64","severity":"WARN","resource":{"service.name":"unknown-agent","service.version":"1.0.0"},"body":{"message":"Alert triggered: execution-failed","alert.type":"execution-failed","step.name":"write_code"},"attributes":{"latency_ms":195,"status":"failed"}}
{"timestamp":"2026-07-01T02:23:50Z","trace_id":"c356f439-77bc-4815-b352-1e0e0187e150","severity":"WARN","resource":{"service.name":"adv","service.version":"1.0.0"},"body":{"message":"Alert triggered: hallucination-suspected","alert.type":"hallucination-suspected","step.name":"step-1-srs"},"attributes":{"latency_ms":197,"status":"success"}}
{"timestamp":"2026-07-01T02:23:56Z","trace_id":"bf9ae0e4-61cc-48e2-b6b6-a6add126c35c","severity":"WARN","resource":{"service.name":"unknown-agent","service.version":"1.0.0"},"body":{"message":"Alert triggered: execution-failed","alert.type":"execution-failed","step.name":"test_timeout"},"attributes":{"latency_ms":2206,"status":"failed"}}