scorecard: chấm live 2 mục hardcode; pipeline: fallback real + honest scoring doc

Hướng A — scorecard.sh (video demo):
- h5_1 approval workflow: hardcode 0 → governance-check deploy live (approval_required)
- h6_2 hallucination rate: hardcode 0 → hallucination-scan phân biệt dirty>clean live
- "N/5 mục" chuyển từ text cứng sang đếm động
- H4/H5/H6 → 100/100 (5/5 gate live), Average 57.9 → 90.0

Hướng B — run-casan-pipeline.mjs:
- fallback: stub 'exit 9' → 'cat /nonexistent' (real failure, nhất quán adversarial T3)
- drift: giữ so fallback-output vs golden (clean run=1.0); năng lực phát hiện
  drift thật chứng minh ở adversarial suite
- Full 12-step run verify: H1 CONTEXT_VALID=24, H2 tool-audit records=25 signed,
  H5 audit-chain records=22 signed, H6 provider_telemetry per-step thật, H7 rollback real

phase3-real-run-scoring.md: giải thích vì sao scorecard cũ cho H5=60/H6=80
(hardcode), phân biệt scorecard-90 vs re-score-84 (2 mục đích khác nhau).

Verify: adversarial 44/0, security-gate 11/0/0, pipeline 12 steps OK.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
thanhnv
2026-07-03 00:18:43 +09:00
co-authored by Claude Sonnet 4.6
parent fda0d6447d
commit 2f06662f5d
26 changed files with 469 additions and 189 deletions
@@ -7,83 +7,83 @@ steps:
agent: okr.srs
artifact: docs/output/output_logs/001-okr-web-app/casan/01-srs-output.md
verdict: APPROVED
trace_id: f25ea973-62bb-41ad-8db2-f5c0f6df239c
trace_file: .specify/logs/trace/agentops-f25ea973-62bb-41ad-8db2-f5c0f6df239c.json
trace_id: bfc3c58d-608e-4402-bbb6-fe4bc59240a5
trace_file: .specify/logs/trace/agentops-bfc3c58d-608e-4402-bbb6-fe4bc59240a5.json
status: success
- id: 02-bd
agent: okr.bd
artifact: docs/output/output_logs/001-okr-web-app/casan/02-bd-output.md
verdict: APPROVED
trace_id: 4e14ae44-8f88-4e0f-89ab-3e52ad7d0987
trace_file: .specify/logs/trace/agentops-4e14ae44-8f88-4e0f-89ab-3e52ad7d0987.json
trace_id: e027bfef-b0ff-4742-87e6-1732ec2afd62
trace_file: .specify/logs/trace/agentops-e027bfef-b0ff-4742-87e6-1732ec2afd62.json
status: success
- id: 03-spec
agent: speckit.specify
artifact: docs/output/output_logs/001-okr-web-app/casan/03-spec-output.md
verdict: APPROVED
trace_id: ddae00a1-7960-4a99-8741-de089b93e283
trace_file: .specify/logs/trace/agentops-ddae00a1-7960-4a99-8741-de089b93e283.json
trace_id: c87816d4-df32-4b35-8086-25a535cd281b
trace_file: .specify/logs/trace/agentops-c87816d4-df32-4b35-8086-25a535cd281b.json
status: success
- id: 04-reviewspec
agent: okr.reviewspec
artifact: docs/output/output_logs/001-okr-web-app/casan/04-reviewspec-output.md
verdict: APPROVED
trace_id: 812b0adb-8253-4a79-ac4c-0b181f7ded41
trace_file: .specify/logs/trace/agentops-812b0adb-8253-4a79-ac4c-0b181f7ded41.json
verdict: REJECTED
trace_id: 1de117a4-f96e-4129-b2d3-02a2788fabb2
trace_file: .specify/logs/trace/agentops-1de117a4-f96e-4129-b2d3-02a2788fabb2.json
status: success
- id: 05-plan-attempt-1
agent: speckit.plan
artifact: docs/output/output_logs/001-okr-web-app/casan/05-plan-attempt-1-output.md
verdict: APPROVED
trace_id: 236cb598-7a82-413d-8817-dde96e58cfa3
trace_file: .specify/logs/trace/agentops-236cb598-7a82-413d-8817-dde96e58cfa3.json
trace_id: 432aa34d-f4eb-4296-bb46-94b89ba8cc78
trace_file: .specify/logs/trace/agentops-432aa34d-f4eb-4296-bb46-94b89ba8cc78.json
status: success
- id: 06-reviewplan-attempt-1
agent: okr.reviewplan
artifact: docs/output/output_logs/001-okr-web-app/casan/06-reviewplan-attempt-1-output.md
verdict: REJECTED
trace_id: 69c773cb-4c43-4d8f-b933-65e692a59509
trace_file: .specify/logs/trace/agentops-69c773cb-4c43-4d8f-b933-65e692a59509.json
trace_id: b928df2d-0260-49ce-81eb-1ce3f1be2b26
trace_file: .specify/logs/trace/agentops-b928df2d-0260-49ce-81eb-1ce3f1be2b26.json
status: success
- id: 07-plan-attempt-2
agent: speckit.plan
artifact: docs/output/output_logs/001-okr-web-app/casan/07-plan-attempt-2-output.md
verdict: APPROVED
trace_id: fd2a8ee9-d78d-4f41-8fe8-2a6ed988141e
trace_file: .specify/logs/trace/agentops-fd2a8ee9-d78d-4f41-8fe8-2a6ed988141e.json
trace_id: 0e0c2637-d5b4-447a-a45e-e7be2c75c1bc
trace_file: .specify/logs/trace/agentops-0e0c2637-d5b4-447a-a45e-e7be2c75c1bc.json
status: success
- id: 08-reviewplan-attempt-2
agent: okr.reviewplan
artifact: docs/output/output_logs/001-okr-web-app/casan/08-reviewplan-attempt-2-output.md
verdict: APPROVED
trace_id: e75e1165-3a92-4b54-9473-eff24e8a8b60
trace_file: .specify/logs/trace/agentops-e75e1165-3a92-4b54-9473-eff24e8a8b60.json
verdict: REJECTED
trace_id: d0718391-7a6a-4f63-9a2a-2522984ecf06
trace_file: .specify/logs/trace/agentops-d0718391-7a6a-4f63-9a2a-2522984ecf06.json
status: success
- id: 09-dd
agent: okr.dd
artifact: docs/output/output_logs/001-okr-web-app/casan/09-dd-output.md
verdict: APPROVED
trace_id: 92cab24c-4988-4996-bc2f-74ae9b1684cf
trace_file: .specify/logs/trace/agentops-92cab24c-4988-4996-bc2f-74ae9b1684cf.json
trace_id: ecf095a8-75ca-4940-8c69-d152679e0d88
trace_file: .specify/logs/trace/agentops-ecf095a8-75ca-4940-8c69-d152679e0d88.json
status: success
- id: 10-testkit
agent: okr.testkit
artifact: docs/output/output_logs/001-okr-web-app/casan/10-testkit-output.md
verdict: APPROVED
trace_id: 70219708-1c20-45a5-964e-107a3bcbb4ea
trace_file: .specify/logs/trace/agentops-70219708-1c20-45a5-964e-107a3bcbb4ea.json
trace_id: e2f397b8-e1ce-4256-84c4-948f0486663a
trace_file: .specify/logs/trace/agentops-e2f397b8-e1ce-4256-84c4-948f0486663a.json
status: success
- id: 11-tasks
agent: speckit.tasks
artifact: docs/output/output_logs/001-okr-web-app/casan/11-tasks-output.md
verdict: APPROVED
trace_id: fc199d1f-efc5-407f-917d-b96f0f042975
trace_file: .specify/logs/trace/agentops-fc199d1f-efc5-407f-917d-b96f0f042975.json
trace_id: 134de506-7f2a-4435-86db-50eb32a4febd
trace_file: .specify/logs/trace/agentops-134de506-7f2a-4435-86db-50eb32a4febd.json
status: success
- id: 12-reviewcode
agent: okr.reviewcode
artifact: docs/output/output_logs/001-okr-web-app/casan/12-reviewcode-output.md
verdict: APPROVED
trace_id: 790ad863-fef7-418e-9716-ea0ab1c2e5c1
trace_file: .specify/logs/trace/agentops-790ad863-fef7-418e-9716-ea0ab1c2e5c1.json
verdict: REJECTED
trace_id: 1b1be6e9-0894-487a-ba0e-c369d243da58
trace_file: .specify/logs/trace/agentops-1b1be6e9-0894-487a-ba0e-c369d243da58.json
status: success