Files
CASAN/docs/plans/CASAN_PLAN_10_TRACEABILITY_EVAL.md
T
thanhnvandClaude Opus 4.8 4918012199 docs(plans): sync roadmap status + paths to post-restructure state
- Plan-01 marked ✅ DONE (INDEX table + P3 tier + BACKLOG row + plan header).
- Plan-06 / Plan-12 dependency on 01 satisfied → 🔓 unblocked (headers + BACKLOG rows).
- Repoint command/path refs in all plans (except Plan-01's migration narrative):
  .specify/{scripts,tests,security,config,templates,governance} -> packages/casan-harness/...;
  golden-runs/traceability-map/docs-input -> apps/okr/domain/...; `cd AINative_OKR_CASAN5`
  -> `cd $(git rev-parse --show-toplevel)`; fix relative links + Plan-13 control-plane location.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 14:57:01 +09:00

2.2 KiB

CASAN PLAN 10 — Traceability REQ→Code→Test + H3 Eval

Status 2026-07-06: MVP implemented + tested. Scope is deterministic traceability for the OKR sample app; broader H3 eval-set expansion remains a platform follow-up.

MVP delivered

Capability Where Verification
Parse FR-* from apps/okr/domain/input/okr-requirement.md packages/casan-harness/scripts/bash/traceability-matrix.py phase10-traceability-tests.sh
Declarative FR→code→test map apps/okr/domain/traceability-map.json gate checks every mapped file exists
Gate every FR has >=1 code file and >=1 test file traceability-matrix.py --gate missing FR test coverage fails
Symbol/line-level refs (object form {file, symbols, lines}, backward compatible with plain path) traceability-matrix.py (resolve_files, symbol_present) phase10 symbol/line checks (missing symbol/line ⇒ gate FAIL)
Evidence Pack includes traceability matrix evidence-pack-build.py phase3-evidence-pack-tests.sh expects 13 files
CI enforcement ci-harness-gate.sh runs phase10-traceability CI fails a new FR without code+test (+symbol) coverage

Verify môi trường: chạy trong WSL/Linux (msys + Python Windows lệch dịch path → fail giả). phase10 = 6/0 trong WSL.

Current result

python3 packages/casan-harness/scripts/bash/traceability-matrix.py --gate
# TRACEABILITY_MATRIX requirements=5 pass=5 fail=0 symbols=2/2 ...

bash packages/casan-harness/tests/phase10-traceability-tests.sh   # chạy trong WSL
# TRACEABILITY SUMMARY: PASS=6 FAIL=0

Generated artifact:

  • docs/output/casan/traceability-matrix.json (nạp thêm symbol_refs, missing_symbols, missing_lines)

Next steps

Priority Work Done when
P1 ✅ Symbol/line-level refs đã làm matrix trỏ tới symbol/line chính xác; missing ⇒ gate FAIL
P3 ✅ Enforce traceability in release CI đã có ci-harness-gate.sh chạy phase10
P2 Expand H3 eval-set beyond the OKR sample app independent eval corpus runs through judge gates
P4 Enrich symbol refs cho các FR còn lại (FR-02→FR-05) mỗi FR có ≥1 symbol ref được verify