CASAN/casan-next-plans/CASAN_PLAN_10_TRACEABILITY_EVAL.md

CASAN PLAN 10 — Traceability REQ→Code→Test + H3 Eval

Status 2026-07-06: MVP implemented + tested. Scope is deterministic traceability for the OKR sample app; broader H3 eval-set expansion remains a platform follow-up.

MVP delivered

Capability	Where	Verification
Parse FR-* from docs/input/okr-requirement.md	.specify/scripts/bash/traceability-matrix.py	phase10-traceability-tests.sh
Declarative FR→code→test map	.specify/traceability-map.json	gate checks every mapped file exists
Gate every FR has >=1 code file and >=1 test file	traceability-matrix.py --gate	missing FR test coverage fails
Evidence Pack includes traceability matrix	evidence-pack-build.py	phase3-evidence-pack-tests.sh expects 13 files

Current result

python3 .specify/scripts/bash/traceability-matrix.py --gate
# TRACEABILITY_MATRIX requirements=5 pass=5 fail=0 ...

bash .specify/tests/phase10-traceability-tests.sh
# TRACEABILITY SUMMARY: PASS=3 FAIL=0

Generated artifact:

• AINative_OKR_CASAN5/docs/output/casan/traceability-matrix.json

Next steps

Priority	Work	Done when
P1	Add traceability line references or symbol references, not only file paths	matrix can point to precise code/test locations
P2	Expand H3 eval-set beyond the OKR sample app	independent eval corpus runs through judge gates
P3	Enforce traceability in release CI	CI blocks a new FR without code+test coverage