# CASAN Phase 3 — Wave 5 Prompt (New Session)

> Copy toàn bộ nội dung bên dưới vào session mới.

---

## Context

Dự án: `Output_CASAN5_REFINED/AINative_OKR_CASAN5` — OKR web app (NestJS + Prisma + SQLite + React + Vite + Tailwind), được đánh giá theo framework **FPT CASAN** gồm 7 Harness H1–H7 (thang điểm 0–100).

Mục tiêu ban đầu: tất cả harness > 80, ideally ~90. **Hiện tại ~84 average, tất cả > 80 — CASAN Level 4 genuine.**

---

## Trạng thái sau Wave 4 + T1/T4 fixes (commit `a0df8fd`, 2026-07-01)

### Điểm

| Harness | Score |
|---|:---:|
| H1 Context | 85 |
| H2 Tool | 84 |
| H3 Evaluation | 78 |
| H4 Security | 86 |
| H5 Governance | 83 |
| H6 AgentOps | 84 |
| H7 Orchestration | 87 |
| **Average** | **~84** |

### Verify commands (tất cả phải PASS trước khi làm gì)

```bash
cd Output_CASAN5_REFINED/AINative_OKR_CASAN5
bash .specify/tests/run-casan4-harness-tests.sh           # 35 PASS / 0 FAIL
bash .specify/tests/adversarial-harness-tests.sh          # 44 PASS / 0 FAIL
bash .specify/scripts/bash/verify-audit-chain.sh          # AUDIT_CHAIN_VALID anchor=signed
bash .specify/scripts/bash/verify-tool-audit.sh           # TOOL_AUDIT_VALID anchor=signed
bash .specify/scripts/bash/security-gate.sh               # PASS=10 FAIL=0 SKIP=0
npm test -w frontend                                       # 16 PASS / 0 FAIL
```

### Environment

- macOS, shell: zsh
- **Ollama local**: `ornith:9b` (qwen3.5 family) tại `127.0.0.1:11434` — UP
- Node 20+, Python 3.x (lệnh: `python` không phải `python3`)
- Không có `ANTHROPIC_API_KEY` / `OPENAI_API_KEY`
- Không có AWS/KMS/Object Lock

---

## Những gì đã xong (Wave 1–4 + T1/T4)

- **H4**: Semantic injection via Ollama ornith:9b, 30-sample corpus (recall=0.85), artifact indirect injection scanner, secrets scan, tool timeout, circuit breaker, no-bypass scan, SSRF guard
- **H5**: RSA-anchored audit chain (SHA-256 + RSA anchor), signed policy manifest, private key NOT in repo (gitignored `*.pem`)
- **H2**: Per-agent least privilege (tool-registry-gate.sh), runtime rate-limit counter, per-call JSON-schema validation (validate-tool-input.sh), tool-exec.sh timeout wired into harness
- **H3**: 16 Vitest frontend tests (Badge, ProgressBar, Zod schema, progress calc, API error), fail-before cycle proven
- **H6**: cost_source=provider_telemetry (real Ollama tokens via CASAN_STEP_NAME propagation), cost-spike detection, hallucination scanner
- **H7**: Rollback wired into pipeline (`casan-step.mjs` → `rollback-manager.sh checkpoint/execute`), drift-detect real difflib, model fallback with real failure (not exit 9 stub)
- **H1**: context-validate CONTEXT_VALID checked=24, TTL/staleness logic implemented
- **Adversarial suite**: 44 PASS / 0 FAIL (up from 22 at start of Phase 3)
- **Security gate**: PASS=10 FAIL=0 SKIP=0 (with local Ollama)

---

## Residual gaps (limiting ~84 → 90)

Những gap này cần infra hoặc scope lớn hơn:

| Task | Harness | Gap | Cần gì |
|---|:---:|---|---|
| **CI gate integration** | H3 | Không có GitHub Actions / local CI chạy tests tự động | Cần CI pipeline config |
| **E2E / browser tests** | H3 | Không có Playwright/Cypress | Cần browser test infra |
| **Cloud model recall** | H4 | Recall=0.85, 15% novel paraphrases vẫn evade 9B model | Cần `ANTHROPIC_API_KEY` hoặc model 70B+ |
| **KMS/HSM signing** | H5 | Private key local, không có cloud key management | Cần AWS KMS hoặc tương đương |
| **Full pipeline re-run** | H6 | Pipeline chưa được re-run end-to-end trong Phase 3 | Cần `run-casan-pipeline.mjs` chạy thành công với real app |
| **Design artifact depth** | H1 | 12/24 references là stub traces (retention_gap) | Cần re-run pipeline để tạo real traces |

**Nếu có ANTHROPIC_API_KEY**: Có thể nâng H4 recall từ 0.85 → ~0.95 bằng cách thêm Anthropic/GPT làm secondary judge.

---

## Stack kỹ thuật quan trọng

### Model call
- Script: `.specify/scripts/bash/model-router.sh` → `.specify/scripts/bash/model-call.py`
- Ollama endpoint: `http://127.0.0.1:11434/api/generate`
- `think: false` required cho ornith:9b (thinking model)
- `num_predict=16`, `temperature=0` cho classify/judge
- SSRF guard: chỉ cho phép `127.0.0.1:11434`

### Pipeline
- Orchestrator: `scripts/casan-step.mjs` (Node.js ESM)
- Harness wrapper: `.specify/scripts/bash/casan-harness.sh`
- Steps: `01-srs → 02-bd → 03-spec → 04-reviewspec → 05-plan → 06-reviewplan → 07-dd → 08-testkit → 09-tasks → 10-reviewcode`
- Model judge gate (WP-B): wired vào steps 04, 06, 10 — AND(rule, model), SKIP non-blocking

### Audit chain
- `.specify/logs/audit/audit.jsonl` — append-only, SHA-256 hash chain
- `.specify/logs/audit/audit-head.sig` — RSA signature của head hash
- Private key: `.specify/level5/central-governance/policy-private.pem` (gitignored, local only)
- Public key: `.specify/level5/central-governance/policy-public.pem` (in repo, cho verify)
- Verify: `bash .specify/scripts/bash/verify-audit-chain.sh`

### Rollback (mới wired - T1)
- `rollback-manager.sh checkpoint <file>` → backup + ghi restore command thật vào `rollback-transactions.jsonl`
- `rollback-manager.sh execute <tx-id>` → chạy `bash -c "cp backup original"` thật
- Sidecar: `docs/output/specs/001-okr-web-app/plan.checkpoint.txid`
- Wired trong `casan-step.mjs` step 05-plan (attempt≠1) và step 06-reviewplan (REJECTED)

### Provider telemetry (mới fix - T4)
- `casan-harness.sh` export `CASAN_STEP_NAME=$ACTION_NAME`
- `model-call.py` đọc `CASAN_STEP_NAME` từ env → ghi vào `provider-usage.jsonl`
- `agent-metrics.sh` → `provider-cost-lookup.py` match by step name → `cost_source=provider_telemetry`

---

## Integrity rules (KHÔNG được vi phạm)

1. **Không fake PASS/APPROVED/verdict/token/cost/rollback/audit** — mọi kết quả phải từ execution thật
2. **Không hardcode expected PASS** — mọi test phải có thể FAIL
3. **Không dùng `.specify/tests/generate-casan-demo-context.py`**
4. **Không gọi regex-only logic là semantic hoặc model-based**
5. **Không claim cloud/hybrid judging nếu không có real API key**
6. **Không claim H5 WORM/KMS** — AWS không được cấu hình
7. **Không claim target scores đạt được nếu chưa có real audit**
8. **Nếu infra thiếu → viết BLOCKED hoặc SKIPPED với lý do rõ ràng**
9. **Mọi fix phải có fail-before proof** (test phải FAIL trước fix, PASS sau fix)
10. **Người build không tự score** (builder ≠ reviewer)

---

## Key files

| File | Vai trò |
|---|---|
| `scripts/casan-step.mjs` | Pipeline orchestrator (246+18 dòng, có checkpoint/rollback T1) |
| `.specify/scripts/bash/casan-harness.sh` | Unified harness (H4→H5→H2→H6→H4) |
| `.specify/scripts/bash/model-call.py` | Model router workhorse (Ollama, SSRF guard, fail-closed) |
| `.specify/scripts/bash/security-check.sh` | H4 input/output filter + semantic injection |
| `.specify/scripts/bash/governance-check.sh` | H5 policy enforcement |
| `.specify/scripts/bash/tool-registry-gate.sh` | H2 per-agent permission + rate-limit |
| `.specify/scripts/bash/rollback-manager.sh` | H7 checkpoint/execute real restore |
| `.specify/scripts/bash/agent-metrics.sh` | H6 token/cost/latency/hallucination |
| `.specify/scripts/bash/drift-detect.sh` | H7 difflib similarity |
| `.specify/scripts/bash/model-fallback.sh` | H7 primary/fallback runner |
| `.specify/scripts/bash/security-gate.sh` | One-command gate (10 gates) |
| `.specify/tests/adversarial-harness-tests.sh` | 44-test adversarial suite |
| `.specify/tests/run-casan4-harness-tests.sh` | 35-test harness suite |
| `.specify/security/redteam-corpus.jsonl` | 30-sample red team (20 injection + 10 benign) |
| `frontend/src/__tests__/okr.test.tsx` | 16 Vitest frontend tests |
| `docs/output/casan/phase3-final-rescore.md` | Final re-score với evidence |
| `docs/output/casan/TEAM-HANDOFF-PLAN.md` | Task list T1–T11, model assignment guide |

---

## Gợi ý việc tiếp theo (nếu muốn tiếp tục push → 90)

Theo thứ tự ROI cao nhất:

1. **CI gate (H3 +3~5)**: Thêm `.github/workflows/ci.yml` chạy `npm test -w frontend` + `bash adversarial-harness-tests.sh` tự động trên mỗi PR
2. **Pipeline re-run (H6 +2~3)**: Chạy `run-casan-pipeline.mjs` đầy đủ (macOS, Ollama lên, node available) để lấy real per-step telemetry
3. **E2E tests (H3 +2~3)**: Viết Playwright tests cho login, create objective, update progress (cần npm run dev -w backend, dev -w frontend)
4. **Cloud recall (H4 +3)**: Nếu có ANTHROPIC_API_KEY → thêm Anthropic làm secondary judge, consensus voting 2/3

Mỗi task phải có **fail-before test → fix → pass-after test** cycle, không hardcode.