Complete cloud provider patch MVP

casan-next-plans/CASAN_BACKLOG_STATUS.md

@@ -5,7 +5,7 @@
 > cũng làm tiếp được ngay**. Cập nhật mỗi khi hoàn thành một mục.
 >
 > Cập nhật lần cuối: 2026-07-06 · Nhánh làm tiếp từ handoff Claude.
-> Test hiện tại: **218 PASS / 0 FAIL** trên 13 core harness suite; local-prod Docker infra lab **2 PASS / 0 FAIL** (`infra-lab verify`: 7/0 internal checks); `phase3-model-router` riêng **10 PASS / 0 FAIL**; frontend Vitest **16 PASS / 0 FAIL**. Backend `npm test` còn bị chặn bởi test-infra cũ (`schema.prisma` MySQL nhưng `setup-sqlite.mjs` chạy SQLite).
+> Test hiện tại: **218 PASS / 0 FAIL** trên 13 core harness suite; local-prod Docker infra lab **2 PASS / 0 FAIL** (`infra-lab verify`: 7/0 internal checks); `phase3-model-router` riêng **11 PASS / 0 FAIL**; frontend Vitest **16 PASS / 0 FAIL**. Backend `npm test` còn bị chặn bởi test-infra cũ (`schema.prisma` MySQL nhưng `setup-sqlite.mjs` chạy SQLite).
 > Điểm công tâm vẫn quanh **~81/100**, harness thấp nhất 80; TIER 2 infra thật vẫn là trần Strong.
 > Nguồn liên quan: `CASAN_HARDENING_STATUS.md` (chi tiết control) · `evidence/scoring-run-report.md` (điểm).
 
@@ -40,8 +40,8 @@
 | Plan | Trạng thái | Lõi cần làm (bước tiếp theo cho AI kế) |
 |---|:--:|---|
 | **10 Traceability + H3 Eval** | ✅ MVP done+test | Đã nối traceability vào Evidence Pack. Sau MVP: line/symbol-level traceability + H3 eval-set độc lập (nhiều model). |
+| **03 Cloud patch** | 🟡 MVP done+test | `model-call.py` đã hỗ trợ `openai:<model>` và `anthropic:<model>` qua endpoint hard-pin + API key env; parser token usage và malformed payload có deterministic test. Còn live smoke với key thật + billing usage API ground truth. |
 | **02 LLM source-gen** | 📋 chưa bắt đầu | Thay template bằng LLM thật sinh source qua `model-router.sh`; đi qua wrapper H4→H7. Phụ thuộc 03. Bước 1: định contract prompt→file cho 1 module (objectives), gate bằng H3 judge + traceability. |
-| **03 Cloud patch** | 📋 chưa bắt đầu | Bỏ stub trong `model-call.py` cho OpenAI/Anthropic; test bằng key thật (🔌). Bước 1: env `CASAN_MODEL_PRIMARY=openai:…`, xác thực round-trip + ghi provider-usage thật. |
 | **04 Self-improve** | 📋 chưa bắt đầu | Khép vòng `casan improve`: đọc metrics/drift/hallucination → đề xuất vá → chạy lại gate. Phụ thuộc 02, 05. Bước 1: script đọc `metrics.jsonl` + `drift-report.json` → sinh backlog vá tự động. |
 | **05 CI/CD** | 📋 một phần (act_runner/deploy có) | Chuẩn hoá pipeline phát hành package `fpt-casan-sdd-harness` + chạy 12 suite trong CI (Vault+Docker service). Bước 1: `.gitea/workflows/harness-ci.yml` chạy toàn bộ suite + security-gate. |
 | **06 Onboard dự án 2** | 📋 chưa bắt đầu | Chứng minh reuse: cắm 1 repo khác + golden/corpus/input, đăng ký qua `verify-harness-reuse.sh`, không sửa gate. Phụ thuộc 01. |

casan-next-plans/CASAN_HARDENING_STATUS.md

@@ -88,7 +88,7 @@
 | `phase-h4-split-inject-tests.sh` | 8 | **New** — split-injection assembly scan + classifier-inject (B2) |
 | `phase10-traceability-tests.sh` | 3 | **New** — Plan-10 FR→code→test matrix + fail-able missing-test gate |
 | `phase-prod-infra-lab-tests.sh` | 2 | **New optional/local-prod** — Docker Compose infra lab starts + verifies Vault/IdP/MinIO/dashboard/alert/billing |
-| **Total** | **218 core + 2 local-prod infra lab** | Baseline 79 preserved; +139 new hardening/traceability checks. Last full harness run 2026-07-06, 0 fail. Direct `phase3-model-router-tests.sh` adds 10/0; `infra-lab verify` adds 7 internal infra checks. |
+| **Total** | **218 core + 2 local-prod infra lab** | Baseline 79 preserved; +139 new hardening/traceability checks. Last full harness run 2026-07-06, 0 fail. Direct `phase3-model-router-tests.sh` adds 11/0 including deterministic cloud provider parser coverage; `infra-lab verify` adds 7 internal infra checks. |
 
 Run order note: `run-casan4-harness-tests.sh` does `rm -rf .specify/logs`, so run it
 **first** and never concurrently with the other suites.

casan-next-plans/CASAN_PLAN_00_INDEX.md

@@ -21,7 +21,7 @@
 |---|---|---|---|---|
 | 01 | `CASAN_PLAN_01_RESTRUCTURE.md` | Tai cau truc thu muc Phase 0-6 | **File chi tiet dang thieu**; cong viec chua lam | Cao, nhung nen lam tren nhanh rieng |
 | 02 | `CASAN_PLAN_02_LLM_SOURCEGEN.md` | Noi LLM that vao sinh source thay template | Chua bat dau | Cao sau 03 |
-| 03 | `CASAN_PLAN_03_CLOUD_PATCH.md` | Patch cloud/OpenAI/Anthropic, bo stub | **File chi tiet dang thieu**; cong viec chua lam | Cao, can key/provider |
+| 03 | `CASAN_PLAN_03_CLOUD_PATCH.md` | Patch cloud/OpenAI/Anthropic, bo stub | MVP da co + deterministic test; live smoke can key that | Cao cho live provider |
 | 04 | `CASAN_PLAN_04_SELFIMPROVE.md` | Khep vong `casan improve` | Chua bat dau | Trung-Cao sau 02/05 |
 | 05 | `CASAN_PLAN_05_CICD.md` | CI/CD + release package | **File chi tiet dang thieu**; moi co mot phan runner/deploy | Cao cho release gate |
 | 06 | `CASAN_PLAN_06_ONBOARD.md` | Onboard du an that thu 2 | Chua bat dau | Cao de chung minh reuse |
@@ -36,7 +36,7 @@
 
 | Muc | Ket qua | Verify |
 |---|---|---|
-| B4 model-digest pinning | Pin digest `ornith:9b`; mismatch BLOCK mac dinh, WARN rollout | `phase3-model-router-tests.sh` 10/10 |
+| B4 model-digest pinning + cloud parser coverage | Pin digest `ornith:9b`; mismatch BLOCK mac dinh, WARN rollout; OpenAI/Anthropic parser deterministic test | `phase3-model-router-tests.sh` 11/11 |
 | C4 approval IdP/OIDC MVP | `approval-verify.sh` verify RS256 JWT, role, expiry, request binding; co mock IdP/JWKS | `phase-h5-approval-tests.sh` 12/12 |
 | Plan-10 traceability MVP | Parse `FR-*`, map requirement -> code -> test, gate thieu coverage | `phase10-traceability-tests.sh` 3/3 |
 | Docker local-prod infra lab | Vault, IdP/JWKS, MinIO Object Lock, dashboard nginx auth, alert webhook, billing API mock | `phase-prod-infra-lab-tests.sh` 2/2; `infra-lab verify` 7/7 |
@@ -45,8 +45,8 @@
 
 | Uu tien | Viec con lai | Ly do | Buoc dau tien de lam tiep |
 |---|---|---|---|
-| P1 | Plan-03 Cloud patch/OpenAI/Anthropic | De LLM source-gen va billing telemetry that co y nghia | Tao/bo sung `CASAN_PLAN_03_CLOUD_PATCH.md`; wire `CASAN_MODEL_PRIMARY=openai/anthropic` qua `model-call.py`, test round-trip co key |
-| P2 | Plan-02 LLM source-gen dot A | Hien pipeline van sinh artifact bang template deterministic | Dinh contract `generate(step, ctx)` cho `01-srs`/`02-bd`, goi `model-router.sh --role generate`, fallback template |
+| P1 | Plan-02 LLM source-gen dot A | Hien pipeline van sinh artifact bang template deterministic | Dinh contract `generate(step, ctx)` cho `01-srs`/`02-bd`, goi `model-router.sh --role generate`, fallback template |
+| P2 | Plan-03 live cloud smoke | MVP cloud patch da test offline; chua co bang chung key/provider that tren may nay | Khi co `OPENAI_API_KEY`/`ANTHROPIC_API_KEY`, chay smoke va luu evidence token usage that |
 | P3 | Managed prod infra T2 | Docker lab da chay, nhung chua the claim production Strong | Cau hinh enterprise IdP/JWKS, S3 Object Lock/QLDB, KMS default/HSM, dashboard TLS/OIDC, Slack/PagerDuty, billing API that |
 | P4 | Plan-05 CI/CD release gate | Can CI chay tat ca suite va goi package | Tao/bo sung `CASAN_PLAN_05_CICD.md`; them workflow chay 218 core + optional Docker infra lab |
 | P5 | Plan-06 onboard project 2 | Chung minh harness reuse that | Chon domain thu 2, tao app/domain data/golden/corpus, dang ky registry, chay `verify-harness-reuse.sh` |
@@ -109,6 +109,7 @@ bash .specify/scripts/bash/infra-lab.sh verify
 
 - `CASAN_BACKLOG_STATUS.md` la handoff chi tiet nhat ve viec con lai.
 - `CASAN_HARDENING_STATUS.md` la nguon cho control nao da implemented+tested.
+- `CASAN_PLAN_03_CLOUD_PATCH.md` la nguon cho cloud provider patch va cach chay live khi co key.
 - Khong stage cac file audit log runtime neu chi thay doi do chay verify.
 - Backend `npm test` toan repo con blocker cu: Prisma schema MySQL nhung `setup-sqlite.mjs`
   ap migration vao SQLite. Day khong phai loi cua cac plan hardening vua lam.

casan-next-plans/CASAN_PLAN_03_CLOUD_PATCH.md

@@ -0,0 +1,55 @@
+# CASAN PLAN 03 — Cloud Provider Patch (OpenAI/Anthropic)
+
+> Status 2026-07-06: **MVP implemented + deterministic tests added**. The router
+> can call OpenAI/Anthropic when real API keys are present, while local tests
+> verify endpoint pinning, provider usage parsing, and fail-closed malformed
+> payload handling without requiring paid keys.
+
+## Delivered
+
+| Capability | Where | Verification |
+|---|---|---|
+| Model spec prefixes `openai:<model>` and `anthropic:<model>` | `.specify/scripts/bash/model-call.py` | `phase3-model-router-tests.sh` |
+| Endpoint allowlist/SSRF posture | Cloud hosts are hard-pinned to `api.openai.com` and `api.anthropic.com`; Ollama remains `127.0.0.1:11434` only | router SSRF test |
+| Honest unavailable state | Missing `OPENAI_API_KEY` / `ANTHROPIC_API_KEY` exits non-zero; no fake PASS or fake token usage | router cloud-unavailable test |
+| Real provider token usage path | OpenAI `usage.prompt_tokens/completion_tokens`; Anthropic `usage.input_tokens/output_tokens` | deterministic monkeypatched cloud parser test |
+| Malformed provider payload fail-closed | Missing usage/schema exits non-zero via `provider_usage_invalid` | deterministic parser test |
+| Provider telemetry source tags | Usage rows write `openai_api_real_tokens` or `anthropic_api_real_tokens` | `model-call.py` provider log path |
+
+## How to run live when keys exist
+
+```bash
+cd AINative_OKR_CASAN5
+export CASAN_MODEL_PRIMARY=openai:gpt-4o-mini
+export OPENAI_API_KEY=...
+bash .specify/scripts/bash/model-router.sh /tmp/prompt.txt /tmp/out.json --role classify
+
+export CASAN_MODEL_PRIMARY=anthropic:claude-sonnet-4-5
+export ANTHROPIC_API_KEY=...
+bash .specify/scripts/bash/model-router.sh /tmp/prompt.txt /tmp/out.json --role judge
+```
+
+Expected evidence:
+
+- command exits `0` for well-formed provider output;
+- `/tmp/out.json` has non-zero `input_tokens` and `output_tokens`;
+- `.specify/logs/level5/provider-usage.jsonl` gets a row with provider
+  `openai` or `anthropic` and cost source `*_api_real_tokens`;
+- malformed or unreachable provider exits non-zero and does not fabricate a
+  successful verdict.
+
+## Remaining production work
+
+| Priority | Work | Done when |
+|---|---|---|
+| P1 | Run live smoke with real org keys on the target Mac/CI | Evidence file records real provider response, token counts, and no secret leakage |
+| P2 | Add provider cost lookup table for selected models | `cost_usd` is computed from current provider pricing instead of `0.0` |
+| P3 | Wire real provider billing usage API, not only per-call response usage | H6 reconcile uses OpenAI/Anthropic ground truth APIs with schema-versioned fetchers |
+| P4 | Add model allowlist policy for approved cloud models | Unknown cloud model names require approval or block |
+| P5 | Decide default failover order for local -> cloud or cloud -> local | `model-fallback.sh` policy is explicit per role |
+
+## Notes for Plan-02
+
+Plan-02 source generation should call `model-router.sh --role generate`, not
+provider SDKs directly. This keeps H4 prompt handling, H6 telemetry, endpoint
+allowlisting, and provider usage logging in one path.

casan-next-plans/CASAN_PLAN_07_PRODUCTION_HARDENING.md

@@ -19,7 +19,7 @@
 
 ## 2. Thang điểm sẵn sàng production (0–5, cao = tốt)
 
-> ✅ **CẬP NHẬT 2026-07-06 — Track A + C-MVP + Evidence Pack + H5/H6/deep-gap hardening + Plan-10 traceability ĐÃ LÀM + TEST (218 core checks, 0 fail; model-router riêng 10/0). Local production-like Docker infra lab cũng đã có (2/0, verify nội bộ 7/0).**
+> ✅ **CẬP NHẬT 2026-07-06 — Track A + C-MVP + Evidence Pack + H5/H6/deep-gap hardening + Plan-10 traceability ĐÃ LÀM + TEST (218 core checks, 0 fail; model-router riêng 11/0). Local production-like Docker infra lab cũng đã có (2/0, verify nội bộ 7/0).**
 > Bảng dưới có cột **Baseline → Nay**. Điểm chấm CÔNG TÂM (0–100, theo `casan_harness_assessment.md`):
 > **H4 = 80 · H5 = 76→80 ⬆ · H6 = 79→80 ⬆ · trung bình 7 harness ~80.9/100 · không còn harness nào dưới 80 → CASAN Level 4 (vững ngưỡng)**.
 > Nguồn: `00_SUBMISSION_PACKAGE/evidence/scoring-run-report.md`. Chi tiết implemented-vs-planned: `CASAN_HARDENING_STATUS.md`.