thanhnvandClaude Opus 4.8
4cc78f74eb
- CASAN_HARDENING_STATUS.md: canonical implemented/scaffold/planned record with per-control test mapping and test inventory (baseline 79 + 61 new = 140 checks). - README claim boundary: separate implemented+tested controls from planned; explicitly does NOT claim full production-readiness (Track B, C-Gov/Ops, true sandbox isolation, IdP/WORM still planned). - INDEX status table: Plan-07 Track A + C-MVP done, Plan-09 MVP done. - video guide: core demo battery counts unchanged (hardening lives in separate suites); added deep-dive commands + 3 Q&A rows + sandbox honesty note. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
75 lines
5.3 KiB
Markdown
75 lines
5.3 KiB
Markdown
# CASAN Production Hardening — Implementation Status
|
||
|
||
> Live record of what has been **implemented and tested** from Plan-07
|
||
> (Production Hardening) and Plan-09 (Evidence Pack), versus what is still
|
||
> **planned**. Honest by design: a control is only listed as done when it has an
|
||
> executable test that would fail if the control were removed.
|
||
>
|
||
> Labels: **[implemented+tested]** · **[scaffold+tested]** (works, but a stronger
|
||
> production form is documented as TODO) · **[planned]** (not built).
|
||
|
||
## 1. What is implemented now
|
||
|
||
### Phase 1 — Plan-07 Track A (low-risk hardening) — [implemented+tested]
|
||
|
||
| ID | Control | Where | Test |
|
||
|---|---|---|---|
|
||
| A1 | H4 semantic **strict fail-closed** (`CASAN_SECURITY_STRICT=1`): model unavailable ⇒ BLOCK, never silent SKIP; non-strict logs `SEMANTIC_SKIPPED` loudly | `security-check.sh` | phase1 A1 |
|
||
| A2 | H4 **unicode/encoding normalization**: NFKC + zero-width strip + homoglyph fold + base64/hex decode & rescan | `security-check.sh`, `unicode-normalize.py`, `decode-suspicious.py` | phase1 A2 |
|
||
| A3 | H4 **tool-output injection scan** before output re-enters model context (mode off/warn/block; strict⇒block) | `tool-output-scan.sh`, `casan-harness.sh` | phase1 A3 |
|
||
| A4 | H5 **telemetry integrity**: provider-usage + cost metrics bound to a signed manifest head (tamper ⇒ mismatch; re-forge ⇒ signature invalid) | `telemetry-integrity.sh`, `sign-audit-head.sh` | phase1 A4 |
|
||
| A5 | H6 **cost controls**: absolute per-call cap + cumulative budget + cold-start protection, keeping median×mult spike | `cost-spike-detect.sh` | phase1 A5 |
|
||
| A6 | **Benign corpus (EN/VI/JA, 95 samples) + FP budget gate**: FP ≤ 3%, adversarial block ≥ 95%, CRITICAL = 100% | `benign-fp-report.sh`, `benign-corpus/`, `redteam-vectors.jsonl` | phase1 A6 |
|
||
|
||
### Phase 2 — Plan-07 Track C-MVP (production minimum bar) — mixed
|
||
|
||
| ID | Control | Status | Where | Test |
|
||
|---|---|---|---|---|
|
||
| C1 | **Tool authorization / action gating** (ALLOW/WARN/REQUIRE_APPROVAL/BLOCK): sensitive-file writes, destructive/remote-exec commands, dep-install & network egress | [implemented+tested] | `action-gate.sh` | phase2 C1 |
|
||
| C2 | **Supply-chain gate**: manifest diff, denylist + typosquat + dangerous lifecycle-script block, new-dep approval, dep-diff report | [implemented+tested] | `supply-chain-gate.sh`, `supply-chain-scan.py` | phase2 C2 |
|
||
| C3 | **Data-exfil guard**: secret-to-cloud & artifact-leaks-env ⇒ BLOCK; PII-in-audit ⇒ mask | [implemented+tested] | `data-exfil-guard.sh` | phase2 C3 |
|
||
| C6 | **Runtime sandbox**: static policy block (read ~/.ssh, net egress, fork bomb, write-outside-workspace, huge-file) + ulimit/timeout backstops | [scaffold+tested] | `sandbox-run.sh` | phase2 C6 |
|
||
|
||
### Phase 3 — Plan-09 Evidence Pack (MVP) — [implemented+tested]
|
||
|
||
| Control | Where | Test |
|
||
|---|---|---|
|
||
| `casan pack` / `casan verify-pack` (mapped to `evidence-pack.sh`): standard 12-file pack, hash manifest, signed head, tamper-evident verify, certified-run gate (no false certification, no silent skip) | `evidence-pack.sh`, `evidence-pack-build.py`, `evidence-pack-verify.py` | phase3-evidence-pack |
|
||
|
||
## 2. Test inventory (all suites)
|
||
|
||
| Suite | Checks | Purpose |
|
||
|---|---|---|
|
||
| `run-casan4-harness-tests.sh` | 35 | Baseline happy-path + Level-5 evidence (unchanged) |
|
||
| `adversarial-harness-tests.sh` | 44 | Original adversarial battery (unchanged) |
|
||
| `phase1-track-a-tests.sh` | 25 | **New** — Track A hardening |
|
||
| `phase2-track-c-tests.sh` | 29 | **New** — Track C-MVP |
|
||
| `phase3-evidence-pack-tests.sh` | 7 | **New** — Evidence Pack MVP |
|
||
| **Total** | **140** | Baseline 79 preserved; +61 new hardening checks |
|
||
|
||
Run order note: `run-casan4-harness-tests.sh` does `rm -rf .specify/logs`, so run it
|
||
**first** and never concurrently with the other suites.
|
||
|
||
## 3. What is NOT done (still planned — do not claim as production-ready)
|
||
|
||
| Area | Status | Plan ref |
|
||
|---|---|---|
|
||
| H4 multilingual detection (VI/JA injection block-patterns) | [planned] | Plan-07 B1 (V2) |
|
||
| Classifier-inject / split-injection resistance | [planned] | Plan-07 B2 (V5,V6) |
|
||
| HSM/rotation key management, TOCTOU re-verify | [planned] | Plan-07 B3 (V10,V11) |
|
||
| Model-digest pinning, sliding-window circuit breaker | [planned] | Plan-07 B4 (V15,V16) |
|
||
| **True runtime isolation** (container `--network=none --read-only --pids-limit`, nsjail) | [planned] — C6 is a static+ulimit scaffold only | Plan-07 C6 (V22) |
|
||
| Policy versioning + mandatory reviewer approval + rollback | [planned] | Plan-07 C4 (V20) |
|
||
| External append-only (WORM) audit | [planned] | Plan-07 C5 (V21) |
|
||
| Incident severity/kill-switch/runbook | [planned] | Plan-07 C7 (V23) |
|
||
| Telemetry signing under Vault KMS path | [partial] — local-key path done; KMS path signs unsigned | Plan-07 A4 follow-up |
|
||
| Live CVE/OSV scanning wired in | [partial] — availability detected; local denylist authoritative offline | Plan-07 C2 follow-up |
|
||
|
||
## 4. Honest claim
|
||
|
||
Track A + Track C-MVP + Evidence Pack raise H4/H5/H6 from "PoC/demo (~3.0/5)" toward
|
||
**early internal-production hardening (~3.8–4.0)**, with executable adversarial tests
|
||
for every control. This is **not** full production readiness: serious production
|
||
still needs Track B, Track C-Governance/Ops, true sandbox isolation, live IdP/WORM,
|
||
and hosted telemetry — enumerated above and in `CASAN_PLAN_07_PRODUCTION_HARDENING.md`.
|