Files

T

thanhnvandClaude Opus 4.8 4cc78f74eb docs: honest hardening status + claim boundary (Plan-07 A/C-MVP, Plan-09)

- CASAN_HARDENING_STATUS.md: canonical implemented/scaffold/planned record with
  per-control test mapping and test inventory (baseline 79 + 61 new = 140 checks).
- README claim boundary: separate implemented+tested controls from planned;
  explicitly does NOT claim full production-readiness (Track B, C-Gov/Ops, true
  sandbox isolation, IdP/WORM still planned).
- INDEX status table: Plan-07 Track A + C-MVP done, Plan-09 MVP done.
- video guide: core demo battery counts unchanged (hardening lives in separate
  suites); added deep-dive commands + 3 Q&A rows + sandbox honesty note.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-07-03 23:07:31 +09:00

3.9 KiB

Raw Blame History

HarnessAthon Submission Package

One-Line Positioning

We upgraded the SDD Speckit OKR pipeline from the CASAN Level 3→4 transition to CASAN Level 4 — proven by attack: H4/H5/H6 (former GAPs 20/25/30) now defeat ~23 live adversarial vectors with on-screen exit codes. Level 5 is stated as roadmap (IdP, WORM log storage, provider-telemetry API, hosted dashboard) — not claimed as achieved.

Open These First

presentation/HarnessAthon_CASAN_Level5_Demo.pptx
docs/00_task_breakdown.md
docs/01_submission_checklist.md
docs/02_pitch_script.md
docs/03_judge_qna.md
docs/04_evidence_map.md
video/01_video_recording_guide.md
ai_context/AI_README.md

Main Evidence

Evidence	Source
Harness test report	`../AINative_OKR_CASAN5/docs/output/casan/evidence/harness-test-report.md`
CASAN assessment	`../AINative_OKR_CASAN5/docs/output/casan/casan-level4-assessment.md`
Before/after scorecard	`../AINative_OKR_CASAN5/docs/output/casan/before-after-scorecard.md`
Demo pipeline context	`../AINative_OKR_CASAN5/docs/output/output_logs/casan-demo/pipeline-context.yaml`
Central dashboard	`../AINative_OKR_CASAN5/docs/output/casan/central-agentops-dashboard.html`
Final zip	`../AINative_OKR_CASAN5.zip`

Folder Purpose

Folder	Purpose
`docs/`	Checklist, pitch script, judge Q&A, evidence map, AI-optimized structure notes
`presentation/`	PPT deck and deterministic generation source
`video/`	Screen-recording guide and narration outline
`evidence/`	Evidence index pointing to canonical generated evidence
`ai_context/`	One-page context for AI/teammate review

Verification Command

cd ../AINative_OKR_CASAN5
bash .specify/tests/run-casan4-harness-tests.sh

Expected: all PASS, including H4/H5/H6 and Level 5 evidence checks.

Claim Boundary

CASAN Level 4: achieved.
CASAN Level 5: demonstrated locally with signed policy, provider telemetry import, shared harness registry, fallback, drift detection, rollback, KPI feedback, and central dashboard.
Enterprise production Level 5 still needs live IdP, WORM storage, live provider API integration, and hosted dashboard.

Production hardening — implemented vs planned

Following Plan-07/Plan-09, the following hardening is implemented and tested (each control has an executable adversarial test that fails if the control is removed). Full status: casan-next-plans/CASAN_HARDENING_STATUS.md.

Implemented + tested (Plan-07 Track A): H4 strict semantic fail-closed (CASAN_SECURITY_STRICT), unicode/encoding normalization (homoglyph, zero-width, fullwidth, base64/hex), tool-output injection scan, H5 telemetry-integrity signing, H6 absolute + cumulative + cold-start cost controls, benign/false-positive budget gate (FP ≤ 3%, adversarial block ≥ 95%, CRITICAL = 100%).
Implemented + tested (Plan-07 Track C-MVP): tool-authorization / action gating, supply-chain gate, data-exfiltration guard.
Scaffold + tested (Track C-MVP): runtime sandbox — static policy + ulimit backstops. Not kernel isolation; production needs container --network=none --read-only --pids-limit / nsjail.
Implemented + tested (Plan-09 MVP): Evidence Pack — casan pack / casan verify-pack (tamper-evident manifest + signed head + certified-run gate).
Planned (NOT done — do not claim as production-ready): multilingual H4, classifier/split-injection resistance, HSM key management, policy-approval workflow, external WORM audit, incident kill-switch, true sandbox isolation.

Test totals: baseline 79 (run-casan4 35 + adversarial 44) preserved, +61 new hardening checks (Track A 25, Track C-MVP 29, Evidence Pack 7) = 140 — see CASAN_HARDENING_STATUS.md. Because these live in separate suites, the demo attack battery counts in video/01_video_recording_guide.md are unchanged.

3.9 KiB Raw Blame History