update first - 84

2026-06-30 02:21:39 +09:00
commit 07ac1bdcdd
561 changed files with 88164 additions and 0 deletions
@@ -0,0 +1,66 @@
+# CASAN Refined Re-Assessment - AINative_OKR_CASAN5
+
+Generated: 2026-06-28
+
+## Summary
+
+This package merges the stable CASAN4 submission with the strongest additions from `AINative_OKR_Claude_GHCP`, then fixes the regressions found during review.
+
+Positioning:
+
+- CASAN Level 4: achieved.
+- Level 5: stronger local demonstration, not full enterprise production.
+- Evidence score: 88.7 / 100.
+
+## Scorecard
+
+| Harness | CASAN4 | Refined | Delta | Reason |
+|---|---:|---:|---:|---|
+| H1 Context | 90 | 91 | +1 | Context schema now exposes Level 5 paths for tool registry, drift, rollback, and AgentOps evidence. |
+| H2 Tool | 84 | 88 | +4 | Tool registry expanded, side-effect idempotency tested, per-call audit added, cache hit no longer bypasses CASAN trace generation. |
+| H3 Evaluation | 85 | 86 | +1 | Hallucination checklist added for review gates. |
+| H4 Security | 88 | 90 | +2 | Jailbreak and private key tests added; private key leakage bug fixed with explicit regex and `grep --`. |
+| H5 Governance | 87 | 88 | +1 | Risk policy and registry auto-update guidance added; audit chain remains verified. |
+| H6 AgentOps | 89 | 90 | +1 | JSON alert format and tool-call audit evidence added. |
+| H7 Orchestration | 87 | 88 | +1 | Refined wrapper preserves evidence on cache hit and keeps rollback/drift/fallback evidence. |
+| **Average** | **87.1** | **88.7** | **+1.6** | Stronger Level 5 evidence path with verified regression fixes. |
+
+## Verified Improvements
+
+The test harness now checks:
+
+- Prompt injection block.
+- PII masking.
+- Jailbreak block.
+- Private key material block.
+- High-risk deny-by-default.
+- Explicit approval path.
+- Tool-call audit log.
+- Cache hit still records fresh CASAN traces.
+- Audit hash-chain verification.
+- Drift, fallback, tool registry, rollback, KPI, signed policy, provider telemetry, harness reuse, and central dashboard evidence.
+
+## Regressions Fixed from AINative_OKR_Claude_GHCP
+
+| Issue | Fix |
+|---|---|
+| Bash scripts were not executable in the submitted folder | New folder is copied from CASAN4 baseline with executable bits preserved. |
+| Private key regex caused `grep` option parsing and leaked private key text | `security-check.sh` now uses `grep --` and explicit private key, DB connection string, and AWS key regexes. |
+| Idempotency cache bypassed H4/H5/H6 on cache hit | `casan-harness.sh` now runs H4 input and H5 before cache use, records H6 cache-copy metrics, and runs H4 output. |
+| Tool registry did not write central tool-call audit | `tool-registry-gate.sh` now writes `.specify/logs/audit/tool-calls.jsonl`. |
+| Submission package still referenced CASAN4 paths | Paths were updated to `AINative_OKR_CASAN5`. |
+
+## Claim Boundary
+
+This package is stronger than CASAN4 for hackathon evidence. It should still be described as:
+
+> Level 4 achieved; Level 5 demonstrated locally with verifiable controls.
+
+Do not claim full production Level 5 until these are connected to enterprise services:
+
+- IdP-backed approval and separation of duties.
+- WORM or external signed log storage.
+- Live provider telemetry APIs.
+- Hosted dashboard with real-time ingestion.
+- Shared harness package reused across independent production projects.
+