# CASAN Refined Re-Assessment - AINative_OKR_CASAN5

Generated: 2026-06-28

## Summary

This package merges the stable CASAN4 submission with the strongest additions from `AINative_OKR_Claude_GHCP`, then fixes the regressions found during review.

Positioning:

- CASAN Level 4: achieved.
- Level 5: stronger local demonstration, not full enterprise production.
- Evidence score: 88.7 / 100.

## Scorecard

| Harness | CASAN4 | Refined | Delta | Reason |
|---|---:|---:|---:|---|
| H1 Context | 90 | 91 | +1 | Context schema now exposes Level 5 paths for tool registry, drift, rollback, and AgentOps evidence. |
| H2 Tool | 84 | 88 | +4 | Tool registry expanded, side-effect idempotency tested, per-call audit added, cache hit no longer bypasses CASAN trace generation. |
| H3 Evaluation | 85 | 86 | +1 | Hallucination checklist added for review gates. |
| H4 Security | 88 | 90 | +2 | Jailbreak and private key tests added; private key leakage bug fixed with explicit regex and `grep --`. |
| H5 Governance | 87 | 88 | +1 | Risk policy and registry auto-update guidance added; audit chain remains verified. |
| H6 AgentOps | 89 | 90 | +1 | JSON alert format and tool-call audit evidence added. |
| H7 Orchestration | 87 | 88 | +1 | Refined wrapper preserves evidence on cache hit and keeps rollback/drift/fallback evidence. |
| **Average** | **87.1** | **88.7** | **+1.6** | Stronger Level 5 evidence path with verified regression fixes. |

## Verified Improvements

The test harness now checks:

- Prompt injection block.
- PII masking.
- Jailbreak block.
- Private key material block.
- High-risk deny-by-default.
- Explicit approval path.
- Tool-call audit log.
- Cache hit still records fresh CASAN traces.
- Audit hash-chain verification.
- Drift, fallback, tool registry, rollback, KPI, signed policy, provider telemetry, harness reuse, and central dashboard evidence.

## Regressions Fixed from AINative_OKR_Claude_GHCP

| Issue | Fix |
|---|---|
| Bash scripts were not executable in the submitted folder | New folder is copied from CASAN4 baseline with executable bits preserved. |
| Private key regex caused `grep` option parsing and leaked private key text | `security-check.sh` now uses `grep --` and explicit private key, DB connection string, and AWS key regexes. |
| Idempotency cache bypassed H4/H5/H6 on cache hit | `casan-harness.sh` now runs H4 input and H5 before cache use, records H6 cache-copy metrics, and runs H4 output. |
| Tool registry did not write central tool-call audit | `tool-registry-gate.sh` now writes `.specify/logs/audit/tool-calls.jsonl`. |
| Submission package still referenced CASAN4 paths | Paths were updated to `AINative_OKR_CASAN5`. |

## Claim Boundary

This package is stronger than CASAN4 for hackathon evidence. It should still be described as:

> Level 4 achieved; Level 5 demonstrated locally with verifiable controls.

Do not claim full production Level 5 until these are connected to enterprise services:

- IdP-backed approval and separation of duties.
- WORM or external signed log storage.
- Live provider telemetry APIs.
- Hosted dashboard with real-time ingestion.
- Shared harness package reused across independent production projects.