thanhnvandClaude Opus 4.8
18997caf49
Update agent protocols (.claude/agents, .github/agents), guides, runbooks and READMEs
to the post-restructure layout: .specify/{scripts,tests,security,config,templates,
governance,memory,level5-config} -> packages/casan-harness/...; docs/input +
golden-runs + traceability-map -> apps/okr/domain/...; drop AINative_OKR_CASAN5/ prefix.
Runtime-state paths (.specify/logs, .specify/agentops, .specify/level5/central-governance)
kept as-is. Historical evidence under docs/output/ left untouched (immutable run records).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
109 lines
5.0 KiB
Markdown
109 lines
5.0 KiB
Markdown
# CASAN Level 4 Harness Protocol
|
|
|
|
This protocol is mandatory for the Boss orchestrator and every delegated agent step.
|
|
|
|
## Objective
|
|
|
|
Raise the SDD Speckit OKR pipeline from CASAN Level 3 to Level 4 by making Security, Governance, and AgentOps enforced runtime controls, not documentation-only artifacts.
|
|
|
|
## Harness Controls
|
|
|
|
| Harness | Runtime control | Script | Evidence |
|
|
|---|---|---|---|
|
|
| H4 Security | Prompt injection block, PII masking, secret redaction, output filtering | `packages/casan-harness/scripts/bash/security-check.sh` | `.specify/logs/trace/security-*.json`, `.specify/logs/audit/security.jsonl` |
|
|
| H5 Governance | Risk scoring, non-interactive approval policy, append-only hash-chain audit | `packages/casan-harness/scripts/bash/governance-check.sh` | `.specify/logs/trace/governance-*.json`, `.specify/logs/audit/audit.jsonl` |
|
|
| H6 AgentOps | Latency, retry, token, cost, status, alert tracking | `packages/casan-harness/scripts/bash/agent-metrics.sh` | `.specify/logs/trace/agentops-*.json`, `.specify/logs/cost/metrics.jsonl`, `.specify/agentops/alerts.log` |
|
|
| H2 Tool Registry | Side-effect registry, idempotency key, per-call audit | `packages/casan-harness/scripts/bash/tool-registry-gate.sh` | `.specify/logs/level5/tool-registry.jsonl`, `.specify/logs/audit/tool-calls.jsonl` |
|
|
|
|
## Mandatory Gate Pattern
|
|
|
|
Before every delegated step:
|
|
|
|
```bash
|
|
packages/casan-harness/scripts/bash/security-check.sh "$STEP_INPUT" "$SAFE_INPUT" input
|
|
packages/casan-harness/scripts/bash/governance-check.sh "$SAFE_INPUT" "$APPROVED_INPUT" "$ACTION_NAME"
|
|
```
|
|
|
|
Around every execution or agent/tool invocation:
|
|
|
|
```bash
|
|
CASAN_AGENT_NAME="<agent>" CASAN_STEP_NAME="<step>" \
|
|
packages/casan-harness/scripts/bash/agent-metrics.sh "$APPROVED_INPUT" "$RAW_OUTPUT" -- <real command>
|
|
```
|
|
|
|
After every generated artifact or agent response:
|
|
|
|
```bash
|
|
packages/casan-harness/scripts/bash/security-check.sh "$RAW_OUTPUT" "$FINAL_OUTPUT" output
|
|
```
|
|
|
|
When a step can be represented as a single command, Boss MAY use the wrapper:
|
|
|
|
```bash
|
|
CASAN_AGENT_NAME="<agent>" CASAN_STEP_NAME="<step>" \
|
|
packages/casan-harness/scripts/bash/casan-harness.sh "$STEP_INPUT" "$FINAL_OUTPUT" "$ACTION_NAME" -- <real command>
|
|
```
|
|
|
|
Cache hits MUST NOT bypass CASAN evidence. A cached wrapper run still has to produce fresh H4 input, H5 governance, H6 metrics, and H4 output traces for the current execution.
|
|
|
|
## Governance Defaults
|
|
|
|
- Low risk: auto-approved and audited.
|
|
- Medium risk: auto-approved with audit marker.
|
|
- High risk: denied unless both are present:
|
|
- `CASAN_APPROVAL_DECISION=approve`
|
|
- `CASAN_APPROVER=<responsible architect or reviewer>`
|
|
|
|
The pipeline MUST NOT use interactive `read` prompts. All approval decisions must be deterministic and auditable.
|
|
|
|
## Required Pipeline Context Fields
|
|
|
|
After each step, Boss updates `pipeline-context.yaml` with:
|
|
|
|
```yaml
|
|
casan:
|
|
h4-security:
|
|
status: PASS | BLOCKED
|
|
trace: .specify/logs/trace/security-<id>.json
|
|
h5-governance:
|
|
decision: approved | denied
|
|
risk-level: low | medium | high
|
|
trace: .specify/logs/trace/governance-<id>.json
|
|
audit-log: .specify/logs/audit/audit.jsonl
|
|
h6-agentops:
|
|
status: success | failed
|
|
trace: .specify/logs/trace/agentops-<id>.json
|
|
metrics-log: .specify/logs/cost/metrics.jsonl
|
|
```
|
|
|
|
## Level 4 Hard Gates
|
|
|
|
The pipeline is not CASAN Level 4 compliant unless all conditions are true:
|
|
|
|
1. Every step has at least one H4 trace and one H6 trace.
|
|
2. Every side-effecting action has an H5 governance decision.
|
|
3. High-risk actions are denied by default or approved with approver identity.
|
|
4. Audit log is append-only JSONL with `previous_record_hash` and `record_hash`.
|
|
5. Security tests prove prompt injection is blocked and PII/secrets are masked or redacted.
|
|
6. Metrics tests prove latency, token, cost, retry, status, and alert fields are recorded.
|
|
7. Final `docs/output/casan/casan-level4-assessment.md` links each claim to concrete evidence files.
|
|
8. Tool registry tests prove side-effecting tools require idempotency and write per-call audit evidence.
|
|
|
|
## CASAN Higher-Level Criteria
|
|
|
|
Use these criteria when extending beyond Level 4:
|
|
|
|
| Level | Extra criteria beyond current controls |
|
|
|---|---|
|
|
| Level 4 Automated | All H4/H5/H6 gates enforced automatically with audit and recovery evidence. |
|
|
| Level 5 Native | Multi-agent workflows self-optimize using drift detection, model fallback, business KPI feedback, reusable enterprise harness registry, and cross-project governance. |
|
|
|
|
## Level 5 Self-Enforcement Additions
|
|
|
|
Boss orchestrator also enforces:
|
|
|
|
1. **Self-Security Audit:** Every agent reads `protocols/agent-self-security-rules.md` before output is accepted.
|
|
2. **Self-Governance Audit:** Every agent reads `protocols/governance-risk-policy.md` before high-risk action planning.
|
|
3. **Hallucination Checklist:** Every review agent reads `protocols/hallucination-prevention-checklist.md` during review gates.
|
|
4. **Tool Registry Policy:** Every side-effecting tool is checked against `protocols/tool-registry-policy.md` and `packages/casan-harness/level5/tool-registry.yaml`.
|