CASAN/.github/agents/protocols/casan-harness-protocol.md

CASAN Level 4 Harness Protocol

This protocol is mandatory for the Boss orchestrator and every delegated agent step.

Objective

Raise the SDD Speckit OKR pipeline from CASAN Level 3 to Level 4 by making Security, Governance, and AgentOps enforced runtime controls, not documentation-only artifacts.

Harness Controls

Harness	Runtime control	Script	Evidence
H4 Security	Prompt injection block, PII masking, secret redaction, output filtering	packages/casan-harness/scripts/bash/security-check.sh	.specify/logs/trace/security-*.json, .specify/logs/audit/security.jsonl
H5 Governance	Risk scoring, non-interactive approval policy, append-only hash-chain audit	packages/casan-harness/scripts/bash/governance-check.sh	.specify/logs/trace/governance-*.json, .specify/logs/audit/audit.jsonl
H6 AgentOps	Latency, retry, token, cost, status, alert tracking	packages/casan-harness/scripts/bash/agent-metrics.sh	.specify/logs/trace/agentops-*.json, .specify/logs/cost/metrics.jsonl, .specify/agentops/alerts.log
H2 Tool Registry	Side-effect registry, idempotency key, per-call audit	packages/casan-harness/scripts/bash/tool-registry-gate.sh	.specify/logs/level5/tool-registry.jsonl, .specify/logs/audit/tool-calls.jsonl

Mandatory Gate Pattern

Before every delegated step:

packages/casan-harness/scripts/bash/security-check.sh "$STEP_INPUT" "$SAFE_INPUT" input
packages/casan-harness/scripts/bash/governance-check.sh "$SAFE_INPUT" "$APPROVED_INPUT" "$ACTION_NAME"

Around every execution or agent/tool invocation:

CASAN_AGENT_NAME="<agent>" CASAN_STEP_NAME="<step>" \
  packages/casan-harness/scripts/bash/agent-metrics.sh "$APPROVED_INPUT" "$RAW_OUTPUT" -- <real command>

After every generated artifact or agent response:

packages/casan-harness/scripts/bash/security-check.sh "$RAW_OUTPUT" "$FINAL_OUTPUT" output

When a step can be represented as a single command, Boss MAY use the wrapper:

CASAN_AGENT_NAME="<agent>" CASAN_STEP_NAME="<step>" \
  packages/casan-harness/scripts/bash/casan-harness.sh "$STEP_INPUT" "$FINAL_OUTPUT" "$ACTION_NAME" -- <real command>

Cache hits MUST NOT bypass CASAN evidence. A cached wrapper run still has to produce fresh H4 input, H5 governance, H6 metrics, and H4 output traces for the current execution.

Governance Defaults

• Low risk: auto-approved and audited.
• Medium risk: auto-approved with audit marker.
• High risk: denied unless both are present:
  • CASAN_APPROVAL_DECISION=approve
  • CASAN_APPROVER=<responsible architect or reviewer>

The pipeline MUST NOT use interactive read prompts. All approval decisions must be deterministic and auditable.

Required Pipeline Context Fields

After each step, Boss updates pipeline-context.yaml with:

casan:
  h4-security:
    status: PASS | BLOCKED
    trace: .specify/logs/trace/security-<id>.json
  h5-governance:
    decision: approved | denied
    risk-level: low | medium | high
    trace: .specify/logs/trace/governance-<id>.json
    audit-log: .specify/logs/audit/audit.jsonl
  h6-agentops:
    status: success | failed
    trace: .specify/logs/trace/agentops-<id>.json
    metrics-log: .specify/logs/cost/metrics.jsonl

Level 4 Hard Gates

The pipeline is not CASAN Level 4 compliant unless all conditions are true:

1. Every step has at least one H4 trace and one H6 trace.
2. Every side-effecting action has an H5 governance decision.
3. High-risk actions are denied by default or approved with approver identity.
4. Audit log is append-only JSONL with previous_record_hash and record_hash.
5. Security tests prove prompt injection is blocked and PII/secrets are masked or redacted.
6. Metrics tests prove latency, token, cost, retry, status, and alert fields are recorded.
7. Final docs/output/casan/casan-level4-assessment.md links each claim to concrete evidence files.
8. Tool registry tests prove side-effecting tools require idempotency and write per-call audit evidence.

CASAN Higher-Level Criteria

Use these criteria when extending beyond Level 4:

Level	Extra criteria beyond current controls
Level 4 Automated	All H4/H5/H6 gates enforced automatically with audit and recovery evidence.
Level 5 Native	Multi-agent workflows self-optimize using drift detection, model fallback, business KPI feedback, reusable enterprise harness registry, and cross-project governance.

Level 5 Self-Enforcement Additions

Boss orchestrator also enforces:

1. Self-Security Audit: Every agent reads protocols/agent-self-security-rules.md before output is accepted.
2. Self-Governance Audit: Every agent reads protocols/governance-risk-policy.md before high-risk action planning.
3. Hallucination Checklist: Every review agent reads protocols/hallucination-prevention-checklist.md during review gates.
4. Tool Registry Policy: Every side-effecting tool is checked against protocols/tool-registry-policy.md and packages/casan-harness/level5/tool-registry.yaml.