feat: add governed chat agent selection

This commit is contained in:
thanhnv
2026-07-08 22:19:51 +09:00
parent 06d8d31b16
commit 004afa73c9
18 changed files with 677 additions and 39 deletions
+1 -1
View File
@@ -61,7 +61,7 @@
| Future B1–B6 | 💤 vision | `CASAN_PLAN_FUTURE_PHASES.md` — approval workflow nâng cao · state machine · model benchmark · governed memory · auto-remediation · platform KPI. |
| **17 Loop Engineering** | � T1–T6 done+test (offline) | **Agentic Loop Governance** — đủ 5 primitive + orchestrator (97/0 WSL, nối CI). T1 **Governor** (`loop-governor.py`; deny-by-default, no/corrupt policy→strict/HALT, on_exceed halt/escalate) 15/0; T2 **Convergence** (`loop-convergence.py`; repeat/thrash→OSCILLATING, flat→STALLED, fail-closed) 15/0; T3 **Verify Contract** (`loop-gate.py`; H4→DENY, unmet→FAIL, correction bounded→ESCALATE, no self-declared DONE) 20/0; T4 **Trace/Replay** (`loop-trace.py`; append-only hash-linked, edited→BREAK, tampered artifact→replay DRIFT) 16/0; T5 **Meta-loop** (`loop-metaloop.py`; propose≠apply, SoD, loosen>org_ceiling refused, apply qua governed CP store→đổi thật ceiling + rollback) 15/0; T6 **Orchestrator** (`loop-run.sh`; gate→governor→convergence→trace/turn, secure-by-default opt-out, nén giữa vòng) 16/0. State qua `CASAN_LOOP_STATE_ROOT` (repo `.specify/state` sạch). **Còn (infra):** T4 KMS-anchor head (A7 Vault), T6 widget Command Center (17.22, C5), live H3-judge. Chi tiết: `CASAN_PLAN_17_LOOP_ENGINEERING.md`. |
| **16 Security audit remediation** | � P0/P1/P2 phần lớn done+test | **Remediation đã thực thi:** 28 SEC suite (151/0 WSL, nối `ci-harness-gate.sh`). Done: SEC-01..10, 12, 13, **14** (model-digest bỏ env-override ở prod/strict), 15, 16..21, **22** (trusted-time JWT `exp` ARCH-06 + tag proposal nguồn-không-tin ARCH-08), **26** (stored/second-order injection scan), 27..30, **23 Phase 1–5 offline** (multi-tenant: tenant-store+guard · per-tenant CP/audit/telemetry · RBAC data-boundary · tenant kill-switch/quota · ký registry · crypt at-rest per-tenant), **24 offline** (image digest-pin + ký workflow), **25 offline** (artifact attestation tested==deployed); **SEC-11 gộp vào SEC-17** (`CASAN_PROFILE=prod` enforce-by-default). **Còn 📋 planned (hạ tầng/process):** SEC-22 ARCH-10 (attestation ngoài) · SEC-23 23.11 (crypt qua Vault Transit) · **SEC-24 còn** (live CVE/OSV + scan image thật — offline image-pin/ký-workflow đã done) · **SEC-25 còn** (signed-commit enrollment + SLSA chain — offline artifact-attestation đã done). Chi tiết: `CASAN_PLAN_16` §0a/§2d. |
| **18 Chat Console** | 🟡 **MVP-0 + MVP-1 done+test** · target arch còn planned | **Governed Chat Console** (cắt lát MVP chống lan man). **MVP-0 Ask CASAN read-only DONE**: `prompt-mode-router.py`, `chat-readonly.py`, `chat-session.schema.json`, H4 input/output scan, H5 chat audit hash-chain, H6 token telemetry, answer kèm evidence sources. **MVP-1 Operator DONE**: `operator-actions.yaml`, `chat-operator.py`, `chat-turn.py`, registered actions `run tests`/`build pack`/`verify pack`, all through `action-gate`, no free-command, action artifacts with provenance, Control Panel `POST /api/v1/chat/ask`, `GET /api/v1/chat/actions`, `/chat` UI. Test: chat suites **24/0** (`phase-chat-prompt-router` 8/0, `phase-chat-readonly` 5/0, `phase-chat-session-audit` 3/0, `phase-chat-operator` 8/0), Control Panel **24/0** + build xanh. Sau: MVP-2 chat-as-loop + agent/skill (**cần Plan-17+14**) → MVP-3 multi-tenant (**cần SEC-23**). |
| **18 Chat Console** | 🟡 **MVP-0 + MVP-1 done+test; MVP-2 Track 4 foundation done** · target arch còn planned | **Governed Chat Console** (cắt lát MVP chống lan man). **MVP-0 Ask CASAN read-only DONE**: `prompt-mode-router.py`, `chat-readonly.py`, `chat-session.schema.json`, H4 input/output scan, H5 chat audit hash-chain, H6 token telemetry, answer kèm evidence sources. **MVP-1 Operator DONE**: `operator-actions.yaml`, `chat-operator.py`, `chat-turn.py`, registered actions `run tests`/`build pack`/`verify pack`, all through `action-gate`, no free-command, action artifacts with provenance. **MVP-2 Track 4 foundation DONE**: `agent-registry.yaml`, `chat-agent-resolver.py`, `chat:select_agent` RBAC, delegation hold, tool allowlist BLOCK, Control Panel `GET /api/v1/chat/agents` + `/chat` agent/skill/delegation picker. Test: chat suites **32/0** (`phase-chat-prompt-router` 8/0, `phase-chat-readonly` 5/0, `phase-chat-session-audit` 3/0, `phase-chat-operator` 8/0, `phase-chat-agent-select` 8/0), Control Panel **25/0** + build xanh. Next: MVP-2 Track 5/6 chat-as-loop + streaming + replay → MVP-3 multi-tenant. |
---
## Trần điểm & điều kiện lên "Strong (81+)"
+1 -1
View File
@@ -35,7 +35,7 @@
| 15 | `CASAN_PLAN_15_RESPONSIBLE_AI_DATA_GOV.md` | Responsible AI & Data Governance (FPT §14.3–14.4) | 📋 |
| 16 | `CASAN_PLAN_16_SECURITY_AUDIT_REMEDIATION.md` | Security audit core harness + kế hoạch vá (tamper-evidence/injection/fail-open) | � P0/P1/P2 done |
| 17 | `CASAN_PLAN_17_LOOP_ENGINEERING.md` | Loop Engineering / Agentic Loop Governance (budget governor, convergence, verify-contract, loop trace/replay, meta-loop) | 📋 |
| 18 | `CASAN_PLAN_18_CHAT_CONSOLE.md` | Governed Chat Console (target arch; **MVP-0 Ask CASAN + MVP-1 Operator done** → chat-as-loop → multi-tenant) | 🟡 MVP-0/1 done |
| 18 | `CASAN_PLAN_18_CHAT_CONSOLE.md` | Governed Chat Console (target arch; **MVP-0 Ask CASAN + MVP-1 Operator done; MVP-2 Track 4 foundation done** → chat-as-loop/streaming/replay → multi-tenant) | 🟡 MVP-0/1 + Track 4 done |
| Future | `CASAN_PLAN_FUTURE_PHASES.md` | Approval workflow, state machine, benchmark, memory, remediation, KPI | 💤 vision |
> **Không có plan số 11:** số 11 được bỏ trống có chủ ý — nhánh eval/traceability đã
+23 -12
View File
@@ -1,12 +1,15 @@
# KẾ HOẠCH 18 — Governed Chat Console (Chat-as-Loop qua Control Plane)
> Status 2026-07-08: **🟡 MVP-0 + MVP-1 done+test — target architecture còn planned.**
> Status 2026-07-08: **🟡 MVP-0 + MVP-1 done+test; MVP-2 Track 4 foundation done — target architecture còn planned.**
> Đã implement **Ask CASAN — Read-only Evidence Assistant** qua harness + Control
> Panel (`/api/v1/chat/ask`, `/chat` UI): Prompt Router deterministic
> `READ_ONLY/OPERATOR/BLOCK/NOT_SUPPORTED`, context whitelist, H4 scan in/out, H5
> chat audit hash-chain, H6 token telemetry, answer kèm evidence sources. **MVP-1
> Operator** đã có registered actions `run tests` / `build pack` / `verify pack`
> qua `action-gate`, không free-command, kết quả có artifact provenance. Mục tiêu
> qua `action-gate`, không free-command, kết quả có artifact provenance. **MVP-2
> Track 4 foundation** đã có `agent-registry.yaml`, agent/skill resolver, RBAC
> gate chọn agent, delegation hold, tool allowlist block, Control Panel agent picker.
> Mục tiêu
> tổng thể vẫn là thêm **cửa sổ chat** vào Control Plane (Plan-13) như một
> **bề mặt tương tác của core harness** — mỗi lượt chat là **một loop-run được
> governance** (Plan-17), đi qua đúng H1→H7, không có đường vòng. Đây **không** phải
@@ -250,10 +253,10 @@ flowchart TD
### Track 4 — Agent/Skill selection governance `[MVP-2]`
| Task | Việc | File | Verify (WSL) |
|---|---|---|---|
| 18.4.1 | `agent-registry.yaml` versioned (tool_allowlist, max_delegation, roles_allowed, model_role) + resolver bind theo turn | mới `config/agent-registry.yaml` | chọn agent → allowlist đúng |
| 18.4.2 | RBAC gate chọn agent (14): role không đủ → DENY; vượt `max_delegation` → security-sensitive → approval | nối `rbac-check.py` | role thấp chọn agent tự chủ → DENY |
| 18.4.3 | Tool-call runtime chỉ tool **trong allowlist**; ngoài → `action-gate` BLOCK | nối `action-gate.sh` | tool ngoài allowlist → BLOCK |
| 18.4.4 | Skill code-gen chạy trong allowlist + H4 artifact-scan (Plan-02) trước output | nối `artifact-scan.sh` | skill sinh mã có injection → BLOCK |
| 18.4.1 | ✅ `agent-registry.yaml` versioned (tool_allowlist, max_delegation, roles_allowed, model_role) + resolver bind theo turn | `packages/casan-harness/config/agent-registry.yaml` + `chat-agent-resolver.py` | chọn agent → allowlist đúng |
| 18.4.2 | ✅ RBAC gate chọn agent (14): role không đủ → DENY; vượt `max_delegation` → security-sensitive → approval hold | nối `rbac-check.py` (`chat:select_agent`) | role thấp chọn agent tự chủ → DENY |
| 18.4.3 | ✅ Tool-call runtime chỉ tool **trong allowlist**; ngoài → BLOCK trước runtime | `chat-agent-resolver.py` | tool ngoài allowlist → BLOCK |
| 18.4.4 | ⏳ Skill code-gen chạy trong allowlist + H4 artifact-scan (Plan-02) trước output | nối `artifact-scan.sh` | skill sinh mã có injection → BLOCK |
### Track 5 — Chat-as-loop pipeline `[MVP-2]`
| Task | Việc | File | Verify (WSL) |
@@ -274,7 +277,7 @@ flowchart TD
|---|---|---|---|
| 18.7.1 | ✅ Chat API (NestJS) **bọc** harness (MVP-0/1: read-only + operator; single-source, không verdict riêng) | `packages/casan-control-panel/backend/src/chat/*` | API gọi `chat-turn.py`; `console:test` xanh |
| 18.7.2 | ✅ Chat panel React + hiển thị **evidence sources**, registered actions, action-gate + badge `mode/risk` | `packages/casan-control-panel/frontend/src/pages/Chat.tsx` | `console:build` xanh |
| 18.7.3 | (MVP-2) agent/skill picker theo RBAC + badge `UNCERTIFIED/CERTIFIED` + nút **loop-breaker** + delegation-level | Control Plane UI | picker ẩn agent ngoài quyền |
| 18.7.3 | 🟡 (MVP-2 partial) agent/skill picker theo RBAC + delegation-level đã có; còn badge `UNCERTIFIED/CERTIFIED` + nút **loop-breaker** khi Track 5/6 mở | Control Plane UI | picker hiện locked agent, backend DENY agent ngoài quyền |
| 18.7.4 | Fail-loud: API/telemetry chết → UI `STALE/503` (tái dùng D3 `/healthz`) | nối Plan-07 D3 | ngắt backend → UI báo stale |
### Track 8 — Replay / Evidence / Command Center widgets `[MVP-2 → MVP-3]`
@@ -313,8 +316,8 @@ flowchart TD
**File test (theo MVP):** MVP-0 — `phase-chat-prompt-router-tests.sh`,
`phase-chat-readonly-tests.sh`, `phase-chat-session-audit-tests.sh`; MVP-1 —
`phase-chat-operator-tests.sh`; MVP-2 — `phase-chat-pipeline-tests.sh`,
`phase-chat-agent-select-tests.sh`, `phase-chat-stream-hold-tests.sh`,
`phase-chat-operator-tests.sh`; MVP-2 — `phase-chat-agent-select-tests.sh`,
`phase-chat-pipeline-tests.sh`, `phase-chat-stream-hold-tests.sh`,
`phase-chat-replay-tests.sh`; MVP-3 — `phase-chat-tenant-tests.sh`. Chạy **WSL**
(deterministic, mock model; nhánh live cần Ollama SKIP-aware). Nối `ci-harness-gate.sh`;
cập nhật tổng test ở `CASAN_HARDENING_STATUS.md`.
@@ -336,6 +339,13 @@ cập nhật tổng test ở `CASAN_HARDENING_STATUS.md`.
- [x] Kết quả action ghi artifact có provenance và được hiển thị lại trong `/chat`.
- [x] Suite MVP-1 xanh và nối CI. Verify: `phase-chat-operator` 8/0, prompt-router 8/0, read-only 5/0, session-audit 3/0, `console:test` 24/0, `console:build` xanh.
**MVP-2 Track 4 foundation (Agent/Skill selection governance):**
- [x] Agent registry versioned có `tool_allowlist`, `max_delegation_level`, `roles_allowed`, `model_role`.
- [x] Agent resolver bind theo turn, ghi audit hash-chain, fail-closed khi registry lỗi.
- [x] RBAC gate `chat:select_agent`; role thấp chọn agent tự chủ → DENY; cross-tenant → DENY.
- [x] Delegation vượt agent max → `REQUIRES_APPROVAL`; tool ngoài allowlist → BLOCK trước runtime.
- [x] Control Panel `GET /api/v1/chat/agents`, `/chat` agent/skill/delegation picker, response có `agent_binding`. Verify: `phase-chat-agent-select` 8/0, RBAC 12/0, SEC-23 RBAC 6/0, `console:test` 25/0, `console:build` xanh.
**Full (target architecture) — DoD tổng:**
- [ ] Turn chạy **đúng như một loop-run Plan-17** (không định nghĩa vòng lặp riêng).
- [ ] **Không đường vòng:** test chứng minh bỏ bất kỳ gate nào (preflight/verify) → FAIL.
@@ -349,9 +359,10 @@ cập nhật tổng test ở `CASAN_HARDENING_STATUS.md`.
---
## 7. Ghi chú trung thực & Non-goals (không lan man)
- **[MVP-0 + MVP-1] — 🟡 done+test.** Ask CASAN read-only và Operator registered
actions đã có harness CLI + Control Panel API/UI. Các phase Chat-as-loop /
Agent-selection / multi-tenant production vẫn chưa mở.
- **[MVP-0 + MVP-1 + MVP-2 Track 4 foundation] — 🟡 done+test.** Ask CASAN read-only,
Operator registered actions, và agent/skill governance foundation đã có harness CLI
+ Control Panel API/UI. Các phase Chat-as-loop / streaming / replay / multi-tenant
production vẫn chưa mở.
- **MVP-first (chống lan man):** **MVP-0 (Ask CASAN read-only) KHÔNG phụ thuộc Plan-17/
14/SEC-23** — làm được ngay trên nền H4/H5/H6 hiện có. Chỉ **MVP-2 trở đi** (chat-as-
loop + agent) mới cần Plan-17 (T1–T3) + Plan-14 RBAC; **MVP-3** mới cần SEC-23 tenant.
+1 -1
View File
@@ -77,7 +77,7 @@ approval JWT thật ([16 SEC-07](CASAN_PLAN_16_SECURITY_AUDIT_REMEDIATION.md)),
|---|---|---|:--:|:--:|---|
| B9 | ✅ **done** — **MVP-0 Ask CASAN read-only**: Prompt Router (Track 0) + Read-only Ask CASAN (Track 1) + session/audit (Track 2) + Control Panel API/UI (Track 7). H4 in/out, H5 audit hash-chain, H6 token, evidence sources; chat suites 14/0, Control Panel 23/0 + build xanh. Real model provider binding remains Track M follow-up when a provider is enabled. | [18 §1b, Track 0/1/2/7/M](CASAN_PLAN_18_CHAT_CONSOLE.md) | L | ✅ | done |
| B10 | ✅ **done** — **MVP-1 Operator mode**: registered actions `run tests` / `build pack` / `verify pack`, no free-command, all through `action-gate`, action artifacts with provenance, Control Panel quick actions. `phase-chat-operator` 8/0; total chat suites 24/0; Control Panel 24/0 + build xanh. | [18 Track 3](CASAN_PLAN_18_CHAT_CONSOLE.md) | M | ✅ | done |
| B11 | **MVP-2** Chat-as-loop (Track 5) + Agent/Skill selection (Track 4) + streaming draft-hold (Track 6) | [18 Track 4/5/6](CASAN_PLAN_18_CHAT_CONSOLE.md) | L | 🔗 | B6, C7(RBAC), B10 |
| B11 | 🟡 **partial done** — Track 4 Agent/Skill governance foundation: `agent-registry.yaml`, `chat-agent-resolver.py`, RBAC `chat:select_agent`, delegation hold, tool allowlist BLOCK, Control Panel agent/skill/delegation picker. `phase-chat-agent-select` 8/0; Control Panel 25/0 + build xanh. **Remaining MVP-2:** Track 5 chat-as-loop + Track 6 streaming draft-hold + Track 8 replay widget. | [18 Track 4/5/6](CASAN_PLAN_18_CHAT_CONSOLE.md) | L | 🔗 | B6, C7(RBAC), B10 |
| B12 | **Widget Loop/Chat** trên Command Center (loop ticker/budget/replay drawer) | [17 (17.22)](CASAN_PLAN_17_LOOP_ENGINEERING.md) · [18 Track 8](CASAN_PLAN_18_CHAT_CONSOLE.md) | M | 🔗 | C5 ✅ baseline |
| B13 | **MVP-3** multi-tenant chat hardening (Track 9) | [18 Track 9](CASAN_PLAN_18_CHAT_CONSOLE.md) | M | 🔗 | A3(SEC-23) |
+7 -4
View File
@@ -56,7 +56,7 @@ Approval inbox / HITL:
- `POST /api/v1/approvals/decide` — approve/reject with SoD and reason; approved
settings proposals apply through `control-plane-settings.py`.
Governed Chat (Plan-18 MVP-0/1):
Governed Chat (Plan-18 MVP-0/1 + MVP-2 Track 4 foundation):
- `POST /api/v1/chat/ask` — Ask CASAN endpoint. The API only wraps harness
`chat-turn.py`; router verdicts, H4 input/output scan, action-gate decisions,
@@ -64,11 +64,14 @@ Governed Chat (Plan-18 MVP-0/1):
execution remain harness-owned.
- `GET /api/v1/chat/actions` — list registered operator actions from
`operator-actions.yaml`; no free-command execution is exposed.
- `GET /api/v1/chat/agents` — list governed agents from `agent-registry.yaml`
with role visibility; selected agent/skill/delegation are bound by harness
`chat-agent-resolver.py`.
- `GET /api/v1/chat/audit/verify` — verifies the chat audit hash chain.
- `/chat` UI shows actor/role scope, `mode/risk/decision` badges, certified answer,
evidence sources, registered operator actions, action-gate status, router details,
and audit hash. Side-effect requests outside registered actions return governed
`BLOCK` or `NOT_SUPPORTED` responses.
evidence sources, registered operator actions, agent binding, action-gate status,
router details, and audit hash. Side-effect requests outside registered actions
return governed `BLOCK` or `NOT_SUPPORTED` responses.
FinOps/SLO:
@@ -21,4 +21,9 @@ export class ChatController {
actions(@Headers() headers: Record<string, string | string[] | undefined>) {
return ok(this.svc.listActions(actorFromHeaders(headers)));
}
@Get('agents')
agents(@Headers() headers: Record<string, string | string[] | undefined>) {
return ok(this.svc.listAgents(actorFromHeaders(headers)));
}
}
@@ -7,6 +7,9 @@ import type { SettingsActor } from '../settings/settings.service.js';
export interface ChatAskInput {
message: string;
chatId?: string;
agentId?: string;
skillId?: string;
delegationLevel?: number;
}
interface CommandResult {
@@ -18,6 +21,7 @@ interface CommandResult {
const HARNESS_BIN = join(APP_ROOT, 'packages', 'casan-harness', 'scripts', 'bash');
const CHAT_CLI = join(HARNESS_BIN, 'chat-turn.py');
const OPERATOR_CLI = join(HARNESS_BIN, 'chat-operator.py');
const AGENT_CLI = join(HARNESS_BIN, 'chat-agent-resolver.py');
const RBAC_CLI = join(HARNESS_BIN, 'rbac-check.py');
function runPython(script: string, args: string[]): CommandResult {
@@ -55,17 +59,25 @@ export class ChatService {
}
this.requireRead(actor);
const res = runPython(CHAT_CLI, [
const args = [
'ask',
'--message',
input.message,
'--actor',
actor.actor,
'--role',
actor.role,
'--project',
actor.project,
'--chat-id',
input.chatId || 'chat-default',
'--tenant',
actor.tenant,
]);
];
if (input.agentId) args.push('--agent', input.agentId);
if (input.skillId) args.push('--skill', input.skillId);
if (input.delegationLevel !== undefined) args.push('--delegation-level', String(input.delegationLevel));
const res = runPython(CHAT_CLI, args);
const parsed = parseJson<Record<string, any>>(res.stdout);
if (parsed) {
return { ...parsed, actor, audit_verify: this.verifyAudit() };
@@ -89,6 +101,14 @@ export class ChatService {
throw new InternalServerErrorException(res.stderr || res.stdout || 'CHAT_OPERATOR_ACTIONS_FAILED');
}
listAgents(actor: SettingsActor) {
this.requireRead(actor);
const res = runPython(AGENT_CLI, ['list-agents', '--role', actor.role]);
const parsed = parseJson<Record<string, any>>(res.stdout);
if (parsed) return parsed;
throw new InternalServerErrorException(res.stderr || res.stdout || 'CHAT_AGENTS_FAILED');
}
private requireRead(actor: SettingsActor) {
const res = runPython(RBAC_CLI, [
'check',
@@ -6,6 +6,7 @@ import { join } from 'node:path';
import { ChatService } from '../src/chat/chat.service.js';
const viewer = { actor: 'chat-viewer', role: 'viewer', project: 'default', tenant: 'default' };
const operator = { actor: 'chat-operator', role: 'operator', project: 'default', tenant: 'default' };
function withTempChatState(fn: () => void) {
const saved = {
@@ -66,15 +67,38 @@ test('chat ask marks side-effect requests unsupported in MVP-0', () => {
test('chat ask executes registered operator action through action-gate', () => {
withTempChatState(() => {
const svc = new ChatService();
const actions = svc.listActions(viewer) as any;
const actions = svc.listActions(operator) as any;
assert.ok(actions.actions.some((a: any) => a.id === 'run-chat-tests'));
const agents = svc.listAgents(operator) as any;
assert.ok(agents.agents.some((a: any) => a.id === 'ops-operator' && a.allowed_for_role === true));
const res = svc.ask({ message: 'run tests', chatId: 'operator-chat' }, viewer) as any;
const res = svc.ask({
message: 'run tests',
chatId: 'operator-chat',
agentId: 'ops-operator',
skillId: 'registered-actions',
}, operator) as any;
assert.equal(res.success, true);
assert.equal(res.mode, 'OPERATOR');
assert.equal(res.decision, 'ACTION_COMPLETED');
assert.equal(res.action.id, 'run-chat-tests');
assert.equal(res.action_gate.outcome, 'ALLOW');
assert.equal(res.agent_binding.agent_selected, 'ops-operator');
assert.equal(res.agent_binding.skill_selected, 'registered-actions');
assert.equal(res.audit_verify.ok, true);
});
});
test('chat ask denies selected agent outside actor role', () => {
withTempChatState(() => {
const svc = new ChatService();
const agents = svc.listAgents(viewer) as any;
assert.ok(agents.agents.some((a: any) => a.id === 'ops-operator' && a.allowed_for_role === false));
const res = svc.ask({ message: 'run tests', agentId: 'ops-operator' }, viewer) as any;
assert.equal(res.success, false);
assert.equal(res.decision, 'DENIED');
assert.equal(res.agent_selected, 'ops-operator');
assert.equal(res.reason, 'role_not_allowed_for_agent');
});
});
@@ -108,6 +108,19 @@ export interface ChatAnswer {
};
action?: { id: string; label: string; description: string } | null;
action_gate?: { outcome?: string; reason?: string; exit_code?: number };
agent_binding?: {
success: boolean;
decision: string;
reason: string;
agent_selected: string;
skill_selected: string;
delegation_level: number;
model_role?: string;
mode?: string;
tool_allowlist: string[];
requires_approval: boolean;
audit?: { record_hash?: string; head?: string; seq?: number };
};
actor: SettingsActor;
}
@@ -118,6 +131,19 @@ export interface ChatAction {
triggers: string[];
}
export interface ChatAgent {
id: string;
label: string;
mode: string;
model_role: string;
roles_allowed: string[];
skills_allowed: string[];
tool_allowlist: string[];
max_delegation_level: number;
requires_approval?: boolean;
allowed_for_role?: boolean;
}
export interface SettingsState {
actor: SettingsActor;
capabilities: {
@@ -164,10 +190,11 @@ export const api = {
post<{ proposal: any; audit_verify: { ok: boolean; output: string } }>('approvals/submit', body, actorHeaders(actor)),
decideApproval: (actor: SettingsActor, body: { id: string; decision: 'approve' | 'reject'; reason: string }) =>
post<{ proposal: any; applied: any; audit_verify: { ok: boolean; output: string } }>('approvals/decide', body, actorHeaders(actor)),
askChat: (actor: SettingsActor, body: { message: string; chatId?: string }) =>
askChat: (actor: SettingsActor, body: { message: string; chatId?: string; agentId?: string; skillId?: string; delegationLevel?: number }) =>
post<ChatAnswer>('chat/ask', body, actorHeaders(actor)),
verifyChatAudit: () => get<{ ok: boolean; output: string }>('chat/audit/verify'),
chatActions: (actor: SettingsActor) => getWithHeaders<{ success: boolean; actions: ChatAction[] }>('chat/actions', actorHeaders(actor)),
chatAgents: (actor: SettingsActor) => getWithHeaders<{ success: boolean; agents: ChatAgent[] }>('chat/agents', actorHeaders(actor)),
};
// Health is raw (not enveloped) + carries HTTP status.
@@ -1,6 +1,6 @@
import { useState } from 'react';
import { useMutation, useQuery } from '@tanstack/react-query';
import { api, ChatAction, ChatAnswer, SettingsActor } from '../lib/api';
import { api, ChatAction, ChatAgent, ChatAnswer, SettingsActor } from '../lib/api';
import { Card, StatusBadge } from '../components/ui/Card';
const ROLES = ['viewer', 'auditor', 'operator', 'project-admin', 'org-admin'];
@@ -26,6 +26,9 @@ export function Chat() {
tenant: 'default',
});
const [chatId, setChatId] = useState('chat-default');
const [agentId, setAgentId] = useState('evidence-reader');
const [skillId, setSkillId] = useState('evidence-summary');
const [delegationLevel, setDelegationLevel] = useState(0);
const [message, setMessage] = useState('Summarize Plan 18 MVP-0 status');
const [last, setLast] = useState<ChatAnswer | null>(null);
const [error, setError] = useState<string | null>(null);
@@ -40,9 +43,25 @@ export function Chat() {
queryFn: () => api.chatActions(actor),
retry: false,
});
const agentsQuery = useQuery({
queryKey: ['chat-agents', actor],
queryFn: () => api.chatAgents(actor),
retry: false,
});
const agents = agentsQuery.data?.agents ?? [];
const selectedAgent = agents.find((a) => a.id === agentId) ?? agents.find((a) => a.allowed_for_role) ?? agents[0];
const selectedSkill = selectedAgent?.skills_allowed.includes(skillId)
? skillId
: (selectedAgent?.skills_allowed[0] ?? '');
const ask = useMutation({
mutationFn: (override?: { message?: string }) => api.askChat(actor, { message: override?.message ?? message, chatId }),
mutationFn: (override?: { message?: string; agentId?: string; skillId?: string }) => api.askChat(actor, {
message: override?.message ?? message,
chatId,
agentId: override?.agentId ?? selectedAgent?.id ?? agentId,
skillId: override?.skillId ?? selectedSkill,
delegationLevel,
}),
onSuccess: (res) => {
setLast(res);
setError(null);
@@ -111,6 +130,33 @@ export function Chat() {
<input className="w-full rounded border border-gray-300 px-3 py-2" value={chatId}
onChange={(e) => setChatId(e.target.value)} />
</label>
<label className="space-y-1 md:col-span-2 xl:col-span-1">
<span className="text-gray-500">Agent</span>
<select className="w-full rounded border border-gray-300 px-3 py-2" value={selectedAgent?.id ?? agentId}
onChange={(e) => {
const next = agents.find((a) => a.id === e.target.value);
setAgentId(e.target.value);
setSkillId(next?.skills_allowed[0] ?? '');
}}>
{agents.map((a: ChatAgent) => (
<option key={a.id} value={a.id}>
{a.label}{a.allowed_for_role ? '' : ' (locked)'}
</option>
))}
</select>
</label>
<label className="space-y-1">
<span className="text-gray-500">Skill</span>
<select className="w-full rounded border border-gray-300 px-3 py-2" value={selectedSkill}
onChange={(e) => setSkillId(e.target.value)}>
{(selectedAgent?.skills_allowed ?? []).map((s) => <option key={s} value={s}>{s}</option>)}
</select>
</label>
<label className="space-y-1">
<span className="text-gray-500">Delegation</span>
<input className="w-full rounded border border-gray-300 px-3 py-2" type="number" min={0} max={5}
value={delegationLevel} onChange={(e) => setDelegationLevel(Number(e.target.value || 0))} />
</label>
</div>
</div>
</Card>
@@ -125,6 +171,8 @@ export function Chat() {
<StatusBadge value={last.mode} />
<StatusBadge value={last.risk} />
<StatusBadge value={last.decision} />
{last.agent_binding && <StatusBadge value={last.agent_binding.agent_selected} />}
{last.agent_binding && <StatusBadge value={`L${last.agent_binding.delegation_level}`} />}
<StatusBadge value={last.audit_verify.ok ? 'audit ok' : 'audit fail'} />
</div>
<div className="whitespace-pre-wrap text-sm leading-6 text-gray-800">{last.answer}</div>
@@ -174,6 +222,18 @@ export function Chat() {
</div>
</div>
)}
{last.agent_binding && (
<div className="rounded border border-gray-200 p-3">
<div className="text-xs font-semibold uppercase text-gray-400">Agent binding</div>
<div className="mt-1 font-medium text-gray-800">{last.agent_binding.agent_selected}</div>
<div className="mt-1 text-xs text-gray-500">{last.agent_binding.skill_selected || 'no skill'}</div>
<div className="mt-2 flex flex-wrap gap-2">
<StatusBadge value={last.agent_binding.decision} />
<StatusBadge value={last.agent_binding.model_role ?? 'model'} />
<StatusBadge value={`tools ${last.agent_binding.tool_allowlist.length}`} />
</div>
</div>
)}
<div>
<div className="text-xs font-semibold uppercase text-gray-400">Matched rules</div>
<div className="mt-1 flex flex-wrap gap-2">
@@ -202,7 +262,7 @@ export function Chat() {
disabled={ask.isPending}
onClick={() => {
setMessage(trigger);
ask.mutate({ message: trigger });
ask.mutate({ message: trigger, agentId: 'ops-operator', skillId: 'registered-actions' });
}}
className="text-left rounded border border-gray-200 p-3 hover:border-blue-300 hover:bg-blue-50 disabled:opacity-50"
>
@@ -0,0 +1,36 @@
{
"version": 1,
"agents": [
{
"id": "evidence-reader",
"label": "Evidence Reader",
"mode": "READ_ONLY",
"model_role": "read_only",
"roles_allowed": ["viewer", "auditor", "operator", "project-admin", "org-admin"],
"skills_allowed": ["evidence-summary"],
"tool_allowlist": [],
"max_delegation_level": 0
},
{
"id": "ops-operator",
"label": "Ops Operator",
"mode": "OPERATOR",
"model_role": "operator",
"roles_allowed": ["operator", "project-admin", "org-admin"],
"skills_allowed": ["registered-actions"],
"tool_allowlist": ["run-chat-tests", "build-evidence-pack", "verify-evidence-pack"],
"max_delegation_level": 1
},
{
"id": "codegen-draft",
"label": "Codegen Draft",
"mode": "CODEGEN",
"model_role": "codegen",
"roles_allowed": ["project-admin", "org-admin"],
"skills_allowed": ["sourcegen-draft"],
"tool_allowlist": ["artifact-scan", "tool-output-scan"],
"max_delegation_level": 2,
"requires_approval": true
}
]
}
@@ -0,0 +1,268 @@
#!/usr/bin/env python3
"""Plan-18 MVP-2 agent/skill governance resolver.
This is a harness-owned policy primitive. It binds a chat turn to an agent,
skill, delegation level, model role, and tool allowlist before any runtime is
invoked. Unknown policy, RBAC failure, role mismatch, delegation escalation, or
tool outside the agent allowlist all fail closed.
"""
import argparse
import datetime
import hashlib
import json
import os
import subprocess
import sys
def project_root() -> str:
d = os.path.abspath(os.path.dirname(__file__))
p = d
while p != os.path.dirname(p):
if os.path.isdir(os.path.join(p, ".specify")) or os.path.isdir(os.path.join(p, "packages/casan-harness")):
return p
p = os.path.dirname(p)
return os.path.abspath(os.path.join(d, "..", "..", ".."))
ROOT = project_root()
HARNESS_ROOT = os.path.join(ROOT, "packages", "casan-harness")
HARNESS_BIN = os.path.join(HARNESS_ROOT, "scripts", "bash")
RBAC = os.path.join(HARNESS_BIN, "rbac-check.py")
GENESIS_HASH = "0" * 64
def now_iso() -> str:
return datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
def sha(text: str) -> str:
return hashlib.sha256(text.encode("utf-8")).hexdigest()
def state_root() -> str:
return os.environ.get("CASAN_STATE_ROOT") or os.path.join(ROOT, ".specify")
def registry_path() -> str:
return os.environ.get("CASAN_AGENT_REGISTRY_FILE") or os.path.join(HARNESS_ROOT, "config", "agent-registry.yaml")
def audit_path() -> str:
return os.environ.get("CASAN_AGENT_BIND_AUDIT_LOG") or os.path.join(state_root(), "logs", "chat", "agent-bindings.jsonl")
def head_path() -> str:
return os.environ.get("CASAN_AGENT_BIND_AUDIT_HEAD") or os.path.join(state_root(), "logs", "chat", "agent-bindings-head.txt")
def load_registry():
try:
with open(registry_path(), encoding="utf-8") as fh:
data = json.load(fh)
agents = data.get("agents")
if not isinstance(agents, list):
raise ValueError("agents_not_list")
return data
except Exception as exc:
return {"_error": f"agent_registry_unreadable:{exc}", "agents": []}
def find_agent(registry, agent_id: str):
return next((a for a in registry.get("agents", []) if a.get("id") == agent_id), None)
def load_head() -> str:
try:
return open(head_path(), encoding="utf-8").read().strip() or GENESIS_HASH
except OSError:
return GENESIS_HASH
def append_audit(base):
path = audit_path()
os.makedirs(os.path.dirname(path), exist_ok=True)
seq = 1
if os.path.isfile(path):
with open(path, encoding="utf-8") as fh:
seq = sum(1 for line in fh if line.strip()) + 1
prev = load_head()
core = {"seq": seq, **base, "prev_hash": prev}
record_hash = sha(json.dumps(core, sort_keys=True, ensure_ascii=False))
rec = {**core, "record_hash": record_hash}
with open(path, "a", encoding="utf-8") as fh:
fh.write(json.dumps(rec, ensure_ascii=False) + "\n")
os.makedirs(os.path.dirname(head_path()), exist_ok=True)
with open(head_path(), "w", encoding="utf-8") as fh:
fh.write(record_hash + "\n")
return rec
def decision_payload(args, decision: str, reason: str, agent=None, skill="", tools=None):
tools = tools or []
success = decision == "BOUND"
payload = {
"success": success,
"decision": decision,
"reason": reason,
"agent_selected": (agent or {}).get("id") or args.agent,
"skill_selected": skill or "",
"delegation_level": args.delegation_level,
"model_role": (agent or {}).get("model_role"),
"mode": (agent or {}).get("mode"),
"tool_allowlist": (agent or {}).get("tool_allowlist", []),
"requested_tools": tools,
"side_effect_allowed": bool((agent or {}).get("tool_allowlist")),
"requires_approval": bool((agent or {}).get("requires_approval")) or decision == "REQUIRES_APPROVAL",
}
rec = append_audit({
"timestamp": now_iso(),
"harness": "H5-chat-agent-bind",
"actor": args.actor,
"role": args.role,
"project": args.project,
"tenant": args.tenant,
"agent_selected": payload["agent_selected"],
"skill_selected": payload["skill_selected"],
"delegation_level": args.delegation_level,
"decision": decision,
"reason": reason,
"tool_allowlist_ref": sha(json.dumps(payload["tool_allowlist"], sort_keys=True)),
"requested_tools_ref": sha(json.dumps(tools, sort_keys=True)),
})
payload["audit"] = {"seq": rec["seq"], "record_hash": rec["record_hash"], "head": rec["record_hash"]}
return payload
def run_rbac(args):
r = subprocess.run([
"python3", RBAC, "check",
"--role", args.role,
"--resource", "chat",
"--action", "select_agent",
"--role-project", args.project,
"--target-project", args.target_project or args.project,
"--role-tenant", args.tenant,
"--target-tenant", args.target_tenant or args.tenant,
], cwd=ROOT, capture_output=True, text=True)
return r.returncode, (r.stdout + r.stderr).strip()
def bind(args) -> int:
registry = load_registry()
if registry.get("_error"):
out = decision_payload(args, "DENIED", registry["_error"])
print(json.dumps(out, ensure_ascii=False))
return 2
agent = find_agent(registry, args.agent)
if not agent:
out = decision_payload(args, "DENIED", "unknown_agent")
print(json.dumps(out, ensure_ascii=False))
return 2
rc, msg = run_rbac(args)
if rc != 0:
out = decision_payload(args, "DENIED", f"rbac_denied:{msg}", agent)
print(json.dumps(out, ensure_ascii=False))
return 2
if args.role not in agent.get("roles_allowed", []):
out = decision_payload(args, "DENIED", "role_not_allowed_for_agent", agent)
print(json.dumps(out, ensure_ascii=False))
return 2
skill = args.skill or (agent.get("skills_allowed", [""])[0] if agent.get("skills_allowed") else "")
if skill and skill not in agent.get("skills_allowed", []):
out = decision_payload(args, "DENIED", "skill_not_allowed_for_agent", agent, skill)
print(json.dumps(out, ensure_ascii=False))
return 2
max_level = int(agent.get("max_delegation_level", 0))
if args.delegation_level > max_level:
out = decision_payload(args, "REQUIRES_APPROVAL", "delegation_exceeds_agent_max", agent, skill)
print(json.dumps(out, ensure_ascii=False))
return 3
requested_tools = list(args.tool or [])
allow = set(agent.get("tool_allowlist", []))
outside = [t for t in requested_tools if t not in allow]
if outside:
out = decision_payload(args, "DENIED", "tool_not_allowlisted", agent, skill, requested_tools)
out["action_gate"] = {"outcome": "BLOCK", "reason": f"tool_not_allowlisted:{','.join(outside)}"}
print(json.dumps(out, ensure_ascii=False))
return 2
out = decision_payload(args, "BOUND", "agent_bound", agent, skill, requested_tools)
print(json.dumps(out, ensure_ascii=False))
return 0
def list_agents(args) -> int:
registry = load_registry()
if registry.get("_error"):
print(json.dumps({"success": False, "error": registry["_error"], "agents": []}, ensure_ascii=False))
return 2
agents = []
for agent in registry.get("agents", []):
item = {k: agent.get(k) for k in (
"id", "label", "mode", "model_role", "roles_allowed", "skills_allowed",
"tool_allowlist", "max_delegation_level", "requires_approval",
)}
item["allowed_for_role"] = args.role in agent.get("roles_allowed", []) if args.role else None
agents.append(item)
print(json.dumps({"success": True, "version": registry.get("version"), "agents": agents}, ensure_ascii=False))
return 0
def verify_audit() -> int:
prev = GENESIS_HASH
count = 0
try:
fh = open(audit_path(), encoding="utf-8")
except OSError:
print("CHAT_AGENT_AUDIT ok=true records=0 head=" + prev)
return 0
with fh:
for line in fh:
if not line.strip():
continue
count += 1
rec = json.loads(line)
got = rec.get("record_hash")
rest = {k: v for k, v in rec.items() if k != "record_hash"}
if rest.get("prev_hash") != prev or sha(json.dumps(rest, sort_keys=True, ensure_ascii=False)) != got:
print(f"CHAT_AGENT_AUDIT ok=false brokenAt={count}")
return 1
prev = got
print(f"CHAT_AGENT_AUDIT ok=true records={count} head={prev}")
return 0
def main() -> int:
ap = argparse.ArgumentParser()
sub = ap.add_subparsers(dest="cmd", required=True)
b = sub.add_parser("bind")
b.add_argument("--agent", required=True)
b.add_argument("--skill", default="")
b.add_argument("--actor", default="anonymous")
b.add_argument("--role", default="viewer")
b.add_argument("--project", default="default")
b.add_argument("--tenant", default="default")
b.add_argument("--target-project", default="")
b.add_argument("--target-tenant", default="")
b.add_argument("--delegation-level", type=int, default=0)
b.add_argument("--tool", action="append", default=[])
b.set_defaults(func=bind)
l = sub.add_parser("list-agents")
l.add_argument("--role", default="")
l.set_defaults(func=list_agents)
sub.add_parser("verify-audit").set_defaults(func=lambda _args: verify_audit())
args = ap.parse_args()
return args.func(args)
if __name__ == "__main__":
raise SystemExit(main())
@@ -24,6 +24,7 @@ def project_root() -> str:
ROOT = project_root()
BIN = os.path.join(ROOT, "packages", "casan-harness", "scripts", "bash")
ROUTER = os.path.join(BIN, "prompt-mode-router.py")
AGENT_RESOLVER = os.path.join(BIN, "chat-agent-resolver.py")
READONLY = os.path.join(BIN, "chat-readonly.py")
OPERATOR = os.path.join(BIN, "chat-operator.py")
@@ -36,13 +37,69 @@ def classify(message: str):
return {"mode": "BLOCK", "risk": "high", "reason": "router_invalid_json", "matched_rules": [r.stderr.strip()]}
def run_mode(args, binding):
r = subprocess.run(args, cwd=ROOT, capture_output=True, text=True)
try:
payload = json.loads(r.stdout)
payload["agent_binding"] = binding
print(json.dumps(payload, ensure_ascii=False))
except Exception:
if r.stdout:
sys.stdout.write(r.stdout)
if r.stderr:
sys.stderr.write(r.stderr)
return r.returncode
def run_and_passthrough(args):
r = subprocess.run(args, cwd=ROOT, text=True)
return r.returncode
def default_agent_for_mode(mode: str) -> str:
if mode == "OPERATOR":
return "ops-operator"
if mode == "CODEGEN":
return "codegen-draft"
return "evidence-reader"
def bind_agent(args, router):
agent = args.agent or default_agent_for_mode(router.get("mode", "READ_ONLY"))
tools = []
if router.get("mode") == "OPERATOR":
# The operator primitive resolves the exact registered action later; the
# agent bind still proves this turn is allowed to use the operator class.
tools.append("run-chat-tests")
cmd = [
"python3", AGENT_RESOLVER, "bind",
"--agent", agent,
"--actor", args.actor,
"--role", args.role,
"--project", args.project,
"--tenant", args.tenant,
"--delegation-level", str(args.delegation_level),
]
if args.skill:
cmd += ["--skill", args.skill]
for tool in tools:
cmd += ["--tool", tool]
r = subprocess.run(cmd, cwd=ROOT, capture_output=True, text=True)
if r.returncode != 0:
sys.stdout.write((r.stdout or r.stderr).strip() + "\n")
return r.returncode, None
try:
return 0, json.loads(r.stdout)
except Exception:
print(json.dumps({"success": False, "decision": "DENIED", "reason": "agent_resolver_invalid_json"}, ensure_ascii=False))
return 2, None
def ask(args) -> int:
router = classify(args.message)
bind_rc, binding = bind_agent(args, router)
if bind_rc != 0:
return bind_rc
common = [
"--message", args.message,
"--actor", args.actor,
@@ -51,8 +108,8 @@ def ask(args) -> int:
"--tenant", args.tenant,
]
if router.get("mode") == "OPERATOR":
return run_and_passthrough(["python3", OPERATOR, "run", *common])
return run_and_passthrough(["python3", READONLY, "ask", *common])
return run_mode(["python3", OPERATOR, "run", *common], binding)
return run_mode(["python3", READONLY, "ask", *common], binding)
def verify_audit() -> int:
@@ -65,9 +122,14 @@ def main() -> int:
askp = sub.add_parser("ask")
askp.add_argument("--message", required=True)
askp.add_argument("--actor", default="anonymous")
askp.add_argument("--role", default="viewer")
askp.add_argument("--project", default="default")
askp.add_argument("--chat-id", default="")
askp.add_argument("--turn-id", default="")
askp.add_argument("--tenant", default="default")
askp.add_argument("--agent", default="")
askp.add_argument("--skill", default="")
askp.add_argument("--delegation-level", type=int, default=0)
askp.set_defaults(func=ask)
sub.add_parser("verify-audit").set_defaults(func=lambda _args: verify_audit())
args = ap.parse_args()
@@ -149,6 +149,7 @@ run "phase-chat-prompt-router" bash "$TESTS/phase-chat-prompt-router-tests.sh"
run "phase-chat-readonly" bash "$TESTS/phase-chat-readonly-tests.sh"
run "phase-chat-session-audit" bash "$TESTS/phase-chat-session-audit-tests.sh"
run "phase-chat-operator" bash "$TESTS/phase-chat-operator-tests.sh"
run "phase-chat-agent-select" bash "$TESTS/phase-chat-agent-select-tests.sh"
# ARCH-02: coverage cannot silently drop; ARCH-01: harness/policy cannot silently
# drift. Both SKIP cleanly when no manifest is provisioned (non-strict dev/CI).
@@ -17,11 +17,11 @@ import sys
# scope: "org" (all projects) or "project" (must match the acted-on project)
PERMISSIONS = {
"org-admin": {"scope": "org", "allow": {"*"}},
"project-admin": {"scope": "project", "allow": {"settings:read", "settings:write", "monitoring:read", "audit:read"}},
"approver": {"scope": "project", "allow": {"settings:read", "monitoring:read", "approval:grant"}},
"operator": {"scope": "project", "allow": {"monitoring:read", "kill_switch:engage"}},
"viewer": {"scope": "project", "allow": {"settings:read", "monitoring:read"}},
"auditor": {"scope": "org", "allow": {"settings:read", "monitoring:read", "audit:read"}},
"project-admin": {"scope": "project", "allow": {"settings:read", "settings:write", "monitoring:read", "audit:read", "chat:select_agent"}},
"approver": {"scope": "project", "allow": {"settings:read", "monitoring:read", "approval:grant", "chat:select_agent"}},
"operator": {"scope": "project", "allow": {"monitoring:read", "kill_switch:engage", "chat:select_agent"}},
"viewer": {"scope": "project", "allow": {"settings:read", "monitoring:read", "chat:select_agent"}},
"auditor": {"scope": "org", "allow": {"settings:read", "monitoring:read", "audit:read", "chat:select_agent"}},
}
# Maps an IdP-issued claim (role name / group) to an RBAC role. The IdP (Plan-07
@@ -0,0 +1,120 @@
#!/usr/bin/env bash
set -uo pipefail
# Plan-18 MVP-2 Track 4: agent/skill selection is governed by registry + RBAC.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../scripts/bash/casan-paths.sh"
RESOLVER="$CASAN_HARNESS_ROOT/scripts/bash/chat-agent-resolver.py"
TURN="$CASAN_HARNESS_ROOT/scripts/bash/chat-turn.py"
WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT
export CASAN_STATE_ROOT="$WORK/state"
PASS=0; FAIL=0
pass() { echo "PASS: $1"; PASS=$((PASS + 1)); }
fail() { echo "FAIL: $1"; FAIL=$((FAIL + 1)); }
echo "===== Plan-18 MVP-2 agent/skill selection ====="
python3 "$RESOLVER" list-agents --role viewer > "$WORK/list.json"
python3 - "$WORK/list.json" <<'PY' \
&& pass "agent registry lists versioned agents with role visibility" || fail "agent registry listing invalid"
import json, sys
d = json.load(open(sys.argv[1]))
ids = {a["id"]: a for a in d["agents"]}
assert d["success"] is True
assert {"evidence-reader", "ops-operator", "codegen-draft"}.issubset(ids)
assert ids["evidence-reader"]["allowed_for_role"] is True
assert ids["ops-operator"]["allowed_for_role"] is False
PY
python3 "$RESOLVER" bind --agent evidence-reader --skill evidence-summary --role viewer --actor alice --project default --tenant tenant-a > "$WORK/viewer.json"
python3 - "$WORK/viewer.json" <<'PY' \
&& pass "viewer can bind read-only evidence-reader" || fail "viewer could not bind read-only agent"
import json, sys
d = json.load(open(sys.argv[1]))
assert d["success"] is True
assert d["decision"] == "BOUND"
assert d["agent_selected"] == "evidence-reader"
assert d["delegation_level"] == 0
assert d["tool_allowlist"] == []
PY
set +e
python3 "$RESOLVER" bind --agent ops-operator --role viewer --actor alice --project default --tenant tenant-a > "$WORK/viewer-op.json"
RC=$?
set -e 2>/dev/null || true
python3 - "$WORK/viewer-op.json" "$RC" <<'PY' \
&& pass "viewer cannot select operator agent" || fail "viewer selected operator agent"
import json, sys
d = json.load(open(sys.argv[1]))
assert int(sys.argv[2]) == 2
assert d["success"] is False
assert d["decision"] == "DENIED"
assert d["reason"] == "role_not_allowed_for_agent"
PY
set +e
python3 "$RESOLVER" bind --agent evidence-reader --role viewer --actor alice --project default --tenant tenant-a --target-tenant tenant-b > "$WORK/cross-tenant.json"
RC=$?
set -e 2>/dev/null || true
python3 - "$WORK/cross-tenant.json" "$RC" <<'PY' \
&& pass "cross-tenant agent binding is denied by RBAC" || fail "cross-tenant agent binding was allowed"
import json, sys
d = json.load(open(sys.argv[1]))
assert int(sys.argv[2]) == 2
assert d["success"] is False
assert d["decision"] == "DENIED"
assert "rbac_denied" in d["reason"]
PY
set +e
python3 "$RESOLVER" bind --agent evidence-reader --role viewer --actor alice --project default --tenant tenant-a --delegation-level 1 > "$WORK/delegation.json"
RC=$?
set -e 2>/dev/null || true
python3 - "$WORK/delegation.json" "$RC" <<'PY' \
&& pass "delegation above agent max requires approval" || fail "delegation escalation was not held"
import json, sys
d = json.load(open(sys.argv[1]))
assert int(sys.argv[2]) == 3
assert d["success"] is False
assert d["decision"] == "REQUIRES_APPROVAL"
assert d["requires_approval"] is True
PY
set +e
python3 "$RESOLVER" bind --agent ops-operator --role operator --actor bob --project default --tenant tenant-a --tool shell > "$WORK/tool.json"
RC=$?
set -e 2>/dev/null || true
python3 - "$WORK/tool.json" "$RC" <<'PY' \
&& pass "tool outside agent allowlist is blocked" || fail "tool outside allowlist was accepted"
import json, sys
d = json.load(open(sys.argv[1]))
assert int(sys.argv[2]) == 2
assert d["success"] is False
assert d["decision"] == "DENIED"
assert d["action_gate"]["outcome"] == "BLOCK"
PY
set +e
python3 "$TURN" ask --message "run tests" --agent ops-operator --role viewer --actor alice --chat-id agent-chat --tenant tenant-a > "$WORK/turn-deny.json"
RC=$?
set -e 2>/dev/null || true
python3 - "$WORK/turn-deny.json" "$RC" <<'PY' \
&& pass "chat-turn denies unauthorized selected agent before operator runtime" || fail "chat-turn bypassed agent resolver"
import json, sys
d = json.load(open(sys.argv[1]))
assert int(sys.argv[2]) == 2
assert d["success"] is False
assert d["decision"] == "DENIED"
assert d["agent_selected"] == "ops-operator"
PY
python3 "$RESOLVER" verify-audit > "$WORK/audit.out" \
&& grep -q "CHAT_AGENT_AUDIT ok=true" "$WORK/audit.out" \
&& pass "agent binding audit hash chain verifies" || fail "agent binding audit invalid"
echo ""
echo "===== CHAT AGENT SUMMARY: PASS=$PASS FAIL=$FAIL ====="
[[ "$FAIL" -eq 0 ]] || exit 1
@@ -37,7 +37,7 @@ ids = {a["id"] for a in d["actions"]}
assert {"run-chat-tests", "build-evidence-pack", "verify-evidence-pack"}.issubset(ids)
PY
python3 "$TURN" ask --message "run tests" --actor bob --chat-id op1 > "$WORK/run.json"
python3 "$TURN" ask --message "run tests" --actor bob --role operator --chat-id op1 > "$WORK/run.json"
python3 - "$WORK/run.json" <<'PY' \
&& pass "registered run tests action executes through chat-turn" || fail "registered action did not complete"
import json, os, sys
+5 -4
View File
@@ -1,7 +1,8 @@
# CASAN Platform (Level 3 — Productization UI) · **PREVIEW**
> Status: **PREVIEW.** The AgentOps dashboard and Plan-13 Control Panel, including
> Command Center baseline and Plan-18 MVP-0/1 Chat Console, exist today.
> Command Center baseline and Plan-18 MVP-0/1 Chat Console plus MVP-2 agent
> selection foundation, exist today.
> Evidence/attack viewers, Gitea integration, and managed production rollout remain
> planned. `package-release.sh platform` builds a clearly-stamped
> `casan-platform-preview-*` bundle containing only what exists.
@@ -13,12 +14,12 @@ Optional layer for teams that want UI / dashboard / visibility. Packages: `casan
| Component | Status | Where |
|---|---|---|
| AgentOps Dashboard | ✅ exists | `packages/casan-harness/scripts/bash/dashboard-server.py` + `dashboard-serve.sh` (`casan dashboard`) |
| **Ops Console (Control Panel)** | ✅ **monitoring + governed settings + HITL + kill-switch + FinOps/SLO + Command Center + Chat MVP-0/1 + local-prod TLS/OIDC smoke** | `packages/casan-control-panel/` — NestJS API + React UI over harness telemetry, settings management, approval inbox, kill-switch, FinOps/SLO, Command Center, Ask CASAN read-only, and registered Operator actions (`npm run console:api` + `console:ui`). Includes Run History + verdicts + governance + security + incidents + traceability + FinOps + Approvals + role-aware Settings page + `/command` evidence-backed executive view + `/chat` governed read-only/operator console. |
| **Ops Console (Control Panel)** | ✅ **monitoring + governed settings + HITL + kill-switch + FinOps/SLO + Command Center + Chat MVP-0/1 + agent selection foundation + local-prod TLS/OIDC smoke** | `packages/casan-control-panel/` — NestJS API + React UI over harness telemetry, settings management, approval inbox, kill-switch, FinOps/SLO, Command Center, Ask CASAN read-only, registered Operator actions, and governed agent/skill selection (`npm run console:api` + `console:ui`). Includes Run History + verdicts + governance + security + incidents + traceability + FinOps + Approvals + role-aware Settings page + `/command` evidence-backed executive view + `/chat` governed read-only/operator console. |
| Management / settings writes | ✅ done+test | Plan-13 Track 2: wraps `control-plane-settings.py`, calls `rbac-check.py`, supports set/rollback/audit verify. |
| RBAC + approval inbox | ✅ local-prod done | Settings + kill-switch API RBAC enforcement, approval inbox/delegation/oversight, SoD, governed setting proposal apply, and local OIDC claim→role mapping smoke are done. Enterprise IdP rollout remains production follow-up. |
| Evidence Pack Viewer | 📋 planned | reads `docs/output/casan/evidence-packs/` |
| Attack Battery Viewer | 📋 planned | reads red-team corpus + H4 recall results |
| Read-only Ask CASAN + Operator | ✅ MVP-0/1 done+test | Plan-18 MVP-0/1: `POST /api/v1/chat/ask`, `GET /api/v1/chat/actions` + `/chat`, backed by harness `chat-turn.py` |
| Read-only Ask CASAN + Operator + agent selection | 🟡 MVP-0/1 done+test; MVP-2 Track 4 foundation done | Plan-18: `POST /api/v1/chat/ask`, `GET /api/v1/chat/actions`, `GET /api/v1/chat/agents` + `/chat`, backed by harness `chat-turn.py` and `chat-agent-resolver.py` |
| Gitea webhook integration | 📋 planned | trigger gate / publish evidence on push |
## Build (preview)
@@ -28,6 +29,6 @@ scripts/package-release.sh platform # → dist/casan-platform-preview-vX.Y.Z
The bundle includes a `PREVIEW-INCOMPLETE.txt` marker. Do not treat it as a finished product.
## To implement later
Start from Plan-15 RAI view or Plan-18 MVP-2 (Chat-as-loop + agent/skill). Keep new numbers
Start from Plan-15 RAI view or Plan-18 MVP-2 Track 5/6 (Chat-as-loop + streaming). Keep new numbers
evidence-backed with provenance; any write/governed action must continue to route through
harness RBAC, approval, and audit primitives.