feat: add governed chat agent selection
This commit is contained in:
@@ -61,7 +61,7 @@
|
||||
| Future B1–B6 | 💤 vision | `CASAN_PLAN_FUTURE_PHASES.md` — approval workflow nâng cao · state machine · model benchmark · governed memory · auto-remediation · platform KPI. |
|
||||
| **17 Loop Engineering** | � T1–T6 done+test (offline) | **Agentic Loop Governance** — đủ 5 primitive + orchestrator (97/0 WSL, nối CI). T1 **Governor** (`loop-governor.py`; deny-by-default, no/corrupt policy→strict/HALT, on_exceed halt/escalate) 15/0; T2 **Convergence** (`loop-convergence.py`; repeat/thrash→OSCILLATING, flat→STALLED, fail-closed) 15/0; T3 **Verify Contract** (`loop-gate.py`; H4→DENY, unmet→FAIL, correction bounded→ESCALATE, no self-declared DONE) 20/0; T4 **Trace/Replay** (`loop-trace.py`; append-only hash-linked, edited→BREAK, tampered artifact→replay DRIFT) 16/0; T5 **Meta-loop** (`loop-metaloop.py`; propose≠apply, SoD, loosen>org_ceiling refused, apply qua governed CP store→đổi thật ceiling + rollback) 15/0; T6 **Orchestrator** (`loop-run.sh`; gate→governor→convergence→trace/turn, secure-by-default opt-out, nén giữa vòng) 16/0. State qua `CASAN_LOOP_STATE_ROOT` (repo `.specify/state` sạch). **Còn (infra):** T4 KMS-anchor head (A7 Vault), T6 widget Command Center (17.22, C5), live H3-judge. Chi tiết: `CASAN_PLAN_17_LOOP_ENGINEERING.md`. |
|
||||
| **16 Security audit remediation** | � P0/P1/P2 phần lớn done+test | **Remediation đã thực thi:** 28 SEC suite (151/0 WSL, nối `ci-harness-gate.sh`). Done: SEC-01..10, 12, 13, **14** (model-digest bỏ env-override ở prod/strict), 15, 16..21, **22** (trusted-time JWT `exp` ARCH-06 + tag proposal nguồn-không-tin ARCH-08), **26** (stored/second-order injection scan), 27..30, **23 Phase 1–5 offline** (multi-tenant: tenant-store+guard · per-tenant CP/audit/telemetry · RBAC data-boundary · tenant kill-switch/quota · ký registry · crypt at-rest per-tenant), **24 offline** (image digest-pin + ký workflow), **25 offline** (artifact attestation tested==deployed); **SEC-11 gộp vào SEC-17** (`CASAN_PROFILE=prod` enforce-by-default). **Còn 📋 planned (hạ tầng/process):** SEC-22 ARCH-10 (attestation ngoài) · SEC-23 23.11 (crypt qua Vault Transit) · **SEC-24 còn** (live CVE/OSV + scan image thật — offline image-pin/ký-workflow đã done) · **SEC-25 còn** (signed-commit enrollment + SLSA chain — offline artifact-attestation đã done). Chi tiết: `CASAN_PLAN_16` §0a/§2d. |
|
||||
| **18 Chat Console** | 🟡 **MVP-0 + MVP-1 done+test** · target arch còn planned | **Governed Chat Console** (cắt lát MVP chống lan man). **MVP-0 Ask CASAN read-only DONE**: `prompt-mode-router.py`, `chat-readonly.py`, `chat-session.schema.json`, H4 input/output scan, H5 chat audit hash-chain, H6 token telemetry, answer kèm evidence sources. **MVP-1 Operator DONE**: `operator-actions.yaml`, `chat-operator.py`, `chat-turn.py`, registered actions `run tests`/`build pack`/`verify pack`, all through `action-gate`, no free-command, action artifacts with provenance, Control Panel `POST /api/v1/chat/ask`, `GET /api/v1/chat/actions`, `/chat` UI. Test: chat suites **24/0** (`phase-chat-prompt-router` 8/0, `phase-chat-readonly` 5/0, `phase-chat-session-audit` 3/0, `phase-chat-operator` 8/0), Control Panel **24/0** + build xanh. Sau: MVP-2 chat-as-loop + agent/skill (**cần Plan-17+14**) → MVP-3 multi-tenant (**cần SEC-23**). |
|
||||
| **18 Chat Console** | 🟡 **MVP-0 + MVP-1 done+test; MVP-2 Track 4 foundation done** · target arch còn planned | **Governed Chat Console** (cắt lát MVP chống lan man). **MVP-0 Ask CASAN read-only DONE**: `prompt-mode-router.py`, `chat-readonly.py`, `chat-session.schema.json`, H4 input/output scan, H5 chat audit hash-chain, H6 token telemetry, answer kèm evidence sources. **MVP-1 Operator DONE**: `operator-actions.yaml`, `chat-operator.py`, `chat-turn.py`, registered actions `run tests`/`build pack`/`verify pack`, all through `action-gate`, no free-command, action artifacts with provenance. **MVP-2 Track 4 foundation DONE**: `agent-registry.yaml`, `chat-agent-resolver.py`, `chat:select_agent` RBAC, delegation hold, tool allowlist BLOCK, Control Panel `GET /api/v1/chat/agents` + `/chat` agent/skill/delegation picker. Test: chat suites **32/0** (`phase-chat-prompt-router` 8/0, `phase-chat-readonly` 5/0, `phase-chat-session-audit` 3/0, `phase-chat-operator` 8/0, `phase-chat-agent-select` 8/0), Control Panel **25/0** + build xanh. Next: MVP-2 Track 5/6 chat-as-loop + streaming + replay → MVP-3 multi-tenant. |
|
||||
---
|
||||
|
||||
## Trần điểm & điều kiện lên "Strong (81+)"
|
||||
|
||||
@@ -35,7 +35,7 @@
|
||||
| 15 | `CASAN_PLAN_15_RESPONSIBLE_AI_DATA_GOV.md` | Responsible AI & Data Governance (FPT §14.3–14.4) | 📋 |
|
||||
| 16 | `CASAN_PLAN_16_SECURITY_AUDIT_REMEDIATION.md` | Security audit core harness + kế hoạch vá (tamper-evidence/injection/fail-open) | � P0/P1/P2 done |
|
||||
| 17 | `CASAN_PLAN_17_LOOP_ENGINEERING.md` | Loop Engineering / Agentic Loop Governance (budget governor, convergence, verify-contract, loop trace/replay, meta-loop) | 📋 |
|
||||
| 18 | `CASAN_PLAN_18_CHAT_CONSOLE.md` | Governed Chat Console (target arch; **MVP-0 Ask CASAN + MVP-1 Operator done** → chat-as-loop → multi-tenant) | 🟡 MVP-0/1 done |
|
||||
| 18 | `CASAN_PLAN_18_CHAT_CONSOLE.md` | Governed Chat Console (target arch; **MVP-0 Ask CASAN + MVP-1 Operator done; MVP-2 Track 4 foundation done** → chat-as-loop/streaming/replay → multi-tenant) | 🟡 MVP-0/1 + Track 4 done |
|
||||
| Future | `CASAN_PLAN_FUTURE_PHASES.md` | Approval workflow, state machine, benchmark, memory, remediation, KPI | 💤 vision |
|
||||
|
||||
> **Không có plan số 11:** số 11 được bỏ trống có chủ ý — nhánh eval/traceability đã
|
||||
|
||||
@@ -1,12 +1,15 @@
|
||||
# KẾ HOẠCH 18 — Governed Chat Console (Chat-as-Loop qua Control Plane)
|
||||
|
||||
> Status 2026-07-08: **🟡 MVP-0 + MVP-1 done+test — target architecture còn planned.**
|
||||
> Status 2026-07-08: **🟡 MVP-0 + MVP-1 done+test; MVP-2 Track 4 foundation done — target architecture còn planned.**
|
||||
> Đã implement **Ask CASAN — Read-only Evidence Assistant** qua harness + Control
|
||||
> Panel (`/api/v1/chat/ask`, `/chat` UI): Prompt Router deterministic
|
||||
> `READ_ONLY/OPERATOR/BLOCK/NOT_SUPPORTED`, context whitelist, H4 scan in/out, H5
|
||||
> chat audit hash-chain, H6 token telemetry, answer kèm evidence sources. **MVP-1
|
||||
> Operator** đã có registered actions `run tests` / `build pack` / `verify pack`
|
||||
> qua `action-gate`, không free-command, kết quả có artifact provenance. Mục tiêu
|
||||
> qua `action-gate`, không free-command, kết quả có artifact provenance. **MVP-2
|
||||
> Track 4 foundation** đã có `agent-registry.yaml`, agent/skill resolver, RBAC
|
||||
> gate chọn agent, delegation hold, tool allowlist block, Control Panel agent picker.
|
||||
> Mục tiêu
|
||||
> tổng thể vẫn là thêm **cửa sổ chat** vào Control Plane (Plan-13) như một
|
||||
> **bề mặt tương tác của core harness** — mỗi lượt chat là **một loop-run được
|
||||
> governance** (Plan-17), đi qua đúng H1→H7, không có đường vòng. Đây **không** phải
|
||||
@@ -250,10 +253,10 @@ flowchart TD
|
||||
### Track 4 — Agent/Skill selection governance `[MVP-2]`
|
||||
| Task | Việc | File | Verify (WSL) |
|
||||
|---|---|---|---|
|
||||
| 18.4.1 | `agent-registry.yaml` versioned (tool_allowlist, max_delegation, roles_allowed, model_role) + resolver bind theo turn | mới `config/agent-registry.yaml` | chọn agent → allowlist đúng |
|
||||
| 18.4.2 | RBAC gate chọn agent (14): role không đủ → DENY; vượt `max_delegation` → security-sensitive → approval | nối `rbac-check.py` | role thấp chọn agent tự chủ → DENY |
|
||||
| 18.4.3 | Tool-call runtime chỉ tool **trong allowlist**; ngoài → `action-gate` BLOCK | nối `action-gate.sh` | tool ngoài allowlist → BLOCK |
|
||||
| 18.4.4 | Skill code-gen chạy trong allowlist + H4 artifact-scan (Plan-02) trước output | nối `artifact-scan.sh` | skill sinh mã có injection → BLOCK |
|
||||
| 18.4.1 | ✅ `agent-registry.yaml` versioned (tool_allowlist, max_delegation, roles_allowed, model_role) + resolver bind theo turn | `packages/casan-harness/config/agent-registry.yaml` + `chat-agent-resolver.py` | chọn agent → allowlist đúng |
|
||||
| 18.4.2 | ✅ RBAC gate chọn agent (14): role không đủ → DENY; vượt `max_delegation` → security-sensitive → approval hold | nối `rbac-check.py` (`chat:select_agent`) | role thấp chọn agent tự chủ → DENY |
|
||||
| 18.4.3 | ✅ Tool-call runtime chỉ tool **trong allowlist**; ngoài → BLOCK trước runtime | `chat-agent-resolver.py` | tool ngoài allowlist → BLOCK |
|
||||
| 18.4.4 | ⏳ Skill code-gen chạy trong allowlist + H4 artifact-scan (Plan-02) trước output | nối `artifact-scan.sh` | skill sinh mã có injection → BLOCK |
|
||||
|
||||
### Track 5 — Chat-as-loop pipeline `[MVP-2]`
|
||||
| Task | Việc | File | Verify (WSL) |
|
||||
@@ -274,7 +277,7 @@ flowchart TD
|
||||
|---|---|---|---|
|
||||
| 18.7.1 | ✅ Chat API (NestJS) **bọc** harness (MVP-0/1: read-only + operator; single-source, không verdict riêng) | `packages/casan-control-panel/backend/src/chat/*` | API gọi `chat-turn.py`; `console:test` xanh |
|
||||
| 18.7.2 | ✅ Chat panel React + hiển thị **evidence sources**, registered actions, action-gate + badge `mode/risk` | `packages/casan-control-panel/frontend/src/pages/Chat.tsx` | `console:build` xanh |
|
||||
| 18.7.3 | (MVP-2) agent/skill picker theo RBAC + badge `UNCERTIFIED/CERTIFIED` + nút **loop-breaker** + delegation-level | Control Plane UI | picker ẩn agent ngoài quyền |
|
||||
| 18.7.3 | 🟡 (MVP-2 partial) agent/skill picker theo RBAC + delegation-level đã có; còn badge `UNCERTIFIED/CERTIFIED` + nút **loop-breaker** khi Track 5/6 mở | Control Plane UI | picker hiện locked agent, backend DENY agent ngoài quyền |
|
||||
| 18.7.4 | Fail-loud: API/telemetry chết → UI `STALE/503` (tái dùng D3 `/healthz`) | nối Plan-07 D3 | ngắt backend → UI báo stale |
|
||||
|
||||
### Track 8 — Replay / Evidence / Command Center widgets `[MVP-2 → MVP-3]`
|
||||
@@ -313,8 +316,8 @@ flowchart TD
|
||||
|
||||
**File test (theo MVP):** MVP-0 — `phase-chat-prompt-router-tests.sh`,
|
||||
`phase-chat-readonly-tests.sh`, `phase-chat-session-audit-tests.sh`; MVP-1 —
|
||||
`phase-chat-operator-tests.sh`; MVP-2 — `phase-chat-pipeline-tests.sh`,
|
||||
`phase-chat-agent-select-tests.sh`, `phase-chat-stream-hold-tests.sh`,
|
||||
`phase-chat-operator-tests.sh`; MVP-2 — `phase-chat-agent-select-tests.sh`,
|
||||
`phase-chat-pipeline-tests.sh`, `phase-chat-stream-hold-tests.sh`,
|
||||
`phase-chat-replay-tests.sh`; MVP-3 — `phase-chat-tenant-tests.sh`. Chạy **WSL**
|
||||
(deterministic, mock model; nhánh live cần Ollama SKIP-aware). Nối `ci-harness-gate.sh`;
|
||||
cập nhật tổng test ở `CASAN_HARDENING_STATUS.md`.
|
||||
@@ -336,6 +339,13 @@ cập nhật tổng test ở `CASAN_HARDENING_STATUS.md`.
|
||||
- [x] Kết quả action ghi artifact có provenance và được hiển thị lại trong `/chat`.
|
||||
- [x] Suite MVP-1 xanh và nối CI. Verify: `phase-chat-operator` 8/0, prompt-router 8/0, read-only 5/0, session-audit 3/0, `console:test` 24/0, `console:build` xanh.
|
||||
|
||||
**MVP-2 Track 4 foundation (Agent/Skill selection governance):**
|
||||
- [x] Agent registry versioned có `tool_allowlist`, `max_delegation_level`, `roles_allowed`, `model_role`.
|
||||
- [x] Agent resolver bind theo turn, ghi audit hash-chain, fail-closed khi registry lỗi.
|
||||
- [x] RBAC gate `chat:select_agent`; role thấp chọn agent tự chủ → DENY; cross-tenant → DENY.
|
||||
- [x] Delegation vượt agent max → `REQUIRES_APPROVAL`; tool ngoài allowlist → BLOCK trước runtime.
|
||||
- [x] Control Panel `GET /api/v1/chat/agents`, `/chat` agent/skill/delegation picker, response có `agent_binding`. Verify: `phase-chat-agent-select` 8/0, RBAC 12/0, SEC-23 RBAC 6/0, `console:test` 25/0, `console:build` xanh.
|
||||
|
||||
**Full (target architecture) — DoD tổng:**
|
||||
- [ ] Turn chạy **đúng như một loop-run Plan-17** (không định nghĩa vòng lặp riêng).
|
||||
- [ ] **Không đường vòng:** test chứng minh bỏ bất kỳ gate nào (preflight/verify) → FAIL.
|
||||
@@ -349,9 +359,10 @@ cập nhật tổng test ở `CASAN_HARDENING_STATUS.md`.
|
||||
---
|
||||
|
||||
## 7. Ghi chú trung thực & Non-goals (không lan man)
|
||||
- **[MVP-0 + MVP-1] — 🟡 done+test.** Ask CASAN read-only và Operator registered
|
||||
actions đã có harness CLI + Control Panel API/UI. Các phase Chat-as-loop /
|
||||
Agent-selection / multi-tenant production vẫn chưa mở.
|
||||
- **[MVP-0 + MVP-1 + MVP-2 Track 4 foundation] — 🟡 done+test.** Ask CASAN read-only,
|
||||
Operator registered actions, và agent/skill governance foundation đã có harness CLI
|
||||
+ Control Panel API/UI. Các phase Chat-as-loop / streaming / replay / multi-tenant
|
||||
production vẫn chưa mở.
|
||||
- **MVP-first (chống lan man):** **MVP-0 (Ask CASAN read-only) KHÔNG phụ thuộc Plan-17/
|
||||
14/SEC-23** — làm được ngay trên nền H4/H5/H6 hiện có. Chỉ **MVP-2 trở đi** (chat-as-
|
||||
loop + agent) mới cần Plan-17 (T1–T3) + Plan-14 RBAC; **MVP-3** mới cần SEC-23 tenant.
|
||||
|
||||
@@ -77,7 +77,7 @@ approval JWT thật ([16 SEC-07](CASAN_PLAN_16_SECURITY_AUDIT_REMEDIATION.md)),
|
||||
|---|---|---|:--:|:--:|---|
|
||||
| B9 | ✅ **done** — **MVP-0 Ask CASAN read-only**: Prompt Router (Track 0) + Read-only Ask CASAN (Track 1) + session/audit (Track 2) + Control Panel API/UI (Track 7). H4 in/out, H5 audit hash-chain, H6 token, evidence sources; chat suites 14/0, Control Panel 23/0 + build xanh. Real model provider binding remains Track M follow-up when a provider is enabled. | [18 §1b, Track 0/1/2/7/M](CASAN_PLAN_18_CHAT_CONSOLE.md) | L | ✅ | done |
|
||||
| B10 | ✅ **done** — **MVP-1 Operator mode**: registered actions `run tests` / `build pack` / `verify pack`, no free-command, all through `action-gate`, action artifacts with provenance, Control Panel quick actions. `phase-chat-operator` 8/0; total chat suites 24/0; Control Panel 24/0 + build xanh. | [18 Track 3](CASAN_PLAN_18_CHAT_CONSOLE.md) | M | ✅ | done |
|
||||
| B11 | **MVP-2** Chat-as-loop (Track 5) + Agent/Skill selection (Track 4) + streaming draft-hold (Track 6) | [18 Track 4/5/6](CASAN_PLAN_18_CHAT_CONSOLE.md) | L | 🔗 | B6, C7(RBAC), B10 |
|
||||
| B11 | 🟡 **partial done** — Track 4 Agent/Skill governance foundation: `agent-registry.yaml`, `chat-agent-resolver.py`, RBAC `chat:select_agent`, delegation hold, tool allowlist BLOCK, Control Panel agent/skill/delegation picker. `phase-chat-agent-select` 8/0; Control Panel 25/0 + build xanh. **Remaining MVP-2:** Track 5 chat-as-loop + Track 6 streaming draft-hold + Track 8 replay widget. | [18 Track 4/5/6](CASAN_PLAN_18_CHAT_CONSOLE.md) | L | 🔗 | B6, C7(RBAC), B10 |
|
||||
| B12 | **Widget Loop/Chat** trên Command Center (loop ticker/budget/replay drawer) | [17 (17.22)](CASAN_PLAN_17_LOOP_ENGINEERING.md) · [18 Track 8](CASAN_PLAN_18_CHAT_CONSOLE.md) | M | 🔗 | C5 ✅ baseline |
|
||||
| B13 | **MVP-3** multi-tenant chat hardening (Track 9) | [18 Track 9](CASAN_PLAN_18_CHAT_CONSOLE.md) | M | 🔗 | A3(SEC-23) |
|
||||
|
||||
|
||||
@@ -56,7 +56,7 @@ Approval inbox / HITL:
|
||||
- `POST /api/v1/approvals/decide` — approve/reject with SoD and reason; approved
|
||||
settings proposals apply through `control-plane-settings.py`.
|
||||
|
||||
Governed Chat (Plan-18 MVP-0/1):
|
||||
Governed Chat (Plan-18 MVP-0/1 + MVP-2 Track 4 foundation):
|
||||
|
||||
- `POST /api/v1/chat/ask` — Ask CASAN endpoint. The API only wraps harness
|
||||
`chat-turn.py`; router verdicts, H4 input/output scan, action-gate decisions,
|
||||
@@ -64,11 +64,14 @@ Governed Chat (Plan-18 MVP-0/1):
|
||||
execution remain harness-owned.
|
||||
- `GET /api/v1/chat/actions` — list registered operator actions from
|
||||
`operator-actions.yaml`; no free-command execution is exposed.
|
||||
- `GET /api/v1/chat/agents` — list governed agents from `agent-registry.yaml`
|
||||
with role visibility; selected agent/skill/delegation are bound by harness
|
||||
`chat-agent-resolver.py`.
|
||||
- `GET /api/v1/chat/audit/verify` — verifies the chat audit hash chain.
|
||||
- `/chat` UI shows actor/role scope, `mode/risk/decision` badges, certified answer,
|
||||
evidence sources, registered operator actions, action-gate status, router details,
|
||||
and audit hash. Side-effect requests outside registered actions return governed
|
||||
`BLOCK` or `NOT_SUPPORTED` responses.
|
||||
evidence sources, registered operator actions, agent binding, action-gate status,
|
||||
router details, and audit hash. Side-effect requests outside registered actions
|
||||
return governed `BLOCK` or `NOT_SUPPORTED` responses.
|
||||
|
||||
FinOps/SLO:
|
||||
|
||||
|
||||
@@ -21,4 +21,9 @@ export class ChatController {
|
||||
actions(@Headers() headers: Record<string, string | string[] | undefined>) {
|
||||
return ok(this.svc.listActions(actorFromHeaders(headers)));
|
||||
}
|
||||
|
||||
@Get('agents')
|
||||
agents(@Headers() headers: Record<string, string | string[] | undefined>) {
|
||||
return ok(this.svc.listAgents(actorFromHeaders(headers)));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -7,6 +7,9 @@ import type { SettingsActor } from '../settings/settings.service.js';
|
||||
export interface ChatAskInput {
|
||||
message: string;
|
||||
chatId?: string;
|
||||
agentId?: string;
|
||||
skillId?: string;
|
||||
delegationLevel?: number;
|
||||
}
|
||||
|
||||
interface CommandResult {
|
||||
@@ -18,6 +21,7 @@ interface CommandResult {
|
||||
const HARNESS_BIN = join(APP_ROOT, 'packages', 'casan-harness', 'scripts', 'bash');
|
||||
const CHAT_CLI = join(HARNESS_BIN, 'chat-turn.py');
|
||||
const OPERATOR_CLI = join(HARNESS_BIN, 'chat-operator.py');
|
||||
const AGENT_CLI = join(HARNESS_BIN, 'chat-agent-resolver.py');
|
||||
const RBAC_CLI = join(HARNESS_BIN, 'rbac-check.py');
|
||||
|
||||
function runPython(script: string, args: string[]): CommandResult {
|
||||
@@ -55,17 +59,25 @@ export class ChatService {
|
||||
}
|
||||
this.requireRead(actor);
|
||||
|
||||
const res = runPython(CHAT_CLI, [
|
||||
const args = [
|
||||
'ask',
|
||||
'--message',
|
||||
input.message,
|
||||
'--actor',
|
||||
actor.actor,
|
||||
'--role',
|
||||
actor.role,
|
||||
'--project',
|
||||
actor.project,
|
||||
'--chat-id',
|
||||
input.chatId || 'chat-default',
|
||||
'--tenant',
|
||||
actor.tenant,
|
||||
]);
|
||||
];
|
||||
if (input.agentId) args.push('--agent', input.agentId);
|
||||
if (input.skillId) args.push('--skill', input.skillId);
|
||||
if (input.delegationLevel !== undefined) args.push('--delegation-level', String(input.delegationLevel));
|
||||
const res = runPython(CHAT_CLI, args);
|
||||
const parsed = parseJson<Record<string, any>>(res.stdout);
|
||||
if (parsed) {
|
||||
return { ...parsed, actor, audit_verify: this.verifyAudit() };
|
||||
@@ -89,6 +101,14 @@ export class ChatService {
|
||||
throw new InternalServerErrorException(res.stderr || res.stdout || 'CHAT_OPERATOR_ACTIONS_FAILED');
|
||||
}
|
||||
|
||||
listAgents(actor: SettingsActor) {
|
||||
this.requireRead(actor);
|
||||
const res = runPython(AGENT_CLI, ['list-agents', '--role', actor.role]);
|
||||
const parsed = parseJson<Record<string, any>>(res.stdout);
|
||||
if (parsed) return parsed;
|
||||
throw new InternalServerErrorException(res.stderr || res.stdout || 'CHAT_AGENTS_FAILED');
|
||||
}
|
||||
|
||||
private requireRead(actor: SettingsActor) {
|
||||
const res = runPython(RBAC_CLI, [
|
||||
'check',
|
||||
|
||||
@@ -6,6 +6,7 @@ import { join } from 'node:path';
|
||||
import { ChatService } from '../src/chat/chat.service.js';
|
||||
|
||||
const viewer = { actor: 'chat-viewer', role: 'viewer', project: 'default', tenant: 'default' };
|
||||
const operator = { actor: 'chat-operator', role: 'operator', project: 'default', tenant: 'default' };
|
||||
|
||||
function withTempChatState(fn: () => void) {
|
||||
const saved = {
|
||||
@@ -66,15 +67,38 @@ test('chat ask marks side-effect requests unsupported in MVP-0', () => {
|
||||
test('chat ask executes registered operator action through action-gate', () => {
|
||||
withTempChatState(() => {
|
||||
const svc = new ChatService();
|
||||
const actions = svc.listActions(viewer) as any;
|
||||
const actions = svc.listActions(operator) as any;
|
||||
assert.ok(actions.actions.some((a: any) => a.id === 'run-chat-tests'));
|
||||
const agents = svc.listAgents(operator) as any;
|
||||
assert.ok(agents.agents.some((a: any) => a.id === 'ops-operator' && a.allowed_for_role === true));
|
||||
|
||||
const res = svc.ask({ message: 'run tests', chatId: 'operator-chat' }, viewer) as any;
|
||||
const res = svc.ask({
|
||||
message: 'run tests',
|
||||
chatId: 'operator-chat',
|
||||
agentId: 'ops-operator',
|
||||
skillId: 'registered-actions',
|
||||
}, operator) as any;
|
||||
assert.equal(res.success, true);
|
||||
assert.equal(res.mode, 'OPERATOR');
|
||||
assert.equal(res.decision, 'ACTION_COMPLETED');
|
||||
assert.equal(res.action.id, 'run-chat-tests');
|
||||
assert.equal(res.action_gate.outcome, 'ALLOW');
|
||||
assert.equal(res.agent_binding.agent_selected, 'ops-operator');
|
||||
assert.equal(res.agent_binding.skill_selected, 'registered-actions');
|
||||
assert.equal(res.audit_verify.ok, true);
|
||||
});
|
||||
});
|
||||
|
||||
test('chat ask denies selected agent outside actor role', () => {
|
||||
withTempChatState(() => {
|
||||
const svc = new ChatService();
|
||||
const agents = svc.listAgents(viewer) as any;
|
||||
assert.ok(agents.agents.some((a: any) => a.id === 'ops-operator' && a.allowed_for_role === false));
|
||||
|
||||
const res = svc.ask({ message: 'run tests', agentId: 'ops-operator' }, viewer) as any;
|
||||
assert.equal(res.success, false);
|
||||
assert.equal(res.decision, 'DENIED');
|
||||
assert.equal(res.agent_selected, 'ops-operator');
|
||||
assert.equal(res.reason, 'role_not_allowed_for_agent');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -108,6 +108,19 @@ export interface ChatAnswer {
|
||||
};
|
||||
action?: { id: string; label: string; description: string } | null;
|
||||
action_gate?: { outcome?: string; reason?: string; exit_code?: number };
|
||||
agent_binding?: {
|
||||
success: boolean;
|
||||
decision: string;
|
||||
reason: string;
|
||||
agent_selected: string;
|
||||
skill_selected: string;
|
||||
delegation_level: number;
|
||||
model_role?: string;
|
||||
mode?: string;
|
||||
tool_allowlist: string[];
|
||||
requires_approval: boolean;
|
||||
audit?: { record_hash?: string; head?: string; seq?: number };
|
||||
};
|
||||
actor: SettingsActor;
|
||||
}
|
||||
|
||||
@@ -118,6 +131,19 @@ export interface ChatAction {
|
||||
triggers: string[];
|
||||
}
|
||||
|
||||
export interface ChatAgent {
|
||||
id: string;
|
||||
label: string;
|
||||
mode: string;
|
||||
model_role: string;
|
||||
roles_allowed: string[];
|
||||
skills_allowed: string[];
|
||||
tool_allowlist: string[];
|
||||
max_delegation_level: number;
|
||||
requires_approval?: boolean;
|
||||
allowed_for_role?: boolean;
|
||||
}
|
||||
|
||||
export interface SettingsState {
|
||||
actor: SettingsActor;
|
||||
capabilities: {
|
||||
@@ -164,10 +190,11 @@ export const api = {
|
||||
post<{ proposal: any; audit_verify: { ok: boolean; output: string } }>('approvals/submit', body, actorHeaders(actor)),
|
||||
decideApproval: (actor: SettingsActor, body: { id: string; decision: 'approve' | 'reject'; reason: string }) =>
|
||||
post<{ proposal: any; applied: any; audit_verify: { ok: boolean; output: string } }>('approvals/decide', body, actorHeaders(actor)),
|
||||
askChat: (actor: SettingsActor, body: { message: string; chatId?: string }) =>
|
||||
askChat: (actor: SettingsActor, body: { message: string; chatId?: string; agentId?: string; skillId?: string; delegationLevel?: number }) =>
|
||||
post<ChatAnswer>('chat/ask', body, actorHeaders(actor)),
|
||||
verifyChatAudit: () => get<{ ok: boolean; output: string }>('chat/audit/verify'),
|
||||
chatActions: (actor: SettingsActor) => getWithHeaders<{ success: boolean; actions: ChatAction[] }>('chat/actions', actorHeaders(actor)),
|
||||
chatAgents: (actor: SettingsActor) => getWithHeaders<{ success: boolean; agents: ChatAgent[] }>('chat/agents', actorHeaders(actor)),
|
||||
};
|
||||
|
||||
// Health is raw (not enveloped) + carries HTTP status.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { useState } from 'react';
|
||||
import { useMutation, useQuery } from '@tanstack/react-query';
|
||||
import { api, ChatAction, ChatAnswer, SettingsActor } from '../lib/api';
|
||||
import { api, ChatAction, ChatAgent, ChatAnswer, SettingsActor } from '../lib/api';
|
||||
import { Card, StatusBadge } from '../components/ui/Card';
|
||||
|
||||
const ROLES = ['viewer', 'auditor', 'operator', 'project-admin', 'org-admin'];
|
||||
@@ -26,6 +26,9 @@ export function Chat() {
|
||||
tenant: 'default',
|
||||
});
|
||||
const [chatId, setChatId] = useState('chat-default');
|
||||
const [agentId, setAgentId] = useState('evidence-reader');
|
||||
const [skillId, setSkillId] = useState('evidence-summary');
|
||||
const [delegationLevel, setDelegationLevel] = useState(0);
|
||||
const [message, setMessage] = useState('Summarize Plan 18 MVP-0 status');
|
||||
const [last, setLast] = useState<ChatAnswer | null>(null);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
@@ -40,9 +43,25 @@ export function Chat() {
|
||||
queryFn: () => api.chatActions(actor),
|
||||
retry: false,
|
||||
});
|
||||
const agentsQuery = useQuery({
|
||||
queryKey: ['chat-agents', actor],
|
||||
queryFn: () => api.chatAgents(actor),
|
||||
retry: false,
|
||||
});
|
||||
const agents = agentsQuery.data?.agents ?? [];
|
||||
const selectedAgent = agents.find((a) => a.id === agentId) ?? agents.find((a) => a.allowed_for_role) ?? agents[0];
|
||||
const selectedSkill = selectedAgent?.skills_allowed.includes(skillId)
|
||||
? skillId
|
||||
: (selectedAgent?.skills_allowed[0] ?? '');
|
||||
|
||||
const ask = useMutation({
|
||||
mutationFn: (override?: { message?: string }) => api.askChat(actor, { message: override?.message ?? message, chatId }),
|
||||
mutationFn: (override?: { message?: string; agentId?: string; skillId?: string }) => api.askChat(actor, {
|
||||
message: override?.message ?? message,
|
||||
chatId,
|
||||
agentId: override?.agentId ?? selectedAgent?.id ?? agentId,
|
||||
skillId: override?.skillId ?? selectedSkill,
|
||||
delegationLevel,
|
||||
}),
|
||||
onSuccess: (res) => {
|
||||
setLast(res);
|
||||
setError(null);
|
||||
@@ -111,6 +130,33 @@ export function Chat() {
|
||||
<input className="w-full rounded border border-gray-300 px-3 py-2" value={chatId}
|
||||
onChange={(e) => setChatId(e.target.value)} />
|
||||
</label>
|
||||
<label className="space-y-1 md:col-span-2 xl:col-span-1">
|
||||
<span className="text-gray-500">Agent</span>
|
||||
<select className="w-full rounded border border-gray-300 px-3 py-2" value={selectedAgent?.id ?? agentId}
|
||||
onChange={(e) => {
|
||||
const next = agents.find((a) => a.id === e.target.value);
|
||||
setAgentId(e.target.value);
|
||||
setSkillId(next?.skills_allowed[0] ?? '');
|
||||
}}>
|
||||
{agents.map((a: ChatAgent) => (
|
||||
<option key={a.id} value={a.id}>
|
||||
{a.label}{a.allowed_for_role ? '' : ' (locked)'}
|
||||
</option>
|
||||
))}
|
||||
</select>
|
||||
</label>
|
||||
<label className="space-y-1">
|
||||
<span className="text-gray-500">Skill</span>
|
||||
<select className="w-full rounded border border-gray-300 px-3 py-2" value={selectedSkill}
|
||||
onChange={(e) => setSkillId(e.target.value)}>
|
||||
{(selectedAgent?.skills_allowed ?? []).map((s) => <option key={s} value={s}>{s}</option>)}
|
||||
</select>
|
||||
</label>
|
||||
<label className="space-y-1">
|
||||
<span className="text-gray-500">Delegation</span>
|
||||
<input className="w-full rounded border border-gray-300 px-3 py-2" type="number" min={0} max={5}
|
||||
value={delegationLevel} onChange={(e) => setDelegationLevel(Number(e.target.value || 0))} />
|
||||
</label>
|
||||
</div>
|
||||
</div>
|
||||
</Card>
|
||||
@@ -125,6 +171,8 @@ export function Chat() {
|
||||
<StatusBadge value={last.mode} />
|
||||
<StatusBadge value={last.risk} />
|
||||
<StatusBadge value={last.decision} />
|
||||
{last.agent_binding && <StatusBadge value={last.agent_binding.agent_selected} />}
|
||||
{last.agent_binding && <StatusBadge value={`L${last.agent_binding.delegation_level}`} />}
|
||||
<StatusBadge value={last.audit_verify.ok ? 'audit ok' : 'audit fail'} />
|
||||
</div>
|
||||
<div className="whitespace-pre-wrap text-sm leading-6 text-gray-800">{last.answer}</div>
|
||||
@@ -174,6 +222,18 @@ export function Chat() {
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
{last.agent_binding && (
|
||||
<div className="rounded border border-gray-200 p-3">
|
||||
<div className="text-xs font-semibold uppercase text-gray-400">Agent binding</div>
|
||||
<div className="mt-1 font-medium text-gray-800">{last.agent_binding.agent_selected}</div>
|
||||
<div className="mt-1 text-xs text-gray-500">{last.agent_binding.skill_selected || 'no skill'}</div>
|
||||
<div className="mt-2 flex flex-wrap gap-2">
|
||||
<StatusBadge value={last.agent_binding.decision} />
|
||||
<StatusBadge value={last.agent_binding.model_role ?? 'model'} />
|
||||
<StatusBadge value={`tools ${last.agent_binding.tool_allowlist.length}`} />
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
<div>
|
||||
<div className="text-xs font-semibold uppercase text-gray-400">Matched rules</div>
|
||||
<div className="mt-1 flex flex-wrap gap-2">
|
||||
@@ -202,7 +262,7 @@ export function Chat() {
|
||||
disabled={ask.isPending}
|
||||
onClick={() => {
|
||||
setMessage(trigger);
|
||||
ask.mutate({ message: trigger });
|
||||
ask.mutate({ message: trigger, agentId: 'ops-operator', skillId: 'registered-actions' });
|
||||
}}
|
||||
className="text-left rounded border border-gray-200 p-3 hover:border-blue-300 hover:bg-blue-50 disabled:opacity-50"
|
||||
>
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
{
|
||||
"version": 1,
|
||||
"agents": [
|
||||
{
|
||||
"id": "evidence-reader",
|
||||
"label": "Evidence Reader",
|
||||
"mode": "READ_ONLY",
|
||||
"model_role": "read_only",
|
||||
"roles_allowed": ["viewer", "auditor", "operator", "project-admin", "org-admin"],
|
||||
"skills_allowed": ["evidence-summary"],
|
||||
"tool_allowlist": [],
|
||||
"max_delegation_level": 0
|
||||
},
|
||||
{
|
||||
"id": "ops-operator",
|
||||
"label": "Ops Operator",
|
||||
"mode": "OPERATOR",
|
||||
"model_role": "operator",
|
||||
"roles_allowed": ["operator", "project-admin", "org-admin"],
|
||||
"skills_allowed": ["registered-actions"],
|
||||
"tool_allowlist": ["run-chat-tests", "build-evidence-pack", "verify-evidence-pack"],
|
||||
"max_delegation_level": 1
|
||||
},
|
||||
{
|
||||
"id": "codegen-draft",
|
||||
"label": "Codegen Draft",
|
||||
"mode": "CODEGEN",
|
||||
"model_role": "codegen",
|
||||
"roles_allowed": ["project-admin", "org-admin"],
|
||||
"skills_allowed": ["sourcegen-draft"],
|
||||
"tool_allowlist": ["artifact-scan", "tool-output-scan"],
|
||||
"max_delegation_level": 2,
|
||||
"requires_approval": true
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,268 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Plan-18 MVP-2 agent/skill governance resolver.
|
||||
|
||||
This is a harness-owned policy primitive. It binds a chat turn to an agent,
|
||||
skill, delegation level, model role, and tool allowlist before any runtime is
|
||||
invoked. Unknown policy, RBAC failure, role mismatch, delegation escalation, or
|
||||
tool outside the agent allowlist all fail closed.
|
||||
"""
|
||||
import argparse
|
||||
import datetime
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
|
||||
def project_root() -> str:
|
||||
d = os.path.abspath(os.path.dirname(__file__))
|
||||
p = d
|
||||
while p != os.path.dirname(p):
|
||||
if os.path.isdir(os.path.join(p, ".specify")) or os.path.isdir(os.path.join(p, "packages/casan-harness")):
|
||||
return p
|
||||
p = os.path.dirname(p)
|
||||
return os.path.abspath(os.path.join(d, "..", "..", ".."))
|
||||
|
||||
|
||||
ROOT = project_root()
|
||||
HARNESS_ROOT = os.path.join(ROOT, "packages", "casan-harness")
|
||||
HARNESS_BIN = os.path.join(HARNESS_ROOT, "scripts", "bash")
|
||||
RBAC = os.path.join(HARNESS_BIN, "rbac-check.py")
|
||||
GENESIS_HASH = "0" * 64
|
||||
|
||||
|
||||
def now_iso() -> str:
|
||||
return datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
|
||||
|
||||
def sha(text: str) -> str:
|
||||
return hashlib.sha256(text.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def state_root() -> str:
|
||||
return os.environ.get("CASAN_STATE_ROOT") or os.path.join(ROOT, ".specify")
|
||||
|
||||
|
||||
def registry_path() -> str:
|
||||
return os.environ.get("CASAN_AGENT_REGISTRY_FILE") or os.path.join(HARNESS_ROOT, "config", "agent-registry.yaml")
|
||||
|
||||
|
||||
def audit_path() -> str:
|
||||
return os.environ.get("CASAN_AGENT_BIND_AUDIT_LOG") or os.path.join(state_root(), "logs", "chat", "agent-bindings.jsonl")
|
||||
|
||||
|
||||
def head_path() -> str:
|
||||
return os.environ.get("CASAN_AGENT_BIND_AUDIT_HEAD") or os.path.join(state_root(), "logs", "chat", "agent-bindings-head.txt")
|
||||
|
||||
|
||||
def load_registry():
|
||||
try:
|
||||
with open(registry_path(), encoding="utf-8") as fh:
|
||||
data = json.load(fh)
|
||||
agents = data.get("agents")
|
||||
if not isinstance(agents, list):
|
||||
raise ValueError("agents_not_list")
|
||||
return data
|
||||
except Exception as exc:
|
||||
return {"_error": f"agent_registry_unreadable:{exc}", "agents": []}
|
||||
|
||||
|
||||
def find_agent(registry, agent_id: str):
|
||||
return next((a for a in registry.get("agents", []) if a.get("id") == agent_id), None)
|
||||
|
||||
|
||||
def load_head() -> str:
|
||||
try:
|
||||
return open(head_path(), encoding="utf-8").read().strip() or GENESIS_HASH
|
||||
except OSError:
|
||||
return GENESIS_HASH
|
||||
|
||||
|
||||
def append_audit(base):
|
||||
path = audit_path()
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
seq = 1
|
||||
if os.path.isfile(path):
|
||||
with open(path, encoding="utf-8") as fh:
|
||||
seq = sum(1 for line in fh if line.strip()) + 1
|
||||
prev = load_head()
|
||||
core = {"seq": seq, **base, "prev_hash": prev}
|
||||
record_hash = sha(json.dumps(core, sort_keys=True, ensure_ascii=False))
|
||||
rec = {**core, "record_hash": record_hash}
|
||||
with open(path, "a", encoding="utf-8") as fh:
|
||||
fh.write(json.dumps(rec, ensure_ascii=False) + "\n")
|
||||
os.makedirs(os.path.dirname(head_path()), exist_ok=True)
|
||||
with open(head_path(), "w", encoding="utf-8") as fh:
|
||||
fh.write(record_hash + "\n")
|
||||
return rec
|
||||
|
||||
|
||||
def decision_payload(args, decision: str, reason: str, agent=None, skill="", tools=None):
|
||||
tools = tools or []
|
||||
success = decision == "BOUND"
|
||||
payload = {
|
||||
"success": success,
|
||||
"decision": decision,
|
||||
"reason": reason,
|
||||
"agent_selected": (agent or {}).get("id") or args.agent,
|
||||
"skill_selected": skill or "",
|
||||
"delegation_level": args.delegation_level,
|
||||
"model_role": (agent or {}).get("model_role"),
|
||||
"mode": (agent or {}).get("mode"),
|
||||
"tool_allowlist": (agent or {}).get("tool_allowlist", []),
|
||||
"requested_tools": tools,
|
||||
"side_effect_allowed": bool((agent or {}).get("tool_allowlist")),
|
||||
"requires_approval": bool((agent or {}).get("requires_approval")) or decision == "REQUIRES_APPROVAL",
|
||||
}
|
||||
rec = append_audit({
|
||||
"timestamp": now_iso(),
|
||||
"harness": "H5-chat-agent-bind",
|
||||
"actor": args.actor,
|
||||
"role": args.role,
|
||||
"project": args.project,
|
||||
"tenant": args.tenant,
|
||||
"agent_selected": payload["agent_selected"],
|
||||
"skill_selected": payload["skill_selected"],
|
||||
"delegation_level": args.delegation_level,
|
||||
"decision": decision,
|
||||
"reason": reason,
|
||||
"tool_allowlist_ref": sha(json.dumps(payload["tool_allowlist"], sort_keys=True)),
|
||||
"requested_tools_ref": sha(json.dumps(tools, sort_keys=True)),
|
||||
})
|
||||
payload["audit"] = {"seq": rec["seq"], "record_hash": rec["record_hash"], "head": rec["record_hash"]}
|
||||
return payload
|
||||
|
||||
|
||||
def run_rbac(args):
|
||||
r = subprocess.run([
|
||||
"python3", RBAC, "check",
|
||||
"--role", args.role,
|
||||
"--resource", "chat",
|
||||
"--action", "select_agent",
|
||||
"--role-project", args.project,
|
||||
"--target-project", args.target_project or args.project,
|
||||
"--role-tenant", args.tenant,
|
||||
"--target-tenant", args.target_tenant or args.tenant,
|
||||
], cwd=ROOT, capture_output=True, text=True)
|
||||
return r.returncode, (r.stdout + r.stderr).strip()
|
||||
|
||||
|
||||
def bind(args) -> int:
|
||||
registry = load_registry()
|
||||
if registry.get("_error"):
|
||||
out = decision_payload(args, "DENIED", registry["_error"])
|
||||
print(json.dumps(out, ensure_ascii=False))
|
||||
return 2
|
||||
|
||||
agent = find_agent(registry, args.agent)
|
||||
if not agent:
|
||||
out = decision_payload(args, "DENIED", "unknown_agent")
|
||||
print(json.dumps(out, ensure_ascii=False))
|
||||
return 2
|
||||
|
||||
rc, msg = run_rbac(args)
|
||||
if rc != 0:
|
||||
out = decision_payload(args, "DENIED", f"rbac_denied:{msg}", agent)
|
||||
print(json.dumps(out, ensure_ascii=False))
|
||||
return 2
|
||||
|
||||
if args.role not in agent.get("roles_allowed", []):
|
||||
out = decision_payload(args, "DENIED", "role_not_allowed_for_agent", agent)
|
||||
print(json.dumps(out, ensure_ascii=False))
|
||||
return 2
|
||||
|
||||
skill = args.skill or (agent.get("skills_allowed", [""])[0] if agent.get("skills_allowed") else "")
|
||||
if skill and skill not in agent.get("skills_allowed", []):
|
||||
out = decision_payload(args, "DENIED", "skill_not_allowed_for_agent", agent, skill)
|
||||
print(json.dumps(out, ensure_ascii=False))
|
||||
return 2
|
||||
|
||||
max_level = int(agent.get("max_delegation_level", 0))
|
||||
if args.delegation_level > max_level:
|
||||
out = decision_payload(args, "REQUIRES_APPROVAL", "delegation_exceeds_agent_max", agent, skill)
|
||||
print(json.dumps(out, ensure_ascii=False))
|
||||
return 3
|
||||
|
||||
requested_tools = list(args.tool or [])
|
||||
allow = set(agent.get("tool_allowlist", []))
|
||||
outside = [t for t in requested_tools if t not in allow]
|
||||
if outside:
|
||||
out = decision_payload(args, "DENIED", "tool_not_allowlisted", agent, skill, requested_tools)
|
||||
out["action_gate"] = {"outcome": "BLOCK", "reason": f"tool_not_allowlisted:{','.join(outside)}"}
|
||||
print(json.dumps(out, ensure_ascii=False))
|
||||
return 2
|
||||
|
||||
out = decision_payload(args, "BOUND", "agent_bound", agent, skill, requested_tools)
|
||||
print(json.dumps(out, ensure_ascii=False))
|
||||
return 0
|
||||
|
||||
|
||||
def list_agents(args) -> int:
|
||||
registry = load_registry()
|
||||
if registry.get("_error"):
|
||||
print(json.dumps({"success": False, "error": registry["_error"], "agents": []}, ensure_ascii=False))
|
||||
return 2
|
||||
agents = []
|
||||
for agent in registry.get("agents", []):
|
||||
item = {k: agent.get(k) for k in (
|
||||
"id", "label", "mode", "model_role", "roles_allowed", "skills_allowed",
|
||||
"tool_allowlist", "max_delegation_level", "requires_approval",
|
||||
)}
|
||||
item["allowed_for_role"] = args.role in agent.get("roles_allowed", []) if args.role else None
|
||||
agents.append(item)
|
||||
print(json.dumps({"success": True, "version": registry.get("version"), "agents": agents}, ensure_ascii=False))
|
||||
return 0
|
||||
|
||||
|
||||
def verify_audit() -> int:
|
||||
prev = GENESIS_HASH
|
||||
count = 0
|
||||
try:
|
||||
fh = open(audit_path(), encoding="utf-8")
|
||||
except OSError:
|
||||
print("CHAT_AGENT_AUDIT ok=true records=0 head=" + prev)
|
||||
return 0
|
||||
with fh:
|
||||
for line in fh:
|
||||
if not line.strip():
|
||||
continue
|
||||
count += 1
|
||||
rec = json.loads(line)
|
||||
got = rec.get("record_hash")
|
||||
rest = {k: v for k, v in rec.items() if k != "record_hash"}
|
||||
if rest.get("prev_hash") != prev or sha(json.dumps(rest, sort_keys=True, ensure_ascii=False)) != got:
|
||||
print(f"CHAT_AGENT_AUDIT ok=false brokenAt={count}")
|
||||
return 1
|
||||
prev = got
|
||||
print(f"CHAT_AGENT_AUDIT ok=true records={count} head={prev}")
|
||||
return 0
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser()
|
||||
sub = ap.add_subparsers(dest="cmd", required=True)
|
||||
b = sub.add_parser("bind")
|
||||
b.add_argument("--agent", required=True)
|
||||
b.add_argument("--skill", default="")
|
||||
b.add_argument("--actor", default="anonymous")
|
||||
b.add_argument("--role", default="viewer")
|
||||
b.add_argument("--project", default="default")
|
||||
b.add_argument("--tenant", default="default")
|
||||
b.add_argument("--target-project", default="")
|
||||
b.add_argument("--target-tenant", default="")
|
||||
b.add_argument("--delegation-level", type=int, default=0)
|
||||
b.add_argument("--tool", action="append", default=[])
|
||||
b.set_defaults(func=bind)
|
||||
|
||||
l = sub.add_parser("list-agents")
|
||||
l.add_argument("--role", default="")
|
||||
l.set_defaults(func=list_agents)
|
||||
|
||||
sub.add_parser("verify-audit").set_defaults(func=lambda _args: verify_audit())
|
||||
args = ap.parse_args()
|
||||
return args.func(args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -24,6 +24,7 @@ def project_root() -> str:
|
||||
ROOT = project_root()
|
||||
BIN = os.path.join(ROOT, "packages", "casan-harness", "scripts", "bash")
|
||||
ROUTER = os.path.join(BIN, "prompt-mode-router.py")
|
||||
AGENT_RESOLVER = os.path.join(BIN, "chat-agent-resolver.py")
|
||||
READONLY = os.path.join(BIN, "chat-readonly.py")
|
||||
OPERATOR = os.path.join(BIN, "chat-operator.py")
|
||||
|
||||
@@ -36,13 +37,69 @@ def classify(message: str):
|
||||
return {"mode": "BLOCK", "risk": "high", "reason": "router_invalid_json", "matched_rules": [r.stderr.strip()]}
|
||||
|
||||
|
||||
def run_mode(args, binding):
|
||||
r = subprocess.run(args, cwd=ROOT, capture_output=True, text=True)
|
||||
try:
|
||||
payload = json.loads(r.stdout)
|
||||
payload["agent_binding"] = binding
|
||||
print(json.dumps(payload, ensure_ascii=False))
|
||||
except Exception:
|
||||
if r.stdout:
|
||||
sys.stdout.write(r.stdout)
|
||||
if r.stderr:
|
||||
sys.stderr.write(r.stderr)
|
||||
return r.returncode
|
||||
|
||||
|
||||
def run_and_passthrough(args):
|
||||
r = subprocess.run(args, cwd=ROOT, text=True)
|
||||
return r.returncode
|
||||
|
||||
|
||||
def default_agent_for_mode(mode: str) -> str:
|
||||
if mode == "OPERATOR":
|
||||
return "ops-operator"
|
||||
if mode == "CODEGEN":
|
||||
return "codegen-draft"
|
||||
return "evidence-reader"
|
||||
|
||||
|
||||
def bind_agent(args, router):
|
||||
agent = args.agent or default_agent_for_mode(router.get("mode", "READ_ONLY"))
|
||||
tools = []
|
||||
if router.get("mode") == "OPERATOR":
|
||||
# The operator primitive resolves the exact registered action later; the
|
||||
# agent bind still proves this turn is allowed to use the operator class.
|
||||
tools.append("run-chat-tests")
|
||||
cmd = [
|
||||
"python3", AGENT_RESOLVER, "bind",
|
||||
"--agent", agent,
|
||||
"--actor", args.actor,
|
||||
"--role", args.role,
|
||||
"--project", args.project,
|
||||
"--tenant", args.tenant,
|
||||
"--delegation-level", str(args.delegation_level),
|
||||
]
|
||||
if args.skill:
|
||||
cmd += ["--skill", args.skill]
|
||||
for tool in tools:
|
||||
cmd += ["--tool", tool]
|
||||
r = subprocess.run(cmd, cwd=ROOT, capture_output=True, text=True)
|
||||
if r.returncode != 0:
|
||||
sys.stdout.write((r.stdout or r.stderr).strip() + "\n")
|
||||
return r.returncode, None
|
||||
try:
|
||||
return 0, json.loads(r.stdout)
|
||||
except Exception:
|
||||
print(json.dumps({"success": False, "decision": "DENIED", "reason": "agent_resolver_invalid_json"}, ensure_ascii=False))
|
||||
return 2, None
|
||||
|
||||
|
||||
def ask(args) -> int:
|
||||
router = classify(args.message)
|
||||
bind_rc, binding = bind_agent(args, router)
|
||||
if bind_rc != 0:
|
||||
return bind_rc
|
||||
common = [
|
||||
"--message", args.message,
|
||||
"--actor", args.actor,
|
||||
@@ -51,8 +108,8 @@ def ask(args) -> int:
|
||||
"--tenant", args.tenant,
|
||||
]
|
||||
if router.get("mode") == "OPERATOR":
|
||||
return run_and_passthrough(["python3", OPERATOR, "run", *common])
|
||||
return run_and_passthrough(["python3", READONLY, "ask", *common])
|
||||
return run_mode(["python3", OPERATOR, "run", *common], binding)
|
||||
return run_mode(["python3", READONLY, "ask", *common], binding)
|
||||
|
||||
|
||||
def verify_audit() -> int:
|
||||
@@ -65,9 +122,14 @@ def main() -> int:
|
||||
askp = sub.add_parser("ask")
|
||||
askp.add_argument("--message", required=True)
|
||||
askp.add_argument("--actor", default="anonymous")
|
||||
askp.add_argument("--role", default="viewer")
|
||||
askp.add_argument("--project", default="default")
|
||||
askp.add_argument("--chat-id", default="")
|
||||
askp.add_argument("--turn-id", default="")
|
||||
askp.add_argument("--tenant", default="default")
|
||||
askp.add_argument("--agent", default="")
|
||||
askp.add_argument("--skill", default="")
|
||||
askp.add_argument("--delegation-level", type=int, default=0)
|
||||
askp.set_defaults(func=ask)
|
||||
sub.add_parser("verify-audit").set_defaults(func=lambda _args: verify_audit())
|
||||
args = ap.parse_args()
|
||||
|
||||
@@ -149,6 +149,7 @@ run "phase-chat-prompt-router" bash "$TESTS/phase-chat-prompt-router-tests.sh"
|
||||
run "phase-chat-readonly" bash "$TESTS/phase-chat-readonly-tests.sh"
|
||||
run "phase-chat-session-audit" bash "$TESTS/phase-chat-session-audit-tests.sh"
|
||||
run "phase-chat-operator" bash "$TESTS/phase-chat-operator-tests.sh"
|
||||
run "phase-chat-agent-select" bash "$TESTS/phase-chat-agent-select-tests.sh"
|
||||
|
||||
# ARCH-02: coverage cannot silently drop; ARCH-01: harness/policy cannot silently
|
||||
# drift. Both SKIP cleanly when no manifest is provisioned (non-strict dev/CI).
|
||||
|
||||
@@ -17,11 +17,11 @@ import sys
|
||||
# scope: "org" (all projects) or "project" (must match the acted-on project)
|
||||
PERMISSIONS = {
|
||||
"org-admin": {"scope": "org", "allow": {"*"}},
|
||||
"project-admin": {"scope": "project", "allow": {"settings:read", "settings:write", "monitoring:read", "audit:read"}},
|
||||
"approver": {"scope": "project", "allow": {"settings:read", "monitoring:read", "approval:grant"}},
|
||||
"operator": {"scope": "project", "allow": {"monitoring:read", "kill_switch:engage"}},
|
||||
"viewer": {"scope": "project", "allow": {"settings:read", "monitoring:read"}},
|
||||
"auditor": {"scope": "org", "allow": {"settings:read", "monitoring:read", "audit:read"}},
|
||||
"project-admin": {"scope": "project", "allow": {"settings:read", "settings:write", "monitoring:read", "audit:read", "chat:select_agent"}},
|
||||
"approver": {"scope": "project", "allow": {"settings:read", "monitoring:read", "approval:grant", "chat:select_agent"}},
|
||||
"operator": {"scope": "project", "allow": {"monitoring:read", "kill_switch:engage", "chat:select_agent"}},
|
||||
"viewer": {"scope": "project", "allow": {"settings:read", "monitoring:read", "chat:select_agent"}},
|
||||
"auditor": {"scope": "org", "allow": {"settings:read", "monitoring:read", "audit:read", "chat:select_agent"}},
|
||||
}
|
||||
|
||||
# Maps an IdP-issued claim (role name / group) to an RBAC role. The IdP (Plan-07
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
#!/usr/bin/env bash
|
||||
set -uo pipefail
|
||||
|
||||
# Plan-18 MVP-2 Track 4: agent/skill selection is governed by registry + RBAC.
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
source "$SCRIPT_DIR/../scripts/bash/casan-paths.sh"
|
||||
RESOLVER="$CASAN_HARNESS_ROOT/scripts/bash/chat-agent-resolver.py"
|
||||
TURN="$CASAN_HARNESS_ROOT/scripts/bash/chat-turn.py"
|
||||
WORK="$(mktemp -d)"
|
||||
trap 'rm -rf "$WORK"' EXIT
|
||||
export CASAN_STATE_ROOT="$WORK/state"
|
||||
|
||||
PASS=0; FAIL=0
|
||||
pass() { echo "PASS: $1"; PASS=$((PASS + 1)); }
|
||||
fail() { echo "FAIL: $1"; FAIL=$((FAIL + 1)); }
|
||||
|
||||
echo "===== Plan-18 MVP-2 agent/skill selection ====="
|
||||
|
||||
python3 "$RESOLVER" list-agents --role viewer > "$WORK/list.json"
|
||||
python3 - "$WORK/list.json" <<'PY' \
|
||||
&& pass "agent registry lists versioned agents with role visibility" || fail "agent registry listing invalid"
|
||||
import json, sys
|
||||
d = json.load(open(sys.argv[1]))
|
||||
ids = {a["id"]: a for a in d["agents"]}
|
||||
assert d["success"] is True
|
||||
assert {"evidence-reader", "ops-operator", "codegen-draft"}.issubset(ids)
|
||||
assert ids["evidence-reader"]["allowed_for_role"] is True
|
||||
assert ids["ops-operator"]["allowed_for_role"] is False
|
||||
PY
|
||||
|
||||
python3 "$RESOLVER" bind --agent evidence-reader --skill evidence-summary --role viewer --actor alice --project default --tenant tenant-a > "$WORK/viewer.json"
|
||||
python3 - "$WORK/viewer.json" <<'PY' \
|
||||
&& pass "viewer can bind read-only evidence-reader" || fail "viewer could not bind read-only agent"
|
||||
import json, sys
|
||||
d = json.load(open(sys.argv[1]))
|
||||
assert d["success"] is True
|
||||
assert d["decision"] == "BOUND"
|
||||
assert d["agent_selected"] == "evidence-reader"
|
||||
assert d["delegation_level"] == 0
|
||||
assert d["tool_allowlist"] == []
|
||||
PY
|
||||
|
||||
set +e
|
||||
python3 "$RESOLVER" bind --agent ops-operator --role viewer --actor alice --project default --tenant tenant-a > "$WORK/viewer-op.json"
|
||||
RC=$?
|
||||
set -e 2>/dev/null || true
|
||||
python3 - "$WORK/viewer-op.json" "$RC" <<'PY' \
|
||||
&& pass "viewer cannot select operator agent" || fail "viewer selected operator agent"
|
||||
import json, sys
|
||||
d = json.load(open(sys.argv[1]))
|
||||
assert int(sys.argv[2]) == 2
|
||||
assert d["success"] is False
|
||||
assert d["decision"] == "DENIED"
|
||||
assert d["reason"] == "role_not_allowed_for_agent"
|
||||
PY
|
||||
|
||||
set +e
|
||||
python3 "$RESOLVER" bind --agent evidence-reader --role viewer --actor alice --project default --tenant tenant-a --target-tenant tenant-b > "$WORK/cross-tenant.json"
|
||||
RC=$?
|
||||
set -e 2>/dev/null || true
|
||||
python3 - "$WORK/cross-tenant.json" "$RC" <<'PY' \
|
||||
&& pass "cross-tenant agent binding is denied by RBAC" || fail "cross-tenant agent binding was allowed"
|
||||
import json, sys
|
||||
d = json.load(open(sys.argv[1]))
|
||||
assert int(sys.argv[2]) == 2
|
||||
assert d["success"] is False
|
||||
assert d["decision"] == "DENIED"
|
||||
assert "rbac_denied" in d["reason"]
|
||||
PY
|
||||
|
||||
set +e
|
||||
python3 "$RESOLVER" bind --agent evidence-reader --role viewer --actor alice --project default --tenant tenant-a --delegation-level 1 > "$WORK/delegation.json"
|
||||
RC=$?
|
||||
set -e 2>/dev/null || true
|
||||
python3 - "$WORK/delegation.json" "$RC" <<'PY' \
|
||||
&& pass "delegation above agent max requires approval" || fail "delegation escalation was not held"
|
||||
import json, sys
|
||||
d = json.load(open(sys.argv[1]))
|
||||
assert int(sys.argv[2]) == 3
|
||||
assert d["success"] is False
|
||||
assert d["decision"] == "REQUIRES_APPROVAL"
|
||||
assert d["requires_approval"] is True
|
||||
PY
|
||||
|
||||
set +e
|
||||
python3 "$RESOLVER" bind --agent ops-operator --role operator --actor bob --project default --tenant tenant-a --tool shell > "$WORK/tool.json"
|
||||
RC=$?
|
||||
set -e 2>/dev/null || true
|
||||
python3 - "$WORK/tool.json" "$RC" <<'PY' \
|
||||
&& pass "tool outside agent allowlist is blocked" || fail "tool outside allowlist was accepted"
|
||||
import json, sys
|
||||
d = json.load(open(sys.argv[1]))
|
||||
assert int(sys.argv[2]) == 2
|
||||
assert d["success"] is False
|
||||
assert d["decision"] == "DENIED"
|
||||
assert d["action_gate"]["outcome"] == "BLOCK"
|
||||
PY
|
||||
|
||||
set +e
|
||||
python3 "$TURN" ask --message "run tests" --agent ops-operator --role viewer --actor alice --chat-id agent-chat --tenant tenant-a > "$WORK/turn-deny.json"
|
||||
RC=$?
|
||||
set -e 2>/dev/null || true
|
||||
python3 - "$WORK/turn-deny.json" "$RC" <<'PY' \
|
||||
&& pass "chat-turn denies unauthorized selected agent before operator runtime" || fail "chat-turn bypassed agent resolver"
|
||||
import json, sys
|
||||
d = json.load(open(sys.argv[1]))
|
||||
assert int(sys.argv[2]) == 2
|
||||
assert d["success"] is False
|
||||
assert d["decision"] == "DENIED"
|
||||
assert d["agent_selected"] == "ops-operator"
|
||||
PY
|
||||
|
||||
python3 "$RESOLVER" verify-audit > "$WORK/audit.out" \
|
||||
&& grep -q "CHAT_AGENT_AUDIT ok=true" "$WORK/audit.out" \
|
||||
&& pass "agent binding audit hash chain verifies" || fail "agent binding audit invalid"
|
||||
|
||||
echo ""
|
||||
echo "===== CHAT AGENT SUMMARY: PASS=$PASS FAIL=$FAIL ====="
|
||||
[[ "$FAIL" -eq 0 ]] || exit 1
|
||||
@@ -37,7 +37,7 @@ ids = {a["id"] for a in d["actions"]}
|
||||
assert {"run-chat-tests", "build-evidence-pack", "verify-evidence-pack"}.issubset(ids)
|
||||
PY
|
||||
|
||||
python3 "$TURN" ask --message "run tests" --actor bob --chat-id op1 > "$WORK/run.json"
|
||||
python3 "$TURN" ask --message "run tests" --actor bob --role operator --chat-id op1 > "$WORK/run.json"
|
||||
python3 - "$WORK/run.json" <<'PY' \
|
||||
&& pass "registered run tests action executes through chat-turn" || fail "registered action did not complete"
|
||||
import json, os, sys
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
# CASAN Platform (Level 3 — Productization UI) · **PREVIEW**
|
||||
|
||||
> Status: **PREVIEW.** The AgentOps dashboard and Plan-13 Control Panel, including
|
||||
> Command Center baseline and Plan-18 MVP-0/1 Chat Console, exist today.
|
||||
> Command Center baseline and Plan-18 MVP-0/1 Chat Console plus MVP-2 agent
|
||||
> selection foundation, exist today.
|
||||
> Evidence/attack viewers, Gitea integration, and managed production rollout remain
|
||||
> planned. `package-release.sh platform` builds a clearly-stamped
|
||||
> `casan-platform-preview-*` bundle containing only what exists.
|
||||
@@ -13,12 +14,12 @@ Optional layer for teams that want UI / dashboard / visibility. Packages: `casan
|
||||
| Component | Status | Where |
|
||||
|---|---|---|
|
||||
| AgentOps Dashboard | ✅ exists | `packages/casan-harness/scripts/bash/dashboard-server.py` + `dashboard-serve.sh` (`casan dashboard`) |
|
||||
| **Ops Console (Control Panel)** | ✅ **monitoring + governed settings + HITL + kill-switch + FinOps/SLO + Command Center + Chat MVP-0/1 + local-prod TLS/OIDC smoke** | `packages/casan-control-panel/` — NestJS API + React UI over harness telemetry, settings management, approval inbox, kill-switch, FinOps/SLO, Command Center, Ask CASAN read-only, and registered Operator actions (`npm run console:api` + `console:ui`). Includes Run History + verdicts + governance + security + incidents + traceability + FinOps + Approvals + role-aware Settings page + `/command` evidence-backed executive view + `/chat` governed read-only/operator console. |
|
||||
| **Ops Console (Control Panel)** | ✅ **monitoring + governed settings + HITL + kill-switch + FinOps/SLO + Command Center + Chat MVP-0/1 + agent selection foundation + local-prod TLS/OIDC smoke** | `packages/casan-control-panel/` — NestJS API + React UI over harness telemetry, settings management, approval inbox, kill-switch, FinOps/SLO, Command Center, Ask CASAN read-only, registered Operator actions, and governed agent/skill selection (`npm run console:api` + `console:ui`). Includes Run History + verdicts + governance + security + incidents + traceability + FinOps + Approvals + role-aware Settings page + `/command` evidence-backed executive view + `/chat` governed read-only/operator console. |
|
||||
| Management / settings writes | ✅ done+test | Plan-13 Track 2: wraps `control-plane-settings.py`, calls `rbac-check.py`, supports set/rollback/audit verify. |
|
||||
| RBAC + approval inbox | ✅ local-prod done | Settings + kill-switch API RBAC enforcement, approval inbox/delegation/oversight, SoD, governed setting proposal apply, and local OIDC claim→role mapping smoke are done. Enterprise IdP rollout remains production follow-up. |
|
||||
| Evidence Pack Viewer | 📋 planned | reads `docs/output/casan/evidence-packs/` |
|
||||
| Attack Battery Viewer | 📋 planned | reads red-team corpus + H4 recall results |
|
||||
| Read-only Ask CASAN + Operator | ✅ MVP-0/1 done+test | Plan-18 MVP-0/1: `POST /api/v1/chat/ask`, `GET /api/v1/chat/actions` + `/chat`, backed by harness `chat-turn.py` |
|
||||
| Read-only Ask CASAN + Operator + agent selection | 🟡 MVP-0/1 done+test; MVP-2 Track 4 foundation done | Plan-18: `POST /api/v1/chat/ask`, `GET /api/v1/chat/actions`, `GET /api/v1/chat/agents` + `/chat`, backed by harness `chat-turn.py` and `chat-agent-resolver.py` |
|
||||
| Gitea webhook integration | 📋 planned | trigger gate / publish evidence on push |
|
||||
|
||||
## Build (preview)
|
||||
@@ -28,6 +29,6 @@ scripts/package-release.sh platform # → dist/casan-platform-preview-vX.Y.Z
|
||||
The bundle includes a `PREVIEW-INCOMPLETE.txt` marker. Do not treat it as a finished product.
|
||||
|
||||
## To implement later
|
||||
Start from Plan-15 RAI view or Plan-18 MVP-2 (Chat-as-loop + agent/skill). Keep new numbers
|
||||
Start from Plan-15 RAI view or Plan-18 MVP-2 Track 5/6 (Chat-as-loop + streaming). Keep new numbers
|
||||
evidence-backed with provenance; any write/governed action must continue to route through
|
||||
harness RBAC, approval, and audit primitives.
|
||||
|
||||
Reference in New Issue
Block a user