AI coverage: what the MCP server and the Site agent can do, and what is missing

November 12, 2026. The goal: an AI connected over MCP, or the Site agent in the admin, can run a site end to end: page layouts, media, menus, content types, components, settings, kits and modules. This compares what the two AI surfaces can do with what the admin and the REST API can do, proposes the missing tools and their guardrails, and orders the work in phases. Paths are relative to the repository root; line numbers are as of this writing.

Summary

• MCP server (POST /mcp, packages/core/src/routes/mcp.ts:28): 22 tools (20 before #128), 2 prompts, 2 resources plus 2 resource templates. Every tool lives in one array, TOOLS (packages/core/src/mcp/tools.ts:133); tools/list returns all of them to every caller regardless of permission (mcp/server.ts:49, :90), and each handler checks permissions itself with require(cx, ...) (mcp/tools.ts:46). The token's permissions are the token's list intersected with the user's role permissions (auth/bearer.ts:12); 120 calls per minute per token (shared/src/index.ts:1627); every call is audited as mcp.tool_call (mcp/server.ts:68).
• Site agent and Ask from anywhere: one pipeline. The Ask panel is the site agent's chat (packages/admin/src/shell/AskPanel.tsx:30, useAgentChat('site')). The agent reuses the MCP tool objects (agents/tools.ts:12, fromMcp at :205, ALL at :228) and adds 6 of its own (propose_task, list_tasks, open_screen, search_manual, get_analytics, audit_site; agents/tools.ts:101). That makes 28 tools (26 before #128), handed out per "allowed action" through ACTION_TOOLS (agents/tools.ts:22). The agent's role tops out at Manager (AgentRole, shared/src/index.ts:1339), and a chat run gets the lesser of the agent's and the person's permissions (agents/run.ts:51). The system prompt says outright: "You cannot change site settings, install modules, kits or themes, or manage users and tasks yet" (agents/run.ts:246).
• AI tasks (packages/core/src/ai/tasks.ts): pageFromBrief (:33), assist (:72), typeFromDescription (:83), altText (:183), summarize (:214) and translateEntityData (ai/translate.ts:29). MCP exposes only the first two (draft_page_from_brief, assist_text).
• Module AI tools are never called. Modules can register them (modules/module-api.ts:49, manifest aiTools, modules/manifest.ts:131). The only thing that reads registry.aiTools is the contributions listing (routes/modules.ts:62).
• What it amounts to: the REST API has 443 route handlers in 64 files (225 of them mutating), and docs/reference/openapi.json documents 452 operations. The AI surfaces cover one domain well: entities. You can create, update, submit, publish, schedule and unpublish, check, list revisions and comment. Everything else is read-only (content types, collections, media list, site settings, components, review queue) or not covered at all. The AI cannot touch menus, media uploads, taxonomy, content types, forms, redirects, brand/theme, kits, modules, users, releases, alerts, personalization, CDN, imports or backups.

Fixed in kwall-llc/spillway-cms#128

• schedule_entity and unpublish_entity for MCP and the Site agent (Publish action), list_entities with status: scheduled, and publishAt, unpublishAt and lastScheduleError in entity summaries. Both new tools are audited (CHANGING_TOOLS).
• Publishing content dated in the future is refused with future_date unless the caller says publishNow.
• get_site_settings and spillway://site mask the CAPTCHA secret (they returned it to any token).
• update_entity refuses with lock_held while someone else is editing the item.

Coverage by area

"MCP" and "Agent" list today's tools. A dash means nothing. RO means read-only.

Proposed tools

Names follow mcp/tools.ts: verb_noun, snake case, list_/get_ for reads.

Legend: RO = read-only, M = mutating, D = destructive, HR = high-risk (needs a guardrail from the next section).

Content (11)

• delete_entity: move to trash, or delete; content.delete; D, HR.
• archive_entity: take it off the site and archive it; content.unpublish; M.
• clone_entity: copy as a new draft (/entities/:id/clone); content.create; M.
• bulk_update_entities: one bulk action on up to N ids (content/bulk.ts:19); per-action permission; D when the action is delete, HR.
• get_entity_by_path: RO; content.view_drafts.
• restore_revision: edit access; M.
• diff_revisions: content.view_drafts; RO.
• reject_entity: send back from review with a reason; content.publish; M.
• list_comments: content.view_drafts; RO.
• resolve_comment: content.view_drafts; M.
• get_lock: who is editing; content.view_drafts; RO.

Suggestions (4)

• suggest_changes: save field changes as a suggestion, not a revision. This is the safest write for an agent. content.submit_review; M.
• list_suggestions: RO.
• apply_suggestion / decline_suggestion: content.publish or edit_any; M.

Page layouts and components (11)

• get_component: one block's fields, slots, options, examples, styles and aiHint (ComponentInfo, shared/src/index.ts:268); content.view_drafts; RO.
• get_block_schema: every allowed block in blockSchemaText form (ai/layout.ts:228), filtered by the site's hidden blocks and the caller's styles. Also a resource, spillway://blocks. RO.
• validate_layout: dry run. Applies normaliseLayout (ai/layout.ts:40), the type's validateData and the style allowlist, then returns problems and a normalized layout without saving; RO.
• edit_layout: block-level operations on an entity's layout field (insert, update_props, move, remove, duplicate; by block id and slot path); saves a new revision; edit access; M.
• render_block_preview: HTML of one block with props (/components/:name/render); RO.
• set_block_set: hide, show or favorite blocks (PUT /site builder); site.settings; M.
• list_shared_blocks, get_shared_block: RO.
• create_shared_block, update_shared_block, delete_shared_block: types.manage; M/M/D.

Media (6)

• upload_media: from a URL (fetched server-side with the importer's SSRF guard) or base64 up to the upload limit; media.upload; M.
• update_media: alt, title, focal point; media.upload; M.
• delete_media: media.delete; D, HR. Refuse when the item is in use unless force.
• generate_alt_text: altText (ai/tasks.ts:183); ai.use + media.upload; RO (returns text) or M with save: true.
• search_dam, import_from_dam: media.upload; RO/M.

Structure: menus, placements, sections, collections, taxonomy, types (25)

• Menus: list_menus, get_menu (RO); create_menu, update_menu (whole item tree; site.settings or section menu; M); delete_menu (D).
• Placements: list_placements (RO); upsert_placement, delete_placement (site.settings; M/D).
• Sections: list_sections (RO); upsert_section, delete_section (site.settings; M/D).
• Collections: preview_collection (RO); create_collection, update_collection, delete_collection (types.manage; M/M/D).
• Taxonomy: list_vocabularies, list_terms (RO); upsert_vocabulary, upsert_term, reorder_terms, delete_term (types.manage; M/M/M/D).
• Content types: propose_content_type (/ai/type; RO); create_content_type, update_content_type (types.manage; M, HR when it removes or retypes fields that hold data), delete_content_type (D, HR).

Forms, redirects, search (11)

• list_forms, get_form (RO); create_form, update_form (forms.manage; M); delete_form (D).
• list_form_submissions (forms.view_submissions; RO, HR for PII: mask emails by default and keep it off for task runs unless switched on).
• list_redirects, list_redirect_misses (RO); upsert_redirect, delete_redirect (site.settings; M/D).
• reindex_search (site.settings; M).

Site settings, brand, theme (13)

• update_site_settings: a patch limited to safe sections (name, tagline, SEO defaults, locales, search, embeds); secrets and credentials refused; site.settings; M.
• export_site_structure (RO); apply_site_structure (config-as-code, /site/structure; M, HR, dry run first).
• get_site_lock (RO); set_site_lock (site.settings; HR: it can lock visitors out or open a site that is not ready).
• Brand: get_brand (RO); update_brand_draft and set_brand_styles (site.settings; M); publish_brand_draft (M, HR: site-wide visual change, preview first); list_brand_revisions (RO); restore_brand_revision (M); add_google_font (M).
• import_brand: from a URL or PDF into the draft; ai.use + site.settings; M.

Kits, modules, updates (12)

• list_kits (RO); plan_kit_install (dry run; RO); install_kit (types.manage + site.settings; M, HR).
• search_registry: modules, themes and kits; RO.
• list_modules, get_module_logs, list_updates (RO).
• install_module (site.settings, trusted tier users.manage; HR: runs third-party code, may restart the process).
• enable_module / disable_module (M, HR for trusted).
• configure_module: settings, with secret keys write-only; M.
• remove_module: D, HR.

People and access (7; proposals only for the Site agent)

• list_users, list_api_tokens (users.manage; RO; never return secrets).
• invite_user, set_user_roles (incl. site and section roles), deactivate_user (users.manage; HR).
• revoke_api_token (users.manage; D, HR).
• get_sso_status: read-only provider list and SCIM status; users.manage; RO.
• No tool creates API tokens, edits SSO providers or rotates SCIM tokens: each one returns or holds a secret.

Localization (4)

• create_translation: AI-translated draft (/entities/:id/translations); content.create + ai.use; M.
• get_translation_status: RO.
• send_to_tms, pull_tms_job: content.translate; M.

Releases (6)

• list_releases, diff_release (RO).
• create_release (content.create; M).
• merge_release, schedule_release (content.publish; M, HR: merging publishes many changes at once).
• discard_release (D).
• Plus an optional release argument on every entity, menu and site write tool (REST takes X-Spillway-Release).

Alerts, personalization, CDN, analytics, notifications (17)

• list_alerts (RO); upsert_alert, delete_alert (content.publish; M/D).
• list_audiences, list_experiments, get_experiment_results (RO); upsert_audience, upsert_experiment (site.settings; M); start_experiment / stop_experiment (M).
• purge_cdn (site.settings; M; rate-limited), list_cdn_purges (RO).
• get_analytics (move the agent's tool into MCP), get_page_analytics, get_search_analytics (RO).
• list_notification_rules (RO), upsert_notification_rule (site.settings; M).

Operations, agents, guidance (9)

• get_go_live_checklist, list_backups (users.manage; RO).
• run_backup (users.manage; M). No restore tool: restore stays CLI-only (cli.ts:51).
• list_import_jobs, start_import (imports.manage, admin; M, HR: crawls an outside site and spends AI tokens; estimate first).
• MCP gets search_manual, list_tasks and run_task (agents.use; M).
• call_module_tool: runs a module-registered AI tool (modules/module-api.ts:49); M, per-tool switch as the spec says (docs/dev/site-agents.md, "Module tools").

Total: 136 entries: 133 new tool names, plus 3 agent-only tools ported to MCP (get_analytics, search_manual, list_tasks). That is on top of today's 28 (MCP 22 plus agent-only 6), and the upsert_* pattern keeps the number down. About 56 are read-only, 63 mutating and 17 destructive; 21 are flagged HR.

Guardrails for high-risk tools

1. Proposal cards for anything the Site agent should not do alone. Generalize AgentProposal (today kind: 'task' only, shared/src/index.ts:1536) into kind: 'change' with a tool name, arguments and a human-readable diff. The person accepts through POST /agent-proposals/:runId/:id/accept (routes/agents.ts:306), which then runs the tool with the accepting person's permissions. That lets a Manager-capped agent prepare admin work (invite_user, install_module, set_site_lock) without ever holding admin itself. In a task run, nobody is there to accept, so these tools are refused, just as propose_task refuses today (agents/tools.ts:109).
2. A preview for every risky write.
   • dryRun: true returns what would change: plan for kits, a diff for structure, validate_layout, the delete_* impact (usage counts, inbound links, redirects that would break).
   • The REST already has kits/:key/plan, collections/preview, releases/:id/diff, restore --dry-run.
3. Two-step confirmation over MCP. A destructive call without confirm returns { needsConfirmation, summary, confirmToken }; repeating it with the token (short-lived, bound to the arguments) executes. MCP clients render the summary to the human. Mark these tools destructiveHint: true (already the pattern, mcp/tools.ts:284).
4. Scoped tokens and agent actions.
   • Add new AgentActions per area (structure, media, brand, setup, people), with the risky ones adminOnly like publish (shared/src/index.ts:1333).
   • Let API tokens carry a toolset allowlist beside permissions, so an "editorial bot" token cannot reach install_module even when its user is an admin.
5. Audit.
   • Derive CHANGING_TOOLS from readOnlyHint instead of a hand list (#128 nearly missed the two new tools there, agents/run.ts:29).
   • Record arguments (masked) and a risk level on mcp.tool_call rows. Today they hold only { tool, ok }, mcp/server.ts:73.
6. Hard exclusions: no tool that returns or sets a secret (API token create, SSO client secrets, SCIM tokens, mail passwords, module secret settings readback), no restore, no site delete, no registry moderation.
7. Rate and blast-radius limits:
   • per-token caps on destructive calls (for example 10 per hour)
   • bulk_update_entities limited to 50 ids
   • purge_cdn debounced
   • start_import needs an estimate the caller accepted

Structural notes

1. One implementation, five hand-maintained lists. The MCP and agent tools share handlers (good: agents/tools.ts:2-5, :205). But each tool's metadata is spread across TOOLS (mcp/tools.ts), ACTION_TOOLS (agents/tools.ts:22), LABELS (agents/tools.ts:68), the entity-collecting list in collectFrom (agents/tools.ts:222) and CHANGING_TOOLS (agents/run.ts:29). Adding the schedule tools in #128 meant touching all five.
   • Recommendation: extend ToolDef with toolset, agentAction, risk, label(input) and returnsEntity, and derive the rest.
   • Split mcp/tools.ts into mcp/tools/<toolset>.ts before adding about 120 tools.
2. Parity with REST is by convention, not by construction. MCP handlers call services directly and re-implement the route's checks, and the two have already drifted:
   • Editing locks: PUT /entities/:id refuses another user's lock and handles releases (routes/entities.ts:180-198). MCP update_entity did not check the lock until #128.
   • Block-style allowlist: assertStylesAllowed (routes/entities.ts:102) is not applied by MCP.
   • Section-scoped permissions: routes use requireEntityPermission/canOnEntity (auth/guards.ts:64-74); MCP uses a flat permission list (mcp/tools.ts:45-55).
   • Style access in AI drafts: draft_page_from_brief calls pageFromBrief without the access argument the REST route passes (mcp/tools.ts:399 vs routes/ai.ts:128), so style availability falls back to global roles.

   Recommendation: move each route's guard and validation into a service function (entityService.update(actor, scope, ...)) that both REST and tools call. Generating tools straight from routes is tempting, since docs/reference/openapi.json already has 452 operations with zod-derived request schemas. But permissions appear only as prose in descriptions, many routes are multipart or streaming, and tool descriptions need model-oriented wording. Generate the skeleton (schema, permission, risk) from a route registry, hand-write the description. A test should fail when a mutating route has neither a tool nor an explicit aiExcluded entry.

3. Secret leak in get_site_settings (fixed in #128). MCP get_site_settings and the spillway://site resource returned cx.site without maskSiteSecrets (repos/site.ts:245), so any token, a viewer's included, could read the CAPTCHA secret. DAM and CDN credentials are encrypted at rest but still go out as ciphertext; new settings tools must return only what GET /api/site returns.
4. Tool count. About 150 tools would cost a model roughly 40-60k tokens of schema per turn, and it degrades tool choice. Options, best first:
   • Toolsets with lazy loading. Keep a core set (about 20: read, draft, checks, search, list_toolsets, load_toolset) and load toolsets on demand. MCP: declare tools: { listChanged: true } (today false, mcp/server.ts:84). Since the server is stateless, take the toolset from a ?toolsets=content,structure query or header on /mcp, or from the token's toolset allowlist. The Site agent: choose toolsets from the screen context (AgentChatContext.path, agents/run.ts:232) plus a load_tools meta-tool within the run.
   • Filter tools/list by permission so a viewer token does not see publish_entity (mcp/server.ts:49 lists all).
   • Consolidate CRUD into manage_<thing> tools with an action enum only where schemas are near-identical (menus, redirects, alerts). Keep destructive actions as separate tools so annotations stay truthful.
5. Layout editing for an AI is underspecified.
   • The only way to write a layout is to send the whole Puck document in data.layout through create_entity/update_entity (description at mcp/tools.ts:~123). Validation only checks that content is an array (content/validate.ts:35-36), so unknown block types, bad props, hidden blocks and missing ids are all accepted. normaliseLayout, which repairs and filters, runs only on AI briefs (ai/tasks.ts:64).
   • No tool or resource gives the block prop schema. list_components strips fields, slots, examples and aiHint (mcp/tools.ts:453), although ComponentInfo carries them and describeBlocks/blockSchemaText (ai/layout.ts:203, :228) already produce model-ready text.
   • The write-page prompt hard-codes nine block names (mcp/prompts.ts:55) that will drift from the registry (which already has Section, Gallery, Buttons, Menu, Calendar, Shared and others; docs/reference/blocks.md).
   • Recommendation: get_block_schema and spillway://blocks, get_component, validate_layout (dry run, same normalizer) and a block-level edit_layout, so a model can change one Hero headline without resending a 30 KB document. Apply normaliseLayout plus the style allowlist on every tool write.
6. The Site agent's ceiling. AgentRole is author, editor or manager (shared/src/index.ts:1339), and the system prompt forbids settings, installs and users (agents/run.ts:246). Even with all the tools above, "manage a site completely" from Ask means admin operations go through proposal cards (guardrail 1). That matches the phase 2 plan in docs/dev/site-agents.md ("setup proposals (install, settings, structure) with Ask an admin"; tools propose_install, propose_settings_change, propose_structure_change). Another gap: the "read" action is labeled "Read content, media, menus and settings" (shared/src/index.ts:1326), but no menu tool exists.
7. MCP has no release or multisite dimension. Tools are bound to the request's site (routes/mcp.ts:33) and always write the main line. Large AI changes (a redesign, a new section) belong in a release so a human reviews one diff; add a release argument and the release tools.
8. Media upload over MCP needs a non-multipart path. Use upload_media from a URL, reusing the importer's fetch guards, or base64 with a small cap.
9. Docs to update with any of this: docs/agents/connect.md (tool list), docs/api.md, docs/dev/site-agents.md (the action table), and CHANGELOG.md.

Top gaps, in priority order

1. Layout authoring: block schema, validation, block-level edits.
2. Media upload, alt text and metadata.
3. Menus.
4. Content type and taxonomy management.
5. Site settings and brand writes.
6. Kits and modules from the registry, with plan and approve.
7. Releases as the container for large AI changes.
8. Forms and redirects.
9. People and access, via proposals only.
10. Agent ceiling: admin work through proposal cards (structural note 6).

Phases

Progress. Phase 1 so far: one registry (core/src/mcp/tools.ts and tools/<toolset>.ts, each tool carrying its toolset, agent action, permissions, label and entity flag; the agent's action map, labels and audit list derive from it), tools/list filtered by the caller's permissions, permission checks in one place (runTool), arguments on the audit rows of changing calls, and mcp/coverage.ts with test/ai-coverage.test.ts: every changing route has a tool, a planned phase or a reason. Shared guards: editing locks and the block style rule (theme/styles.ts assertStylesAllowed) apply to the AI tools as to the admin, and the AI's page drafts get the caller's style access. Section roles do not reach tokens (a token carries its user's site-wide roles only), so tools cannot act through a section role; working in a workstream is phase 4. Toolsets: an MCP client lists only the toolsets it asks for (?toolsets= or X-Spillway-Toolsets). Change proposals and two-step confirmation moved to phase 4, where the admin-only and destructive tools that need them arrive; built now, they would have nothing to guard.

Each phase ships tools for MCP and the Site agent together, with tests, docs and the guardrails it needs.

1. Foundation. One tool registry: ToolDef carries toolset, agent action, risk, label and whether it returns an entity, and the five lists derive from it; mcp/tools.ts splits into one file per toolset. Shared service functions so REST and tools apply the same guards (locks, style allowlist, section permissions, releases). tools/list filtered by the caller's permissions; toolsets with lazy loading (listChanged). A test that fails when a mutating route has no tool and no aiExcluded entry.
2. Pages. Started: get_block_schema, get_layout_outline, validate_layout and edit_layout in the pages toolset (mcp/tools/pages.ts, content/layout-edit.ts, ai/blockset.ts); upload_media, update_media, generate_alt_text (media toolset, sharing media/create.ts with the upload route) and list_menus, get_menu, create_menu, update_menu (menus). Left for later: delete_media (with the phase 4 confirmation), placements, DAM search and import, component previews. get_block_schema and spillway://blocks, get_component, validate_layout, block-level edit_layout, render_block_preview; normaliseLayout and the style allowlist on every tool write. Media: upload_media (URL or base64), update_media, generate_alt_text, delete_media. Menus and placements.
3. Structure. Content types (with propose_content_type), taxonomy, collections, shared blocks, sections, forms, redirects, alerts. Content extras: delete, archive, clone, bulk, restore and diff revisions, reject, suggestions.
4. Site. Proposal cards of kind change (a person accepts with their own permissions) and two-step confirmation for destructive MCP calls, built with the first tools that need them. Site settings (safe sections only), site lock, brand and fonts, structure export and apply, kits (plan, then install), modules and themes from the registry (search, install, enable, configure, remove), updates. Admin-only ones reach the Site agent as proposals. Releases, with a release argument on every write.
5. People and operations. Users and roles, token listing and revocation, SSO status, translations and TMS, personalization, CDN purge, analytics parity, notifications, backups (no restore), the importer, tasks over MCP, module AI tools.

Phases 2 and 3 cover what most site work needs. Phase 1 comes first because it keeps around 130 new tools consistent, safe and affordable for a model to use.

Spillway 0.1.0 · Edit this page on GitHub