forked from farhoodlabs/paperclip
3c73ed26b5
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - The plugin system is the extension boundary for optional product capabilities > - Rich plugins need more than a worker entrypoint: they need scoped database storage, local project folders, managed agents/routines, host navigation, and reusable UI components > - The LLM Wiki work exposed those missing host surfaces while keeping plugin code outside the core control plane > - This pull request expands the core plugin host, SDK, server APIs, and UI bridge so plugins can declare and use those surfaces > - The benefit is that future plugins can integrate with Paperclip through documented, validated contracts instead of bespoke server or UI imports ## What Changed - Added plugin-managed database namespaces and migration tracking, including Drizzle schema/migration files and SQL validation for namespace isolation. - Added server support for plugin local folders, managed agents, managed routines, scoped plugin APIs, and plugin operation visibility. - Expanded shared plugin manifest/types/validators and SDK host/testing/UI exports for richer plugin surfaces. - Added reusable UI pieces for file trees, managed routines, resizable sidebars, route sidebars, and plugin bridge initialization. - Updated plugin docs and example plugins to use the expanded host and SDK surface. ## Verification - `pnpm install --frozen-lockfile` - `pnpm run preflight:workspace-links && pnpm exec vitest run packages/shared/src/validators/plugin.test.ts server/src/__tests__/plugin-database.test.ts server/src/__tests__/plugin-local-folders.test.ts server/src/__tests__/plugin-managed-agents.test.ts server/src/__tests__/plugin-managed-routines.test.ts server/src/__tests__/plugin-orchestration-apis.test.ts ui/src/api/plugins.test.ts ui/src/components/FileTree.test.tsx ui/src/components/ResizableSidebarPane.test.tsx ui/src/pages/PluginPage.test.tsx ui/src/plugins/bridge.test.ts` passed: 11 files, 67 tests. - Confirmed this PR changes 89 files and does not include `pnpm-lock.yaml` or `.github/workflows/*`. ## Risks - Medium: this expands plugin host contracts across db/shared/server/ui and includes a new core migration (`0076_useful_elektra.sql`). - The plugin database namespace validator is intentionally restrictive; plugin authors may need follow-up affordances for SQL patterns that remain blocked. - Merge this before the LLM Wiki plugin PR so the plugin can resolve the new SDK and host APIs. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent, tool-enabled shell/git/GitHub workflow. Context window size was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
393 lines
13 KiB
Markdown
393 lines
13 KiB
Markdown
# Plugin Authoring Guide
|
|
|
|
This guide describes the current, implemented way to create a Paperclip plugin in this repo.
|
|
|
|
It is intentionally narrower than [PLUGIN_SPEC.md](./PLUGIN_SPEC.md). The spec includes future ideas; this guide only covers the alpha surface that exists now.
|
|
|
|
## Current reality
|
|
|
|
- Treat plugin workers and plugin UI as trusted code.
|
|
- Plugin UI runs as same-origin JavaScript inside the main Paperclip app.
|
|
- Worker-side host APIs are capability-gated.
|
|
- Plugin UI is not sandboxed by manifest capabilities.
|
|
- Plugin database migrations are restricted to a host-derived plugin namespace.
|
|
- Plugin-owned JSON API routes must be declared in the manifest and are mounted
|
|
only under `/api/plugins/:pluginId/api/*`.
|
|
- The host provides a small shared React component kit through
|
|
`@paperclipai/plugin-sdk/ui`; use it for common Paperclip controls before
|
|
building custom versions.
|
|
- `ctx.assets` is not supported in the current runtime.
|
|
|
|
## Scaffold a plugin
|
|
|
|
Use the scaffold package:
|
|
|
|
```bash
|
|
pnpm --filter @paperclipai/create-paperclip-plugin build
|
|
node packages/plugins/create-paperclip-plugin/dist/index.js @yourscope/plugin-name --output ./packages/plugins/examples
|
|
```
|
|
|
|
For a plugin that lives outside the Paperclip repo:
|
|
|
|
```bash
|
|
pnpm --filter @paperclipai/create-paperclip-plugin build
|
|
node packages/plugins/create-paperclip-plugin/dist/index.js @yourscope/plugin-name \
|
|
--output /absolute/path/to/plugin-repos \
|
|
--sdk-path /absolute/path/to/paperclip/packages/plugins/sdk
|
|
```
|
|
|
|
That creates a package with:
|
|
|
|
- `src/manifest.ts`
|
|
- `src/worker.ts`
|
|
- `src/ui/index.tsx`
|
|
- `tests/plugin.spec.ts`
|
|
- `esbuild.config.mjs`
|
|
- `rollup.config.mjs`
|
|
|
|
Inside this monorepo, the scaffold uses `workspace:*` for `@paperclipai/plugin-sdk`.
|
|
|
|
Outside this monorepo, the scaffold snapshots `@paperclipai/plugin-sdk` from the local Paperclip checkout into a `.paperclip-sdk/` tarball so you can build and test a plugin without publishing anything to npm first.
|
|
|
|
## Recommended local workflow
|
|
|
|
From the generated plugin folder:
|
|
|
|
```bash
|
|
pnpm install
|
|
pnpm typecheck
|
|
pnpm test
|
|
pnpm build
|
|
```
|
|
|
|
For local development, install it into Paperclip from an absolute local path through the plugin manager or API. The server supports local filesystem installs and watches local-path plugins for file changes so worker restarts happen automatically after rebuilds.
|
|
|
|
Example:
|
|
|
|
```bash
|
|
curl -X POST http://127.0.0.1:3100/api/plugins/install \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"packageName":"/absolute/path/to/your-plugin","isLocalPath":true}'
|
|
```
|
|
|
|
## Supported alpha surface
|
|
|
|
Worker:
|
|
|
|
- config
|
|
- events
|
|
- jobs
|
|
- launchers
|
|
- http
|
|
- secrets
|
|
- activity
|
|
- state
|
|
- database namespace via `ctx.db`
|
|
- scoped JSON API routes declared with `apiRoutes`
|
|
- entities
|
|
- projects and project workspaces
|
|
- companies
|
|
- issues, comments, namespaced `plugin:<pluginKey>` origins, blocker relations, checkout assertions, assignment wakeups, and orchestration summaries
|
|
- agents and agent sessions
|
|
- goals
|
|
- data/actions
|
|
- streams
|
|
- tools
|
|
- metrics
|
|
- logger
|
|
|
|
### Plugin database declarations
|
|
|
|
First-party or otherwise trusted orchestration plugins can declare:
|
|
|
|
```ts
|
|
database: {
|
|
migrationsDir: "migrations",
|
|
coreReadTables: ["issues"],
|
|
}
|
|
```
|
|
|
|
Required capabilities are `database.namespace.migrate` and
|
|
`database.namespace.read`; add `database.namespace.write` for runtime mutations.
|
|
The host derives `ctx.db.namespace`, runs SQL files in filename order before the
|
|
worker starts, records checksums in `plugin_migrations`, and rejects changed
|
|
already-applied migrations.
|
|
|
|
Migration SQL may create or alter objects only inside `ctx.db.namespace`. It may
|
|
reference whitelisted `public` core tables for foreign keys or read-only views,
|
|
but may not mutate/alter/drop/truncate public tables, create extensions,
|
|
triggers, untrusted languages, or runtime multi-statement SQL. Runtime
|
|
`ctx.db.query()` is restricted to `SELECT`; runtime `ctx.db.execute()` is
|
|
restricted to namespace-local `INSERT`, `UPDATE`, and `DELETE`.
|
|
|
|
### Scoped plugin API routes
|
|
|
|
Plugins can expose JSON-only routes under their own namespace:
|
|
|
|
```ts
|
|
apiRoutes: [
|
|
{
|
|
routeKey: "initialize",
|
|
method: "POST",
|
|
path: "/issues/:issueId/smoke",
|
|
auth: "board-or-agent",
|
|
capability: "api.routes.register",
|
|
checkoutPolicy: "required-for-agent-in-progress",
|
|
companyResolution: { from: "issue", param: "issueId" },
|
|
},
|
|
]
|
|
```
|
|
|
|
The host resolves the plugin, checks that it is ready, enforces
|
|
`api.routes.register`, matches the declared method/path, resolves company access,
|
|
and applies checkout policy before dispatching to the worker's `onApiRequest`
|
|
handler. The worker receives sanitized headers, route params, query, parsed JSON
|
|
body, actor context, and company id. Do not use plugin routes to claim core
|
|
paths; they always remain under `/api/plugins/:pluginId/api/*`.
|
|
|
|
UI:
|
|
|
|
- `usePluginData`
|
|
- `usePluginAction`
|
|
- `usePluginStream`
|
|
- `usePluginToast`
|
|
- `useHostContext`
|
|
- typed slot props from `@paperclipai/plugin-sdk/ui`
|
|
|
|
Mount surfaces currently wired in the host include:
|
|
|
|
- `page`
|
|
- `settingsPage`
|
|
- `dashboardWidget`
|
|
- `sidebar`
|
|
- `sidebarPanel`
|
|
- `detailTab`
|
|
- `taskDetailView`
|
|
- `projectSidebarItem`
|
|
- `globalToolbarButton`
|
|
- `toolbarButton`
|
|
- `contextMenuItem`
|
|
- `commentAnnotation`
|
|
- `commentContextMenuItem`
|
|
|
|
## Shared host components
|
|
|
|
Use shared components from `@paperclipai/plugin-sdk/ui` when the plugin needs a
|
|
Paperclip-native control. The host owns the implementation, so plugins inherit
|
|
the board's current styling, ordering, recent selections, and dark-mode behavior
|
|
without importing `ui/src` internals.
|
|
|
|
Currently exposed components include:
|
|
|
|
- `MarkdownBlock` and `MarkdownEditor` for rendered and editable markdown.
|
|
- `FileTree` for serializable file and directory trees.
|
|
- `IssuesList` for a native company-scoped issue table.
|
|
- `AssigneePicker` for the same agent/user selector used in the new issue pane.
|
|
Use the controlled `value` format `agent:<id>`, `user:<id>`, or `""`.
|
|
- `ProjectPicker` for the same project selector used in the new issue pane.
|
|
Use the controlled project id value, or `""` for no project.
|
|
- `ManagedRoutinesList` for plugin-owned routine settings pages.
|
|
|
|
```tsx
|
|
import { AssigneePicker, ProjectPicker } from "@paperclipai/plugin-sdk/ui";
|
|
|
|
export function PluginAssignmentControls({ companyId }: { companyId: string }) {
|
|
const [assignee, setAssignee] = useState("");
|
|
const [projectId, setProjectId] = useState("");
|
|
|
|
return (
|
|
<>
|
|
<AssigneePicker
|
|
companyId={companyId}
|
|
value={assignee}
|
|
onChange={(value) => setAssignee(value)}
|
|
/>
|
|
<ProjectPicker
|
|
companyId={companyId}
|
|
value={projectId}
|
|
onChange={setProjectId}
|
|
/>
|
|
</>
|
|
);
|
|
}
|
|
```
|
|
|
|
## File and path UI
|
|
|
|
Plugin UI often needs to render a file tree, accept a folder path, or browse a
|
|
project workspace. There are three different surfaces for that, and they map to
|
|
different trust and data-flow boundaries. Pick the surface that matches the
|
|
data the plugin actually has.
|
|
|
|
### When to use the shared `FileTree`
|
|
|
|
Use `FileTree` from `@paperclipai/plugin-sdk/ui` whenever the plugin only needs
|
|
to render a serializable file/directory list and react to selection or
|
|
expand/collapse. The host owns the implementation, so plugin UI inherits the
|
|
board's icons, indent, focus ring, and dark-mode styling without importing host
|
|
internals.
|
|
|
|
```tsx
|
|
import {
|
|
FileTree,
|
|
type FileTreeNode,
|
|
} from "@paperclipai/plugin-sdk/ui";
|
|
|
|
const nodes: FileTreeNode[] = [
|
|
{ name: "AGENTS.md", path: "AGENTS.md", kind: "file", children: [] },
|
|
{
|
|
name: "wiki",
|
|
path: "wiki",
|
|
kind: "dir",
|
|
children: [
|
|
{ name: "index.md", path: "wiki/index.md", kind: "file", children: [] },
|
|
],
|
|
},
|
|
];
|
|
|
|
export function WikiTree() {
|
|
const [expanded, setExpanded] = useState<Set<string>>(() => new Set(["wiki"]));
|
|
const [selected, setSelected] = useState<string | null>(null);
|
|
|
|
return (
|
|
<FileTree
|
|
nodes={nodes}
|
|
selectedFile={selected}
|
|
expandedPaths={expanded}
|
|
onSelectFile={(path) => setSelected(path)}
|
|
onToggleDir={(path) =>
|
|
setExpanded((current) => {
|
|
const next = new Set(current);
|
|
next.has(path) ? next.delete(path) : next.add(path);
|
|
return next;
|
|
})
|
|
}
|
|
/>
|
|
);
|
|
}
|
|
```
|
|
|
|
Good fits:
|
|
|
|
- LLM Wiki page navigation in `packages/plugins/plugin-llm-wiki` builds a
|
|
`FileTreeNode[]` from worker query results and renders it through `FileTree`.
|
|
- The example `plugin-file-browser-example` lazily fetches a directory's
|
|
children through a `loadFileList` action when `onToggleDir` fires, then
|
|
merges the children into the local tree state — letting the shared component
|
|
handle rendering and selection.
|
|
|
|
Boundary rules:
|
|
|
|
- Keep the prop surface serializable (`nodes`, `expandedPaths`, `checkedPaths`,
|
|
`fileBadges`, `fileTones`). Do not pass arbitrary render functions across the
|
|
plugin/host boundary in v1; the supported escape hatches are
|
|
`fileBadges` (status pill keyed by path) and `fileTones` (row tone keyed by
|
|
path).
|
|
- Do not import the host's `FileTree.tsx` or any `ui/src/*` module. The SDK
|
|
declaration is the only supported import path for plugin UI.
|
|
- The shared `FileTree` is for rendering and selection. Plugin-specific editors,
|
|
ingest flows, query forms, and lint runs stay inside the plugin and do not
|
|
belong as `FileTree` props.
|
|
|
|
### When to declare `localFolders`
|
|
|
|
When the plugin needs operator-configured filesystem roots — typically for
|
|
trusted local plugins like wiki tooling — declare `localFolders[]` on the
|
|
manifest and add the `local.folders` capability. The host renders a settings
|
|
surface for the operator to set the absolute path, validates the path
|
|
server-side (containment, symlinks, required files/directories), and exposes
|
|
`ctx.localFolders.readText()` and `ctx.localFolders.writeTextAtomic()` in the
|
|
worker.
|
|
|
|
```ts
|
|
export const manifest = {
|
|
capabilities: ["local.folders"],
|
|
localFolders: [
|
|
{
|
|
folderKey: "content-root",
|
|
displayName: "Content root",
|
|
access: "readWrite",
|
|
requiredDirectories: ["sources", "pages"],
|
|
requiredFiles: ["schema.md"],
|
|
},
|
|
],
|
|
};
|
|
```
|
|
|
|
Use this when:
|
|
|
|
- The data lives outside any project workspace.
|
|
- Reads and writes need company-scoped configuration.
|
|
- The operator picks the path once in plugin settings and the worker resolves
|
|
files relative to that root.
|
|
|
|
Do not use `localFolders` to grant the UI direct browser-side access to the
|
|
filesystem — there is no such capability. The browser still goes through the
|
|
worker via `getData` / `performAction`, and the worker only exposes paths it
|
|
chose to expose.
|
|
|
|
### When to keep worker-mediated project workspace browsing
|
|
|
|
When the data lives inside an existing project workspace, keep the browsing
|
|
flow worker-mediated:
|
|
|
|
- The worker uses `ctx.projects.listWorkspaces()` to resolve the workspace
|
|
path, then reads its filesystem with normal Node APIs.
|
|
- The plugin UI calls a `getData` handler for the root listing and an action
|
|
for lazy children, then renders them through `FileTree`.
|
|
- The worker is the only side that touches the disk. The browser receives a
|
|
serializable tree and never sees raw absolute paths it can replay.
|
|
|
|
The example `plugin-file-browser-example` is the reference for this pattern:
|
|
the worker registers `fileList` (data) and `loadFileList` (action) over the
|
|
same handler, and the UI uses the action for on-toggle directory loading so the
|
|
shared `FileTree` stays the rendering surface.
|
|
|
|
### Mixing surfaces
|
|
|
|
A single plugin can use more than one of these. The LLM Wiki uses
|
|
`localFolders` for its content root, then renders the resulting page list
|
|
through `FileTree`. The file browser example uses `ctx.projects.listWorkspaces`
|
|
to pick a workspace and renders its on-disk tree through `FileTree` with lazy
|
|
loading. Pick the boundary per data source, not per plugin.
|
|
|
|
## Company routes
|
|
|
|
Plugins may declare a `page` slot with `routePath` to own a company route like:
|
|
|
|
```text
|
|
/:companyPrefix/<routePath>
|
|
```
|
|
|
|
Rules:
|
|
|
|
- `routePath` must be a single lowercase slug
|
|
- it cannot collide with reserved host routes
|
|
- it cannot duplicate another installed plugin page route
|
|
|
|
## Publishing guidance
|
|
|
|
- Use npm packages as the deployment artifact.
|
|
- Treat repo-local example installs as a development workflow only.
|
|
- Prefer keeping plugin UI self-contained inside the package.
|
|
- Do not rely on host design-system components or undocumented app internals.
|
|
- GitHub repository installs are not a first-class workflow today. For local development, use a checked-out local path. For production, publish to npm or a private npm-compatible registry.
|
|
|
|
## Verification before handoff
|
|
|
|
At minimum:
|
|
|
|
```bash
|
|
pnpm --filter <your-plugin-package> typecheck
|
|
pnpm --filter <your-plugin-package> test
|
|
pnpm --filter <your-plugin-package> build
|
|
```
|
|
|
|
If you changed host integration too, also run:
|
|
|
|
```bash
|
|
pnpm -r typecheck
|
|
pnpm test:run
|
|
pnpm build
|
|
```
|