[codex] Add LLM Wiki plugin host support (#5597)

## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies.
> - The plugin system needs host contracts and runtime support before
large plugins can integrate cleanly.
> - The source branch mixed the LLM Wiki package with supporting
host/runtime work, managed plugin skills, root-level storage spaces, and
a bookmarks reference plugin.
> - [PAP-9173](/PAP/issues/PAP-9173) asked for the current branch to be
split by file boundary: plugin package separately from everything else.
> - [PAP-9188](/PAP/issues/PAP-9188) clarified that LLM Wiki may have
plugin-local spaces, but Paperclip core should not reorganize top-level
local storage into spaces.
> - Follow-up review clarified that the bookmarks example should not
ship in this PR either.
> - This pull request contains the
non-`packages/plugins/plugin-llm-wiki/` host/runtime work, keeps runtime
state under the selected Paperclip instance root, and no longer includes
the bookmarks example.

## What Changed

- Added/updated plugin host contracts, SDK types, worker RPC plumbing,
managed plugin skill support, and related server tests.
- Removed the bookmarks example plugin package and its
bundled-example/workspace references.
- Removed the root-level local spaces CLI/migration surface and restored
instance-root runtime defaults for config, db, logs, storage, secrets,
workspaces, projects, and adapter homes.
- Replaced shared root `space-paths` helpers with `home-paths` helpers
for core runtime storage.
- Tightened stranded recovery unique-conflict detection so concurrent
recovery scans reuse the raced recovery issue when Postgres errors are
wrapped.
- Kept `packages/plugins/plugin-llm-wiki/` out of this PR diff;
plugin-local spaces remain in the stacked plugin-only PR.

## Verification

- `pnpm exec vitest run cli/src/__tests__/data-dir.test.ts
cli/src/__tests__/home-paths.test.ts cli/src/__tests__/onboard.test.ts
packages/shared/src/home-paths.test.ts
packages/db/src/runtime-config.test.ts
server/src/__tests__/agent-instructions-service.test.ts
server/src/__tests__/claude-local-execute.test.ts
server/src/__tests__/codex-local-execute.test.ts`
- `pnpm exec vitest run packages/db/src/runtime-config.test.ts`
- `pnpm exec vitest run
server/src/__tests__/plugin-routes-authz.test.ts`
- `pnpm --filter @paperclipai/server typecheck`
- `pnpm exec vitest run
server/src/__tests__/heartbeat-process-recovery.test.ts -t "reuses the
raced stranded recovery issue"` skipped locally because embedded
Postgres did not initialize on this macOS temp host; the code path was
typechecked and is covered by Linux CI.
- Boundary check: no core references remain for `PAPERCLIP_SPACE_ID`,
`spaces migrate-default`, `@paperclipai/shared/space-paths`,
`registerSpacesCommands`, or the removed bookmarks example.
- Previous PR head `4f23e034` had green GitHub checks: `verify`, all
four serialized server shards, `e2e`, `Canary Dry Run`, `policy`, Snyk,
and `Greptile Review`. Current head `582f466d` is re-running checks
after the bookmarks deletion.

## Risks

- Plugin host changes touch shared runtime paths, so regressions would
most likely appear in adapter startup, plugin loading, or local dev path
defaults.
- Removing the bookmarks example also removes one demonstration of
plugin database namespaces plus local-folder persistence; remaining
plugin examples still cover bundled example discovery and plugin host
flows.
- The plugin package itself is intentionally deferred to the stacked
plugin-only PR, where LLM Wiki plugin-local spaces live.
- Existing installs that tested the transient root-level spaces CLI
should stop using it; this PR intentionally removes that unsupported
migration surface before merge.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI GPT-5 Codex via Codex CLI, tool use and local code execution
enabled; context window not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass, except where noted above
for host-specific embedded Postgres initialization
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

Stacked follow-up: PR #5592 contains only
`packages/plugins/plugin-llm-wiki/` and targets this branch.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta
2026-05-10 07:34:12 -05:00
committed by GitHub
parent eb12c42009
commit 0096b56a1c
40 changed files with 1892 additions and 224 deletions
+64 -10
View File
@@ -126,6 +126,33 @@ function applyInstructionTemplateVariables(
return next;
}
function declaredInstructionFiles(
declaration: PluginManagedAgentDeclaration,
variables: Record<string, string | null | undefined>,
) {
const instructionDeclaration = declaration.instructions;
if (!instructionDeclaration?.content && !instructionDeclaration?.files) return null;
const entryFile = instructionDeclaration.entryFile ?? "AGENTS.md";
const files = { ...(instructionDeclaration.files ?? {}) };
if (instructionDeclaration.content !== undefined) {
files[entryFile] = instructionDeclaration.content;
}
if (files[entryFile] === undefined) {
files[entryFile] = "";
}
return {
entryFile,
files: Object.fromEntries(
Object.entries(files).map(([filePath, content]) => [
filePath,
applyInstructionTemplateVariables(content, variables),
]),
),
};
}
function rowIsManagedAgent(
row: typeof agents.$inferSelect,
pluginKey: string,
@@ -299,19 +326,18 @@ export function pluginManagedAgentService(
companyId: string,
agent: Agent,
declaration: PluginManagedAgentDeclaration,
options: { replaceExisting: boolean },
materializeOptions: { replaceExisting: boolean },
): Promise<Agent> {
const instructionDeclaration = declaration.instructions;
if (!instructionDeclaration?.content) return agent;
const entryFile = instructionDeclaration.entryFile ?? "AGENTS.md";
const variables = await optionsForInstructionVariables(companyId);
const declared = declaredInstructionFiles(declaration, variables);
if (!declared) return agent;
const materialized = await instructions.materializeManagedBundle(
agent,
{ [entryFile]: applyInstructionTemplateVariables(instructionDeclaration.content, variables) },
declared.files,
{
entryFile,
replaceExisting: options.replaceExisting,
entryFile: declared.entryFile,
replaceExisting: materializeOptions.replaceExisting,
clearLegacyPromptTemplate: true,
},
);
@@ -325,6 +351,33 @@ export function pluginManagedAgentService(
return (updated as Agent | null) ?? { ...agent, adapterConfig: materialized.adapterConfig };
}
async function managedInstructionDefaultDrift(
companyId: string,
agent: Agent | null,
declaration: PluginManagedAgentDeclaration,
): Promise<PluginManagedAgentResolution["defaultDrift"]> {
if (!agent) return null;
const variables = await optionsForInstructionVariables(companyId);
const declared = declaredInstructionFiles(declaration, variables);
if (!declared) return null;
let exported: Awaited<ReturnType<typeof instructions.exportFiles>>;
try {
exported = await instructions.exportFiles(agent);
} catch {
return { entryFile: declared.entryFile, changedFiles: [declared.entryFile] };
}
const paths = new Set([...Object.keys(declared.files), ...Object.keys(exported.files)]);
const changedFiles = [...paths]
.filter((filePath) => (exported.files[filePath] ?? null) !== (declared.files[filePath] ?? null))
.sort((left, right) => left.localeCompare(right));
if (exported.entryFile !== declared.entryFile && !changedFiles.includes(declared.entryFile)) {
changedFiles.unshift(declared.entryFile);
}
return changedFiles.length > 0 ? { entryFile: declared.entryFile, changedFiles } : null;
}
async function optionsForInstructionVariables(companyId: string) {
return options.instructionTemplateVariables ? options.instructionTemplateVariables(companyId) : {};
}
@@ -333,13 +386,13 @@ export function pluginManagedAgentService(
return options.pluginKey;
}
function resolution(
async function resolution(
companyId: string,
declaration: PluginManagedAgentDeclaration,
agent: Agent | null,
status: PluginManagedAgentResolution["status"],
approvalId?: string | null,
): PluginManagedAgentResolution {
): Promise<PluginManagedAgentResolution> {
return {
pluginKey: options.pluginKey,
resourceKind: "agent",
@@ -349,6 +402,7 @@ export function pluginManagedAgentService(
agent,
status,
approvalId: approvalId ?? null,
defaultDrift: await managedInstructionDefaultDrift(companyId, agent, declaration),
};
}