forked from farhoodlabs/paperclip
[codex] Add LLM Wiki plugin host support (#5597)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - The plugin system needs host contracts and runtime support before large plugins can integrate cleanly. > - The source branch mixed the LLM Wiki package with supporting host/runtime work, managed plugin skills, root-level storage spaces, and a bookmarks reference plugin. > - [PAP-9173](/PAP/issues/PAP-9173) asked for the current branch to be split by file boundary: plugin package separately from everything else. > - [PAP-9188](/PAP/issues/PAP-9188) clarified that LLM Wiki may have plugin-local spaces, but Paperclip core should not reorganize top-level local storage into spaces. > - Follow-up review clarified that the bookmarks example should not ship in this PR either. > - This pull request contains the non-`packages/plugins/plugin-llm-wiki/` host/runtime work, keeps runtime state under the selected Paperclip instance root, and no longer includes the bookmarks example. ## What Changed - Added/updated plugin host contracts, SDK types, worker RPC plumbing, managed plugin skill support, and related server tests. - Removed the bookmarks example plugin package and its bundled-example/workspace references. - Removed the root-level local spaces CLI/migration surface and restored instance-root runtime defaults for config, db, logs, storage, secrets, workspaces, projects, and adapter homes. - Replaced shared root `space-paths` helpers with `home-paths` helpers for core runtime storage. - Tightened stranded recovery unique-conflict detection so concurrent recovery scans reuse the raced recovery issue when Postgres errors are wrapped. - Kept `packages/plugins/plugin-llm-wiki/` out of this PR diff; plugin-local spaces remain in the stacked plugin-only PR. ## Verification - `pnpm exec vitest run cli/src/__tests__/data-dir.test.ts cli/src/__tests__/home-paths.test.ts cli/src/__tests__/onboard.test.ts packages/shared/src/home-paths.test.ts packages/db/src/runtime-config.test.ts server/src/__tests__/agent-instructions-service.test.ts server/src/__tests__/claude-local-execute.test.ts server/src/__tests__/codex-local-execute.test.ts` - `pnpm exec vitest run packages/db/src/runtime-config.test.ts` - `pnpm exec vitest run server/src/__tests__/plugin-routes-authz.test.ts` - `pnpm --filter @paperclipai/server typecheck` - `pnpm exec vitest run server/src/__tests__/heartbeat-process-recovery.test.ts -t "reuses the raced stranded recovery issue"` skipped locally because embedded Postgres did not initialize on this macOS temp host; the code path was typechecked and is covered by Linux CI. - Boundary check: no core references remain for `PAPERCLIP_SPACE_ID`, `spaces migrate-default`, `@paperclipai/shared/space-paths`, `registerSpacesCommands`, or the removed bookmarks example. - Previous PR head `4f23e034` had green GitHub checks: `verify`, all four serialized server shards, `e2e`, `Canary Dry Run`, `policy`, Snyk, and `Greptile Review`. Current head `582f466d` is re-running checks after the bookmarks deletion. ## Risks - Plugin host changes touch shared runtime paths, so regressions would most likely appear in adapter startup, plugin loading, or local dev path defaults. - Removing the bookmarks example also removes one demonstration of plugin database namespaces plus local-folder persistence; remaining plugin examples still cover bundled example discovery and plugin host flows. - The plugin package itself is intentionally deferred to the stacked plugin-only PR, where LLM Wiki plugin-local spaces live. - Existing installs that tested the transient root-level spaces CLI should stop using it; this PR intentionally removes that unsupported migration surface before merge. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI GPT-5 Codex via Codex CLI, tool use and local code execution enabled; context window not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass, except where noted above for host-specific embedded Postgres initialization - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge Stacked follow-up: PR #5592 contains only `packages/plugins/plugin-llm-wiki/` and targets this branch. --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
@@ -126,6 +126,33 @@ function applyInstructionTemplateVariables(
|
||||
return next;
|
||||
}
|
||||
|
||||
function declaredInstructionFiles(
|
||||
declaration: PluginManagedAgentDeclaration,
|
||||
variables: Record<string, string | null | undefined>,
|
||||
) {
|
||||
const instructionDeclaration = declaration.instructions;
|
||||
if (!instructionDeclaration?.content && !instructionDeclaration?.files) return null;
|
||||
|
||||
const entryFile = instructionDeclaration.entryFile ?? "AGENTS.md";
|
||||
const files = { ...(instructionDeclaration.files ?? {}) };
|
||||
if (instructionDeclaration.content !== undefined) {
|
||||
files[entryFile] = instructionDeclaration.content;
|
||||
}
|
||||
if (files[entryFile] === undefined) {
|
||||
files[entryFile] = "";
|
||||
}
|
||||
|
||||
return {
|
||||
entryFile,
|
||||
files: Object.fromEntries(
|
||||
Object.entries(files).map(([filePath, content]) => [
|
||||
filePath,
|
||||
applyInstructionTemplateVariables(content, variables),
|
||||
]),
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
function rowIsManagedAgent(
|
||||
row: typeof agents.$inferSelect,
|
||||
pluginKey: string,
|
||||
@@ -299,19 +326,18 @@ export function pluginManagedAgentService(
|
||||
companyId: string,
|
||||
agent: Agent,
|
||||
declaration: PluginManagedAgentDeclaration,
|
||||
options: { replaceExisting: boolean },
|
||||
materializeOptions: { replaceExisting: boolean },
|
||||
): Promise<Agent> {
|
||||
const instructionDeclaration = declaration.instructions;
|
||||
if (!instructionDeclaration?.content) return agent;
|
||||
|
||||
const entryFile = instructionDeclaration.entryFile ?? "AGENTS.md";
|
||||
const variables = await optionsForInstructionVariables(companyId);
|
||||
const declared = declaredInstructionFiles(declaration, variables);
|
||||
if (!declared) return agent;
|
||||
|
||||
const materialized = await instructions.materializeManagedBundle(
|
||||
agent,
|
||||
{ [entryFile]: applyInstructionTemplateVariables(instructionDeclaration.content, variables) },
|
||||
declared.files,
|
||||
{
|
||||
entryFile,
|
||||
replaceExisting: options.replaceExisting,
|
||||
entryFile: declared.entryFile,
|
||||
replaceExisting: materializeOptions.replaceExisting,
|
||||
clearLegacyPromptTemplate: true,
|
||||
},
|
||||
);
|
||||
@@ -325,6 +351,33 @@ export function pluginManagedAgentService(
|
||||
return (updated as Agent | null) ?? { ...agent, adapterConfig: materialized.adapterConfig };
|
||||
}
|
||||
|
||||
async function managedInstructionDefaultDrift(
|
||||
companyId: string,
|
||||
agent: Agent | null,
|
||||
declaration: PluginManagedAgentDeclaration,
|
||||
): Promise<PluginManagedAgentResolution["defaultDrift"]> {
|
||||
if (!agent) return null;
|
||||
const variables = await optionsForInstructionVariables(companyId);
|
||||
const declared = declaredInstructionFiles(declaration, variables);
|
||||
if (!declared) return null;
|
||||
|
||||
let exported: Awaited<ReturnType<typeof instructions.exportFiles>>;
|
||||
try {
|
||||
exported = await instructions.exportFiles(agent);
|
||||
} catch {
|
||||
return { entryFile: declared.entryFile, changedFiles: [declared.entryFile] };
|
||||
}
|
||||
|
||||
const paths = new Set([...Object.keys(declared.files), ...Object.keys(exported.files)]);
|
||||
const changedFiles = [...paths]
|
||||
.filter((filePath) => (exported.files[filePath] ?? null) !== (declared.files[filePath] ?? null))
|
||||
.sort((left, right) => left.localeCompare(right));
|
||||
if (exported.entryFile !== declared.entryFile && !changedFiles.includes(declared.entryFile)) {
|
||||
changedFiles.unshift(declared.entryFile);
|
||||
}
|
||||
return changedFiles.length > 0 ? { entryFile: declared.entryFile, changedFiles } : null;
|
||||
}
|
||||
|
||||
async function optionsForInstructionVariables(companyId: string) {
|
||||
return options.instructionTemplateVariables ? options.instructionTemplateVariables(companyId) : {};
|
||||
}
|
||||
@@ -333,13 +386,13 @@ export function pluginManagedAgentService(
|
||||
return options.pluginKey;
|
||||
}
|
||||
|
||||
function resolution(
|
||||
async function resolution(
|
||||
companyId: string,
|
||||
declaration: PluginManagedAgentDeclaration,
|
||||
agent: Agent | null,
|
||||
status: PluginManagedAgentResolution["status"],
|
||||
approvalId?: string | null,
|
||||
): PluginManagedAgentResolution {
|
||||
): Promise<PluginManagedAgentResolution> {
|
||||
return {
|
||||
pluginKey: options.pluginKey,
|
||||
resourceKind: "agent",
|
||||
@@ -349,6 +402,7 @@ export function pluginManagedAgentService(
|
||||
agent,
|
||||
status,
|
||||
approvalId: approvalId ?? null,
|
||||
defaultDrift: await managedInstructionDefaultDrift(companyId, agent, declaration),
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user