Files
paperclip/packages/adapters/codex-local/src/server/codex-home.ts
T
Dotta 0096b56a1c [codex] Add LLM Wiki plugin host support (#5597)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies.
> - The plugin system needs host contracts and runtime support before
large plugins can integrate cleanly.
> - The source branch mixed the LLM Wiki package with supporting
host/runtime work, managed plugin skills, root-level storage spaces, and
a bookmarks reference plugin.
> - [PAP-9173](/PAP/issues/PAP-9173) asked for the current branch to be
split by file boundary: plugin package separately from everything else.
> - [PAP-9188](/PAP/issues/PAP-9188) clarified that LLM Wiki may have
plugin-local spaces, but Paperclip core should not reorganize top-level
local storage into spaces.
> - Follow-up review clarified that the bookmarks example should not
ship in this PR either.
> - This pull request contains the
non-`packages/plugins/plugin-llm-wiki/` host/runtime work, keeps runtime
state under the selected Paperclip instance root, and no longer includes
the bookmarks example.

## What Changed

- Added/updated plugin host contracts, SDK types, worker RPC plumbing,
managed plugin skill support, and related server tests.
- Removed the bookmarks example plugin package and its
bundled-example/workspace references.
- Removed the root-level local spaces CLI/migration surface and restored
instance-root runtime defaults for config, db, logs, storage, secrets,
workspaces, projects, and adapter homes.
- Replaced shared root `space-paths` helpers with `home-paths` helpers
for core runtime storage.
- Tightened stranded recovery unique-conflict detection so concurrent
recovery scans reuse the raced recovery issue when Postgres errors are
wrapped.
- Kept `packages/plugins/plugin-llm-wiki/` out of this PR diff;
plugin-local spaces remain in the stacked plugin-only PR.

## Verification

- `pnpm exec vitest run cli/src/__tests__/data-dir.test.ts
cli/src/__tests__/home-paths.test.ts cli/src/__tests__/onboard.test.ts
packages/shared/src/home-paths.test.ts
packages/db/src/runtime-config.test.ts
server/src/__tests__/agent-instructions-service.test.ts
server/src/__tests__/claude-local-execute.test.ts
server/src/__tests__/codex-local-execute.test.ts`
- `pnpm exec vitest run packages/db/src/runtime-config.test.ts`
- `pnpm exec vitest run
server/src/__tests__/plugin-routes-authz.test.ts`
- `pnpm --filter @paperclipai/server typecheck`
- `pnpm exec vitest run
server/src/__tests__/heartbeat-process-recovery.test.ts -t "reuses the
raced stranded recovery issue"` skipped locally because embedded
Postgres did not initialize on this macOS temp host; the code path was
typechecked and is covered by Linux CI.
- Boundary check: no core references remain for `PAPERCLIP_SPACE_ID`,
`spaces migrate-default`, `@paperclipai/shared/space-paths`,
`registerSpacesCommands`, or the removed bookmarks example.
- Previous PR head `4f23e034` had green GitHub checks: `verify`, all
four serialized server shards, `e2e`, `Canary Dry Run`, `policy`, Snyk,
and `Greptile Review`. Current head `582f466d` is re-running checks
after the bookmarks deletion.

## Risks

- Plugin host changes touch shared runtime paths, so regressions would
most likely appear in adapter startup, plugin loading, or local dev path
defaults.
- Removing the bookmarks example also removes one demonstration of
plugin database namespaces plus local-folder persistence; remaining
plugin examples still cover bundled example discovery and plugin host
flows.
- The plugin package itself is intentionally deferred to the stacked
plugin-only PR, where LLM Wiki plugin-local spaces live.
- Existing installs that tested the transient root-level spaces CLI
should stop using it; this PR intentionally removes that unsupported
migration surface before merge.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI GPT-5 Codex via Codex CLI, tool use and local code execution
enabled; context window not exposed.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass, except where noted above
for host-specific embedded Postgres initialization
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

Stacked follow-up: PR #5592 contains only
`packages/plugins/plugin-llm-wiki/` and targets this branch.

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-05-10 07:34:12 -05:00

145 lines
5.0 KiB
TypeScript

import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import type { AdapterExecutionContext } from "@paperclipai/adapter-utils";
import { resolvePaperclipInstanceRootForAdapter } from "@paperclipai/adapter-utils/server-utils";
const TRUTHY_ENV_RE = /^(1|true|yes|on)$/i;
const COPIED_SHARED_FILES = ["config.json", "config.toml", "instructions.md"] as const;
const SYMLINKED_SHARED_FILES = ["auth.json"] as const;
function nonEmpty(value: string | undefined): string | null {
return typeof value === "string" && value.trim().length > 0 ? value.trim() : null;
}
export async function pathExists(candidate: string): Promise<boolean> {
return fs.access(candidate).then(() => true).catch(() => false);
}
export function resolveSharedCodexHomeDir(
env: NodeJS.ProcessEnv = process.env,
): string {
const fromEnv = nonEmpty(env.CODEX_HOME);
return fromEnv ? path.resolve(fromEnv) : path.join(os.homedir(), ".codex");
}
function isWorktreeMode(env: NodeJS.ProcessEnv): boolean {
return TRUTHY_ENV_RE.test(env.PAPERCLIP_IN_WORKTREE ?? "");
}
export function resolveManagedCodexHomeDir(
env: NodeJS.ProcessEnv,
companyId?: string,
): string {
const instanceRoot = resolvePaperclipInstanceRootForAdapter({
homeDir: nonEmpty(env.PAPERCLIP_HOME) ?? undefined,
instanceId: nonEmpty(env.PAPERCLIP_INSTANCE_ID) ?? undefined,
env,
});
return companyId
? path.resolve(instanceRoot, "companies", companyId, "codex-home")
: path.resolve(instanceRoot, "codex-home");
}
async function ensureParentDir(target: string): Promise<void> {
await fs.mkdir(path.dirname(target), { recursive: true });
}
async function ensureSymlink(target: string, source: string): Promise<void> {
const existing = await fs.lstat(target).catch(() => null);
if (!existing) {
await ensureParentDir(target);
await fs.symlink(source, target);
return;
}
if (!existing.isSymbolicLink()) {
return;
}
const linkedPath = await fs.readlink(target).catch(() => null);
if (!linkedPath) return;
const resolvedLinkedPath = path.resolve(path.dirname(target), linkedPath);
if (resolvedLinkedPath === source) return;
await fs.unlink(target);
await fs.symlink(source, target);
}
async function ensureCopiedFile(target: string, source: string): Promise<void> {
const existing = await fs.lstat(target).catch(() => null);
if (existing) return;
await ensureParentDir(target);
await fs.copyFile(source, target);
}
/**
* Writes an `auth.json` containing only `OPENAI_API_KEY` so the codex CLI can
* authenticate via API key. Overwrites any existing file or symlink at that
* path. Required because the codex CLI (>= 0.122) ignores the `OPENAI_API_KEY`
* environment variable and only reads credentials from `$CODEX_HOME/auth.json`.
*/
export async function writeApiKeyAuthJson(home: string, apiKey: string): Promise<void> {
await fs.mkdir(home, { recursive: true });
const target = path.join(home, "auth.json");
await fs.rm(target, { force: true });
await fs.writeFile(target, JSON.stringify({ OPENAI_API_KEY: apiKey }), { mode: 0o600 });
}
export async function prepareManagedCodexHome(
env: NodeJS.ProcessEnv,
onLog: AdapterExecutionContext["onLog"],
companyId?: string,
options: { apiKey?: string | null } = {},
): Promise<string> {
const targetHome = resolveManagedCodexHomeDir(env, companyId);
const apiKey = nonEmpty(options.apiKey ?? undefined);
const sourceHome = resolveSharedCodexHomeDir(env);
const seedFromShared = path.resolve(sourceHome) !== path.resolve(targetHome);
await fs.mkdir(targetHome, { recursive: true });
// If a previous run wrote an apikey-mode auth.json (regular file) and this
// run has no apiKey, remove it so the chatgpt-mode symlink can be restored.
// Without this cleanup, ensureSymlink bails on a non-symlink and Codex keeps
// authenticating with the stale key after it is removed from configuration.
if (!apiKey && seedFromShared) {
const authPath = path.join(targetHome, "auth.json");
const existing = await fs.lstat(authPath).catch(() => null);
if (existing && !existing.isSymbolicLink()) {
await fs.rm(authPath, { force: true });
}
}
if (seedFromShared) {
for (const name of SYMLINKED_SHARED_FILES) {
const source = path.join(sourceHome, name);
if (!(await pathExists(source))) continue;
await ensureSymlink(path.join(targetHome, name), source);
}
for (const name of COPIED_SHARED_FILES) {
const source = path.join(sourceHome, name);
if (!(await pathExists(source))) continue;
await ensureCopiedFile(path.join(targetHome, name), source);
}
await onLog(
"stdout",
`[paperclip] Using ${isWorktreeMode(env) ? "worktree-isolated" : "Paperclip-managed"} Codex home "${targetHome}" (seeded from "${sourceHome}").\n`,
);
}
if (apiKey) {
await writeApiKeyAuthJson(targetHome, apiKey);
await onLog(
"stdout",
`[paperclip] Wrote API-key auth.json into Codex home "${targetHome}" from configured OPENAI_API_KEY.\n`,
);
}
return targetHome;
}