forked from farhoodlabs/paperclip
Add ACPX local adapter runtime (#4893)
## Thinking Path > - Paperclip orchestrates AI-agent companies through a control plane that can start, supervise, and recover agent runs. > - Local adapters are the bridge between Paperclip issues and concrete agent runtimes such as Claude, Codex, and other ACP-compatible tools. > - The roadmap calls out broader “bring your own agent” and claw-style agent support, and ACPX gives Paperclip one path to normalize multiple ACP agents behind a single adapter. > - The branch needed to become one reviewable PR against current `paperclipai/paperclip:master`, without carrying stale base conflicts or generated lockfile churn. > - This pull request adds an experimental built-in `acpx_local` adapter, integrates it through the server/CLI/UI adapter surfaces, and adds regression coverage for runtime execution, skill sync, stream parsing, diagnostics, and log redaction. > - The benefit is that Paperclip can run Claude/Codex/custom ACP agents through ACPX while keeping operator configuration, skills, logging, and transcript rendering inside the existing adapter model. ## What Changed - Added `@paperclipai/adapter-acpx-local` with server execution, config schema, ACPX session handling, CLI formatting, UI config helpers, and stdout parsing. - Registered `acpx_local` across CLI, server, shared constants, UI adapter metadata, adapter capabilities, and agent creation/editing surfaces. - Added ACPX runtime execution support with persistent sessions, local-agent JWT environment handling, skill snapshots, runtime skill materialization, and isolation/security regressions. - Added ACPX adapter diagnostics and marked the adapter experimental in the UI. - Added command/env secret redaction for resolved command metadata in adapter-utils, server event storage, and the Agent Detail invocation UI. - Added Storybook coverage for ACPX config, transcript rendering, and skill states, plus PR screenshots under `docs/pr-screenshots/pap-2944/`. - Rebased the branch onto current `public-gh/master`; `pnpm-lock.yaml` is intentionally not included and there are no migration/schema changes. ## Verification - `pnpm exec vitest run packages/adapters/acpx-local/src/server/execute.test.ts packages/adapters/acpx-local/src/server/test.test.ts packages/adapters/acpx-local/src/cli/format-event.test.ts packages/adapters/acpx-local/src/ui/parse-stdout.test.ts packages/adapter-utils/src/server-utils.test.ts server/src/__tests__/redaction.test.ts server/src/__tests__/acpx-local-execute.test.ts server/src/__tests__/acpx-local-skill-sync.test.ts server/src/__tests__/acpx-local-adapter-environment.test.ts server/src/__tests__/adapter-routes.test.ts server/src/__tests__/agent-skills-routes.test.ts ui/src/adapters/metadata.test.ts` — 12 files, 87 tests passed. - `pnpm --filter @paperclipai/adapter-acpx-local typecheck` — passed. - `pnpm --filter @paperclipai/server typecheck` — passed. - `pnpm --filter @paperclipai/ui typecheck` — passed. - Confirmed PR diff does not include `pnpm-lock.yaml`, database schema files, or migrations. Screenshots:    ## Risks - Medium risk: this introduces a new built-in adapter package and touches runtime execution, adapter registration, agent config, skills, and transcript rendering. - ACPX and ACP agent behavior can vary by installed tool versions; the adapter is marked experimental to set operator expectations. - `pnpm-lock.yaml` is excluded per repository PR policy, so dependency lock refresh must be handled by the repo’s automation or maintainers. - No database migration risk: no schema or migration files changed. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent based on GPT-5, with repository tool use, shell execution, git operations, and local verification. Exact hosted context window was not exposed in this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta
@@ -1,7 +1,9 @@
|
||||
import { spawn, type ChildProcess } from "node:child_process";
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { constants as fsConstants, promises as fs, type Dirent } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { buildSshSpawnTarget, type SshRemoteExecutionSpec } from "./ssh.js";
|
||||
import { redactCommandText } from "./command-redaction.js";
|
||||
import type {
|
||||
AdapterSkillEntry,
|
||||
AdapterSkillSnapshot,
|
||||
@@ -76,10 +78,14 @@ export const MAX_CAPTURE_BYTES = 4 * 1024 * 1024;
|
||||
export const MAX_EXCERPT_BYTES = 32 * 1024;
|
||||
const TERMINAL_RESULT_SCAN_OVERLAP_CHARS = 64 * 1024;
|
||||
const SENSITIVE_ENV_KEY = /(key|token|secret|password|passwd|authorization|cookie)/i;
|
||||
const REDACTED_LOG_VALUE = "***REDACTED***";
|
||||
const PAPERCLIP_SKILL_ROOT_RELATIVE_CANDIDATES = [
|
||||
"../../skills",
|
||||
"../../../../../skills",
|
||||
];
|
||||
const MATERIALIZED_SKILL_SENTINEL = ".paperclip-materialized-skill.json";
|
||||
const MATERIALIZED_SKILL_LOCK_OWNER = "owner.json";
|
||||
const MATERIALIZED_SKILL_LOCK_STALE_MS = 30_000;
|
||||
|
||||
export const DEFAULT_PAPERCLIP_AGENT_PROMPT_TEMPLATE = [
|
||||
"You are agent {{agent.id}} ({{agent.name}}). Continue your Paperclip work.",
|
||||
@@ -111,6 +117,11 @@ export interface InstalledSkillTarget {
|
||||
kind: "symlink" | "directory" | "file";
|
||||
}
|
||||
|
||||
export interface MaterializedPaperclipSkillCopyResult {
|
||||
copiedFiles: number;
|
||||
skippedSymlinks: string[];
|
||||
}
|
||||
|
||||
interface PersistentSkillSnapshotOptions {
|
||||
adapterType: string;
|
||||
availableEntries: PaperclipSkillEntry[];
|
||||
@@ -780,11 +791,15 @@ export function renderPaperclipWakePrompt(
|
||||
export function redactEnvForLogs(env: Record<string, string>): Record<string, string> {
|
||||
const redacted: Record<string, string> = {};
|
||||
for (const [key, value] of Object.entries(env)) {
|
||||
redacted[key] = SENSITIVE_ENV_KEY.test(key) ? "***REDACTED***" : value;
|
||||
redacted[key] = SENSITIVE_ENV_KEY.test(key) ? REDACTED_LOG_VALUE : value;
|
||||
}
|
||||
return redacted;
|
||||
}
|
||||
|
||||
export function redactCommandTextForLogs(command: string): string {
|
||||
return redactCommandText(command, REDACTED_LOG_VALUE);
|
||||
}
|
||||
|
||||
export function buildInvocationEnvForLogs(
|
||||
env: Record<string, string>,
|
||||
options: {
|
||||
@@ -806,7 +821,7 @@ export function buildInvocationEnvForLogs(
|
||||
|
||||
const resolvedCommand = options.resolvedCommand?.trim();
|
||||
if (resolvedCommand) {
|
||||
merged[options.resolvedCommandEnvKey ?? "PAPERCLIP_RESOLVED_COMMAND"] = resolvedCommand;
|
||||
merged[options.resolvedCommandEnvKey ?? "PAPERCLIP_RESOLVED_COMMAND"] = redactCommandTextForLogs(resolvedCommand);
|
||||
}
|
||||
|
||||
return redactEnvForLogs(merged);
|
||||
@@ -1395,6 +1410,190 @@ export async function ensurePaperclipSkillSymlink(
|
||||
return "repaired";
|
||||
}
|
||||
|
||||
async function hashSkillDirectory(root: string): Promise<string> {
|
||||
const hash = createHash("sha256");
|
||||
|
||||
async function visit(candidate: string, relativePath: string): Promise<void> {
|
||||
const stat = await fs.lstat(candidate);
|
||||
if (stat.isSymbolicLink()) {
|
||||
hash.update(`symlink:${relativePath}\n`);
|
||||
return;
|
||||
}
|
||||
if (stat.isDirectory()) {
|
||||
hash.update(`dir:${relativePath}\n`);
|
||||
const entries = await fs.readdir(candidate, { withFileTypes: true });
|
||||
entries.sort((left, right) => left.name.localeCompare(right.name));
|
||||
for (const entry of entries) {
|
||||
const childRelativePath = relativePath ? `${relativePath}/${entry.name}` : entry.name;
|
||||
await visit(path.join(candidate, entry.name), childRelativePath);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (stat.isFile()) {
|
||||
hash.update(`file:${relativePath}:${stat.mode}\n`);
|
||||
hash.update(await fs.readFile(candidate));
|
||||
hash.update("\n");
|
||||
return;
|
||||
}
|
||||
hash.update(`other:${relativePath}:${stat.mode}\n`);
|
||||
}
|
||||
|
||||
await visit(root, "");
|
||||
return hash.digest("hex");
|
||||
}
|
||||
|
||||
async function materializedSkillFingerprintMatches(targetRoot: string, sourceFingerprint: string): Promise<boolean> {
|
||||
try {
|
||||
const raw = JSON.parse(await fs.readFile(path.join(targetRoot, MATERIALIZED_SKILL_SENTINEL), "utf8")) as unknown;
|
||||
const parsed = parseObject(raw);
|
||||
return parsed.version === 1 && parsed.sourceFingerprint === sourceFingerprint;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
async function acquireMaterializeLock(lockDir: string): Promise<() => Promise<void>> {
|
||||
await fs.mkdir(path.dirname(lockDir), { recursive: true });
|
||||
const deadline = Date.now() + MATERIALIZED_SKILL_LOCK_STALE_MS;
|
||||
while (true) {
|
||||
try {
|
||||
await fs.mkdir(lockDir);
|
||||
await fs.writeFile(
|
||||
path.join(lockDir, MATERIALIZED_SKILL_LOCK_OWNER),
|
||||
`${JSON.stringify({ pid: process.pid, createdAt: new Date().toISOString() })}\n`,
|
||||
"utf8",
|
||||
);
|
||||
return async () => {
|
||||
await fs.rm(lockDir, { recursive: true, force: true });
|
||||
};
|
||||
} catch (err) {
|
||||
const code = err && typeof err === "object" ? (err as { code?: unknown }).code : null;
|
||||
if (code !== "EEXIST") throw err;
|
||||
if (await removeStaleMaterializeLock(lockDir, MATERIALIZED_SKILL_LOCK_STALE_MS)) continue;
|
||||
if (Date.now() >= deadline) {
|
||||
throw new Error(`Timed out waiting for Paperclip skill materialization lock at ${lockDir}`);
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function isPidAlive(pid: number): boolean {
|
||||
if (!Number.isInteger(pid) || pid <= 0) return false;
|
||||
try {
|
||||
process.kill(pid, 0);
|
||||
return true;
|
||||
} catch (err) {
|
||||
const code = err && typeof err === "object" ? (err as { code?: unknown }).code : null;
|
||||
return code === "EPERM";
|
||||
}
|
||||
}
|
||||
|
||||
async function removeStaleMaterializeLock(lockDir: string, staleMs: number): Promise<boolean> {
|
||||
const ownerPath = path.join(lockDir, MATERIALIZED_SKILL_LOCK_OWNER);
|
||||
let shouldRemove = false;
|
||||
try {
|
||||
const raw = JSON.parse(await fs.readFile(ownerPath, "utf8")) as unknown;
|
||||
const owner = parseObject(raw);
|
||||
const pid = typeof owner.pid === "number" ? owner.pid : 0;
|
||||
const createdAt = typeof owner.createdAt === "string" ? Date.parse(owner.createdAt) : Number.NaN;
|
||||
const ageMs = Number.isFinite(createdAt) ? Date.now() - createdAt : staleMs + 1;
|
||||
shouldRemove = !isPidAlive(pid) || ageMs > staleMs;
|
||||
} catch {
|
||||
const stat = await fs.stat(lockDir).catch(() => null);
|
||||
shouldRemove = !stat || Date.now() - stat.mtimeMs > staleMs;
|
||||
}
|
||||
if (!shouldRemove) return false;
|
||||
await fs.rm(lockDir, { recursive: true, force: true }).catch(() => {});
|
||||
return true;
|
||||
}
|
||||
|
||||
export async function materializePaperclipSkillCopy(
|
||||
source: string,
|
||||
target: string,
|
||||
): Promise<MaterializedPaperclipSkillCopyResult> {
|
||||
const sourceRoot = path.resolve(source);
|
||||
const targetRoot = path.resolve(target);
|
||||
const relativeTarget = path.relative(sourceRoot, targetRoot);
|
||||
const relativeSource = path.relative(targetRoot, sourceRoot);
|
||||
if (
|
||||
!relativeTarget ||
|
||||
(!relativeTarget.startsWith("..") && !path.isAbsolute(relativeTarget)) ||
|
||||
!relativeSource ||
|
||||
(!relativeSource.startsWith("..") && !path.isAbsolute(relativeSource))
|
||||
) {
|
||||
throw new Error("Refusing to materialize a skill into itself, an ancestor, or one of its descendants.");
|
||||
}
|
||||
|
||||
const rootStat = await fs.lstat(sourceRoot);
|
||||
if (rootStat.isSymbolicLink()) {
|
||||
throw new Error("Refusing to materialize a skill root that is itself a symlink.");
|
||||
}
|
||||
if (!rootStat.isDirectory()) {
|
||||
throw new Error("Paperclip skills must be directories.");
|
||||
}
|
||||
|
||||
const result: MaterializedPaperclipSkillCopyResult = {
|
||||
copiedFiles: 0,
|
||||
skippedSymlinks: [],
|
||||
};
|
||||
|
||||
const lockDir = `${targetRoot}.lock`;
|
||||
const releaseLock = await acquireMaterializeLock(lockDir);
|
||||
const tempRoot = `${targetRoot}.tmp-${process.pid}-${randomUUID()}`;
|
||||
|
||||
async function copyEntry(sourcePath: string, targetPath: string, relativePath: string): Promise<void> {
|
||||
const stat = await fs.lstat(sourcePath);
|
||||
if (stat.isSymbolicLink()) {
|
||||
result.skippedSymlinks.push(relativePath || ".");
|
||||
return;
|
||||
}
|
||||
|
||||
if (stat.isDirectory()) {
|
||||
await fs.mkdir(targetPath, { recursive: true });
|
||||
const entries = await fs.readdir(sourcePath, { withFileTypes: true });
|
||||
entries.sort((left, right) => left.name.localeCompare(right.name));
|
||||
for (const entry of entries) {
|
||||
const childRelativePath = relativePath ? `${relativePath}/${entry.name}` : entry.name;
|
||||
await copyEntry(path.join(sourcePath, entry.name), path.join(targetPath, entry.name), childRelativePath);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (stat.isFile()) {
|
||||
await fs.mkdir(path.dirname(targetPath), { recursive: true });
|
||||
await fs.copyFile(sourcePath, targetPath, fsConstants.COPYFILE_FICLONE).catch(async () => {
|
||||
await fs.copyFile(sourcePath, targetPath);
|
||||
});
|
||||
await fs.chmod(targetPath, stat.mode).catch(() => {});
|
||||
result.copiedFiles += 1;
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
const sourceFingerprint = await hashSkillDirectory(sourceRoot);
|
||||
if (await materializedSkillFingerprintMatches(targetRoot, sourceFingerprint)) return result;
|
||||
await copyEntry(sourceRoot, tempRoot, "");
|
||||
await fs.writeFile(
|
||||
path.join(tempRoot, MATERIALIZED_SKILL_SENTINEL),
|
||||
`${JSON.stringify({
|
||||
version: 1,
|
||||
sourceFingerprint,
|
||||
copiedFiles: result.copiedFiles,
|
||||
skippedSymlinks: result.skippedSymlinks,
|
||||
}, null, 2)}\n`,
|
||||
"utf8",
|
||||
);
|
||||
if (await materializedSkillFingerprintMatches(targetRoot, sourceFingerprint)) return result;
|
||||
await fs.rm(targetRoot, { recursive: true, force: true });
|
||||
await fs.rename(tempRoot, targetRoot);
|
||||
return result;
|
||||
} finally {
|
||||
await fs.rm(tempRoot, { recursive: true, force: true }).catch(() => {});
|
||||
await releaseLock();
|
||||
}
|
||||
}
|
||||
|
||||
export async function removeMaintainerOnlySkillSymlinks(
|
||||
skillsHome: string,
|
||||
allowedSkillNames: Iterable<string>,
|
||||
|
||||
Reference in New Issue
Block a user