forked from farhoodlabs/paperclip
778e775c35
## Thinking Path > - Paperclip orchestrates AI-agent companies and needs secrets handling to work across local development, hosted operators, and governed agent execution. > - The affected subsystem is the company-scoped secrets control plane: database schema, server services/routes, CLI workflows, and the Secrets settings UI. > - The gap was that secrets were local-only and operators could not manage provider vaults or import existing remote references without exposing plaintext. > - This branch adds provider vault configuration plus an AWS Secrets Manager remote-import path while preserving company boundaries, binding context, and audit trails. > - I kept the PR to a single branch PR, removed unrelated lockfile/package drift, rebased the full branch onto the current `public-gh/master`, and addressed fresh Greptile findings. > - The benefit is a reviewable implementation of provider-backed secrets with focused tests covering provider selection, import conflicts, deleted secret reuse, rotation guards, and AWS signing behavior. ## What Changed - Added provider vault support for company secrets, including provider config storage, default vault handling, health checks, binding usage, access events, and remote import preview/commit. - Added an AWS Secrets Manager provider using SigV4 request signing, bounded request timeouts, namespace guardrails, cached runtime credential resolution, and external-reference linking without plaintext reads. - Added Secrets UI surfaces for vault management and remote import, plus CLI/API documentation for setup and operations. - Stabilized routine webhook secret binding paths and SSH environment-driver fixture bindings discovered during verification. - Addressed Greptile and CI findings: no lockfile/package drift, monotonic migration metadata, disabled-vault default races, soft-deleted secret hiding/recreate behavior, remove behavior with disabled vaults, soft-deleted external-reference re-import, non-active rotation guards, managed-secret soft deletion through PATCH, and per-call AWS SDK credential client churn. - Rebased this branch onto `public-gh/master` at `0e1a5828` and force-pushed with lease to keep this as the single PR for the branch. ## Verification - `git fetch public-gh master` - `git rebase public-gh/master` - `git diff --name-only public-gh/master...HEAD | grep '^pnpm-lock\.yaml$' || true` confirmed `pnpm-lock.yaml` is not in the PR diff. - Confirmed migration ordering: master ends at `0081_optimal_dormammu`; this PR adds `0082_dry_vision` and `0083_company_secret_provider_configs`. - Inspected migrations for repeat safety: new tables/indexes use `IF NOT EXISTS`; foreign keys are guarded by `DO $$ ... IF NOT EXISTS`; column additions use `ADD COLUMN IF NOT EXISTS`. - `pnpm -r typecheck` passed before the Greptile follow-up commits. - `pnpm test:run` ran the full stable Vitest path before the Greptile follow-up commits; it completed with 3 timing-related failures under parallel load: `codex-local-execute.test.ts`, `cursor-local-execute.test.ts`, and `environment-service.test.ts`. - `pnpm --filter @paperclipai/server exec vitest run src/__tests__/codex-local-execute.test.ts src/__tests__/cursor-local-execute.test.ts src/__tests__/environment-service.test.ts` passed on targeted rerun (`24/24`). - `pnpm build` passed before the Greptile follow-up commits. Vite reported existing chunk-size/dynamic-import warnings. - After Greptile follow-up commits: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/secrets-service.test.ts` passed (`26/26`). - After Greptile follow-up commits: `pnpm --filter @paperclipai/server exec vitest run src/__tests__/aws-secrets-manager-provider.test.ts src/__tests__/secrets-service.test.ts` passed (`39/39`). - After Greptile follow-up commits: `pnpm --filter @paperclipai/server typecheck` passed. - Captured Storybook screenshots from `ui/storybook-static` for visual review. - Latest PR checks on `5ca3a5cf`: `policy`, serialized server suites 1/4-4/4, `Canary Dry Run`, `e2e`, `security/snyk`, and `Greptile Review` pass; aggregate `verify` is still registering the completed child checks. - Greptile review loop continued through the latest requested pass; all Greptile review threads are resolved and the latest `Greptile Review` check on `5ca3a5cf` passed with 0 comments added. ## Screenshots Before: the provider-vault and remote-import surfaces did not exist on `master`; these are after-state screenshots from the Storybook fixtures.    ## Risks - Migration risk: this adds new secret provider tables and extends existing secret rows. The migrations were checked for monotonic ordering and idempotent guards, but reviewers should still inspect upgrade behavior carefully. - Provider risk: AWS support uses direct SigV4 requests. Automated tests cover signing, request timeouts, vault-config selection, namespace guardrails, pending-version archival, sanitized provider errors, and service-level cleanup paths. A real-vault AWS smoke test remains deployment validation for an operator with AWS credentials rather than an unverified merge blocker in this local branch. - UI risk: the Secrets page and import dialog are large new surfaces; screenshots are included above for reviewer inspection. - Verification risk: the full local stable test command hit parallel-load timing failures, although the exact failed files passed when rerun directly. - Operational risk: remote import intentionally avoids plaintext reads; operators must understand that imported external references resolve at runtime and may fail if AWS permissions change. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent with local shell/tool use in the Paperclip worktree. Exact context-window size was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
449 lines
14 KiB
TypeScript
449 lines
14 KiB
TypeScript
import { describe, expect, it } from "vitest";
|
|
import fs from "node:fs/promises";
|
|
import os from "node:os";
|
|
import path from "node:path";
|
|
import { runChildProcess } from "@paperclipai/adapter-utils/server-utils";
|
|
import { execute } from "@paperclipai/adapter-cursor-local/server";
|
|
|
|
async function writeFakeCursorCommand(commandPath: string): Promise<void> {
|
|
const script = `#!/usr/bin/env node
|
|
const fs = require("node:fs");
|
|
|
|
const capturePath = process.env.PAPERCLIP_TEST_CAPTURE_PATH;
|
|
const payload = {
|
|
argv: process.argv.slice(2),
|
|
prompt: fs.readFileSync(0, "utf8"),
|
|
paperclipEnvKeys: Object.keys(process.env)
|
|
.filter((key) => key.startsWith("PAPERCLIP_"))
|
|
.sort(),
|
|
};
|
|
if (capturePath) {
|
|
fs.writeFileSync(capturePath, JSON.stringify(payload), "utf8");
|
|
}
|
|
console.log(JSON.stringify({
|
|
type: "system",
|
|
subtype: "init",
|
|
session_id: "cursor-session-1",
|
|
model: "auto",
|
|
}));
|
|
console.log(JSON.stringify({
|
|
type: "assistant",
|
|
message: { content: [{ type: "output_text", text: "hello" }] },
|
|
}));
|
|
console.log(JSON.stringify({
|
|
type: "result",
|
|
subtype: "success",
|
|
session_id: "cursor-session-1",
|
|
result: "ok",
|
|
}));
|
|
`;
|
|
await fs.writeFile(commandPath, script, "utf8");
|
|
await fs.chmod(commandPath, 0o755);
|
|
}
|
|
|
|
async function writeFakeSandboxCursorAgent(commandPath: string, capturePath: string): Promise<void> {
|
|
const script = `#!/usr/bin/env node
|
|
const fs = require("node:fs");
|
|
|
|
const payload = {
|
|
command: process.argv[1],
|
|
argv: process.argv.slice(2),
|
|
prompt: fs.readFileSync(0, "utf8"),
|
|
path: process.env.PATH || "",
|
|
};
|
|
fs.writeFileSync(${JSON.stringify(capturePath)}, JSON.stringify(payload), "utf8");
|
|
console.log(JSON.stringify({
|
|
type: "system",
|
|
subtype: "init",
|
|
session_id: "cursor-session-remote-1",
|
|
model: "auto",
|
|
}));
|
|
console.log(JSON.stringify({
|
|
type: "assistant",
|
|
message: { content: [{ type: "output_text", text: "hello" }] },
|
|
}));
|
|
console.log(JSON.stringify({
|
|
type: "result",
|
|
subtype: "success",
|
|
session_id: "cursor-session-remote-1",
|
|
result: "ok",
|
|
}));
|
|
`;
|
|
await fs.mkdir(path.dirname(commandPath), { recursive: true });
|
|
await fs.writeFile(commandPath, script, "utf8");
|
|
await fs.chmod(commandPath, 0o755);
|
|
}
|
|
|
|
function createLocalSandboxRunner() {
|
|
let counter = 0;
|
|
return {
|
|
execute: async (input: {
|
|
command: string;
|
|
args?: string[];
|
|
cwd?: string;
|
|
env?: Record<string, string>;
|
|
stdin?: string;
|
|
timeoutMs?: number;
|
|
onLog?: (stream: "stdout" | "stderr", chunk: string) => Promise<void>;
|
|
onSpawn?: (meta: { pid: number; startedAt: string }) => Promise<void>;
|
|
}) => {
|
|
counter += 1;
|
|
return await runChildProcess(`cursor-sandbox-execute-${counter}`, input.command, input.args ?? [], {
|
|
cwd: input.cwd ?? process.cwd(),
|
|
env: input.env ?? {},
|
|
stdin: input.stdin,
|
|
timeoutSec: Math.max(1, Math.ceil((input.timeoutMs ?? 30_000) / 1000)),
|
|
graceSec: 5,
|
|
onLog: input.onLog ?? (async () => {}),
|
|
onSpawn: input.onSpawn
|
|
? async (meta) => input.onSpawn?.({ pid: meta.pid, startedAt: meta.startedAt })
|
|
: undefined,
|
|
});
|
|
},
|
|
};
|
|
}
|
|
|
|
type CapturePayload = {
|
|
argv: string[];
|
|
prompt: string;
|
|
paperclipEnvKeys: string[];
|
|
};
|
|
|
|
async function createSkillDir(root: string, name: string) {
|
|
const skillDir = path.join(root, name);
|
|
await fs.mkdir(skillDir, { recursive: true });
|
|
await fs.writeFile(path.join(skillDir, "SKILL.md"), `---\nname: ${name}\n---\n`, "utf8");
|
|
return skillDir;
|
|
}
|
|
|
|
describe("cursor execute", () => {
|
|
it("injects paperclip env vars and prompt note by default", async () => {
|
|
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-cursor-execute-"));
|
|
const workspace = path.join(root, "workspace");
|
|
const commandPath = path.join(root, "agent");
|
|
const capturePath = path.join(root, "capture.json");
|
|
await fs.mkdir(workspace, { recursive: true });
|
|
await writeFakeCursorCommand(commandPath);
|
|
|
|
const previousHome = process.env.HOME;
|
|
process.env.HOME = root;
|
|
|
|
let invocationPrompt = "";
|
|
try {
|
|
const result = await execute({
|
|
runId: "run-1",
|
|
agent: {
|
|
id: "agent-1",
|
|
companyId: "company-1",
|
|
name: "Cursor Coder",
|
|
adapterType: "cursor",
|
|
adapterConfig: {},
|
|
},
|
|
runtime: {
|
|
sessionId: null,
|
|
sessionParams: null,
|
|
sessionDisplayId: null,
|
|
taskKey: null,
|
|
},
|
|
config: {
|
|
command: commandPath,
|
|
cwd: workspace,
|
|
model: "auto",
|
|
env: {
|
|
PAPERCLIP_TEST_CAPTURE_PATH: capturePath,
|
|
},
|
|
promptTemplate: "Follow the paperclip heartbeat.",
|
|
},
|
|
context: {},
|
|
authToken: "run-jwt-token",
|
|
onLog: async () => {},
|
|
onMeta: async (meta) => {
|
|
invocationPrompt = meta.prompt ?? "";
|
|
},
|
|
});
|
|
|
|
expect(result.exitCode).toBe(0);
|
|
expect(result.errorMessage).toBeNull();
|
|
|
|
const capture = JSON.parse(await fs.readFile(capturePath, "utf8")) as CapturePayload;
|
|
expect(capture.argv).not.toContain("Follow the paperclip heartbeat.");
|
|
expect(capture.argv).not.toContain("--mode");
|
|
expect(capture.argv).not.toContain("ask");
|
|
expect(capture.paperclipEnvKeys).toEqual(
|
|
expect.arrayContaining([
|
|
"PAPERCLIP_AGENT_ID",
|
|
"PAPERCLIP_API_KEY",
|
|
"PAPERCLIP_API_URL",
|
|
"PAPERCLIP_COMPANY_ID",
|
|
"PAPERCLIP_RUN_ID",
|
|
]),
|
|
);
|
|
expect(capture.prompt).toContain("Paperclip runtime note:");
|
|
expect(capture.prompt).toContain("PAPERCLIP_API_KEY");
|
|
expect(invocationPrompt).toContain("Paperclip runtime note:");
|
|
expect(invocationPrompt).toContain("PAPERCLIP_API_URL");
|
|
} finally {
|
|
if (previousHome === undefined) {
|
|
delete process.env.HOME;
|
|
} else {
|
|
process.env.HOME = previousHome;
|
|
}
|
|
await fs.rm(root, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
it("passes --mode when explicitly configured", async () => {
|
|
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-cursor-execute-mode-"));
|
|
const workspace = path.join(root, "workspace");
|
|
const commandPath = path.join(root, "agent");
|
|
const capturePath = path.join(root, "capture.json");
|
|
await fs.mkdir(workspace, { recursive: true });
|
|
await writeFakeCursorCommand(commandPath);
|
|
|
|
const previousHome = process.env.HOME;
|
|
process.env.HOME = root;
|
|
|
|
try {
|
|
const result = await execute({
|
|
runId: "run-2",
|
|
agent: {
|
|
id: "agent-1",
|
|
companyId: "company-1",
|
|
name: "Cursor Coder",
|
|
adapterType: "cursor",
|
|
adapterConfig: {},
|
|
},
|
|
runtime: {
|
|
sessionId: null,
|
|
sessionParams: null,
|
|
sessionDisplayId: null,
|
|
taskKey: null,
|
|
},
|
|
config: {
|
|
command: commandPath,
|
|
cwd: workspace,
|
|
model: "auto",
|
|
mode: "ask",
|
|
env: {
|
|
PAPERCLIP_TEST_CAPTURE_PATH: capturePath,
|
|
},
|
|
promptTemplate: "Follow the paperclip heartbeat.",
|
|
},
|
|
context: {},
|
|
authToken: "run-jwt-token",
|
|
onLog: async () => {},
|
|
});
|
|
|
|
expect(result.exitCode).toBe(0);
|
|
expect(result.errorMessage).toBeNull();
|
|
|
|
const capture = JSON.parse(await fs.readFile(capturePath, "utf8")) as CapturePayload;
|
|
expect(capture.argv).toContain("--mode");
|
|
expect(capture.argv).toContain("ask");
|
|
} finally {
|
|
if (previousHome === undefined) {
|
|
delete process.env.HOME;
|
|
} else {
|
|
process.env.HOME = previousHome;
|
|
}
|
|
await fs.rm(root, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
it("injects company-library runtime skills into the Cursor skills home before execution", async () => {
|
|
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-cursor-execute-runtime-skill-"));
|
|
const workspace = path.join(root, "workspace");
|
|
const commandPath = path.join(root, "agent");
|
|
const runtimeSkillsRoot = path.join(root, "runtime-skills");
|
|
await fs.mkdir(workspace, { recursive: true });
|
|
await writeFakeCursorCommand(commandPath);
|
|
|
|
const paperclipDir = await createSkillDir(runtimeSkillsRoot, "paperclip");
|
|
const asciiHeartDir = await createSkillDir(runtimeSkillsRoot, "ascii-heart");
|
|
|
|
const previousHome = process.env.HOME;
|
|
process.env.HOME = root;
|
|
|
|
try {
|
|
const result = await execute({
|
|
runId: "run-3",
|
|
agent: {
|
|
id: "agent-1",
|
|
companyId: "company-1",
|
|
name: "Cursor Coder",
|
|
adapterType: "cursor",
|
|
adapterConfig: {},
|
|
},
|
|
runtime: {
|
|
sessionId: null,
|
|
sessionParams: null,
|
|
sessionDisplayId: null,
|
|
taskKey: null,
|
|
},
|
|
config: {
|
|
command: commandPath,
|
|
cwd: workspace,
|
|
model: "auto",
|
|
paperclipRuntimeSkills: [
|
|
{
|
|
name: "paperclip",
|
|
source: paperclipDir,
|
|
required: true,
|
|
requiredReason: "Bundled Paperclip skills are always available for local adapters.",
|
|
},
|
|
{
|
|
name: "ascii-heart",
|
|
source: asciiHeartDir,
|
|
},
|
|
],
|
|
paperclipSkillSync: {
|
|
desiredSkills: ["ascii-heart"],
|
|
},
|
|
promptTemplate: "Follow the paperclip heartbeat.",
|
|
},
|
|
context: {},
|
|
authToken: "run-jwt-token",
|
|
onLog: async () => {},
|
|
onMeta: async () => {},
|
|
});
|
|
|
|
expect(result.exitCode).toBe(0);
|
|
expect(result.errorMessage).toBeNull();
|
|
expect((await fs.lstat(path.join(root, ".cursor", "skills", "ascii-heart"))).isSymbolicLink()).toBe(true);
|
|
expect(await fs.realpath(path.join(root, ".cursor", "skills", "ascii-heart"))).toBe(
|
|
await fs.realpath(asciiHeartDir),
|
|
);
|
|
} finally {
|
|
if (previousHome === undefined) {
|
|
delete process.env.HOME;
|
|
} else {
|
|
process.env.HOME = previousHome;
|
|
}
|
|
await fs.rm(root, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
it("prefers ~/.local/bin/cursor-agent for remote sandbox execution when using the default command", async () => {
|
|
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-cursor-sandbox-execute-"));
|
|
const homeDir = path.join(root, "home");
|
|
const workspace = path.join(root, "workspace");
|
|
const remoteWorkspace = path.join(root, "remote-workspace");
|
|
const capturePath = path.join(root, "capture.json");
|
|
const cursorAgentPath = path.join(homeDir, ".local", "bin", "cursor-agent");
|
|
await fs.mkdir(workspace, { recursive: true });
|
|
await fs.mkdir(remoteWorkspace, { recursive: true });
|
|
await writeFakeSandboxCursorAgent(cursorAgentPath, capturePath);
|
|
|
|
const previousHome = process.env.HOME;
|
|
process.env.HOME = homeDir;
|
|
|
|
try {
|
|
const result = await execute({
|
|
runId: "run-sandbox-1",
|
|
agent: {
|
|
id: "agent-1",
|
|
companyId: "company-1",
|
|
name: "Cursor Coder",
|
|
adapterType: "cursor",
|
|
adapterConfig: {},
|
|
},
|
|
runtime: {
|
|
sessionId: null,
|
|
sessionParams: null,
|
|
sessionDisplayId: null,
|
|
taskKey: null,
|
|
},
|
|
executionTarget: {
|
|
kind: "remote",
|
|
transport: "sandbox",
|
|
remoteCwd: remoteWorkspace,
|
|
runner: createLocalSandboxRunner(),
|
|
timeoutMs: 30_000,
|
|
},
|
|
config: {
|
|
command: "agent",
|
|
cwd: workspace,
|
|
promptTemplate: "Follow the paperclip heartbeat.",
|
|
},
|
|
context: {},
|
|
authToken: "run-jwt-token",
|
|
onLog: async () => {},
|
|
});
|
|
|
|
expect(result.exitCode).toBe(0);
|
|
const capture = JSON.parse(await fs.readFile(capturePath, "utf8")) as {
|
|
command: string;
|
|
argv: string[];
|
|
prompt: string;
|
|
path: string;
|
|
};
|
|
expect(capture.command).toBe(cursorAgentPath);
|
|
expect(capture.path.split(":")[0]).toBe(path.join(homeDir, ".local", "bin"));
|
|
expect(capture.prompt).toContain("Follow the paperclip heartbeat.");
|
|
} finally {
|
|
if (previousHome === undefined) delete process.env.HOME;
|
|
else process.env.HOME = previousHome;
|
|
await fs.rm(root, { recursive: true, force: true });
|
|
}
|
|
}, 10_000);
|
|
|
|
it("keeps explicit command overrides for remote sandbox execution", async () => {
|
|
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-cursor-sandbox-explicit-"));
|
|
const homeDir = path.join(root, "home");
|
|
const workspace = path.join(root, "workspace");
|
|
const remoteWorkspace = path.join(root, "remote-workspace");
|
|
const capturePath = path.join(root, "capture.json");
|
|
const cursorAgentPath = path.join(homeDir, ".local", "bin", "cursor-agent");
|
|
const customCommandPath = path.join(root, "bin", "custom-cursor");
|
|
await fs.mkdir(workspace, { recursive: true });
|
|
await fs.mkdir(remoteWorkspace, { recursive: true });
|
|
await writeFakeSandboxCursorAgent(cursorAgentPath, path.join(root, "unused.json"));
|
|
await writeFakeSandboxCursorAgent(customCommandPath, capturePath);
|
|
|
|
const previousHome = process.env.HOME;
|
|
process.env.HOME = homeDir;
|
|
|
|
try {
|
|
const result = await execute({
|
|
runId: "run-sandbox-2",
|
|
agent: {
|
|
id: "agent-1",
|
|
companyId: "company-1",
|
|
name: "Cursor Coder",
|
|
adapterType: "cursor",
|
|
adapterConfig: {},
|
|
},
|
|
runtime: {
|
|
sessionId: null,
|
|
sessionParams: null,
|
|
sessionDisplayId: null,
|
|
taskKey: null,
|
|
},
|
|
executionTarget: {
|
|
kind: "remote",
|
|
transport: "sandbox",
|
|
remoteCwd: remoteWorkspace,
|
|
runner: createLocalSandboxRunner(),
|
|
timeoutMs: 30_000,
|
|
},
|
|
config: {
|
|
command: customCommandPath,
|
|
cwd: workspace,
|
|
promptTemplate: "Follow the paperclip heartbeat.",
|
|
},
|
|
context: {},
|
|
authToken: "run-jwt-token",
|
|
onLog: async () => {},
|
|
});
|
|
|
|
expect(result.exitCode).toBe(0);
|
|
const capture = JSON.parse(await fs.readFile(capturePath, "utf8")) as { command: string };
|
|
expect(capture.command).toBe(customCommandPath);
|
|
} finally {
|
|
if (previousHome === undefined) delete process.env.HOME;
|
|
else process.env.HOME = previousHome;
|
|
await fs.rm(root, { recursive: true, force: true });
|
|
}
|
|
});
|
|
});
|