forked from farhoodlabs/paperclip
486fb88a15
> _Stacked on top of #5685 → #5686. Diff against master includes commits from earlier PRs in the stack — review focuses on the two new commits (`Extend sandbox callback bridge for Worker-hosted plugins` + `Add Cloudflare sandbox provider plugin`)._ ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Each agent runs in a sandbox environment, and operators choose which provider backs that sandbox — today E2B and Daytona are bundled with the platform > - Cloudflare Workers + Durable Objects + the Sandbox SDK offer a credible new option: globally distributed, cheap idle, and operator-deployable as a single Worker > - To plug it in, Paperclip needs (a) a provider plugin that speaks the `PaperclipPluginManifestV1` lifecycle and (b) a small operator-deployed Worker — the **bridge** — that adapts Paperclip's runtime RPCs to the Cloudflare Sandbox SDK > - The plugin extends the existing sandbox-callback-bridge with a `bridge.transport: "worker"` discriminator so the platform routes runtime RPCs through the Worker bridge instead of the in-process runner > - This pull request adds the plugin, the bridge Worker template, and the supporting adapter-utils + server hooks the new transport needs > - The benefit is that operators can run sandboxes on Cloudflare's edge with no new platform code beyond installing the plugin and deploying the Worker ## What Changed **Shared support (`Extend sandbox callback bridge for Worker-hosted plugins`):** - `packages/adapter-utils/src/sandbox-callback-bridge.{ts,test.ts}`: expose `expectedHostHeader` so plugin-side bridge clients can verify the canonical request envelope before forwarding. - `packages/adapter-utils/src/command-managed-runtime.{ts,test.ts}`: relax the always-fresh runner construction so callers can re-use a runner across exec calls (Worker-hosted bridges hold the runner inside a Durable Object). - `server/src/services/environment-runtime.ts` + `environment-runtime.test.ts`: route Worker-hosted bridges through the same env-shaping path as E2B and pin the `requestEnv` contract. - `server/src/services/plugin-environment-driver.ts`: thread an optional `issueId` through the runtime descriptor so bridges can scope leases to the originating issue (used by Cloudflare to map a sandbox to the issue/workflow for billing and audit). - `packages/plugins/sdk/src/protocol.ts`: add `issueId?` to `PluginEnvironmentDriverBaseParams` and the new `bridge.transport: "worker"` discriminator that the new plugin declares. - `server/__tests__/heartbeat-plugin-environment.test.ts`: pin the heartbeat path against the new runtime descriptor. **The Cloudflare plugin itself (`Add Cloudflare sandbox provider plugin`):** - `packages/plugins/sandbox-providers/cloudflare/`: plugin entry, manifest, plugin runtime (lifecycle + bridge client), config parsing, and Vitest coverage. Manifest declares `bridge.transport: "worker"` so the platform routes runtime RPCs through the bridge client. - `bridge-template/`: a Worker template the operator deploys with `wrangler`. Owns Durable Object-backed sessions (`sessions.ts`), exec/stream routes (`exec.ts`, `routes.ts`), and an HMAC auth layer (`auth.ts`) that pins the `Host` header surface. Includes the SDK-contract-correct exec implementation, lease recovery, and chunked stdout/stderr streaming. - Tests cover lease/session handoff (`bridge-template/src/exec.test.ts`, `routes.test.ts`), bridge client request shaping (`src/bridge-client.test.ts`), and end-to-end plugin behavior (`src/plugin.test.ts`) including streamed exec output. 27 tests in total. - `README.md` walks the operator through deploying the bridge Worker, registering the plugin, and configuring the runtime. ## Verification - `pnpm typecheck` - `pnpm exec vitest run --no-coverage packages/adapter-utils/src/sandbox-callback-bridge.test.ts packages/adapter-utils/src/command-managed-runtime.test.ts server/src/__tests__/environment-runtime.test.ts server/src/__tests__/heartbeat-plugin-environment.test.ts` - `(cd packages/plugins/sandbox-providers/cloudflare && pnpm test)` — 27 passing For an operator-side smoke test: 1. Deploy the bridge: `cd packages/plugins/sandbox-providers/cloudflare/bridge-template && wrangler deploy` 2. Register the plugin in your Paperclip instance, point its bridge URL at the deployed Worker, set the HMAC shared secret. 3. Create a sandbox environment whose provider is `cloudflare`, then run a Codex or Claude job against it. ## Risks - Adds a new `bridge.transport: "worker"` code path, but the existing E2B / Daytona transports go through the same shaped helpers and have explicit test coverage that pins their behavior unchanged. - The Worker bridge stores session state in a Durable Object; operator instances must be aware of the corresponding Cloudflare costs (DO requests, storage). Documented in the README. - The `issueId` plumbing is optional throughout — existing plugins that don't supply it continue to work. ## Model Used - Provider: Anthropic - Model: Claude Opus 4.7 (1M context) - Capabilities used: extended reasoning, tool use (Read/Edit/Bash/Grep) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots — N/A, no UI change - [x] I have updated relevant documentation to reflect my changes (plugin README, bridge-template README) - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
148 lines
5.0 KiB
TypeScript
148 lines
5.0 KiB
TypeScript
import type { Sandbox as CloudflareSandbox } from "@cloudflare/sandbox";
|
|
import { shellQuote } from "./helpers.js";
|
|
import { isTimeoutError } from "./sandboxes.js";
|
|
import { cleanupTimedOutExecution, resolveExecutionTarget, type SessionStrategy } from "./sessions.js";
|
|
|
|
export interface BridgeExecuteParams {
|
|
sandbox: CloudflareSandbox;
|
|
command: string;
|
|
args?: string[];
|
|
cwd?: string;
|
|
env?: Record<string, string>;
|
|
stdin?: string | null;
|
|
timeoutMs?: number;
|
|
sessionStrategy: SessionStrategy;
|
|
sessionId?: string;
|
|
onOutput?: (stream: "stdout" | "stderr", data: string) => void | Promise<void>;
|
|
}
|
|
|
|
function isValidShellEnvKey(value: string): boolean {
|
|
return /^[A-Za-z_][A-Za-z0-9_]*$/.test(value);
|
|
}
|
|
|
|
function randomToken(): string {
|
|
const uuid = globalThis.crypto?.randomUUID?.();
|
|
if (typeof uuid === "string" && uuid.length > 0) return uuid.replace(/[^a-zA-Z0-9-]/g, "");
|
|
return `${Date.now()}-${Math.random().toString(36).slice(2)}`;
|
|
}
|
|
|
|
export function buildLoginShellScript(input: {
|
|
command: string;
|
|
args: string[];
|
|
cwd?: string;
|
|
env?: Record<string, string>;
|
|
stdinFile?: string | null;
|
|
}): string {
|
|
const env = input.env ?? {};
|
|
for (const key of Object.keys(env)) {
|
|
if (!isValidShellEnvKey(key)) {
|
|
throw new Error(`Invalid sandbox environment variable key: ${key}`);
|
|
}
|
|
}
|
|
|
|
const envArgs = Object.entries(env)
|
|
.filter((entry): entry is [string, string] => typeof entry[1] === "string")
|
|
.map(([key, value]) => `${key}=${shellQuote(value)}`);
|
|
const commandParts = [shellQuote(input.command), ...input.args.map(shellQuote)].join(" ");
|
|
const stdinRedirect = input.stdinFile ? ` < ${shellQuote(input.stdinFile)}` : "";
|
|
const lines = [
|
|
'if [ -f /etc/profile ]; then . /etc/profile >/dev/null 2>&1 || true; fi',
|
|
'if [ -f "$HOME/.profile" ]; then . "$HOME/.profile" >/dev/null 2>&1 || true; fi',
|
|
'if [ -f "$HOME/.bash_profile" ]; then . "$HOME/.bash_profile" >/dev/null 2>&1 || true; elif [ -f "$HOME/.bashrc" ]; then . "$HOME/.bashrc" >/dev/null 2>&1 || true; fi',
|
|
'if [ -f "$HOME/.zprofile" ]; then . "$HOME/.zprofile" >/dev/null 2>&1 || true; fi',
|
|
'export NVM_DIR="${NVM_DIR:-$HOME/.nvm}"',
|
|
'[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" >/dev/null 2>&1 || true',
|
|
];
|
|
if (input.cwd) {
|
|
lines.push(`cd ${shellQuote(input.cwd)}`);
|
|
}
|
|
const execLine = envArgs.length > 0
|
|
? `exec env ${envArgs.join(" ")} ${commandParts}${stdinRedirect}`
|
|
: `exec ${commandParts}${stdinRedirect}`;
|
|
lines.push(execLine);
|
|
return lines.join(" && ");
|
|
}
|
|
|
|
function coerceExecuteResult(result: {
|
|
success?: boolean;
|
|
stdout?: string;
|
|
stderr?: string;
|
|
exitCode?: number | null;
|
|
}) {
|
|
return {
|
|
exitCode:
|
|
typeof result.exitCode === "number" || result.exitCode === null
|
|
? result.exitCode
|
|
: result.success === false
|
|
? 1
|
|
: 0,
|
|
signal: null,
|
|
timedOut: false,
|
|
stdout: result.stdout ?? "",
|
|
stderr: result.stderr ?? "",
|
|
};
|
|
}
|
|
|
|
export async function executeInSandbox(params: BridgeExecuteParams) {
|
|
// The @cloudflare/sandbox SDK's exec() takes a single command string and a
|
|
// narrow option set ({ cwd, env, timeout, ... }) — it does not accept `args`
|
|
// or `stdin`. We compose the full shell command ourselves and stage stdin
|
|
// through a temp file in the sandbox when the caller provides one.
|
|
const stdinPayload = typeof params.stdin === "string" && params.stdin.length > 0
|
|
? params.stdin
|
|
: null;
|
|
const stdinFile = stdinPayload ? `/tmp/.paperclip-bridge-stdin-${randomToken()}` : null;
|
|
|
|
if (stdinFile && stdinPayload) {
|
|
await params.sandbox.writeFile(stdinFile, stdinPayload, { encoding: "utf8" });
|
|
}
|
|
|
|
try {
|
|
const target = await resolveExecutionTarget(params.sandbox, {
|
|
sessionStrategy: params.sessionStrategy,
|
|
sessionId: params.sessionId,
|
|
cwd: params.cwd,
|
|
env: params.env,
|
|
timeoutMs: params.timeoutMs,
|
|
});
|
|
const script = buildLoginShellScript({
|
|
command: params.command,
|
|
args: params.args ?? [],
|
|
cwd: params.cwd,
|
|
env: params.env,
|
|
stdinFile,
|
|
});
|
|
const fullCommand = `sh -lc ${shellQuote(script)}`;
|
|
const result = await target.exec(fullCommand, {
|
|
cwd: "/",
|
|
timeout: params.timeoutMs,
|
|
...(typeof params.onOutput === "function"
|
|
? {
|
|
stream: true,
|
|
onOutput: params.onOutput,
|
|
}
|
|
: {}),
|
|
});
|
|
return coerceExecuteResult(result);
|
|
} catch (error) {
|
|
if (isTimeoutError(error)) {
|
|
await cleanupTimedOutExecution(params.sandbox, {
|
|
sessionStrategy: params.sessionStrategy,
|
|
sessionId: params.sessionId,
|
|
});
|
|
return {
|
|
exitCode: null,
|
|
signal: null,
|
|
timedOut: true,
|
|
stdout: typeof (error as { stdout?: unknown }).stdout === "string" ? (error as { stdout: string }).stdout : "",
|
|
stderr: `${error instanceof Error ? error.message : String(error)}\n`,
|
|
};
|
|
}
|
|
throw error;
|
|
} finally {
|
|
if (stdinFile) {
|
|
await params.sandbox.deleteFile?.(stdinFile).catch(() => undefined);
|
|
}
|
|
}
|
|
}
|