forked from farhoodlabs/paperclip
Add Cloudflare sandbox provider plugin (#5687)
> _Stacked on top of #5685 → #5686. Diff against master includes commits from earlier PRs in the stack — review focuses on the two new commits (`Extend sandbox callback bridge for Worker-hosted plugins` + `Add Cloudflare sandbox provider plugin`)._ ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Each agent runs in a sandbox environment, and operators choose which provider backs that sandbox — today E2B and Daytona are bundled with the platform > - Cloudflare Workers + Durable Objects + the Sandbox SDK offer a credible new option: globally distributed, cheap idle, and operator-deployable as a single Worker > - To plug it in, Paperclip needs (a) a provider plugin that speaks the `PaperclipPluginManifestV1` lifecycle and (b) a small operator-deployed Worker — the **bridge** — that adapts Paperclip's runtime RPCs to the Cloudflare Sandbox SDK > - The plugin extends the existing sandbox-callback-bridge with a `bridge.transport: "worker"` discriminator so the platform routes runtime RPCs through the Worker bridge instead of the in-process runner > - This pull request adds the plugin, the bridge Worker template, and the supporting adapter-utils + server hooks the new transport needs > - The benefit is that operators can run sandboxes on Cloudflare's edge with no new platform code beyond installing the plugin and deploying the Worker ## What Changed **Shared support (`Extend sandbox callback bridge for Worker-hosted plugins`):** - `packages/adapter-utils/src/sandbox-callback-bridge.{ts,test.ts}`: expose `expectedHostHeader` so plugin-side bridge clients can verify the canonical request envelope before forwarding. - `packages/adapter-utils/src/command-managed-runtime.{ts,test.ts}`: relax the always-fresh runner construction so callers can re-use a runner across exec calls (Worker-hosted bridges hold the runner inside a Durable Object). - `server/src/services/environment-runtime.ts` + `environment-runtime.test.ts`: route Worker-hosted bridges through the same env-shaping path as E2B and pin the `requestEnv` contract. - `server/src/services/plugin-environment-driver.ts`: thread an optional `issueId` through the runtime descriptor so bridges can scope leases to the originating issue (used by Cloudflare to map a sandbox to the issue/workflow for billing and audit). - `packages/plugins/sdk/src/protocol.ts`: add `issueId?` to `PluginEnvironmentDriverBaseParams` and the new `bridge.transport: "worker"` discriminator that the new plugin declares. - `server/__tests__/heartbeat-plugin-environment.test.ts`: pin the heartbeat path against the new runtime descriptor. **The Cloudflare plugin itself (`Add Cloudflare sandbox provider plugin`):** - `packages/plugins/sandbox-providers/cloudflare/`: plugin entry, manifest, plugin runtime (lifecycle + bridge client), config parsing, and Vitest coverage. Manifest declares `bridge.transport: "worker"` so the platform routes runtime RPCs through the bridge client. - `bridge-template/`: a Worker template the operator deploys with `wrangler`. Owns Durable Object-backed sessions (`sessions.ts`), exec/stream routes (`exec.ts`, `routes.ts`), and an HMAC auth layer (`auth.ts`) that pins the `Host` header surface. Includes the SDK-contract-correct exec implementation, lease recovery, and chunked stdout/stderr streaming. - Tests cover lease/session handoff (`bridge-template/src/exec.test.ts`, `routes.test.ts`), bridge client request shaping (`src/bridge-client.test.ts`), and end-to-end plugin behavior (`src/plugin.test.ts`) including streamed exec output. 27 tests in total. - `README.md` walks the operator through deploying the bridge Worker, registering the plugin, and configuring the runtime. ## Verification - `pnpm typecheck` - `pnpm exec vitest run --no-coverage packages/adapter-utils/src/sandbox-callback-bridge.test.ts packages/adapter-utils/src/command-managed-runtime.test.ts server/src/__tests__/environment-runtime.test.ts server/src/__tests__/heartbeat-plugin-environment.test.ts` - `(cd packages/plugins/sandbox-providers/cloudflare && pnpm test)` — 27 passing For an operator-side smoke test: 1. Deploy the bridge: `cd packages/plugins/sandbox-providers/cloudflare/bridge-template && wrangler deploy` 2. Register the plugin in your Paperclip instance, point its bridge URL at the deployed Worker, set the HMAC shared secret. 3. Create a sandbox environment whose provider is `cloudflare`, then run a Codex or Claude job against it. ## Risks - Adds a new `bridge.transport: "worker"` code path, but the existing E2B / Daytona transports go through the same shaped helpers and have explicit test coverage that pins their behavior unchanged. - The Worker bridge stores session state in a Durable Object; operator instances must be aware of the corresponding Cloudflare costs (DO requests, storage). Documented in the README. - The `issueId` plumbing is optional throughout — existing plugins that don't supply it continue to work. ## Model Used - Provider: Anthropic - Model: Claude Opus 4.7 (1M context) - Capabilities used: extended reasoning, tool use (Read/Edit/Bash/Grep) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots — N/A, no UI change - [x] I have updated relevant documentation to reflect my changes (plugin README, bridge-template README) - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Devin Foley
@@ -531,7 +531,7 @@ describeEmbeddedPostgres("environmentRuntimeService", () => {
|
||||
expect(released).toHaveLength(1);
|
||||
expect(released[0]?.lease.status).toBe("released");
|
||||
expect(workerManager.call).toHaveBeenCalledWith(pluginId, "environmentExecute", expect.anything(), 31000);
|
||||
expect(workerManager.call).toHaveBeenCalledWith(pluginId, "environmentReleaseLease", expect.anything());
|
||||
expect(workerManager.call).toHaveBeenCalledWith(pluginId, "environmentReleaseLease", expect.anything(), 31234);
|
||||
});
|
||||
|
||||
it("uses resolved secret-ref config for plugin-backed sandbox execute and release", async () => {
|
||||
@@ -682,7 +682,7 @@ describeEmbeddedPostgres("environmentRuntimeService", () => {
|
||||
config: expect.objectContaining({
|
||||
apiKey: "resolved-provider-key",
|
||||
}),
|
||||
}));
|
||||
}), 31234);
|
||||
});
|
||||
|
||||
it("waits briefly for a ready sandbox provider plugin worker to come online", async () => {
|
||||
@@ -774,7 +774,104 @@ describeEmbeddedPostgres("environmentRuntimeService", () => {
|
||||
|
||||
expect(acquired.lease.providerLeaseId).toBe("sandbox-1");
|
||||
expect(workerManager.isRunning).toHaveBeenCalledTimes(3);
|
||||
expect(workerManager.call).toHaveBeenCalledWith(pluginId, "environmentAcquireLease", expect.anything());
|
||||
expect(workerManager.call).toHaveBeenCalledWith(pluginId, "environmentAcquireLease", expect.anything(), 31234);
|
||||
});
|
||||
|
||||
it("extends plugin-backed sandbox lease RPC timeouts from provider config", async () => {
|
||||
const pluginId = randomUUID();
|
||||
const { companyId, environment: baseEnvironment, runId } = await seedEnvironment();
|
||||
const providerConfig = {
|
||||
provider: "fake-plugin",
|
||||
image: "fake:test",
|
||||
timeoutMs: 1_234,
|
||||
bridgeRequestTimeoutMs: 40_000,
|
||||
reuseLease: false,
|
||||
};
|
||||
const environment = {
|
||||
...baseEnvironment,
|
||||
name: "Long Lease Plugin Sandbox",
|
||||
driver: "sandbox",
|
||||
config: providerConfig,
|
||||
};
|
||||
await environmentService(db).update(environment.id, {
|
||||
driver: "sandbox",
|
||||
name: environment.name,
|
||||
config: providerConfig,
|
||||
});
|
||||
await db.insert(plugins).values({
|
||||
id: pluginId,
|
||||
pluginKey: "acme.long-lease-sandbox-provider",
|
||||
packageName: "@acme/long-lease-sandbox-provider",
|
||||
version: "1.0.0",
|
||||
apiVersion: 1,
|
||||
categories: ["automation"],
|
||||
manifestJson: {
|
||||
id: "acme.long-lease-sandbox-provider",
|
||||
apiVersion: 1,
|
||||
version: "1.0.0",
|
||||
displayName: "Long Lease Sandbox Provider",
|
||||
description: "Test plugin worker acquire timeout",
|
||||
author: "Paperclip",
|
||||
categories: ["automation"],
|
||||
capabilities: ["environment.drivers.register"],
|
||||
entrypoints: { worker: "dist/worker.js" },
|
||||
environmentDrivers: [
|
||||
{
|
||||
driverKey: "fake-plugin",
|
||||
kind: "sandbox_provider",
|
||||
displayName: "Fake Plugin",
|
||||
configSchema: { type: "object" },
|
||||
},
|
||||
],
|
||||
},
|
||||
status: "ready",
|
||||
installOrder: 1,
|
||||
updatedAt: new Date(),
|
||||
} as any);
|
||||
|
||||
const workerManager = {
|
||||
isRunning: vi.fn((id: string) => id === pluginId),
|
||||
call: vi.fn(async (_pluginId: string, method: string) => {
|
||||
if (method === "environmentAcquireLease") {
|
||||
return {
|
||||
providerLeaseId: "sandbox-1",
|
||||
metadata: {
|
||||
provider: "fake-plugin",
|
||||
image: "fake:test",
|
||||
timeoutMs: 1_234,
|
||||
bridgeRequestTimeoutMs: 40_000,
|
||||
reuseLease: false,
|
||||
},
|
||||
};
|
||||
}
|
||||
throw new Error(`Unexpected plugin method: ${method}`);
|
||||
}),
|
||||
} as unknown as PluginWorkerManager;
|
||||
const runtimeWithPlugin = environmentRuntimeService(db, { pluginWorkerManager: workerManager });
|
||||
|
||||
const acquired = await runtimeWithPlugin.acquireRunLease({
|
||||
companyId,
|
||||
environment,
|
||||
issueId: null,
|
||||
heartbeatRunId: runId,
|
||||
persistedExecutionWorkspace: null,
|
||||
});
|
||||
|
||||
expect(acquired.lease.providerLeaseId).toBe("sandbox-1");
|
||||
expect(workerManager.call).toHaveBeenCalledWith(
|
||||
pluginId,
|
||||
"environmentAcquireLease",
|
||||
expect.objectContaining({
|
||||
driverKey: "fake-plugin",
|
||||
config: {
|
||||
image: "fake:test",
|
||||
timeoutMs: 1_234,
|
||||
bridgeRequestTimeoutMs: 40_000,
|
||||
reuseLease: false,
|
||||
},
|
||||
}),
|
||||
70_000,
|
||||
);
|
||||
});
|
||||
|
||||
it("falls back to acquire when plugin-backed sandbox lease resume throws", async () => {
|
||||
@@ -884,7 +981,7 @@ describeEmbeddedPostgres("environmentRuntimeService", () => {
|
||||
expect(workerManager.call).toHaveBeenNthCalledWith(1, pluginId, "environmentResumeLease", expect.objectContaining({
|
||||
driverKey: "fake-plugin",
|
||||
providerLeaseId: "stale-plugin-lease",
|
||||
}));
|
||||
}), 31234);
|
||||
expect(workerManager.call).toHaveBeenNthCalledWith(2, pluginId, "environmentAcquireLease", expect.objectContaining({
|
||||
driverKey: "fake-plugin",
|
||||
config: {
|
||||
@@ -893,7 +990,7 @@ describeEmbeddedPostgres("environmentRuntimeService", () => {
|
||||
reuseLease: true,
|
||||
},
|
||||
runId,
|
||||
}));
|
||||
}), 31234);
|
||||
});
|
||||
|
||||
it("releases a sandbox run lease from metadata after the environment config changes", async () => {
|
||||
@@ -1008,6 +1105,7 @@ describeEmbeddedPostgres("environmentRuntimeService", () => {
|
||||
driverKey: "fake-plugin",
|
||||
companyId,
|
||||
environmentId: environment.id,
|
||||
issueId: null,
|
||||
config: { template: "base" },
|
||||
runId,
|
||||
workspaceMode: undefined,
|
||||
@@ -1043,6 +1141,7 @@ describeEmbeddedPostgres("environmentRuntimeService", () => {
|
||||
driverKey: "fake-plugin",
|
||||
companyId,
|
||||
environmentId: environment.id,
|
||||
issueId: null,
|
||||
config: {},
|
||||
providerLeaseId: "plugin-lease-1",
|
||||
leaseMetadata: expect.objectContaining({
|
||||
@@ -1201,6 +1300,7 @@ describeEmbeddedPostgres("environmentRuntimeService", () => {
|
||||
driverKey: "fake-plugin",
|
||||
companyId,
|
||||
environmentId: environment.id,
|
||||
issueId: null,
|
||||
config: { template: "base" },
|
||||
providerLeaseId: "plugin-lease-full",
|
||||
leaseMetadata: expect.objectContaining({
|
||||
@@ -1231,6 +1331,7 @@ describeEmbeddedPostgres("environmentRuntimeService", () => {
|
||||
driverKey: "fake-plugin",
|
||||
companyId,
|
||||
environmentId: environment.id,
|
||||
issueId: null,
|
||||
config: { template: "base" },
|
||||
providerLeaseId: "plugin-lease-full",
|
||||
leaseMetadata: expect.objectContaining({
|
||||
|
||||
@@ -206,6 +206,7 @@ describeEmbeddedPostgres("heartbeat plugin environments", () => {
|
||||
driverKey: "sandbox",
|
||||
companyId,
|
||||
environmentId,
|
||||
issueId: null,
|
||||
config: { template: "base" },
|
||||
runId: run!.id,
|
||||
workspaceMode: "shared_workspace",
|
||||
@@ -215,6 +216,7 @@ describeEmbeddedPostgres("heartbeat plugin environments", () => {
|
||||
driverKey: "sandbox",
|
||||
companyId,
|
||||
environmentId,
|
||||
issueId: null,
|
||||
config: { template: "base" },
|
||||
providerLeaseId: "plugin-heartbeat-lease",
|
||||
leaseMetadata: expect.objectContaining({
|
||||
@@ -421,6 +423,7 @@ describeEmbeddedPostgres("heartbeat plugin environments", () => {
|
||||
driverKey: "sandbox",
|
||||
companyId,
|
||||
environmentId: newEnvironmentId,
|
||||
issueId,
|
||||
config: { template: "new" },
|
||||
runId: run!.id,
|
||||
workspaceMode: "shared_workspace",
|
||||
|
||||
@@ -120,6 +120,24 @@ export interface EnvironmentDriverReleaseInput {
|
||||
status: Extract<EnvironmentLeaseStatus, "released" | "expired" | "failed">;
|
||||
}
|
||||
|
||||
function resolvePluginSandboxRpcTimeoutMs(config: Record<string, unknown>): number | undefined {
|
||||
const timeoutCandidates = [
|
||||
typeof config.timeoutMs === "number" ? config.timeoutMs : undefined,
|
||||
typeof config.bridgeRequestTimeoutMs === "number" ? config.bridgeRequestTimeoutMs : undefined,
|
||||
]
|
||||
.filter((value): value is number => typeof value === "number" && Number.isFinite(value) && value > 0)
|
||||
.map((value) => Math.trunc(value));
|
||||
|
||||
if (timeoutCandidates.length === 0) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
return resolvePluginExecuteRpcTimeoutMs({
|
||||
requestedTimeoutMs: Math.max(...timeoutCandidates),
|
||||
config,
|
||||
});
|
||||
}
|
||||
|
||||
export interface EnvironmentDriverLeaseInput {
|
||||
environment: Environment;
|
||||
lease: EnvironmentLease;
|
||||
@@ -446,10 +464,12 @@ function createSandboxEnvironmentDriver(
|
||||
driverKey: parsed.config.provider,
|
||||
companyId: input.companyId,
|
||||
environmentId: input.environment.id,
|
||||
issueId: input.issueId,
|
||||
config: workerConfig,
|
||||
providerLeaseId: reusableLease.providerLeaseId,
|
||||
leaseMetadata: reusableLease.metadata ?? undefined,
|
||||
},
|
||||
resolvePluginSandboxRpcTimeoutMs(workerConfig),
|
||||
).then((resumed) =>
|
||||
typeof resumed.providerLeaseId === "string" && resumed.providerLeaseId.length > 0
|
||||
? resumed
|
||||
@@ -463,6 +483,7 @@ function createSandboxEnvironmentDriver(
|
||||
driverKey: parsed.config.provider,
|
||||
companyId: input.companyId,
|
||||
environmentId: input.environment.id,
|
||||
issueId: input.issueId,
|
||||
config: workerConfig,
|
||||
// Plugin SDK requires a string; ad-hoc test leases use a fresh
|
||||
// UUID so providers that validate or persist the runId still see
|
||||
@@ -470,6 +491,7 @@ function createSandboxEnvironmentDriver(
|
||||
runId: input.heartbeatRunId ?? randomUUID(),
|
||||
workspaceMode: input.executionWorkspaceMode ?? undefined,
|
||||
},
|
||||
resolvePluginSandboxRpcTimeoutMs(workerConfig),
|
||||
);
|
||||
|
||||
// Ad-hoc test leases are never publishable for reuse: storing them
|
||||
@@ -616,6 +638,7 @@ function createSandboxEnvironmentDriver(
|
||||
driverKey: providerKey,
|
||||
companyId: input.lease.companyId,
|
||||
environmentId: input.environment.id,
|
||||
issueId: input.lease.issueId,
|
||||
config: stripSandboxProviderEnvelope(config as SandboxEnvironmentConfig),
|
||||
lease: {
|
||||
providerLeaseId: input.lease.providerLeaseId,
|
||||
@@ -623,7 +646,7 @@ function createSandboxEnvironmentDriver(
|
||||
expiresAt: input.lease.expiresAt?.toISOString() ?? null,
|
||||
},
|
||||
workspace: input.workspace,
|
||||
});
|
||||
}, resolvePluginSandboxRpcTimeoutMs(stripSandboxProviderEnvelope(config as SandboxEnvironmentConfig)));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -660,6 +683,7 @@ function createSandboxEnvironmentDriver(
|
||||
driverKey: providerKey,
|
||||
companyId: input.lease.companyId,
|
||||
environmentId: input.environment.id,
|
||||
issueId: input.lease.issueId,
|
||||
config: sanitizedConfig,
|
||||
lease: {
|
||||
providerLeaseId: input.lease.providerLeaseId,
|
||||
@@ -701,10 +725,11 @@ function createSandboxEnvironmentDriver(
|
||||
driverKey: providerKey,
|
||||
companyId: input.lease.companyId,
|
||||
environmentId: input.environment.id,
|
||||
issueId: input.lease.issueId,
|
||||
config: stripSandboxProviderEnvelope(config as SandboxEnvironmentConfig),
|
||||
providerLeaseId: input.lease.providerLeaseId,
|
||||
leaseMetadata: metadata,
|
||||
});
|
||||
}, resolvePluginSandboxRpcTimeoutMs(stripSandboxProviderEnvelope(config as SandboxEnvironmentConfig)));
|
||||
} catch {
|
||||
cleanupStatus = "failed";
|
||||
}
|
||||
@@ -869,6 +894,7 @@ function createPluginEnvironmentDriver(
|
||||
driverKey: parsed.config.driverKey,
|
||||
companyId: input.companyId,
|
||||
environmentId: input.environment.id,
|
||||
issueId: input.issueId,
|
||||
config: parsed.config.driverConfig,
|
||||
runId: input.heartbeatRunId ?? randomUUID(),
|
||||
workspaceMode: input.executionWorkspaceMode ?? undefined,
|
||||
@@ -901,6 +927,7 @@ function createPluginEnvironmentDriver(
|
||||
driverKey,
|
||||
companyId: input.lease.companyId,
|
||||
environmentId: input.environment.id,
|
||||
issueId: input.lease.issueId,
|
||||
config: driverConfig,
|
||||
providerLeaseId: input.lease.providerLeaseId,
|
||||
leaseMetadata: input.lease.metadata ?? undefined,
|
||||
@@ -921,6 +948,7 @@ function createPluginEnvironmentDriver(
|
||||
workerManager,
|
||||
companyId: input.lease.companyId,
|
||||
environmentId: input.environment.id,
|
||||
issueId: input.lease.issueId,
|
||||
config: {
|
||||
pluginKey,
|
||||
driverKey,
|
||||
@@ -941,6 +969,7 @@ function createPluginEnvironmentDriver(
|
||||
workerManager,
|
||||
companyId: input.lease.companyId,
|
||||
environmentId: input.environment.id,
|
||||
issueId: input.lease.issueId,
|
||||
config: {
|
||||
pluginKey,
|
||||
driverKey,
|
||||
@@ -971,6 +1000,7 @@ function createPluginEnvironmentDriver(
|
||||
driverKey,
|
||||
companyId: input.lease.companyId,
|
||||
environmentId: input.environment.id,
|
||||
issueId: input.lease.issueId,
|
||||
config: driverConfig,
|
||||
lease: {
|
||||
providerLeaseId: input.lease.providerLeaseId,
|
||||
@@ -1001,6 +1031,7 @@ function createPluginEnvironmentDriver(
|
||||
driverKey,
|
||||
companyId: input.lease.companyId,
|
||||
environmentId: input.environment.id,
|
||||
issueId: input.lease.issueId,
|
||||
config: driverConfig,
|
||||
lease: {
|
||||
providerLeaseId: input.lease.providerLeaseId,
|
||||
|
||||
@@ -247,6 +247,7 @@ export async function resumePluginEnvironmentLease(input: {
|
||||
workerManager: PluginWorkerManager;
|
||||
companyId: string;
|
||||
environmentId: string;
|
||||
issueId?: string | null;
|
||||
config: PluginEnvironmentConfig;
|
||||
providerLeaseId: string;
|
||||
leaseMetadata?: Record<string, unknown>;
|
||||
@@ -256,6 +257,7 @@ export async function resumePluginEnvironmentLease(input: {
|
||||
driverKey: input.config.driverKey,
|
||||
companyId: input.companyId,
|
||||
environmentId: input.environmentId,
|
||||
issueId: input.issueId ?? null,
|
||||
config: input.config.driverConfig,
|
||||
providerLeaseId: input.providerLeaseId,
|
||||
leaseMetadata: input.leaseMetadata,
|
||||
@@ -267,6 +269,7 @@ export async function destroyPluginEnvironmentLease(input: {
|
||||
workerManager: PluginWorkerManager;
|
||||
companyId: string;
|
||||
environmentId: string;
|
||||
issueId?: string | null;
|
||||
config: PluginEnvironmentConfig;
|
||||
providerLeaseId: string | null;
|
||||
leaseMetadata?: Record<string, unknown>;
|
||||
@@ -276,6 +279,7 @@ export async function destroyPluginEnvironmentLease(input: {
|
||||
driverKey: input.config.driverKey,
|
||||
companyId: input.companyId,
|
||||
environmentId: input.environmentId,
|
||||
issueId: input.issueId ?? null,
|
||||
config: input.config.driverConfig,
|
||||
providerLeaseId: input.providerLeaseId,
|
||||
leaseMetadata: input.leaseMetadata,
|
||||
|
||||
Reference in New Issue
Block a user