forked from farhoodlabs/paperclip
486fb88a15
> _Stacked on top of #5685 → #5686. Diff against master includes commits from earlier PRs in the stack — review focuses on the two new commits (`Extend sandbox callback bridge for Worker-hosted plugins` + `Add Cloudflare sandbox provider plugin`)._ ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Each agent runs in a sandbox environment, and operators choose which provider backs that sandbox — today E2B and Daytona are bundled with the platform > - Cloudflare Workers + Durable Objects + the Sandbox SDK offer a credible new option: globally distributed, cheap idle, and operator-deployable as a single Worker > - To plug it in, Paperclip needs (a) a provider plugin that speaks the `PaperclipPluginManifestV1` lifecycle and (b) a small operator-deployed Worker — the **bridge** — that adapts Paperclip's runtime RPCs to the Cloudflare Sandbox SDK > - The plugin extends the existing sandbox-callback-bridge with a `bridge.transport: "worker"` discriminator so the platform routes runtime RPCs through the Worker bridge instead of the in-process runner > - This pull request adds the plugin, the bridge Worker template, and the supporting adapter-utils + server hooks the new transport needs > - The benefit is that operators can run sandboxes on Cloudflare's edge with no new platform code beyond installing the plugin and deploying the Worker ## What Changed **Shared support (`Extend sandbox callback bridge for Worker-hosted plugins`):** - `packages/adapter-utils/src/sandbox-callback-bridge.{ts,test.ts}`: expose `expectedHostHeader` so plugin-side bridge clients can verify the canonical request envelope before forwarding. - `packages/adapter-utils/src/command-managed-runtime.{ts,test.ts}`: relax the always-fresh runner construction so callers can re-use a runner across exec calls (Worker-hosted bridges hold the runner inside a Durable Object). - `server/src/services/environment-runtime.ts` + `environment-runtime.test.ts`: route Worker-hosted bridges through the same env-shaping path as E2B and pin the `requestEnv` contract. - `server/src/services/plugin-environment-driver.ts`: thread an optional `issueId` through the runtime descriptor so bridges can scope leases to the originating issue (used by Cloudflare to map a sandbox to the issue/workflow for billing and audit). - `packages/plugins/sdk/src/protocol.ts`: add `issueId?` to `PluginEnvironmentDriverBaseParams` and the new `bridge.transport: "worker"` discriminator that the new plugin declares. - `server/__tests__/heartbeat-plugin-environment.test.ts`: pin the heartbeat path against the new runtime descriptor. **The Cloudflare plugin itself (`Add Cloudflare sandbox provider plugin`):** - `packages/plugins/sandbox-providers/cloudflare/`: plugin entry, manifest, plugin runtime (lifecycle + bridge client), config parsing, and Vitest coverage. Manifest declares `bridge.transport: "worker"` so the platform routes runtime RPCs through the bridge client. - `bridge-template/`: a Worker template the operator deploys with `wrangler`. Owns Durable Object-backed sessions (`sessions.ts`), exec/stream routes (`exec.ts`, `routes.ts`), and an HMAC auth layer (`auth.ts`) that pins the `Host` header surface. Includes the SDK-contract-correct exec implementation, lease recovery, and chunked stdout/stderr streaming. - Tests cover lease/session handoff (`bridge-template/src/exec.test.ts`, `routes.test.ts`), bridge client request shaping (`src/bridge-client.test.ts`), and end-to-end plugin behavior (`src/plugin.test.ts`) including streamed exec output. 27 tests in total. - `README.md` walks the operator through deploying the bridge Worker, registering the plugin, and configuring the runtime. ## Verification - `pnpm typecheck` - `pnpm exec vitest run --no-coverage packages/adapter-utils/src/sandbox-callback-bridge.test.ts packages/adapter-utils/src/command-managed-runtime.test.ts server/src/__tests__/environment-runtime.test.ts server/src/__tests__/heartbeat-plugin-environment.test.ts` - `(cd packages/plugins/sandbox-providers/cloudflare && pnpm test)` — 27 passing For an operator-side smoke test: 1. Deploy the bridge: `cd packages/plugins/sandbox-providers/cloudflare/bridge-template && wrangler deploy` 2. Register the plugin in your Paperclip instance, point its bridge URL at the deployed Worker, set the HMAC shared secret. 3. Create a sandbox environment whose provider is `cloudflare`, then run a Codex or Claude job against it. ## Risks - Adds a new `bridge.transport: "worker"` code path, but the existing E2B / Daytona transports go through the same shaped helpers and have explicit test coverage that pins their behavior unchanged. - The Worker bridge stores session state in a Durable Object; operator instances must be aware of the corresponding Cloudflare costs (DO requests, storage). Documented in the README. - The `issueId` plumbing is optional throughout — existing plugins that don't supply it continue to work. ## Model Used - Provider: Anthropic - Model: Claude Opus 4.7 (1M context) - Capabilities used: extended reasoning, tool use (Read/Edit/Bash/Grep) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots — N/A, no UI change - [x] I have updated relevant documentation to reflect my changes (plugin README, bridge-template README) - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
358 lines
11 KiB
TypeScript
358 lines
11 KiB
TypeScript
import type {
|
|
CloudflareBridgeAcquireLeaseRequest,
|
|
CloudflareBridgeExecuteRequest,
|
|
CloudflareBridgeExecuteResponse,
|
|
CloudflareBridgeHealthResponse,
|
|
CloudflareBridgeLeaseResponse,
|
|
CloudflareBridgeProbeRequest,
|
|
CloudflareBridgeProbeResponse,
|
|
CloudflareBridgeReleaseLeaseRequest,
|
|
CloudflareBridgeResumeLeaseRequest,
|
|
CloudflareDriverConfig,
|
|
} from "./types.js";
|
|
|
|
interface BridgeClientHeaders {
|
|
environmentId?: string;
|
|
runId?: string;
|
|
issueId?: string | null;
|
|
}
|
|
|
|
interface BridgeClientOptions {
|
|
config: CloudflareDriverConfig;
|
|
}
|
|
|
|
interface BridgeExecuteOptions {
|
|
onOutput?: (stream: "stdout" | "stderr", chunk: string) => void | Promise<void>;
|
|
}
|
|
|
|
interface BridgeErrorBody {
|
|
error?: string;
|
|
message?: string;
|
|
details?: unknown;
|
|
}
|
|
|
|
function isRecord(value: unknown): value is Record<string, unknown> {
|
|
return typeof value === "object" && value !== null && !Array.isArray(value);
|
|
}
|
|
|
|
export class CloudflareBridgeError extends Error {
|
|
readonly status: number;
|
|
readonly code: string | null;
|
|
readonly details: unknown;
|
|
|
|
constructor(input: { status: number; code?: string | null; message: string; details?: unknown }) {
|
|
super(input.message);
|
|
this.name = "CloudflareBridgeError";
|
|
this.status = input.status;
|
|
this.code = input.code ?? null;
|
|
this.details = input.details;
|
|
}
|
|
}
|
|
|
|
function buildHeaders(config: CloudflareDriverConfig, extra: BridgeClientHeaders = {}): Headers {
|
|
const headers = new Headers();
|
|
headers.set("Authorization", `Bearer ${config.bridgeAuthToken}`);
|
|
headers.set("Content-Type", "application/json");
|
|
if (extra.environmentId) headers.set("X-Paperclip-Environment-Id", extra.environmentId);
|
|
if (extra.runId) headers.set("X-Paperclip-Run-Id", extra.runId);
|
|
if (extra.issueId) headers.set("X-Paperclip-Issue-Id", extra.issueId);
|
|
return headers;
|
|
}
|
|
|
|
async function parseJson(response: Response): Promise<unknown> {
|
|
const contentType = response.headers.get("content-type") ?? "";
|
|
if (!contentType.toLowerCase().includes("application/json")) {
|
|
return null;
|
|
}
|
|
return await response.json();
|
|
}
|
|
|
|
function encodeExecuteRequestBody(body: CloudflareBridgeExecuteRequest, options?: BridgeExecuteOptions): string {
|
|
return JSON.stringify({
|
|
...body,
|
|
streamOutput: typeof options?.onOutput === "function",
|
|
});
|
|
}
|
|
|
|
function parseExecuteTimeoutMs(body: RequestInit["body"]): number | null {
|
|
if (typeof body !== "string") return null;
|
|
try {
|
|
const parsed = JSON.parse(body) as { timeoutMs?: unknown };
|
|
const timeoutMs = Number(parsed.timeoutMs);
|
|
return Number.isFinite(timeoutMs) && timeoutMs > 0 ? Math.trunc(timeoutMs) : null;
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
|
|
export function resolveRequestTimeoutMs(
|
|
config: CloudflareDriverConfig,
|
|
path: string,
|
|
init: RequestInit,
|
|
): number {
|
|
if (!path.endsWith("/exec")) {
|
|
return config.bridgeRequestTimeoutMs;
|
|
}
|
|
const requestedTimeoutMs = parseExecuteTimeoutMs(init.body);
|
|
return requestedTimeoutMs === null
|
|
? config.bridgeRequestTimeoutMs
|
|
: Math.max(config.bridgeRequestTimeoutMs, requestedTimeoutMs);
|
|
}
|
|
|
|
async function requestJson<T>(
|
|
config: CloudflareDriverConfig,
|
|
path: string,
|
|
init: RequestInit,
|
|
extraHeaders: BridgeClientHeaders = {},
|
|
): Promise<T> {
|
|
const controller = new AbortController();
|
|
const requestTimeoutMs = resolveRequestTimeoutMs(config, path, init);
|
|
const timeout = setTimeout(() => controller.abort(), requestTimeoutMs);
|
|
const baseUrl = config.bridgeBaseUrl.replace(/\/+$/, "");
|
|
|
|
try {
|
|
const response = await fetch(`${baseUrl}${path}`, {
|
|
...init,
|
|
headers: buildHeaders(config, extraHeaders),
|
|
signal: controller.signal,
|
|
});
|
|
const body = await parseJson(response);
|
|
if (!response.ok) {
|
|
const errorBody = isRecord(body) ? body as BridgeErrorBody : {};
|
|
throw new CloudflareBridgeError({
|
|
status: response.status,
|
|
code: typeof errorBody.error === "string" ? errorBody.error : null,
|
|
message:
|
|
typeof errorBody.message === "string" && errorBody.message.trim().length > 0
|
|
? errorBody.message
|
|
: `Cloudflare sandbox bridge request failed with HTTP ${response.status}.`,
|
|
details: errorBody.details,
|
|
});
|
|
}
|
|
return body as T;
|
|
} catch (error) {
|
|
if (error instanceof CloudflareBridgeError) throw error;
|
|
if ((error as { name?: string } | null)?.name === "AbortError") {
|
|
throw new Error(
|
|
`Cloudflare sandbox bridge request timed out after ${requestTimeoutMs}ms.`,
|
|
);
|
|
}
|
|
throw error;
|
|
} finally {
|
|
clearTimeout(timeout);
|
|
}
|
|
}
|
|
|
|
async function requestResponse(
|
|
config: CloudflareDriverConfig,
|
|
path: string,
|
|
init: RequestInit,
|
|
extraHeaders: BridgeClientHeaders = {},
|
|
): Promise<Response> {
|
|
const controller = new AbortController();
|
|
const requestTimeoutMs = resolveRequestTimeoutMs(config, path, init);
|
|
const timeout = setTimeout(() => controller.abort(), requestTimeoutMs);
|
|
const baseUrl = config.bridgeBaseUrl.replace(/\/+$/, "");
|
|
|
|
try {
|
|
const response = await fetch(`${baseUrl}${path}`, {
|
|
...init,
|
|
headers: buildHeaders(config, extraHeaders),
|
|
signal: controller.signal,
|
|
});
|
|
if (!response.ok) {
|
|
const body = await parseJson(response);
|
|
const errorBody = isRecord(body) ? body as BridgeErrorBody : {};
|
|
throw new CloudflareBridgeError({
|
|
status: response.status,
|
|
code: typeof errorBody.error === "string" ? errorBody.error : null,
|
|
message:
|
|
typeof errorBody.message === "string" && errorBody.message.trim().length > 0
|
|
? errorBody.message
|
|
: `Cloudflare sandbox bridge request failed with HTTP ${response.status}.`,
|
|
details: errorBody.details,
|
|
});
|
|
}
|
|
return response;
|
|
} catch (error) {
|
|
if (error instanceof CloudflareBridgeError) throw error;
|
|
if ((error as { name?: string } | null)?.name === "AbortError") {
|
|
throw new Error(
|
|
`Cloudflare sandbox bridge request timed out after ${requestTimeoutMs}ms.`,
|
|
);
|
|
}
|
|
throw error;
|
|
} finally {
|
|
clearTimeout(timeout);
|
|
}
|
|
}
|
|
|
|
interface ParsedSseEvent {
|
|
event: string;
|
|
data: string;
|
|
}
|
|
|
|
function parseSseChunk(buffer: string): { events: ParsedSseEvent[]; rest: string } {
|
|
const normalized = buffer.replace(/\r\n/g, "\n");
|
|
const frames = normalized.split("\n\n");
|
|
const rest = frames.pop() ?? "";
|
|
const events: ParsedSseEvent[] = [];
|
|
|
|
for (const frame of frames) {
|
|
let event = "message";
|
|
const dataLines: string[] = [];
|
|
for (const line of frame.split("\n")) {
|
|
if (line.startsWith("event:")) {
|
|
event = line.slice("event:".length).trim() || "message";
|
|
continue;
|
|
}
|
|
if (line.startsWith("data:")) {
|
|
dataLines.push(line.slice("data:".length).trimStart());
|
|
}
|
|
}
|
|
events.push({
|
|
event,
|
|
data: dataLines.join("\n"),
|
|
});
|
|
}
|
|
|
|
return { events, rest };
|
|
}
|
|
|
|
async function consumeExecuteEventStream(
|
|
response: Response,
|
|
options: BridgeExecuteOptions,
|
|
): Promise<CloudflareBridgeExecuteResponse> {
|
|
if (!response.body) {
|
|
throw new Error("Cloudflare sandbox bridge streaming response had no body.");
|
|
}
|
|
|
|
const reader = response.body.getReader();
|
|
const decoder = new TextDecoder();
|
|
let buffer = "";
|
|
let result: CloudflareBridgeExecuteResponse | null = null;
|
|
|
|
while (true) {
|
|
const { done, value } = await reader.read();
|
|
buffer += decoder.decode(value ?? new Uint8Array(), { stream: !done });
|
|
const parsed = parseSseChunk(done && buffer.length > 0 ? `${buffer}\n\n` : buffer);
|
|
buffer = parsed.rest;
|
|
|
|
for (const event of parsed.events) {
|
|
if (event.event === "stdout" || event.event === "stderr") {
|
|
const payload = JSON.parse(event.data) as { data?: unknown };
|
|
const chunk = typeof payload.data === "string" ? payload.data : "";
|
|
if (chunk) {
|
|
await options.onOutput?.(event.event, chunk);
|
|
}
|
|
continue;
|
|
}
|
|
|
|
if (event.event === "complete") {
|
|
result = JSON.parse(event.data) as CloudflareBridgeExecuteResponse;
|
|
continue;
|
|
}
|
|
|
|
if (event.event === "error") {
|
|
const payload = JSON.parse(event.data) as { error?: unknown };
|
|
const message = typeof payload.error === "string" && payload.error.trim().length > 0
|
|
? payload.error
|
|
: "Cloudflare sandbox bridge streaming command failed.";
|
|
throw new Error(message);
|
|
}
|
|
}
|
|
|
|
if (done) break;
|
|
}
|
|
|
|
if (result) return result;
|
|
throw new Error("Cloudflare sandbox bridge streaming response ended without a completion event.");
|
|
}
|
|
|
|
export function createCloudflareBridgeClient(options: BridgeClientOptions) {
|
|
const { config } = options;
|
|
const apiPrefix = "/api/paperclip-sandbox/v1";
|
|
|
|
return {
|
|
health(extraHeaders?: BridgeClientHeaders): Promise<CloudflareBridgeHealthResponse> {
|
|
return requestJson<CloudflareBridgeHealthResponse>(config, `${apiPrefix}/health`, { method: "GET" }, extraHeaders);
|
|
},
|
|
|
|
probe(body: CloudflareBridgeProbeRequest, extraHeaders?: BridgeClientHeaders): Promise<CloudflareBridgeProbeResponse> {
|
|
return requestJson<CloudflareBridgeProbeResponse>(
|
|
config,
|
|
`${apiPrefix}/probe`,
|
|
{ method: "POST", body: JSON.stringify(body) },
|
|
extraHeaders,
|
|
);
|
|
},
|
|
|
|
acquireLease(
|
|
body: CloudflareBridgeAcquireLeaseRequest,
|
|
extraHeaders?: BridgeClientHeaders,
|
|
): Promise<CloudflareBridgeLeaseResponse> {
|
|
return requestJson<CloudflareBridgeLeaseResponse>(
|
|
config,
|
|
`${apiPrefix}/leases/acquire`,
|
|
{ method: "POST", body: JSON.stringify(body) },
|
|
extraHeaders,
|
|
);
|
|
},
|
|
|
|
resumeLease(
|
|
body: CloudflareBridgeResumeLeaseRequest,
|
|
extraHeaders?: BridgeClientHeaders,
|
|
): Promise<CloudflareBridgeLeaseResponse> {
|
|
return requestJson<CloudflareBridgeLeaseResponse>(
|
|
config,
|
|
`${apiPrefix}/leases/resume`,
|
|
{ method: "POST", body: JSON.stringify(body) },
|
|
extraHeaders,
|
|
);
|
|
},
|
|
|
|
releaseLease(
|
|
body: CloudflareBridgeReleaseLeaseRequest,
|
|
extraHeaders?: BridgeClientHeaders,
|
|
): Promise<{ ok: true }> {
|
|
return requestJson<{ ok: true }>(
|
|
config,
|
|
`${apiPrefix}/leases/release`,
|
|
{ method: "POST", body: JSON.stringify(body) },
|
|
extraHeaders,
|
|
);
|
|
},
|
|
|
|
destroyLease(providerLeaseId: string, extraHeaders?: BridgeClientHeaders): Promise<{ ok: true }> {
|
|
return requestJson<{ ok: true }>(
|
|
config,
|
|
`${apiPrefix}/leases/${encodeURIComponent(providerLeaseId)}`,
|
|
{ method: "DELETE" },
|
|
extraHeaders,
|
|
);
|
|
},
|
|
|
|
execute(
|
|
body: CloudflareBridgeExecuteRequest,
|
|
extraHeaders?: BridgeClientHeaders,
|
|
options?: BridgeExecuteOptions,
|
|
): Promise<CloudflareBridgeExecuteResponse> {
|
|
const encodedBody = encodeExecuteRequestBody(body, options);
|
|
if (typeof options?.onOutput === "function") {
|
|
return requestResponse(
|
|
config,
|
|
`${apiPrefix}/exec`,
|
|
{ method: "POST", body: encodedBody },
|
|
extraHeaders,
|
|
).then((response) => consumeExecuteEventStream(response, options));
|
|
}
|
|
return requestJson<CloudflareBridgeExecuteResponse>(
|
|
config,
|
|
`${apiPrefix}/exec`,
|
|
{ method: "POST", body: encodedBody },
|
|
extraHeaders,
|
|
);
|
|
},
|
|
};
|
|
}
|