[codex] Add local Cloud Upstream sync (#6548)

## Thinking Path

> - Paperclip is the control plane for AI-agent companies.
> - Operators need a path to move local company state toward Paperclip
Cloud without losing local-first control.
> - The Cloud Upstream flow needs API, persistence, CLI, and board UI
surfaces that agree on the same manifest/run model.
> - The existing branch had the feature work plus UX and error-handling
follow-ups.
> - This pull request packages the remaining Cloud Upstream sync work
into one standalone branch.
> - The benefit is an inspectable local-to-cloud sync workflow with
preview, conflicts, activation, and captured UX review states.

## What Changed

- Added Cloud Upstream shared types, server routes/services, and
persisted run schema/migration.
- Added Paperclip Cloud CLI sync helpers and local connection storage.
- Added the Cloud Upstream board UI, settings entry points, query keys,
and UX lab page.
- Added preview/activation checklist behavior, redirect handling,
manifest-only preview support, friendly errors, in-flight hints, and
entity count summaries.

## Verification

- `pnpm --filter @paperclipai/plugin-sdk build`
- `NODE_ENV=test pnpm exec vitest run cli/src/__tests__/cloud.test.ts
server/src/__tests__/instance-settings-routes.test.ts
server/src/__tests__/instance-settings-service.test.ts
ui/src/pages/CloudUpstream.test.tsx
ui/src/components/CompanySettingsSidebar.test.tsx`
- `NODE_ENV=test pnpm exec vitest run
server/src/__tests__/cloud-upstreams.test.ts`

Worktree setup note: the isolated worktree install skipped native sqlite
build scripts, so I copied the already-built local sqlite binding from
the main checkout before running
`server/src/__tests__/cloud-upstreams.test.ts`. The test then passed.

## Risks

- Medium: this adds a database migration and a broad feature path across
CLI/server/UI.
- Merge order: this is the only PR in this split with a DB migration;
merge it before any future Cloud Upstream migration follow-up.
- Mitigation: the PR is based directly on current `origin/master`, has
targeted route/service/UI tests, and keeps the feature behind existing
experimental Cloud Sync settings.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI GPT-5 Codex via `codex_local`, tool-enabled coding session;
exact context window not exposed by this runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, screenshot artifacts are
intentionally omitted per reviewer request
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
This commit is contained in:
Dotta
2026-05-22 09:56:22 -05:00
committed by GitHub
parent a1835cfa5e
commit e43b392a79
36 changed files with 5592 additions and 7 deletions
+15
View File
@@ -226,6 +226,21 @@ By default, agents run on scheduled heartbeats and event-based triggers (task as
<br/>
## Paperclip Cloud Sync
Cloud upstream sync is behind the `Cloud Sync` experimental setting. Enable it in Instance Settings before pushing.
```bash
paperclipai cloud connect https://your-stack.paperclip.app
paperclipai cloud connect https://your-stack.paperclip.app --no-browser
paperclipai cloud push --company <local-company-id> --dry-run
paperclipai cloud push --company <local-company-id>
```
`cloud connect` authorizes the local instance against the target stack and stores the upstream token in the local instance secret store. The default path opens a browser for consent; `--no-browser` uses the device-code flow and prints the verification URL and user code.
`cloud push --dry-run` exports the selected local company, sends a preview bundle to the connected Cloud stack, and exits with code `2` when conflicts need user resolution. A schema mismatch exits with code `3`. Running without `--dry-run` stages chunks idempotently, applies the run, and prints the final summary and recent progress events.
## Development
```bash
+243
View File
@@ -0,0 +1,243 @@
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import type { CompanyPortabilityExportResult } from "@paperclipai/shared";
import {
assertDiscoveryCompatible,
buildBundleFromLocalCompany,
cloudCommandExitCodes,
connectCloud,
resolveDeviceCodeExpiresAt,
} from "../commands/client/cloud.js";
import {
LocalUpstreamPushCoordinator,
normalizedContentHash,
type LocalUpstreamExportBundle,
} from "../commands/client/cloud-transfer.js";
import { getCloudConnection } from "../commands/client/cloud-store.js";
const originalEnv = { ...process.env };
const originalFetch = globalThis.fetch;
describe("cloud CLI helpers", () => {
let tempHome: string;
beforeEach(() => {
tempHome = fs.mkdtempSync(path.join(os.tmpdir(), "paperclip-cloud-cli-"));
process.env = { ...originalEnv, PAPERCLIP_HOME: tempHome };
});
afterEach(() => {
process.env = { ...originalEnv };
globalThis.fetch = originalFetch;
vi.restoreAllMocks();
fs.rmSync(tempHome, { recursive: true, force: true });
});
it("connects with the device-code flow and stores the resulting cloud connection", async () => {
globalThis.fetch = vi.fn(async (url, init) => {
const requestUrl = String(url);
if (requestUrl.endsWith("/.well-known/paperclip-upstream")) {
return jsonResponse(discovery());
}
if (requestUrl.endsWith("/api/upstream-sync/device-code")) {
expect(JSON.parse(String(init?.body))).toMatchObject({
stackId: "stack-1",
scopes: ["upstream_import:preview", "upstream_import:write", "upstream_import:read"],
});
return jsonResponse({
deviceCode: "device-1",
userCode: "ABCD-EFGH",
verificationUri: "https://cloud.example.test/api/upstream-sync/device-code/approve",
expiresAt: new Date(Date.now() + 60_000).toISOString(),
intervalSeconds: 0,
});
}
if (requestUrl.endsWith("/api/upstream-sync/token")) {
return jsonResponse({
accessToken: "upt_test",
scopes: ["upstream_import:preview"],
token: {
id: "token-1",
companyStackId: "stack-1",
targetOrigin: "https://cloud.example.test",
sourceInstanceId: "paperclip-local-default",
sourceInstanceFingerprint: "sha256:test",
scopes: ["upstream_import:preview"],
expiresAt: new Date(Date.now() + 60_000).toISOString(),
},
});
}
return jsonResponse({ error: "not_found" }, 404);
}) as typeof fetch;
const connection = await connectCloud("https://cloud.example.test", { noBrowser: true, json: true });
expect(connection.accessToken).toBe("upt_test");
expect(getCloudConnection("https://cloud.example.test")?.token.id).toBe("token-1");
});
it("hard-blocks incompatible transfer schema versions with the stable schema exit code", () => {
expect(() => assertDiscoveryCompatible(discovery({ supportedSchemaMajor: 99 }))).toThrow(/schema mismatch/i);
expect(cloudCommandExitCodes.schemaMismatch).toBe(3);
});
it("falls back to a bounded device-code expiry when the cloud omits or malforms expiresAt", () => {
const now = Date.UTC(2026, 4, 22, 13, 0, 0);
const validExpiry = "2026-05-22T13:05:00.000Z";
expect(resolveDeviceCodeExpiresAt(validExpiry, now)).toBe(Date.parse(validExpiry));
expect(resolveDeviceCodeExpiresAt(undefined, now)).toBe(now + 15 * 60_000);
expect(resolveDeviceCodeExpiresAt("not-a-date", now)).toBe(now + 15 * 60_000);
});
it("builds deterministic chunks with validated payload hashes", async () => {
const bundle = await buildTestBundle();
expect(bundle.chunks).toHaveLength(2);
expect(bundle.chunks[0]?.sha256).toBe(normalizedContentHash(bundle.chunks[0]?.payload));
expect(bundle.manifest.chunks[0]?.manifestHash).toBe(bundle.manifest.manifestHash);
expect(bundle.manifest.idempotencyKey).toBe((await buildTestBundle()).manifest.idempotencyKey);
});
it("reuses the same manifest and chunk identity when an interrupted apply is retried", async () => {
const bundle = await buildTestBundle();
const calls: Array<{ path: string; body: unknown }> = [];
const coordinator = new LocalUpstreamPushCoordinator({
targetOrigin: "https://cloud.example.test",
paperclipCompanyId: "target-company-1",
fetch: async (url, init) => {
const parsed = new URL(String(url));
const body = init?.body ? JSON.parse(String(init.body)) as unknown : {};
calls.push({ path: parsed.pathname, body });
if (parsed.pathname.endsWith("/runs")) return jsonResponse({ run: { id: "run-1" } });
return jsonResponse({ run: { id: "run-1" }, summary: { create: 0, update: 0, adopt: 0, skip: 2, conflict: 0, staleMapping: 0 } });
},
});
await coordinator.apply(bundle);
await coordinator.apply(bundle);
const runBodies = calls.filter((call) => call.path.endsWith("/runs")).map((call) => call.body as { manifest: { idempotencyKey: string } });
const chunkBodies = calls.filter((call) => call.path.endsWith("/chunks")).map((call) => call.body as { chunkIndex: number; sha256: string });
expect(runBodies).toHaveLength(2);
expect(runBodies[0]?.manifest.idempotencyKey).toBe(runBodies[1]?.manifest.idempotencyKey);
expect(chunkBodies[0]).toEqual(chunkBodies[2]);
expect(chunkBodies[1]).toEqual(chunkBodies[3]);
});
});
async function buildTestBundle(): Promise<LocalUpstreamExportBundle> {
return buildBundleFromLocalCompany({
localCompanyId: "local-company-1",
connection: {
id: "conn-1",
remoteUrl: "https://cloud.example.test",
targetOrigin: "https://cloud.example.test",
targetHost: "cloud.example.test",
stackId: "stack-1",
targetCompanyId: "target-company-1",
accessToken: "upt_test",
token: {
id: "token-1",
companyStackId: "stack-1",
targetOrigin: "https://cloud.example.test",
sourceInstanceId: "paperclip-local-default",
sourceInstanceFingerprint: "sha256:test",
scopes: ["upstream_import:preview"],
expiresAt: new Date(Date.now() + 60_000).toISOString(),
},
privateKeyPem: "unused",
sourcePublicKey: "unused",
sourceInstanceId: "paperclip-local-default",
sourceInstanceFingerprint: "sha256:test",
scopes: ["upstream_import:preview"],
createdAt: "2026-05-18T00:00:00.000Z",
updatedAt: "2026-05-18T00:00:00.000Z",
},
discovery: discovery(),
localApi: {
post: async <T>() => portabilityExport() as T,
},
maxEntitiesPerChunk: 1,
mode: "apply",
});
}
function discovery(overrides: Partial<{ supportedSchemaMajor: number }> = {}) {
return {
schema: "paperclip-upstream-discovery-v1",
stack: {
id: "stack-1",
slug: "cloud-test",
displayName: "Cloud Test",
companyId: "target-company-1",
origin: "https://cloud.example.test",
},
auth: {
deviceCode: {
deviceCodeUrl: "https://cloud.example.test/api/upstream-sync/device-code",
verificationUrl: "https://cloud.example.test/api/upstream-sync/device-code/approve",
tokenUrl: "https://cloud.example.test/api/upstream-sync/token",
},
scopes: ["upstream_import:preview", "upstream_import:write", "upstream_import:read"],
},
transfer: {
supportedSchemaMajor: overrides.supportedSchemaMajor ?? 1,
featureFlags: ["cloud_sync"],
},
};
}
function portabilityExport(): CompanyPortabilityExportResult {
return {
rootPath: ".",
paperclipExtensionPath: ".paperclip.yaml",
manifest: {
schemaVersion: 1,
generatedAt: "2026-05-18T00:00:00.000Z",
source: {
companyId: "local-company-1",
companyName: "Local Company",
},
includes: {
company: true,
agents: true,
projects: true,
issues: true,
skills: true,
},
company: {
path: "company.json",
name: "Local Company",
description: null,
brandColor: null,
logoPath: null,
attachmentMaxBytes: null,
requireBoardApprovalForNewAgents: false,
feedbackDataSharingEnabled: false,
feedbackDataSharingConsentAt: null,
feedbackDataSharingConsentByUserId: null,
feedbackDataSharingTermsVersion: null,
},
sidebar: null,
agents: [],
skills: [],
projects: [],
issues: [],
envInputs: [],
},
files: {
"README.md": "Local Company",
},
warnings: [],
};
}
function jsonResponse(body: unknown, status = 200): Response {
return new Response(JSON.stringify(body), {
status,
headers: { "Content-Type": "application/json" },
});
}
+177
View File
@@ -0,0 +1,177 @@
import fs from "node:fs";
import path from "node:path";
import { resolvePaperclipInstanceRoot } from "../../config/home.js";
export interface CloudConnectionTokenRecord {
id: string;
companyStackId: string;
targetOrigin: string;
sourceInstanceId: string;
sourceInstanceFingerprint: string;
scopes: string[];
expiresAt: string;
[key: string]: unknown;
}
export interface CloudConnection {
id: string;
remoteUrl: string;
targetOrigin: string;
targetHost: string;
stackId: string;
stackSlug?: string | null;
stackDisplayName?: string | null;
targetCompanyId: string;
accessToken: string;
token: CloudConnectionTokenRecord;
privateKeyPem: string;
sourcePublicKey: string;
sourceInstanceId: string;
sourceInstanceFingerprint: string;
scopes: string[];
createdAt: string;
updatedAt: string;
}
interface CloudConnectionStore {
version: 1;
connections: Record<string, CloudConnection>;
currentConnectionId?: string;
}
function defaultStore(): CloudConnectionStore {
return {
version: 1,
connections: {},
};
}
export function resolveCloudConnectionStorePath(): string {
return path.resolve(resolvePaperclipInstanceRoot(), "secrets", "cloud-upstream-connections.json");
}
export function readCloudConnectionStore(storePath = resolveCloudConnectionStorePath()): CloudConnectionStore {
if (!fs.existsSync(storePath)) return defaultStore();
const raw = JSON.parse(fs.readFileSync(storePath, "utf8")) as Partial<CloudConnectionStore> | null;
const connections: Record<string, CloudConnection> = {};
if (raw?.connections && typeof raw.connections === "object") {
for (const [id, value] of Object.entries(raw.connections)) {
const normalized = normalizeConnection(value);
if (normalized) connections[id] = normalized;
}
}
const currentConnectionId =
typeof raw?.currentConnectionId === "string" && connections[raw.currentConnectionId]
? raw.currentConnectionId
: Object.values(connections).sort((left, right) => right.updatedAt.localeCompare(left.updatedAt))[0]?.id;
return {
version: 1,
connections,
currentConnectionId,
};
}
export function writeCloudConnectionStore(
store: CloudConnectionStore,
storePath = resolveCloudConnectionStorePath(),
): void {
fs.mkdirSync(path.dirname(storePath), { recursive: true });
fs.writeFileSync(storePath, `${JSON.stringify(store, null, 2)}\n`, { mode: 0o600 });
}
export function upsertCloudConnection(
connection: CloudConnection,
storePath = resolveCloudConnectionStorePath(),
): CloudConnection {
const store = readCloudConnectionStore(storePath);
const existing = store.connections[connection.id];
const now = new Date().toISOString();
const next = {
...connection,
createdAt: existing?.createdAt ?? connection.createdAt ?? now,
updatedAt: now,
};
store.connections[next.id] = next;
store.currentConnectionId = next.id;
writeCloudConnectionStore(store, storePath);
return next;
}
export function getCloudConnection(
remoteUrlOrOrigin?: string,
storePath = resolveCloudConnectionStorePath(),
): CloudConnection | null {
const store = readCloudConnectionStore(storePath);
if (remoteUrlOrOrigin?.trim()) {
const needle = normalizeRemoteLookup(remoteUrlOrOrigin);
return Object.values(store.connections).find((connection) =>
normalizeRemoteLookup(connection.remoteUrl) === needle ||
normalizeRemoteLookup(connection.targetOrigin) === needle
) ?? null;
}
return store.currentConnectionId ? store.connections[store.currentConnectionId] ?? null : null;
}
function normalizeRemoteLookup(value: string): string {
try {
const url = new URL(value);
return url.origin.replace(/\/+$/u, "");
} catch {
return value.trim().replace(/\/+$/u, "");
}
}
function normalizeConnection(value: unknown): CloudConnection | null {
if (typeof value !== "object" || value === null || Array.isArray(value)) return null;
const record = value as Record<string, unknown>;
const id = stringValue(record.id);
const remoteUrl = stringValue(record.remoteUrl);
const targetOrigin = stringValue(record.targetOrigin);
const targetHost = stringValue(record.targetHost);
const stackId = stringValue(record.stackId);
const targetCompanyId = stringValue(record.targetCompanyId);
const accessToken = stringValue(record.accessToken);
const token = typeof record.token === "object" && record.token !== null && !Array.isArray(record.token)
? record.token as CloudConnectionTokenRecord
: null;
const privateKeyPem = stringValue(record.privateKeyPem);
const sourcePublicKey = stringValue(record.sourcePublicKey);
const sourceInstanceId = stringValue(record.sourceInstanceId);
const sourceInstanceFingerprint = stringValue(record.sourceInstanceFingerprint);
const createdAt = stringValue(record.createdAt);
const updatedAt = stringValue(record.updatedAt);
if (
!id || !remoteUrl || !targetOrigin || !targetHost || !stackId || !targetCompanyId ||
!accessToken || !token || !privateKeyPem || !sourcePublicKey || !sourceInstanceId ||
!sourceInstanceFingerprint || !createdAt || !updatedAt
) {
return null;
}
return {
id,
remoteUrl,
targetOrigin,
targetHost,
stackId,
stackSlug: stringValue(record.stackSlug),
stackDisplayName: stringValue(record.stackDisplayName),
targetCompanyId,
accessToken,
token,
privateKeyPem,
sourcePublicKey,
sourceInstanceId,
sourceInstanceFingerprint,
scopes: stringArray(record.scopes),
createdAt,
updatedAt,
};
}
function stringValue(value: unknown): string | null {
return typeof value === "string" && value.trim().length > 0 ? value.trim() : null;
}
function stringArray(value: unknown): string[] {
return Array.isArray(value) ? value.filter((entry): entry is string => typeof entry === "string") : [];
}
+297
View File
@@ -0,0 +1,297 @@
import { createHash } from "node:crypto";
export const upstreamTransferSchema = {
family: "paperclip-upstream-transfer",
version: "1.0.0",
major: 1,
minor: 0,
} as const;
export type NormalizedSha256 = `sha256:${string}`;
export interface SourceEntityKey {
sourceInstanceId: string;
sourceCompanyId: string;
sourceEntityType: string;
sourceEntityId: string;
sourceNaturalKey?: string;
}
export interface UpstreamTransferWarning {
code: string;
severity: "info" | "warning" | "blocker";
message: string;
entity?: SourceEntityKey;
}
export interface UpstreamTransferEntityRecord {
key: SourceEntityKey;
contentHash: NormalizedSha256;
dependencies: SourceEntityKey[];
warnings: UpstreamTransferWarning[];
}
export interface UpstreamTransferManifestSource {
sourceInstanceId: string;
sourceCompanyId: string;
sourceInstanceKeyFingerprint: string;
exporterVersion: string;
sourceSchemaVersion: string;
}
export interface UpstreamTransferManifestTarget {
targetStackId: string;
targetCompanyId: string;
targetOrigin: string;
supportedSchemaMajor: number;
}
export interface UpstreamTransferChunk {
chunkIndex: number;
totalChunks: number;
byteLength: number;
sha256: NormalizedSha256;
manifestHash: NormalizedSha256;
}
export interface UpstreamTransferManifest {
schema: typeof upstreamTransferSchema;
source: UpstreamTransferManifestSource;
target: UpstreamTransferManifestTarget;
runId: string;
idempotencyKey: string;
generatedAt: string;
entityCount: number;
entities: UpstreamTransferEntityRecord[];
chunks: UpstreamTransferChunk[];
warnings: UpstreamTransferWarning[];
featureFlags: string[];
manifestHash: NormalizedSha256;
}
export interface LocalUpstreamExportEntityInput {
key: SourceEntityKey;
body: Record<string, unknown>;
dependencies?: SourceEntityKey[];
warnings?: UpstreamTransferWarning[];
conflictKeys?: string[];
}
export interface LocalUpstreamExportEntity {
record: UpstreamTransferEntityRecord;
body: Record<string, unknown>;
conflictKeys?: string[];
}
export interface LocalUpstreamExportChunk {
chunkIndex: number;
totalChunks: number;
byteLength: number;
sha256: NormalizedSha256;
payload: {
entityKeys: SourceEntityKey[];
};
}
export interface LocalUpstreamExportBundle {
manifest: UpstreamTransferManifest;
entities: LocalUpstreamExportEntity[];
chunks: LocalUpstreamExportChunk[];
}
export interface BuildLocalUpstreamExportBundleInput {
source: UpstreamTransferManifestSource;
target: UpstreamTransferManifestTarget;
runId: string;
idempotencyKey: string;
entities: LocalUpstreamExportEntityInput[];
warnings?: UpstreamTransferWarning[];
featureFlags?: string[];
maxEntitiesPerChunk?: number;
}
export interface LocalUpstreamPushCoordinatorOptions {
targetOrigin: string;
paperclipCompanyId: string;
fetch?: typeof fetch;
headers?: (input: { method: string; path: string }) => HeadersInit | Promise<HeadersInit>;
}
export class UpstreamImportRequestError extends Error {
readonly status: number;
readonly body: unknown;
constructor(status: number, message: string, body: unknown) {
super(message);
this.status = status;
this.body = body;
}
}
export class LocalUpstreamPushCoordinator {
readonly #targetOrigin: string;
readonly #paperclipCompanyId: string;
readonly #fetch: typeof fetch;
readonly #headers: NonNullable<LocalUpstreamPushCoordinatorOptions["headers"]>;
constructor(options: LocalUpstreamPushCoordinatorOptions) {
this.#targetOrigin = options.targetOrigin.replace(/\/+$/u, "");
this.#paperclipCompanyId = options.paperclipCompanyId;
this.#fetch = options.fetch ?? fetch;
this.#headers = options.headers ?? (() => ({}));
}
async preview(bundle: LocalUpstreamExportBundle): Promise<unknown> {
return this.post(`/api/companies/${encodeURIComponent(this.#paperclipCompanyId)}/upstream-imports/preview`, {
manifest: bundle.manifest,
entities: bundle.entities,
});
}
async apply(bundle: LocalUpstreamExportBundle): Promise<unknown> {
const run = await this.post(`/api/companies/${encodeURIComponent(this.#paperclipCompanyId)}/upstream-imports/runs`, {
mode: "apply",
manifest: bundle.manifest,
entities: bundle.entities,
}) as { run?: { id?: unknown } };
const runId = typeof run.run?.id === "string" ? run.run.id : undefined;
if (!runId) {
throw new Error("Remote upstream importer did not return a run id");
}
for (const chunk of bundle.chunks) {
await this.post(`/api/upstream-import-runs/${encodeURIComponent(runId)}/chunks`, chunk);
}
return this.post(`/api/upstream-import-runs/${encodeURIComponent(runId)}/apply`, {});
}
async events(runId: string): Promise<unknown> {
return this.get(`/api/upstream-import-runs/${encodeURIComponent(runId)}/events`);
}
private async get(path: string): Promise<unknown> {
const response = await this.#fetch(`${this.#targetOrigin}${path}`, {
method: "GET",
headers: await this.#headers({ method: "GET", path }),
});
return parseCoordinatorResponse(response);
}
private async post(path: string, body: unknown): Promise<unknown> {
const response = await this.#fetch(`${this.#targetOrigin}${path}`, {
method: "POST",
headers: {
"Content-Type": "application/json",
...(await this.#headers({ method: "POST", path })),
},
body: JSON.stringify(body),
});
return parseCoordinatorResponse(response);
}
}
export function buildLocalUpstreamExportBundle(
input: BuildLocalUpstreamExportBundleInput,
): LocalUpstreamExportBundle {
const entities = input.entities.map<LocalUpstreamExportEntity>((entity) => ({
record: {
key: entity.key,
contentHash: normalizedContentHash(entity.body),
dependencies: entity.dependencies ?? [],
warnings: entity.warnings ?? [],
},
body: entity.body,
conflictKeys: entity.conflictKeys,
}));
const chunks = buildLocalChunks(entities, input.maxEntitiesPerChunk ?? 100);
const manifestWithoutHash = {
schema: upstreamTransferSchema,
source: input.source,
target: input.target,
runId: input.runId,
idempotencyKey: input.idempotencyKey,
generatedAt: new Date(0).toISOString(),
entityCount: entities.length,
entities: entities.map((entity) => entity.record),
chunks: chunks.map(({ payload: _payload, ...chunk }) => chunk),
warnings: input.warnings ?? [],
featureFlags: (input.featureFlags ?? ["cloud_sync"]).slice().sort(),
};
const manifestHash = normalizedContentHash(manifestWithoutHash);
return {
manifest: {
...manifestWithoutHash,
chunks: manifestWithoutHash.chunks.map((chunk) => ({ ...chunk, manifestHash })),
manifestHash,
},
entities,
chunks,
};
}
export function normalizedContentHash(value: unknown): NormalizedSha256 {
return `sha256:${createHash("sha256").update(canonicalJson(value)).digest("hex")}`;
}
export function canonicalJson(value: unknown): string {
return JSON.stringify(sortJson(value));
}
function buildLocalChunks(
entities: LocalUpstreamExportEntity[],
maxEntitiesPerChunk: number,
): LocalUpstreamExportChunk[] {
if (!Number.isInteger(maxEntitiesPerChunk) || maxEntitiesPerChunk < 1) {
throw new Error("maxEntitiesPerChunk must be a positive integer");
}
if (entities.length === 0) return [];
const groups: LocalUpstreamExportEntity[][] = [];
for (let index = 0; index < entities.length; index += maxEntitiesPerChunk) {
groups.push(entities.slice(index, index + maxEntitiesPerChunk));
}
return groups.map((group, index) => {
const payload = {
entityKeys: group.map((entity) => entity.record.key),
};
return {
chunkIndex: index,
totalChunks: groups.length,
byteLength: Buffer.byteLength(canonicalJson(payload)),
sha256: normalizedContentHash(payload),
payload,
};
});
}
function sortJson(value: unknown): unknown {
if (Array.isArray(value)) return value.map(sortJson);
if (typeof value !== "object" || value === null) return value;
return Object.fromEntries(
Object.entries(value as Record<string, unknown>)
.sort(([left], [right]) => left.localeCompare(right))
.map(([key, entry]) => [key, sortJson(entry)]),
);
}
async function parseCoordinatorResponse(response: Response): Promise<unknown> {
const text = await response.text();
const parsed = text.trim() ? safeParseJson(text) : {};
if (!response.ok) {
const message = typeof parsed === "object" && parsed !== null && "error" in parsed
? String((parsed as { error: unknown }).error)
: `Upstream importer request failed with ${response.status}`;
throw new UpstreamImportRequestError(response.status, message, parsed);
}
return parsed;
}
function safeParseJson(text: string): unknown {
try {
return JSON.parse(text);
} catch {
return text;
}
}
+721
View File
@@ -0,0 +1,721 @@
import { createHash, generateKeyPairSync, randomBytes, randomUUID, sign } from "node:crypto";
import { createServer, type Server } from "node:http";
import { URL } from "node:url";
import { Command } from "commander";
import pc from "picocolors";
import type {
CompanyPortabilityExportResult,
CompanyPortabilityFileEntry,
InstanceExperimentalSettings,
} from "@paperclipai/shared";
import { openUrl } from "../../client/board-auth.js";
import { resolvePaperclipInstanceId } from "../../config/home.js";
import {
addCommonClientOptions,
handleCommandError,
printOutput,
resolveCommandContext,
type BaseClientOptions,
} from "./common.js";
import {
buildLocalUpstreamExportBundle,
LocalUpstreamPushCoordinator,
normalizedContentHash,
upstreamTransferSchema,
UpstreamImportRequestError,
type LocalUpstreamExportBundle,
type LocalUpstreamExportEntityInput,
type SourceEntityKey,
type UpstreamTransferManifestSource,
type UpstreamTransferManifestTarget,
type UpstreamTransferWarning,
} from "./cloud-transfer.js";
import {
getCloudConnection,
upsertCloudConnection,
type CloudConnection,
type CloudConnectionTokenRecord,
} from "./cloud-store.js";
const CLOUD_SYNC_CONFLICT_EXIT_CODE = 2;
const CLOUD_SYNC_SCHEMA_MISMATCH_EXIT_CODE = 3;
const CLOUD_SYNC_SCOPES = ["upstream_import:preview", "upstream_import:write", "upstream_import:read"];
const DEVICE_CODE_FALLBACK_EXPIRES_MS = 15 * 60_000;
interface CloudConnectOptions extends BaseClientOptions {
noBrowser?: boolean;
}
interface CloudPushOptions extends BaseClientOptions {
company?: string;
remoteUrl?: string;
dryRun?: boolean;
maxEntitiesPerChunk?: number;
}
interface UpstreamDiscovery {
schema: string;
stack: {
id: string;
slug?: string;
displayName?: string;
companyId: string;
origin: string;
};
auth: {
pkce?: {
authorizeUrl: string;
tokenUrl: string;
codeChallengeMethod: string;
};
deviceCode?: {
deviceCodeUrl: string;
verificationUrl: string;
tokenUrl: string;
};
scopes?: string[];
};
transfer: {
supportedSchemaMajor: number;
featureFlags?: string[];
};
}
interface TokenResponse {
accessToken: string;
token: CloudConnectionTokenRecord;
scopes?: string[];
expiresAt?: string;
}
class CloudAuthRequestError extends Error {
readonly status: number;
readonly body: unknown;
constructor(status: number, message: string, body: unknown) {
super(message);
this.status = status;
this.body = body;
}
}
export function registerCloudCommands(program: Command): void {
const cloud = program.command("cloud").description("Paperclip Cloud upstream sync commands");
addCommonClientOptions(
cloud
.command("connect")
.description("Authorize this local instance to push into a Paperclip Cloud stack")
.argument("<remote-url>", "Paperclip Cloud stack URL")
.option("--no-browser", "Use the device-code flow instead of opening a browser", false)
.action(async (remoteUrl: string, opts: CloudConnectOptions) => {
try {
await connectCloud(remoteUrl, opts);
} catch (err) {
handleCommandError(err);
}
}),
);
addCommonClientOptions(
cloud
.command("push")
.description("Preview or apply a local company push into the connected Paperclip Cloud stack")
.requiredOption("--company <local-company-id>", "Local company ID to export")
.option("--remote-url <remote-url>", "Use a specific stored cloud connection")
.option("--dry-run", "Preview without applying", false)
.option("--max-entities-per-chunk <count>", "Chunk size for upstream uploads", (value) => Number(value), 100)
.action(async (opts: CloudPushOptions) => {
try {
await pushCloud(opts);
} catch (err) {
if (isSchemaMismatchError(err)) {
console.error(pc.red(err instanceof Error ? err.message : String(err)));
process.exitCode = CLOUD_SYNC_SCHEMA_MISMATCH_EXIT_CODE;
return;
}
handleCommandError(err);
}
}),
);
}
export async function connectCloud(remoteUrl: string, opts: CloudConnectOptions = {}): Promise<CloudConnection> {
const ctx = resolveCommandContext(opts);
const discovery = await discoverUpstream(remoteUrl);
assertDiscoveryCompatible(discovery);
const source = createSourceIdentity();
const token = await authorizeConnection(discovery, source, {
noBrowser: Boolean(opts.noBrowser),
});
const targetOrigin = discovery.stack.origin.replace(/\/+$/u, "");
const targetHost = new URL(targetOrigin).host;
const now = new Date().toISOString();
const connection = upsertCloudConnection({
id: connectionId(targetOrigin),
remoteUrl,
targetOrigin,
targetHost,
stackId: discovery.stack.id,
stackSlug: discovery.stack.slug ?? null,
stackDisplayName: discovery.stack.displayName ?? null,
targetCompanyId: discovery.stack.companyId,
accessToken: token.accessToken,
token: token.token,
privateKeyPem: source.privateKeyPem,
sourcePublicKey: source.sourcePublicKey,
sourceInstanceId: source.sourceInstanceId,
sourceInstanceFingerprint: source.sourceInstanceFingerprint,
scopes: token.scopes ?? token.token.scopes ?? CLOUD_SYNC_SCOPES,
createdAt: now,
updatedAt: now,
});
if (ctx.json) {
printOutput(redactConnection(connection), { json: true });
} else {
console.log(pc.bold("Connected to Paperclip Cloud"));
console.log(`stack=${connection.stackDisplayName ?? connection.stackSlug ?? connection.stackId}`);
console.log(`origin=${connection.targetOrigin}`);
console.log(`company=${connection.targetCompanyId}`);
}
return connection;
}
export async function pushCloud(opts: CloudPushOptions): Promise<unknown> {
const ctx = resolveCommandContext(opts, { requireCompany: false });
const localCompanyId = requiredString(opts.company, "--company");
await assertCloudSyncEnabled(ctx.api.get<InstanceExperimentalSettings>("/api/instance/settings/experimental"));
const connection = getCloudConnection(opts.remoteUrl);
if (!connection) {
throw new Error("No cloud connection found. Run `paperclipai cloud connect <remote-url>` first.");
}
const discovery = await discoverUpstream(connection.targetOrigin);
assertDiscoveryCompatible(discovery);
const bundle = await buildBundleFromLocalCompany({
localCompanyId,
connection,
discovery,
localApi: ctx.api,
maxEntitiesPerChunk: opts.maxEntitiesPerChunk,
mode: opts.dryRun ? "preview" : "apply",
});
const coordinator = new LocalUpstreamPushCoordinator({
targetOrigin: connection.targetOrigin,
paperclipCompanyId: connection.targetCompanyId,
headers: ({ method, path }) => cloudProofHeaders(connection, method, path),
});
const result = opts.dryRun ? await coordinator.preview(bundle) : await coordinator.apply(bundle);
const runId = getRunId(result);
const events = !opts.dryRun && runId ? await coordinator.events(runId).catch(() => null) : null;
const summary = summarizeResult(result);
const conflictCount = summary.conflict + summary.staleMapping;
if (ctx.json) {
printOutput({ result, events }, { json: true });
} else {
console.log(pc.bold(opts.dryRun ? "Cloud Push Preview" : "Cloud Push Applied"));
console.log(`run=${runId ?? "-"}`);
console.log(`manifest=${bundle.manifest.manifestHash}`);
console.log(
`create=${summary.create} update=${summary.update} adopt=${summary.adopt} ` +
`skip=${summary.skip} conflict=${summary.conflict} staleMapping=${summary.staleMapping}`,
);
printWarnings(result);
printConflicts(result);
printEvents(events);
}
if (conflictCount > 0) {
process.exitCode = CLOUD_SYNC_CONFLICT_EXIT_CODE;
}
return result;
}
export async function discoverUpstream(remoteUrl: string): Promise<UpstreamDiscovery> {
const base = new URL(remoteUrl);
const discoveryUrl = new URL("/.well-known/paperclip-upstream", base);
return requestCloudJson<UpstreamDiscovery>(discoveryUrl.toString(), { method: "GET" });
}
export function assertDiscoveryCompatible(discovery: UpstreamDiscovery): void {
if (discovery.schema !== "paperclip-upstream-discovery-v1") {
throw new Error("Remote URL is not a Paperclip Cloud upstream target.");
}
if (discovery.transfer.supportedSchemaMajor !== upstreamTransferSchema.major) {
throw new Error(
`Cloud upstream schema mismatch: local major ${upstreamTransferSchema.major}, remote supports ${discovery.transfer.supportedSchemaMajor}.`,
);
}
if (!discovery.transfer.featureFlags?.includes("cloud_sync")) {
throw new Error("Remote Paperclip Cloud stack does not advertise the cloud_sync transfer flag.");
}
}
export function resolveDeviceCodeExpiresAt(expiresAt: string | undefined, nowMs = Date.now()): number {
const parsed = typeof expiresAt === "string" ? Date.parse(expiresAt) : NaN;
return Number.isFinite(parsed) ? parsed : nowMs + DEVICE_CODE_FALLBACK_EXPIRES_MS;
}
export async function buildBundleFromLocalCompany(input: {
localCompanyId: string;
connection: CloudConnection;
discovery: UpstreamDiscovery;
localApi: {
post<T>(path: string, body?: unknown): Promise<T | null>;
};
maxEntitiesPerChunk?: number;
mode: "preview" | "apply";
}): Promise<LocalUpstreamExportBundle> {
const exported = await input.localApi.post<CompanyPortabilityExportResult>(
`/api/companies/${input.localCompanyId}/export`,
{
include: {
company: true,
agents: true,
projects: true,
issues: true,
skills: true,
},
expandReferencedSkills: true,
},
);
if (!exported) throw new Error("Local company export returned no data.");
const sourceHash = normalizedContentHash({
manifest: exported.manifest,
files: exported.files,
});
const source: UpstreamTransferManifestSource = {
sourceInstanceId: input.connection.sourceInstanceId,
sourceCompanyId: input.localCompanyId,
sourceInstanceKeyFingerprint: input.connection.sourceInstanceFingerprint,
exporterVersion: "paperclipai-cli-cloud-v1",
sourceSchemaVersion: "paperclip-local-portability-v1",
};
const target: UpstreamTransferManifestTarget = {
targetStackId: input.discovery.stack.id,
targetCompanyId: input.discovery.stack.companyId,
targetOrigin: input.discovery.stack.origin,
supportedSchemaMajor: input.discovery.transfer.supportedSchemaMajor,
};
const entities = buildEntitiesFromPortableExport(input.localCompanyId, input.connection.sourceInstanceId, exported);
const idempotencyKey = [
input.mode,
input.connection.sourceInstanceId,
input.localCompanyId,
input.discovery.stack.id,
sourceHash,
].join(":");
return buildLocalUpstreamExportBundle({
source,
target,
runId: `local-${input.mode}-${shortHash(idempotencyKey)}`,
idempotencyKey,
entities,
warnings: exported.warnings.map((message): UpstreamTransferWarning => ({
code: "local_company_export_warning",
severity: "warning",
message,
})),
featureFlags: ["cloud_sync"],
maxEntitiesPerChunk: input.maxEntitiesPerChunk,
});
}
async function authorizeConnection(
discovery: UpstreamDiscovery,
source: ReturnType<typeof createSourceIdentity>,
opts: { noBrowser: boolean },
): Promise<TokenResponse> {
if (!opts.noBrowser && canOpenBrowser() && discovery.auth.pkce) {
try {
return await authorizeWithBrowser(discovery, source);
} catch (error) {
console.error(pc.yellow(`Browser authorization failed; falling back to device-code flow. ${errorMessage(error)}`));
}
}
if (!discovery.auth.deviceCode) {
throw new Error("Remote Paperclip Cloud stack does not support device-code authorization.");
}
return authorizeWithDeviceCode(discovery, source, { openBrowser: !opts.noBrowser && canOpenBrowser() });
}
async function authorizeWithBrowser(
discovery: UpstreamDiscovery,
source: ReturnType<typeof createSourceIdentity>,
): Promise<TokenResponse> {
const pkce = discovery.auth.pkce;
if (!pkce) throw new Error("Remote did not advertise PKCE authorization.");
const callback = await startPkceCallbackServer();
const verifier = randomBytes(32).toString("base64url");
const challenge = createHash("sha256").update(verifier).digest("base64url");
const state = randomUUID();
const authorizeUrl = new URL(pkce.authorizeUrl);
authorizeUrl.searchParams.set("redirectUri", callback.redirectUri);
authorizeUrl.searchParams.set("state", state);
authorizeUrl.searchParams.set("codeChallenge", challenge);
authorizeUrl.searchParams.set("codeChallengeMethod", "S256");
authorizeUrl.searchParams.set("sourceInstanceId", source.sourceInstanceId);
authorizeUrl.searchParams.set("sourceInstanceFingerprint", source.sourceInstanceFingerprint);
authorizeUrl.searchParams.set("sourcePublicKey", source.sourcePublicKey);
authorizeUrl.searchParams.set("scopes", CLOUD_SYNC_SCOPES.join(" "));
try {
console.error(`Open this URL to approve cloud sync:\n${authorizeUrl.toString()}`);
if (!openUrl(authorizeUrl.toString())) {
throw new Error("Could not open a browser.");
}
const code = await callback.waitForCode(state);
return requestCloudJson<TokenResponse>(pkce.tokenUrl, {
method: "POST",
body: JSON.stringify({
grantType: "authorization_code",
code,
redirectUri: callback.redirectUri,
codeVerifier: verifier,
}),
});
} finally {
await callback.close();
}
}
async function authorizeWithDeviceCode(
discovery: UpstreamDiscovery,
source: ReturnType<typeof createSourceIdentity>,
opts: { openBrowser: boolean },
): Promise<TokenResponse> {
const device = discovery.auth.deviceCode;
if (!device) throw new Error("Remote did not advertise device-code authorization.");
const response = await requestCloudJson<{
deviceCode: string;
userCode: string;
verificationUri: string;
expiresAt?: string;
intervalSeconds?: number;
}>(device.deviceCodeUrl, {
method: "POST",
body: JSON.stringify({
stackId: discovery.stack.id,
sourceInstanceId: source.sourceInstanceId,
sourceInstanceFingerprint: source.sourceInstanceFingerprint,
sourcePublicKey: source.sourcePublicKey,
scopes: CLOUD_SYNC_SCOPES,
}),
});
console.error(pc.bold("Cloud device authorization required"));
console.error(`Open: ${response.verificationUri}`);
console.error(`Code: ${response.userCode}`);
if (opts.openBrowser) openUrl(response.verificationUri);
const expiresAt = resolveDeviceCodeExpiresAt(response.expiresAt);
const intervalMs = Math.max(500, (response.intervalSeconds ?? 5) * 1000);
while (Date.now() < expiresAt) {
await sleep(intervalMs);
try {
return await requestCloudJson<TokenResponse>(device.tokenUrl, {
method: "POST",
body: JSON.stringify({
grantType: "device_code",
deviceCode: response.deviceCode,
}),
});
} catch (error) {
if (error instanceof CloudAuthRequestError && error.body && typeof error.body === "object") {
const code = (error.body as { error?: unknown }).error;
if (code === "authorization_pending") continue;
}
throw error;
}
}
throw new Error("Device-code authorization expired before it was approved.");
}
function buildEntitiesFromPortableExport(
localCompanyId: string,
sourceInstanceId: string,
exported: CompanyPortabilityExportResult,
): LocalUpstreamExportEntityInput[] {
const companyKey: SourceEntityKey = {
sourceInstanceId,
sourceCompanyId: localCompanyId,
sourceEntityType: "company",
sourceEntityId: localCompanyId,
sourceNaturalKey: exported.manifest.company?.name ?? localCompanyId,
};
const entities: LocalUpstreamExportEntityInput[] = [
{
key: companyKey,
body: {
kind: "paperclip_company_portability_manifest",
manifest: exported.manifest,
rootPath: exported.rootPath,
paperclipExtensionPath: exported.paperclipExtensionPath,
fileCount: Object.keys(exported.files).length,
},
conflictKeys: [`company:${companyKey.sourceNaturalKey ?? localCompanyId}`],
},
];
for (const [filePath, entry] of Object.entries(exported.files).sort(([left], [right]) => left.localeCompare(right))) {
entities.push({
key: {
sourceInstanceId,
sourceCompanyId: localCompanyId,
sourceEntityType: "company_setting",
sourceEntityId: shortHash(filePath),
sourceNaturalKey: filePath,
},
body: {
kind: "paperclip_portable_file",
path: filePath,
entry: normalizePortableFileEntry(entry),
},
dependencies: [companyKey],
conflictKeys: [`portable_file:${filePath}`],
});
}
return entities;
}
function normalizePortableFileEntry(entry: CompanyPortabilityFileEntry): Record<string, unknown> {
if (typeof entry === "string") {
return { encoding: "utf8", data: entry };
}
return { ...entry };
}
async function assertCloudSyncEnabled(settingsPromise: Promise<InstanceExperimentalSettings | null>): Promise<void> {
const settings = await settingsPromise;
if (settings?.enableCloudSync !== true) {
throw new Error(
"Cloud sync is disabled. Enable the cloud sync experimental setting before running `paperclipai cloud push`.",
);
}
}
function cloudProofHeaders(connection: CloudConnection, method: string, pathAndSearch: string): Record<string, string> {
const timestamp = new Date().toISOString();
const nonce = randomUUID();
const payload = [
method,
connection.targetHost.toLowerCase(),
pathAndSearch,
connection.token.id,
connection.sourceInstanceId,
timestamp,
nonce,
].join("\n");
return {
Authorization: `Bearer ${connection.accessToken}`,
"X-Paperclip-Upstream-Source-Instance-Id": connection.sourceInstanceId,
"X-Paperclip-Upstream-Proof-Timestamp": timestamp,
"X-Paperclip-Upstream-Proof-Nonce": nonce,
"X-Paperclip-Upstream-Proof-Signature": sign(
null,
Buffer.from(payload, "utf8"),
connection.privateKeyPem,
).toString("base64url"),
};
}
async function requestCloudJson<T>(url: string, init: RequestInit): Promise<T> {
const headers = new Headers(init.headers);
headers.set("accept", "application/json");
if (init.body !== undefined && !headers.has("content-type")) {
headers.set("content-type", "application/json");
}
const response = await fetch(url, { ...init, headers });
const text = await response.text();
const parsed = text.trim() ? JSON.parse(text) as unknown : {};
if (!response.ok) {
const message = typeof parsed === "object" && parsed !== null && "error" in parsed
? String((parsed as { error: unknown }).error)
: `Cloud request failed with ${response.status}`;
throw new CloudAuthRequestError(response.status, message, parsed);
}
return parsed as T;
}
function createSourceIdentity() {
const { publicKey, privateKey } = generateKeyPairSync("ed25519");
const sourcePublicKey = publicKey.export({ type: "spki", format: "pem" }).toString();
const sourceInstanceFingerprint = `sha256:${createHash("sha256")
.update(publicKey.export({ type: "spki", format: "der" }))
.digest("hex")}`;
return {
sourceInstanceId: `paperclip-local-${resolvePaperclipInstanceId()}`,
sourceInstanceFingerprint,
sourcePublicKey,
privateKeyPem: privateKey.export({ type: "pkcs8", format: "pem" }).toString(),
};
}
async function startPkceCallbackServer(): Promise<{
redirectUri: string;
waitForCode: (state: string) => Promise<string>;
close: () => Promise<void>;
}> {
let resolveCode: ((code: string) => void) | null = null;
let rejectCode: ((error: Error) => void) | null = null;
let expectedState = "";
const codePromise = new Promise<string>((resolve, reject) => {
resolveCode = resolve;
rejectCode = reject;
});
const server = createServer((req, res) => {
const url = new URL(req.url ?? "/", "http://127.0.0.1");
const code = url.searchParams.get("code");
const state = url.searchParams.get("state");
if (!code || state !== expectedState) {
res.writeHead(400, { "Content-Type": "text/plain" });
res.end("Paperclip Cloud authorization failed. You can close this tab.");
rejectCode?.(new Error("Authorization callback was missing a valid code or state."));
return;
}
res.writeHead(200, { "Content-Type": "text/plain" });
res.end("Paperclip Cloud authorization complete. You can close this tab.");
resolveCode?.(code);
});
await listenOnLoopback(server);
const address = server.address();
if (typeof address !== "object" || !address?.port) {
throw new Error("Failed to start local authorization callback server.");
}
return {
redirectUri: `http://127.0.0.1:${address.port}/cloud/callback`,
waitForCode: (state: string) => {
expectedState = state;
return codePromise;
},
close: () => closeServer(server),
};
}
function listenOnLoopback(server: Server): Promise<void> {
return new Promise((resolve, reject) => {
server.once("error", reject);
server.listen(0, "127.0.0.1", () => {
server.off("error", reject);
resolve();
});
});
}
function closeServer(server: Server): Promise<void> {
return new Promise((resolve, reject) => {
server.close((error) => error ? reject(error) : resolve());
});
}
function canOpenBrowser(): boolean {
if (process.platform === "darwin" || process.platform === "win32") return true;
return Boolean(process.env.DISPLAY || process.env.WAYLAND_DISPLAY);
}
function summarizeResult(result: unknown): {
create: number;
update: number;
adopt: number;
skip: number;
conflict: number;
staleMapping: number;
} {
const summary = asRecord(asRecord(result)?.summary);
return {
create: numberValue(summary?.create),
update: numberValue(summary?.update),
adopt: numberValue(summary?.adopt),
skip: numberValue(summary?.skip),
conflict: numberValue(summary?.conflict),
staleMapping: numberValue(summary?.staleMapping),
};
}
function printWarnings(result: unknown): void {
const warnings = Array.isArray(asRecord(result)?.warnings) ? asRecord(result)?.warnings as unknown[] : [];
for (const warning of warnings) {
const record = asRecord(warning);
console.log(pc.yellow(`warning=${record?.code ?? "warning"} ${record?.message ?? ""}`.trim()));
}
}
function printConflicts(result: unknown): void {
const conflicts = Array.isArray(asRecord(result)?.conflicts) ? asRecord(result)?.conflicts as unknown[] : [];
for (const conflict of conflicts.slice(0, 10)) {
const record = asRecord(conflict);
console.log(pc.red(`conflict=${record?.conflictKind ?? "target_conflict"} target=${record?.targetEntityId ?? "-"}`));
}
if (conflicts.length > 10) console.log(pc.red(`conflicts_truncated=${conflicts.length - 10}`));
}
function printEvents(events: unknown): void {
const rows = Array.isArray(asRecord(events)?.events) ? asRecord(events)?.events as unknown[] : [];
for (const row of rows.slice(-10)) {
const event = asRecord(row);
console.log(pc.dim(`event=${event?.action ?? "-"} target=${event?.targetEntityId ?? "-"}`));
}
}
function getRunId(result: unknown): string | null {
const run = asRecord(asRecord(result)?.run);
return typeof run?.id === "string" ? run.id : null;
}
function redactConnection(connection: CloudConnection): Record<string, unknown> {
return {
id: connection.id,
remoteUrl: connection.remoteUrl,
targetOrigin: connection.targetOrigin,
stackId: connection.stackId,
targetCompanyId: connection.targetCompanyId,
scopes: connection.scopes,
expiresAt: connection.token.expiresAt,
};
}
function connectionId(targetOrigin: string): string {
return `cloud-${shortHash(targetOrigin)}`;
}
function shortHash(value: string): string {
return createHash("sha256").update(value).digest("hex").slice(0, 16);
}
function requiredString(value: unknown, label: string): string {
if (typeof value === "string" && value.trim()) return value.trim();
throw new Error(`${label} is required.`);
}
function numberValue(value: unknown): number {
return typeof value === "number" && Number.isFinite(value) ? value : 0;
}
function asRecord(value: unknown): Record<string, unknown> | null {
return typeof value === "object" && value !== null && !Array.isArray(value)
? value as Record<string, unknown>
: null;
}
function isSchemaMismatchError(error: unknown): boolean {
if (error instanceof UpstreamImportRequestError) {
return JSON.stringify(error.body).toLowerCase().includes("schema");
}
return error instanceof Error && error.message.toLowerCase().includes("schema mismatch");
}
function errorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
function sleep(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
export const cloudCommandExitCodes = {
conflict: CLOUD_SYNC_CONFLICT_EXIT_CODE,
schemaMismatch: CLOUD_SYNC_SCHEMA_MISMATCH_EXIT_CODE,
} as const;
+2
View File
@@ -19,6 +19,7 @@ import { registerDashboardCommands } from "./commands/client/dashboard.js";
import { registerRoutineCommands } from "./commands/routines.js";
import { registerFeedbackCommands } from "./commands/client/feedback.js";
import { registerSecretCommands } from "./commands/client/secrets.js";
import { registerCloudCommands } from "./commands/client/cloud.js";
import { applyDataDirOverride, type DataDirOptionLike } from "./config/data-dir.js";
import { loadPaperclipEnvFile } from "./config/env.js";
import { initTelemetryFromConfigFile, flushTelemetry } from "./telemetry.js";
@@ -149,6 +150,7 @@ registerDashboardCommands(program);
registerRoutineCommands(program);
registerFeedbackCommands(program);
registerSecretCommands(program);
registerCloudCommands(program);
registerWorktreeCommands(program);
registerEnvLabCommands(program);
registerPluginCommands(program);
@@ -0,0 +1,71 @@
CREATE TABLE IF NOT EXISTS "cloud_upstream_connections" (
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
"company_id" uuid NOT NULL,
"remote_url" text NOT NULL,
"source_instance_id" text NOT NULL,
"source_instance_fingerprint" text NOT NULL,
"source_public_key" text NOT NULL,
"private_key_pem" text NOT NULL,
"token_status" text NOT NULL,
"scopes" text[] DEFAULT '{}' NOT NULL,
"authorized_global_user_id" text,
"access_token" text,
"token_id" text,
"token_expires_at" timestamp with time zone,
"target_stack_id" text NOT NULL,
"target_stack_slug" text,
"target_stack_display_name" text,
"target_company_id" text NOT NULL,
"target_origin" text NOT NULL,
"target_primary_host" text NOT NULL,
"target_product" text NOT NULL,
"target_schema_major" integer NOT NULL,
"target_max_chunk_bytes" integer NOT NULL,
"pending_state" text,
"pending_code_verifier" text,
"pending_redirect_uri" text,
"pending_token_url" text,
"last_run_id" uuid,
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
);--> statement-breakpoint
CREATE TABLE IF NOT EXISTS "cloud_upstream_runs" (
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
"connection_id" uuid NOT NULL,
"company_id" uuid NOT NULL,
"remote_run_id" text,
"status" text NOT NULL,
"active_step" text NOT NULL,
"progress_percent" integer DEFAULT 0 NOT NULL,
"dry_run" boolean DEFAULT false NOT NULL,
"retry_of_run_id" uuid,
"summary" jsonb DEFAULT '[]'::jsonb NOT NULL,
"warnings" jsonb DEFAULT '[]'::jsonb NOT NULL,
"conflicts" jsonb DEFAULT '[]'::jsonb NOT NULL,
"events" jsonb DEFAULT '[]'::jsonb NOT NULL,
"report" jsonb DEFAULT '{}'::jsonb NOT NULL,
"idempotency_key" text NOT NULL,
"manifest_hash" text NOT NULL,
"target_url" text,
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
"updated_at" timestamp with time zone DEFAULT now() NOT NULL,
"completed_at" timestamp with time zone
);--> statement-breakpoint
DO $$ BEGIN
IF NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conname = 'cloud_upstream_connections_company_id_companies_id_fk') THEN
ALTER TABLE "cloud_upstream_connections" ADD CONSTRAINT "cloud_upstream_connections_company_id_companies_id_fk" FOREIGN KEY ("company_id") REFERENCES "public"."companies"("id") ON DELETE cascade ON UPDATE no action;
END IF;
END $$;--> statement-breakpoint
DO $$ BEGIN
IF NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conname = 'cloud_upstream_runs_connection_id_cloud_upstream_connections_id_fk') THEN
ALTER TABLE "cloud_upstream_runs" ADD CONSTRAINT "cloud_upstream_runs_connection_id_cloud_upstream_connections_id_fk" FOREIGN KEY ("connection_id") REFERENCES "public"."cloud_upstream_connections"("id") ON DELETE cascade ON UPDATE no action;
END IF;
END $$;--> statement-breakpoint
DO $$ BEGIN
IF NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conname = 'cloud_upstream_runs_company_id_companies_id_fk') THEN
ALTER TABLE "cloud_upstream_runs" ADD CONSTRAINT "cloud_upstream_runs_company_id_companies_id_fk" FOREIGN KEY ("company_id") REFERENCES "public"."companies"("id") ON DELETE cascade ON UPDATE no action;
END IF;
END $$;--> statement-breakpoint
CREATE INDEX IF NOT EXISTS "cloud_upstream_connections_company_idx" ON "cloud_upstream_connections" USING btree ("company_id");--> statement-breakpoint
CREATE INDEX IF NOT EXISTS "cloud_upstream_runs_company_created_idx" ON "cloud_upstream_runs" USING btree ("company_id","created_at");--> statement-breakpoint
CREATE INDEX IF NOT EXISTS "cloud_upstream_runs_connection_idx" ON "cloud_upstream_runs" USING btree ("connection_id");
@@ -624,6 +624,13 @@
"when": 1779446400000,
"tag": "0088_backfill_principal_access_compatibility",
"breakpoints": true
},
{
"idx": 89,
"version": "7",
"when": 1779129600000,
"tag": "0089_cloud_upstreams",
"breakpoints": true
}
]
}
+75
View File
@@ -0,0 +1,75 @@
import { boolean, index, integer, jsonb, pgTable, text, timestamp, uuid } from "drizzle-orm/pg-core";
import { companies } from "./companies.js";
export const cloudUpstreamConnections = pgTable(
"cloud_upstream_connections",
{
id: uuid("id").primaryKey().defaultRandom(),
companyId: uuid("company_id").notNull().references(() => companies.id, { onDelete: "cascade" }),
remoteUrl: text("remote_url").notNull(),
sourceInstanceId: text("source_instance_id").notNull(),
sourceInstanceFingerprint: text("source_instance_fingerprint").notNull(),
sourcePublicKey: text("source_public_key").notNull(),
// Stored through the Cloud Upstream service as an encrypted credential envelope.
privateKeyPem: text("private_key_pem").notNull(),
tokenStatus: text("token_status").notNull(),
scopes: text("scopes").array().notNull().default([]),
authorizedGlobalUserId: text("authorized_global_user_id"),
// Stored through the Cloud Upstream service as an encrypted credential envelope.
accessToken: text("access_token"),
tokenId: text("token_id"),
tokenExpiresAt: timestamp("token_expires_at", { withTimezone: true }),
targetStackId: text("target_stack_id").notNull(),
targetStackSlug: text("target_stack_slug"),
targetStackDisplayName: text("target_stack_display_name"),
targetCompanyId: text("target_company_id").notNull(),
targetOrigin: text("target_origin").notNull(),
targetPrimaryHost: text("target_primary_host").notNull(),
targetProduct: text("target_product").notNull(),
targetSchemaMajor: integer("target_schema_major").notNull(),
targetMaxChunkBytes: integer("target_max_chunk_bytes").notNull(),
pendingState: text("pending_state"),
pendingCodeVerifier: text("pending_code_verifier"),
pendingRedirectUri: text("pending_redirect_uri"),
pendingTokenUrl: text("pending_token_url"),
lastRunId: uuid("last_run_id"),
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
updatedAt: timestamp("updated_at", { withTimezone: true }).notNull().defaultNow(),
},
(table) => [
index("cloud_upstream_connections_company_idx").on(table.companyId),
],
);
export const cloudUpstreamRuns = pgTable(
"cloud_upstream_runs",
{
id: uuid("id").primaryKey().defaultRandom(),
connectionId: uuid("connection_id").notNull().references(() => cloudUpstreamConnections.id, { onDelete: "cascade" }),
companyId: uuid("company_id").notNull().references(() => companies.id, { onDelete: "cascade" }),
remoteRunId: text("remote_run_id"),
status: text("status").notNull(),
activeStep: text("active_step").notNull(),
progressPercent: integer("progress_percent").notNull().default(0),
dryRun: boolean("dry_run").notNull().default(false),
retryOfRunId: uuid("retry_of_run_id"),
summary: jsonb("summary").$type<import("@paperclipai/shared").CloudUpstreamSummaryCount[]>().notNull().default([]),
warnings: jsonb("warnings").$type<import("@paperclipai/shared").CloudUpstreamWarning[]>().notNull().default([]),
conflicts: jsonb("conflicts").$type<import("@paperclipai/shared").CloudUpstreamConflict[]>().notNull().default([]),
events: jsonb("events").$type<import("@paperclipai/shared").CloudUpstreamRunEvent[]>().notNull().default([]),
report: jsonb("report").$type<Record<string, unknown>>().notNull().default({}),
idempotencyKey: text("idempotency_key").notNull(),
manifestHash: text("manifest_hash").notNull(),
targetUrl: text("target_url"),
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
updatedAt: timestamp("updated_at", { withTimezone: true }).notNull().defaultNow(),
completedAt: timestamp("completed_at", { withTimezone: true }),
},
(table) => [
index("cloud_upstream_runs_company_created_idx").on(table.companyId, table.createdAt),
index("cloud_upstream_runs_connection_idx").on(table.connectionId),
],
);
+1
View File
@@ -2,6 +2,7 @@ export { companies } from "./companies.js";
export { companyLogos } from "./company_logos.js";
export { authUsers, authSessions, authAccounts, authVerifications } from "./auth.js";
export { instanceSettings } from "./instance_settings.js";
export { cloudUpstreamConnections, cloudUpstreamRuns } from "./cloud_upstreams.js";
export { instanceUserRoles } from "./instance_user_roles.js";
export { userSidebarPreferences } from "./user_sidebar_preferences.js";
export { agents } from "./agents.js";
+16
View File
@@ -688,6 +688,22 @@ export {
MAX_ISSUE_GRAPH_LIVENESS_AUTO_RECOVERY_LOOKBACK_HOURS,
} from "./types/instance.js";
export type {
CloudUpstreamConnectStartResponse,
CloudUpstreamActivationDecision,
CloudUpstreamActivationEntityType,
CloudUpstreamConnection,
CloudUpstreamConflict,
CloudUpstreamPreview,
CloudUpstreamRun,
CloudUpstreamRunEvent,
CloudUpstreamsState,
CloudUpstreamStep,
CloudUpstreamSummaryCount,
CloudUpstreamTarget,
CloudUpstreamWarning,
} from "./types/cloud-upstream.js";
export {
getClosedIsolatedExecutionWorkspaceMessage,
isClosedIsolatedExecutionWorkspace,
+110
View File
@@ -0,0 +1,110 @@
export type CloudUpstreamStep = "connect" | "scan" | "preview" | "push" | "verify" | "activate";
export type CloudUpstreamRunStatus = "previewed" | "running" | "succeeded" | "failed" | "cancelled";
export type CloudUpstreamActivationEntityType = "agents" | "routines" | "monitors";
export interface CloudUpstreamActivationDecision {
entityType: CloudUpstreamActivationEntityType;
count: number;
status: "paused" | "activated";
activatedAt: string | null;
}
export interface CloudUpstreamTarget {
stackId: string;
stackSlug: string | null;
stackDisplayName: string | null;
companyId: string;
primaryHost: string;
origin: string;
product: string;
schemaMajor: number;
maxChunkBytes: number;
}
export interface CloudUpstreamConnection {
id: string;
companyId: string;
remoteUrl: string;
target: CloudUpstreamTarget;
tokenStatus: "pending" | "connected" | "expired" | "revoked";
scopes: string[];
authorizedGlobalUserId: string | null;
expiresAt: string | null;
createdAt: string;
updatedAt: string;
lastRunId: string | null;
}
export interface CloudUpstreamSummaryCount {
key: string;
label: string;
count: number;
}
export interface CloudUpstreamWarning {
code: string;
severity: "warning" | "blocker";
title: string;
detail: string;
}
export interface CloudUpstreamConflict {
id: string;
entityType: string;
sourceLabel: string;
targetLabel: string;
plannedAction: "create" | "update" | "skip" | "blocked";
reason: string;
}
export interface CloudUpstreamPreview {
connectionId: string;
sourceCompanyId: string;
target: CloudUpstreamTarget;
schemaCompatible: boolean;
summary: CloudUpstreamSummaryCount[];
warnings: CloudUpstreamWarning[];
conflicts: CloudUpstreamConflict[];
generatedAt: string;
}
export interface CloudUpstreamRunEvent {
id: string;
at: string;
phase: CloudUpstreamStep;
type: "created" | "updated" | "skipped" | "conflict" | "retrying" | "failed" | "completed";
message: string;
}
export interface CloudUpstreamRun {
id: string;
connectionId: string;
companyId: string;
status: CloudUpstreamRunStatus;
activeStep: CloudUpstreamStep;
progressPercent: number;
dryRun: boolean;
summary: CloudUpstreamSummaryCount[];
warnings: CloudUpstreamWarning[];
conflicts: CloudUpstreamConflict[];
events: CloudUpstreamRunEvent[];
targetUrl: string | null;
report: Record<string, unknown>;
retryOfRunId: string | null;
createdAt: string;
updatedAt: string;
completedAt: string | null;
}
export interface CloudUpstreamsState {
connections: CloudUpstreamConnection[];
runs: CloudUpstreamRun[];
}
export interface CloudUpstreamConnectStartResponse {
pendingConnectionId: string;
authorizationUrl: string;
connection: CloudUpstreamConnection;
}
+1
View File
@@ -29,6 +29,7 @@ export interface InstanceGeneralSettings {
export interface InstanceExperimentalSettings {
enableEnvironments: boolean;
enableIsolatedWorkspaces: boolean;
enableCloudSync: boolean;
autoRestartDevServerWhenIdle: boolean;
enableIssueGraphLivenessAutoRecovery: boolean;
issueGraphLivenessAutoRecoveryLookbackHours: number;
@@ -38,6 +38,7 @@ export const patchInstanceGeneralSettingsSchema = instanceGeneralSettingsSchema.
export const instanceExperimentalSettingsSchema = z.object({
enableEnvironments: z.boolean().default(false),
enableIsolatedWorkspaces: z.boolean().default(false),
enableCloudSync: z.boolean().default(false),
autoRestartDevServerWhenIdle: z.boolean().default(false),
enableIssueGraphLivenessAutoRecovery: z.boolean().default(false),
issueGraphLivenessAutoRecoveryLookbackHours: z
@@ -9,6 +9,7 @@ import {
createDb,
documents,
documentRevisions,
heartbeatRunEvents,
heartbeatRuns,
issueComments,
issueDocuments,
@@ -42,6 +43,7 @@ describeEmbeddedPostgres("cleanup removal services", () => {
}, 20_000);
afterEach(async () => {
await db.delete(heartbeatRunEvents);
await db.delete(activityLog);
await db.delete(issueReadStates);
await db.delete(issueComments);
@@ -228,4 +230,32 @@ describeEmbeddedPostgres("cleanup removal services", () => {
await expect(db.select().from(issueReadStates).where(eq(issueReadStates.companyId, companyId))).resolves.toHaveLength(0);
await expect(db.select().from(activityLog).where(eq(activityLog.companyId, companyId))).resolves.toHaveLength(0);
});
it("removes heartbeat events by run id before deleting company-owned runs", async () => {
const { agentId, companyId, runId } = await seedFixture();
const otherCompanyId = randomUUID();
await db.insert(companies).values({
id: otherCompanyId,
name: "Other Company",
issuePrefix: `O${otherCompanyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
requireBoardApprovalForNewAgents: false,
});
await db.insert(heartbeatRunEvents).values({
companyId: otherCompanyId,
runId,
agentId,
seq: 1,
eventType: "output",
message: "event with mismatched company scope",
});
const removed = await companyService(db).remove(companyId);
expect(removed?.id).toBe(companyId);
await expect(db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId))).resolves.toHaveLength(0);
await expect(db.select().from(heartbeatRunEvents).where(eq(heartbeatRunEvents.runId, runId))).resolves.toHaveLength(0);
await expect(db.select().from(companies).where(eq(companies.id, otherCompanyId))).resolves.toHaveLength(1);
});
});
@@ -0,0 +1,334 @@
import { generateKeyPairSync, randomUUID } from "node:crypto";
import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from "vitest";
import { companies, cloudUpstreamConnections, cloudUpstreamRuns, companySkills, createDb } from "@paperclipai/db";
import { HttpError } from "../errors.js";
import {
cloudUpstreamRemoteFailureReport,
cloudUpstreamService,
reconcileCloudUpstreamRunsOnStartup,
sealCloudUpstreamCredential,
unsealCloudUpstreamCredential,
} from "../services/cloud-upstreams.js";
import {
getEmbeddedPostgresTestSupport,
startEmbeddedPostgresTestDatabase,
} from "./helpers/embedded-postgres.js";
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
if (!embeddedPostgresSupport.supported) {
console.warn(
`Skipping embedded Postgres cloud upstream tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`,
);
}
describe("cloud upstream remote failures", () => {
it("preserves the cloud response body and message on run reports", () => {
const body = {
error: "bad_request",
message: "entities[42].body must be an object",
errors: [{ path: "entities[42].body" }],
};
expect(cloudUpstreamRemoteFailureReport(new HttpError(400, "bad_request", body))).toEqual({
error: "bad_request",
errorMessage: "entities[42].body must be an object",
details: body,
});
});
it("falls back to the thrown error message for non-remote failures", () => {
expect(cloudUpstreamRemoteFailureReport(new Error("network failed"))).toEqual({
error: "network failed",
});
});
});
describe("cloud upstream credential storage", () => {
const previousMasterKey = process.env.PAPERCLIP_SECRETS_MASTER_KEY;
afterEach(() => {
if (previousMasterKey === undefined) {
delete process.env.PAPERCLIP_SECRETS_MASTER_KEY;
} else {
process.env.PAPERCLIP_SECRETS_MASTER_KEY = previousMasterKey;
}
});
it("stores new credentials as encrypted envelopes and preserves legacy plaintext reads", async () => {
process.env.PAPERCLIP_SECRETS_MASTER_KEY = "12345678901234567890123456789012";
const sealed = await sealCloudUpstreamCredential("cloud-access-token");
expect(sealed).toMatch(/^paperclip-cloud-credential:/);
expect(sealed).not.toContain("cloud-access-token");
await expect(unsealCloudUpstreamCredential(sealed)).resolves.toBe("cloud-access-token");
await expect(unsealCloudUpstreamCredential("legacy-plaintext-token")).resolves.toBe("legacy-plaintext-token");
});
});
describeEmbeddedPostgres("cloud upstream persistence", () => {
let db!: ReturnType<typeof createDb>;
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
const previousMasterKey = process.env.PAPERCLIP_SECRETS_MASTER_KEY;
beforeAll(async () => {
process.env.PAPERCLIP_SECRETS_MASTER_KEY = "12345678901234567890123456789012";
tempDb = await startEmbeddedPostgresTestDatabase("paperclip-cloud-upstreams-");
db = createDb(tempDb.connectionString);
}, 20_000);
afterEach(async () => {
vi.restoreAllMocks();
await db.delete(cloudUpstreamRuns);
await db.delete(cloudUpstreamConnections);
await db.delete(companySkills);
await db.delete(companies);
});
afterAll(async () => {
if (previousMasterKey === undefined) {
delete process.env.PAPERCLIP_SECRETS_MASTER_KEY;
} else {
process.env.PAPERCLIP_SECRETS_MASTER_KEY = previousMasterKey;
}
await tempDb?.cleanup();
});
it("encrypts stored upstream credentials while keeping connection flows usable", async () => {
const companyId = randomUUID();
await seedCompany(companyId);
const tokenUrl = "https://cloud.example.test/oauth/token";
vi.spyOn(globalThis, "fetch").mockImplementation(async (input, init) => {
const url = String(input);
if (url.startsWith("https://cloud.example.test/.well-known/paperclip-upstream")) {
return jsonResponse({
product: "Paperclip Cloud",
stack: {
id: "stack-1",
companyId: "cloud-company-1",
origin: "https://cloud.example.test",
primaryHost: "cloud.example.test",
},
transfer: {
supportedSchemaMajor: 1,
maxChunkBytes: 8192,
},
auth: {
scopes: ["upstream_import:write"],
pkce: {
authorizeUrl: "https://cloud.example.test/oauth/authorize",
tokenUrl,
},
},
});
}
if (url === tokenUrl && init?.method === "POST") {
const payload = JSON.parse(String(init.body));
expect(payload.codeVerifier).toEqual(expect.any(String));
expect(payload.codeVerifier).not.toContain("paperclip-cloud-credential:");
return jsonResponse({
accessToken: "cloud-access-token",
token: {
id: "token-1",
expiresAt: "2026-05-22T13:00:00.000Z",
globalUserId: "user-1",
},
});
}
throw new Error(`Unexpected fetch: ${url}`);
});
const service = cloudUpstreamService(db, { instanceId: "test" });
const started = await service.startConnect({
companyId,
remoteUrl: "https://cloud.example.test",
redirectUri: "http://localhost:3100/callback",
});
await service.finishConnect({
pendingConnectionId: started.pendingConnectionId,
code: "auth-code",
state: new URL(started.authorizationUrl).searchParams.get("state") ?? "",
});
const [row] = await db.select().from(cloudUpstreamConnections);
expect(row.privateKeyPem).toMatch(/^paperclip-cloud-credential:/);
expect(row.privateKeyPem).not.toContain("BEGIN PRIVATE KEY");
expect(row.accessToken).toMatch(/^paperclip-cloud-credential:/);
expect(row.accessToken).not.toContain("cloud-access-token");
});
it("marks orphaned running runs failed during startup reconciliation", async () => {
const companyId = randomUUID();
const connectionId = randomUUID();
const runningRunId = randomUUID();
const succeededRunId = randomUUID();
const reconciledAt = new Date("2026-05-22T13:00:00.000Z");
await seedCompany(companyId);
await db.insert(cloudUpstreamConnections).values({
id: connectionId,
companyId,
remoteUrl: "https://cloud.example.test",
sourceInstanceId: "source-1",
sourceInstanceFingerprint: "sha256:test",
sourcePublicKey: "public-key",
privateKeyPem: "legacy-private-key",
tokenStatus: "connected",
scopes: ["upstream_import:write"],
authorizedGlobalUserId: "user-1",
accessToken: "legacy-token",
tokenId: "token-1",
targetStackId: "stack-1",
targetCompanyId: "cloud-company-1",
targetOrigin: "https://cloud.example.test",
targetPrimaryHost: "cloud.example.test",
targetProduct: "Paperclip Cloud",
targetSchemaMajor: 1,
targetMaxChunkBytes: 8192,
});
await db.insert(cloudUpstreamRuns).values([
cloudRunRow({ id: runningRunId, connectionId, companyId, status: "running" }),
cloudRunRow({ id: succeededRunId, connectionId, companyId, status: "succeeded", completedAt: reconciledAt }),
]);
await expect(reconcileCloudUpstreamRunsOnStartup(db, reconciledAt)).resolves.toEqual({ reconciled: 1 });
const rows = await db.select().from(cloudUpstreamRuns);
const running = rows.find((row) => row.id === runningRunId);
const succeeded = rows.find((row) => row.id === succeededRunId);
expect(running?.status).toBe("failed");
expect(running?.completedAt?.toISOString()).toBe(reconciledAt.toISOString());
expect(running?.events.at(-1)?.message).toContain("server startup");
expect(running?.report).toMatchObject({
error: "orphaned_running_run",
reconciledAt: reconciledAt.toISOString(),
});
expect(succeeded?.status).toBe("succeeded");
});
it("rejects a new run when the connection already has a running run", async () => {
const companyId = randomUUID();
const connectionId = randomUUID();
const runningRunId = randomUUID();
await seedCompany(companyId);
await db.insert(cloudUpstreamConnections).values(cloudConnectionRow({ id: connectionId, companyId }));
await db.insert(cloudUpstreamRuns).values(
cloudRunRow({ id: runningRunId, connectionId, companyId, status: "running" }),
);
await expect(cloudUpstreamService(db).createRun({ connectionId, companyId })).rejects.toMatchObject({
status: 409,
details: { runId: runningRunId },
});
});
it("preserves a cancelled run when an in-flight createRun tries to finish", async () => {
const companyId = randomUUID();
const connectionId = randomUUID();
await seedCompany(companyId);
await db.insert(cloudUpstreamConnections).values(cloudConnectionRow({ id: connectionId, companyId }));
const service = cloudUpstreamService(db);
const remoteCalls: string[] = [];
globalThis.fetch = vi.fn(async (input) => {
const path = new URL(String(input)).pathname;
remoteCalls.push(path);
if (path.endsWith("/upstream-imports/runs")) {
return jsonResponse({ run: { id: "remote-run-1" } });
}
if (path.endsWith("/chunks")) {
const run = await db.select().from(cloudUpstreamRuns).then((rows) => rows[0]);
expect(run?.status).toBe("running");
await service.cancelRun(connectionId, run.id, companyId);
return jsonResponse({ ok: true });
}
if (path.endsWith("/cancel")) {
return jsonResponse({ ok: true });
}
if (path.endsWith("/apply")) {
return jsonResponse({ ok: true });
}
if (path.endsWith("/events")) {
return jsonResponse({ events: [] });
}
return jsonResponse({ error: "not_found" }, 404);
}) as typeof fetch;
const result = await service.createRun({ connectionId, companyId });
expect(result.status).toBe("cancelled");
expect(remoteCalls.some((path) => path.endsWith("/apply"))).toBe(false);
const rows = await db.select().from(cloudUpstreamRuns);
expect(rows).toHaveLength(1);
expect(rows[0]?.status).toBe("cancelled");
});
async function seedCompany(companyId: string) {
await db.insert(companies).values({
id: companyId,
name: "Paperclip",
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
requireBoardApprovalForNewAgents: false,
});
}
});
function jsonResponse(body: unknown): Response {
return new Response(JSON.stringify(body), {
status: 200,
headers: { "Content-Type": "application/json" },
});
}
function cloudConnectionRow(input: { id: string; companyId: string }) {
const { privateKey } = generateKeyPairSync("ed25519");
return {
id: input.id,
companyId: input.companyId,
remoteUrl: "https://cloud.example.test",
sourceInstanceId: "source-1",
sourceInstanceFingerprint: "sha256:test",
sourcePublicKey: "public-key",
privateKeyPem: privateKey.export({ type: "pkcs8", format: "pem" }).toString(),
tokenStatus: "connected",
scopes: ["upstream_import:write"],
authorizedGlobalUserId: "user-1",
accessToken: "legacy-token",
tokenId: "token-1",
targetStackId: "stack-1",
targetCompanyId: "cloud-company-1",
targetOrigin: "https://cloud.example.test",
targetPrimaryHost: "cloud.example.test",
targetProduct: "Paperclip Cloud",
targetSchemaMajor: 1,
targetMaxChunkBytes: 8192,
};
}
function cloudRunRow(input: {
id: string;
connectionId: string;
companyId: string;
status: string;
completedAt?: Date;
}) {
return {
id: input.id,
connectionId: input.connectionId,
companyId: input.companyId,
status: input.status,
activeStep: "push",
progressPercent: input.status === "running" ? 45 : 100,
dryRun: false,
summary: [],
warnings: [],
conflicts: [],
events: [],
report: {},
idempotencyKey: `key-${input.id}`,
manifestHash: `sha256:${input.id.replace(/-/g, "")}`,
targetUrl: "https://cloud.example.test",
completedAt: input.completedAt,
};
}
@@ -64,6 +64,7 @@ describe("instance settings routes", () => {
mockInstanceSettingsService.getExperimental.mockResolvedValue({
enableEnvironments: false,
enableIsolatedWorkspaces: false,
enableCloudSync: false,
autoRestartDevServerWhenIdle: false,
enableIssueGraphLivenessAutoRecovery: true,
issueGraphLivenessAutoRecoveryLookbackHours: 24,
@@ -81,6 +82,7 @@ describe("instance settings routes", () => {
experimental: {
enableEnvironments: true,
enableIsolatedWorkspaces: true,
enableCloudSync: true,
autoRestartDevServerWhenIdle: false,
enableIssueGraphLivenessAutoRecovery: true,
issueGraphLivenessAutoRecoveryLookbackHours: 24,
@@ -123,6 +125,7 @@ describe("instance settings routes", () => {
expect(getRes.body).toEqual({
enableEnvironments: false,
enableIsolatedWorkspaces: false,
enableCloudSync: false,
autoRestartDevServerWhenIdle: false,
enableIssueGraphLivenessAutoRecovery: true,
issueGraphLivenessAutoRecoveryLookbackHours: 24,
@@ -0,0 +1,23 @@
import { describe, expect, it } from "vitest";
import { normalizeExperimentalSettings } from "../services/instance-settings.js";
describe("instance settings service", () => {
it("ignores retired experimental flags without resetting current settings", () => {
expect(normalizeExperimentalSettings({
enableEnvironments: true,
enableIsolatedWorkspaces: true,
enableCloudSync: true,
autoRestartDevServerWhenIdle: true,
enableIssueGraphLivenessAutoRecovery: true,
issueGraphLivenessAutoRecoveryLookbackHours: 48,
enableNewestFirstIssueThread: true,
})).toEqual({
enableEnvironments: true,
enableIsolatedWorkspaces: true,
enableCloudSync: true,
autoRestartDevServerWhenIdle: true,
enableIssueGraphLivenessAutoRecovery: true,
issueGraphLivenessAutoRecoveryLookbackHours: 48,
});
});
});
@@ -166,6 +166,7 @@ vi.mock("../services/index.js", () => ({
},
})),
})),
reconcileCloudUpstreamRunsOnStartup: vi.fn(async () => ({ reconciled: 0 })),
reconcilePersistedRuntimeServicesOnStartup: vi.fn(async () => ({ reconciled: 0 })),
routineService: vi.fn(() => ({
tickScheduledTriggers: vi.fn(async () => ({ triggered: 0 })),
+14
View File
@@ -33,6 +33,7 @@ import {
backfillPrincipalAccessCompatibility,
heartbeatService,
instanceSettingsService,
reconcileCloudUpstreamRunsOnStartup,
reconcilePersistedRuntimeServicesOnStartup,
routineService,
} from "./services/index.js";
@@ -700,6 +701,19 @@ export async function startServer(): Promise<StartedServer> {
logger.error({ err }, "startup reconciliation of persisted runtime services failed");
});
void reconcileCloudUpstreamRunsOnStartup(db as any)
.then((result) => {
if (result.reconciled > 0) {
logger.warn(
{ reconciled: result.reconciled },
"reconciled cloud upstream runs from a previous server process",
);
}
})
.catch((err) => {
logger.error({ err }, "startup reconciliation of cloud upstream runs failed");
});
if (config.heartbeatSchedulerEnabled) {
const heartbeat = heartbeatService(db as any, { pluginWorkerManager });
const routines = routineService(db as any, { pluginWorkerManager });
+118
View File
@@ -0,0 +1,118 @@
import { Router } from "express";
import type { Db } from "@paperclipai/db";
import { badRequest, notFound } from "../errors.js";
import { assertBoardOrgAccess } from "./authz.js";
import { cloudUpstreamService, instanceSettingsService } from "../services/index.js";
export function cloudUpstreamRoutes(db: Db, options: { instanceId?: string } = {}) {
const router = Router();
const service = cloudUpstreamService(db, options);
const settings = instanceSettingsService(db);
async function assertEnabled() {
const experimental = await settings.getExperimental();
if (experimental.enableCloudSync !== true) {
throw notFound("Cloud sync is not enabled");
}
}
router.get("/cloud-upstreams", async (req, res) => {
assertBoardOrgAccess(req);
await assertEnabled();
const companyId = stringQuery(req.query.companyId, "companyId");
res.json(await service.list(companyId));
});
router.post("/cloud-upstreams/connect/start", async (req, res) => {
assertBoardOrgAccess(req);
await assertEnabled();
const companyId = stringBody(req.body, "companyId");
const remoteUrl = stringBody(req.body, "remoteUrl");
const redirectUri = stringBody(req.body, "redirectUri");
res.json(await service.startConnect({ companyId, remoteUrl, redirectUri }));
});
router.post("/cloud-upstreams/connect/finish", async (req, res) => {
assertBoardOrgAccess(req);
await assertEnabled();
res.json(await service.finishConnect({
pendingConnectionId: stringBody(req.body, "pendingConnectionId"),
code: stringBody(req.body, "code"),
state: stringBody(req.body, "state"),
}));
});
router.post("/cloud-upstreams/:connectionId/push-runs/preview", async (req, res) => {
assertBoardOrgAccess(req);
await assertEnabled();
res.json(await service.preview(req.params.connectionId, stringBody(req.body, "companyId")));
});
router.post("/cloud-upstreams/:connectionId/push-runs", async (req, res) => {
assertBoardOrgAccess(req);
await assertEnabled();
res.json(await service.createRun({
connectionId: req.params.connectionId,
companyId: stringBody(req.body, "companyId"),
retryOfRunId: optionalString(req.body?.retryOfRunId),
}));
});
router.get("/cloud-upstreams/:connectionId/push-runs/:runId", async (req, res) => {
assertBoardOrgAccess(req);
await assertEnabled();
res.json(await service.readRun(req.params.connectionId, req.params.runId, stringQuery(req.query.companyId, "companyId")));
});
router.post("/cloud-upstreams/:connectionId/push-runs/:runId/cancel", async (req, res) => {
assertBoardOrgAccess(req);
await assertEnabled();
res.json(await service.cancelRun(req.params.connectionId, req.params.runId, stringBody(req.body, "companyId")));
});
router.post("/cloud-upstreams/:connectionId/push-runs/:runId/activation", async (req, res) => {
assertBoardOrgAccess(req);
await assertEnabled();
res.json(await service.activateRunEntities({
connectionId: req.params.connectionId,
runId: req.params.runId,
companyId: stringBody(req.body, "companyId"),
entityType: activationEntityTypeBody(req.body),
}));
});
return router;
}
function stringQuery(value: unknown, label: string): string {
if (typeof value !== "string" || value.trim().length === 0) {
throw badRequest(`${label} is required`);
}
return value;
}
function stringBody(body: unknown, key: string): string {
if (!body || typeof body !== "object" || Array.isArray(body)) {
throw badRequest(`${key} is required`);
}
const value = (body as Record<string, unknown>)[key];
if (typeof value !== "string" || value.trim().length === 0) {
throw badRequest(`${key} is required`);
}
return value;
}
function optionalString(value: unknown): string | null {
return typeof value === "string" && value.length > 0 ? value : null;
}
function activationEntityTypeBody(body: unknown): "agents" | "routines" | "monitors" {
if (!body || typeof body !== "object" || Array.isArray(body)) {
throw badRequest("entityType is required");
}
const value = (body as Record<string, unknown>).entityType;
if (value !== "agents" && value !== "routines" && value !== "monitors") {
throw badRequest("entityType must be agents, routines, or monitors");
}
return value;
}
+1
View File
@@ -19,3 +19,4 @@ export { llmRoutes } from "./llms.js";
export { accessRoutes } from "./access.js";
export { instanceSettingsRoutes } from "./instance-settings.js";
export { instanceDatabaseBackupRoutes } from "./instance-database-backups.js";
export { cloudUpstreamRoutes } from "./cloud-upstreams.js";
File diff suppressed because it is too large Load Diff
+10
View File
@@ -265,7 +265,17 @@ export function companyService(db: Db) {
remove: (id: string) =>
db.transaction(async (tx) => {
// Delete from child tables in dependency order
const companyRunIds = await tx
.select({ id: heartbeatRuns.id })
.from(heartbeatRuns)
.where(eq(heartbeatRuns.companyId, id));
await tx.delete(heartbeatRunEvents).where(eq(heartbeatRunEvents.companyId, id));
if (companyRunIds.length > 0) {
await tx
.delete(heartbeatRunEvents)
.where(inArray(heartbeatRunEvents.runId, companyRunIds.map((run) => run.id)));
}
await tx.delete(agentTaskSessions).where(eq(agentTaskSessions.companyId, id));
await tx.delete(activityLog).where(eq(activityLog.companyId, id));
await tx.delete(heartbeatRuns).where(eq(heartbeatRuns.companyId, id));
+1
View File
@@ -59,6 +59,7 @@ export type {
} from "./authorization.js";
export { boardAuthService } from "./board-auth.js";
export { instanceSettingsService } from "./instance-settings.js";
export { cloudUpstreamService, reconcileCloudUpstreamRunsOnStartup } from "./cloud-upstreams.js";
export { companyPortabilityService } from "./company-portability.js";
export { environmentService } from "./environments.js";
export { executionWorkspaceService } from "./execution-workspaces.js";
+7 -3
View File
@@ -15,9 +15,11 @@ import {
import { eq } from "drizzle-orm";
const DEFAULT_SINGLETON_KEY = "default";
const instanceGeneralSettingsStorageSchema = instanceGeneralSettingsSchema.strip();
const instanceExperimentalSettingsStorageSchema = instanceExperimentalSettingsSchema.strip();
function normalizeGeneralSettings(raw: unknown): InstanceGeneralSettings {
const parsed = instanceGeneralSettingsSchema.safeParse(raw ?? {});
const parsed = instanceGeneralSettingsStorageSchema.safeParse(raw ?? {});
if (parsed.success) {
return {
censorUsernameInLogs: parsed.data.censorUsernameInLogs ?? false,
@@ -35,12 +37,13 @@ function normalizeGeneralSettings(raw: unknown): InstanceGeneralSettings {
};
}
function normalizeExperimentalSettings(raw: unknown): InstanceExperimentalSettings {
const parsed = instanceExperimentalSettingsSchema.safeParse(raw ?? {});
export function normalizeExperimentalSettings(raw: unknown): InstanceExperimentalSettings {
const parsed = instanceExperimentalSettingsStorageSchema.safeParse(raw ?? {});
if (parsed.success) {
return {
enableEnvironments: parsed.data.enableEnvironments ?? false,
enableIsolatedWorkspaces: parsed.data.enableIsolatedWorkspaces ?? false,
enableCloudSync: parsed.data.enableCloudSync ?? false,
autoRestartDevServerWhenIdle: parsed.data.autoRestartDevServerWhenIdle ?? false,
enableIssueGraphLivenessAutoRecovery: parsed.data.enableIssueGraphLivenessAutoRecovery ?? false,
issueGraphLivenessAutoRecoveryLookbackHours:
@@ -51,6 +54,7 @@ function normalizeExperimentalSettings(raw: unknown): InstanceExperimentalSettin
return {
enableEnvironments: false,
enableIsolatedWorkspaces: false,
enableCloudSync: false,
autoRestartDevServerWhenIdle: false,
enableIssueGraphLivenessAutoRecovery: false,
issueGraphLivenessAutoRecoveryLookbackHours:
+4
View File
@@ -32,6 +32,8 @@ import { CompanySettings } from "./pages/CompanySettings";
import { CompanyEnvironments } from "./pages/CompanyEnvironments";
import { CompanySettingsPluginPage } from "./pages/CompanySettingsPluginPage";
import { CompanyAccess, CompanyAccessLegacyRoute } from "./pages/CompanyAccess";
import { CloudUpstream } from "./pages/CloudUpstream";
import { CloudUpstreamUxLab } from "./pages/CloudUpstreamUxLab";
import { CompanyInvites } from "./pages/CompanyInvites";
import { CompanySkills } from "./pages/CompanySkills";
import { Secrets } from "./pages/Secrets";
@@ -72,6 +74,7 @@ function boardRoutes() {
<Route path="company/settings/environments" element={<CompanyEnvironments />} />
<Route path="company/settings/members" element={<CompanyAccess />} />
<Route path="company/settings/access" element={<CompanyAccessLegacyRoute />} />
<Route path="company/settings/cloud-upstream" element={<CloudUpstream />} />
<Route path="company/settings/invites" element={<CompanyInvites />} />
<Route path="company/export/*" element={<CompanyExport />} />
<Route path="company/import" element={<CompanyImport />} />
@@ -279,6 +282,7 @@ export function App() {
<Route path="cli-auth/:id" element={<CliAuthPage />} />
<Route path="invite/:token" element={<InviteLandingPage />} />
<Route path="tests/perf/long-thread" element={<IssueChatLongThreadPerf />} />
<Route path="ux-lab/cloud-upstream" element={<CloudUpstreamUxLab />} />
<Route element={<CloudAccessGate />}>
<Route index element={<CompanyRootRedirect />} />
+40
View File
@@ -0,0 +1,40 @@
import type {
CloudUpstreamActivationEntityType,
CloudUpstreamConnectStartResponse,
CloudUpstreamConnection,
CloudUpstreamPreview,
CloudUpstreamRun,
CloudUpstreamsState,
} from "@paperclipai/shared";
import { api } from "./client";
export const cloudUpstreamsApi = {
list: (companyId: string) =>
api.get<CloudUpstreamsState>(`/cloud-upstreams?companyId=${encodeURIComponent(companyId)}`),
startConnect: (input: { companyId: string; remoteUrl: string; redirectUri: string }) =>
api.post<CloudUpstreamConnectStartResponse>("/cloud-upstreams/connect/start", input),
finishConnect: (input: { pendingConnectionId: string; code: string; state: string }) =>
api.post<CloudUpstreamConnection>("/cloud-upstreams/connect/finish", input),
preview: (connectionId: string, input: { companyId: string }) =>
api.post<CloudUpstreamPreview>(`/cloud-upstreams/${encodeURIComponent(connectionId)}/push-runs/preview`, input),
createRun: (connectionId: string, input: { companyId: string; retryOfRunId?: string | null }) =>
api.post<CloudUpstreamRun>(`/cloud-upstreams/${encodeURIComponent(connectionId)}/push-runs`, input ?? {}),
getRun: (connectionId: string, runId: string, companyId: string) =>
api.get<CloudUpstreamRun>(
`/cloud-upstreams/${encodeURIComponent(connectionId)}/push-runs/${encodeURIComponent(runId)}?companyId=${encodeURIComponent(companyId)}`,
),
cancelRun: (connectionId: string, runId: string, input: { companyId: string }) =>
api.post<CloudUpstreamRun>(
`/cloud-upstreams/${encodeURIComponent(connectionId)}/push-runs/${encodeURIComponent(runId)}/cancel`,
input,
),
activateEntities: (
connectionId: string,
runId: string,
input: { companyId: string; entityType: CloudUpstreamActivationEntityType },
) =>
api.post<CloudUpstreamRun>(
`/cloud-upstreams/${encodeURIComponent(connectionId)}/push-runs/${encodeURIComponent(runId)}/activation`,
input,
),
};
@@ -11,6 +11,9 @@ const mockSidebarBadgesApi = vi.hoisted(() => ({
get: vi.fn(),
}));
const mockUsePluginSlots = vi.hoisted(() => vi.fn());
const mockInstanceSettingsApi = vi.hoisted(() => ({
getExperimental: vi.fn(),
}));
vi.mock("@/lib/router", () => ({
Link: ({
@@ -66,6 +69,10 @@ vi.mock("@/plugins/slots", () => ({
usePluginSlots: mockUsePluginSlots,
}));
vi.mock("@/api/instanceSettings", () => ({
instanceSettingsApi: mockInstanceSettingsApi,
}));
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).IS_REACT_ACT_ENVIRONMENT = true;
@@ -93,6 +100,9 @@ describe("CompanySettingsSidebar", () => {
isLoading: false,
errorMessage: null,
});
mockInstanceSettingsApi.getExperimental.mockResolvedValue({
enableCloudSync: false,
});
});
afterEach(() => {
@@ -121,6 +131,7 @@ describe("CompanySettingsSidebar", () => {
expect(container.textContent).toContain("General");
expect(container.textContent).toContain("Environments");
expect(container.textContent).toContain("Members");
expect(container.textContent).not.toContain("Cloud upstream");
expect(container.textContent).toContain("Invites");
expect(container.textContent).toContain("Secrets");
expect(sidebarNavItemMock).toHaveBeenCalledWith(
@@ -210,4 +221,36 @@ describe("CompanySettingsSidebar", () => {
root.unmount();
});
});
it("shows cloud upstream only when cloud sync is enabled", async () => {
mockInstanceSettingsApi.getExperimental.mockResolvedValue({
enableCloudSync: true,
});
const root = createRoot(container);
const queryClient = new QueryClient({
defaultOptions: { queries: { retry: false } },
});
await act(async () => {
root.render(
<QueryClientProvider client={queryClient}>
<CompanySettingsSidebar />
</QueryClientProvider>,
);
});
await flushReact();
expect(container.textContent).toContain("Cloud upstream");
expect(sidebarNavItemMock).toHaveBeenCalledWith(
expect.objectContaining({
to: "/company/settings/cloud-upstream",
label: "Cloud upstream",
end: true,
}),
);
await act(async () => {
root.unmount();
});
});
});
+15 -1
View File
@@ -1,6 +1,7 @@
import { useQuery } from "@tanstack/react-query";
import { ChevronLeft, KeyRound, MailPlus, MonitorCog, Puzzle, Settings, SlidersHorizontal, Users } from "lucide-react";
import { ChevronLeft, CloudUpload, KeyRound, MailPlus, MonitorCog, Puzzle, Settings, SlidersHorizontal, Users } from "lucide-react";
import { sidebarBadgesApi } from "@/api/sidebarBadges";
import { instanceSettingsApi } from "@/api/instanceSettings";
import { ApiError } from "@/api/client";
import { Link } from "@/lib/router";
import { queryKeys } from "@/lib/queryKeys";
@@ -35,6 +36,11 @@ export function CompanySettingsSidebar() {
retry: false,
refetchInterval: 15_000,
});
const { data: experimentalSettings } = useQuery({
queryKey: queryKeys.instance.experimentalSettings,
queryFn: () => instanceSettingsApi.getExperimental(),
});
const showCloudUpstream = experimentalSettings?.enableCloudSync === true;
return (
<aside className="w-full h-full min-h-0 border-r border-border bg-background flex flex-col">
@@ -66,6 +72,14 @@ export function CompanySettingsSidebar() {
icon={MonitorCog}
end
/>
{showCloudUpstream ? (
<SidebarNavItem
to="/company/settings/cloud-upstream"
label="Cloud upstream"
icon={CloudUpload}
end
/>
) : null}
<SidebarNavItem
to="/company/settings/members"
label="Members"
+1
View File
@@ -133,6 +133,7 @@ export const queryKeys = {
schedulerHeartbeats: ["instance", "scheduler-heartbeats"] as const,
experimentalSettings: ["instance", "experimental-settings"] as const,
},
cloudUpstreams: (companyId: string) => ["cloud-upstreams", companyId] as const,
health: ["health"] as const,
secrets: {
list: (companyId: string) => ["secrets", companyId] as const,
+408
View File
@@ -0,0 +1,408 @@
// @vitest-environment jsdom
import { act } from "react";
import { createRoot } from "react-dom/client";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import type { CloudUpstreamRun, CloudUpstreamsState } from "@paperclipai/shared";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { CloudUpstream, buildActivationRows } from "./CloudUpstream";
const mockCloudUpstreamsApi = vi.hoisted(() => ({
list: vi.fn(),
startConnect: vi.fn(),
finishConnect: vi.fn(),
preview: vi.fn(),
createRun: vi.fn(),
getRun: vi.fn(),
cancelRun: vi.fn(),
activateEntities: vi.fn(),
}));
const mockInstanceSettingsApi = vi.hoisted(() => ({
getExperimental: vi.fn(),
}));
const mockSetBreadcrumbs = vi.hoisted(() => vi.fn());
const mockCompanyState = vi.hoisted(() => ({
selectedCompany: { id: "company-1", name: "Paperclip", issuePrefix: "PAP" } as
| { id: string; name: string; issuePrefix: string | null }
| null,
selectedCompanyId: "company-1" as string | null,
}));
const mockLocationState = vi.hoisted(() => ({
pathname: "/PAP/company/settings/cloud-upstream",
search: "",
}));
vi.mock("@/api/cloudUpstreams", () => ({
cloudUpstreamsApi: mockCloudUpstreamsApi,
}));
vi.mock("@/api/instanceSettings", () => ({
instanceSettingsApi: mockInstanceSettingsApi,
}));
vi.mock("@/context/BreadcrumbContext", () => ({
useBreadcrumbs: () => ({
setBreadcrumbs: mockSetBreadcrumbs,
}),
}));
vi.mock("@/context/CompanyContext", () => ({
useCompany: () => ({
selectedCompany: mockCompanyState.selectedCompany,
selectedCompanyId: mockCompanyState.selectedCompanyId,
}),
}));
vi.mock("@/lib/router", () => ({
Link: ({ children, to, className }: { children: React.ReactNode; to: string; className?: string }) => (
<a href={to} className={className}>
{children}
</a>
),
useLocation: () => ({ pathname: mockLocationState.pathname, search: mockLocationState.search }),
}));
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).IS_REACT_ACT_ENVIRONMENT = true;
async function flushReact() {
await act(async () => {
await Promise.resolve();
await new Promise((resolve) => window.setTimeout(resolve, 0));
});
}
describe("CloudUpstream", () => {
let container: HTMLDivElement;
beforeEach(() => {
container = document.createElement("div");
document.body.appendChild(container);
mockCompanyState.selectedCompany = { id: "company-1", name: "Paperclip", issuePrefix: "PAP" };
mockCompanyState.selectedCompanyId = "company-1";
mockLocationState.pathname = "/PAP/company/settings/cloud-upstream";
mockLocationState.search = "";
mockInstanceSettingsApi.getExperimental.mockResolvedValue({ enableCloudSync: true });
mockCloudUpstreamsApi.list.mockResolvedValue(stateWithRun(buildRun({ status: "succeeded" })));
mockCloudUpstreamsApi.activateEntities.mockImplementation((_connectionId, _runId, input) =>
Promise.resolve(buildRun({
status: "succeeded",
report: {
activationChecklist: {
[input.entityType]: {
entityType: input.entityType,
count: input.entityType === "agents" ? 2 : 1,
status: "activated",
activatedAt: "2026-05-18T19:00:00.000Z",
},
},
},
})),
);
mockCloudUpstreamsApi.createRun.mockResolvedValue(buildRun({ status: "running" }));
});
afterEach(() => {
container.remove();
document.body.innerHTML = "";
vi.clearAllMocks();
});
it("binds the succeeded run activation checklist to imported category counts", async () => {
const root = createRoot(container);
const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } });
await act(async () => {
root.render(
<QueryClientProvider client={queryClient}>
<CloudUpstream />
</QueryClientProvider>,
);
});
await flushReact();
await flushReact();
expect(container.textContent).toContain("Re-run");
expect(container.textContent).not.toContain("Retry");
expect(container.textContent).toContain("Activation checklist");
expect(container.textContent).toContain("2 paused");
expect(container.textContent).toContain("1 paused");
expect(container.textContent).toContain("0 imported monitors in this run.");
expect(container.textContent).toContain("Keep paused");
const activateButton = Array.from(container.querySelectorAll("button"))
.find((button) => button.textContent?.trim() === "Activate") as HTMLButtonElement | undefined;
expect(activateButton).toBeTruthy();
await act(async () => {
activateButton?.dispatchEvent(new MouseEvent("click", { bubbles: true }));
});
await flushReact();
expect(mockCloudUpstreamsApi.activateEntities).toHaveBeenCalledWith(
"connection-1",
"run-1",
{ companyId: "company-1", entityType: "agents" },
);
await act(async () => {
root.unmount();
});
});
it("sends a company-prefixed redirectUri when starting Connect", async () => {
mockCloudUpstreamsApi.list.mockResolvedValue({ connections: [], runs: [] });
mockCloudUpstreamsApi.startConnect.mockResolvedValue({
pendingConnectionId: "pending-1",
authorizationUrl: "https://cloud.example/upstream-consent?state=abc",
});
const root = createRoot(container);
const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } });
await act(async () => {
root.render(
<QueryClientProvider client={queryClient}>
<CloudUpstream />
</QueryClientProvider>,
);
});
await flushReact();
await flushReact();
const input = container.querySelector<HTMLInputElement>("input[aria-label='Paperclip Cloud stack URL']");
expect(input).toBeTruthy();
await act(async () => {
const setter = Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "value")!.set!;
setter.call(input!, "https://cloud.example/PAP/dashboard");
input!.dispatchEvent(new Event("input", { bubbles: true }));
});
await flushReact();
const connectButton = Array.from(container.querySelectorAll("button"))
.find((button) => button.textContent?.trim() === "Connect") as HTMLButtonElement | undefined;
expect(connectButton).toBeTruthy();
await act(async () => {
connectButton?.dispatchEvent(new MouseEvent("click", { bubbles: true }));
});
await flushReact();
expect(mockCloudUpstreamsApi.startConnect).toHaveBeenCalledWith({
companyId: "company-1",
remoteUrl: "https://cloud.example/PAP/dashboard",
redirectUri: `${window.location.origin}/PAP/company/settings/cloud-upstream`,
});
await act(async () => {
root.unmount();
});
});
it("uses the URL pathname prefix when cleaning up the callback URL with no company context", async () => {
mockCompanyState.selectedCompany = null;
mockCompanyState.selectedCompanyId = null;
mockLocationState.pathname = "/PAP/company/settings/cloud-upstream";
mockLocationState.search = "?code=cb-code&state=cb-state";
mockCloudUpstreamsApi.list.mockResolvedValue({ connections: [], runs: [] });
mockCloudUpstreamsApi.finishConnect.mockResolvedValue({
id: "connection-1",
companyId: "company-1",
remoteUrl: "https://cloud.example/PAP",
target: {
stackId: "stack-1",
stackSlug: "stack",
stackDisplayName: "Paperclip Cloud",
companyId: "cloud-company-1",
primaryHost: "cloud.example",
origin: "https://cloud.example",
product: "Paperclip Cloud",
schemaMajor: 1,
maxChunkBytes: 1024,
},
tokenStatus: "connected",
scopes: ["upstream_import:write"],
authorizedGlobalUserId: "user-1",
expiresAt: null,
createdAt: "2026-05-18T18:00:00.000Z",
updatedAt: "2026-05-18T18:00:00.000Z",
lastRunId: null,
});
window.localStorage.setItem("paperclip-cloud-upstream-pending-connection", "pending-1");
const replaceStateSpy = vi.spyOn(window.history, "replaceState");
try {
const root = createRoot(container);
const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } });
await act(async () => {
root.render(
<QueryClientProvider client={queryClient}>
<CloudUpstream />
</QueryClientProvider>,
);
});
await flushReact();
await flushReact();
expect(mockCloudUpstreamsApi.finishConnect).toHaveBeenCalledWith({
pendingConnectionId: "pending-1",
code: "cb-code",
state: "cb-state",
});
expect(replaceStateSpy).toHaveBeenCalledWith(null, "", "/PAP/company/settings/cloud-upstream");
await act(async () => {
root.unmount();
});
} finally {
replaceStateSpy.mockRestore();
window.localStorage.removeItem("paperclip-cloud-upstream-pending-connection");
}
});
it("does not retry the OAuth callback finish mutation after an error", async () => {
mockLocationState.pathname = "/PAP/company/settings/cloud-upstream";
mockLocationState.search = "?code=cb-code&state=cb-state";
mockCloudUpstreamsApi.list.mockResolvedValue({ connections: [], runs: [] });
mockCloudUpstreamsApi.finishConnect.mockRejectedValue(new Error("state expired"));
window.localStorage.setItem("paperclip-cloud-upstream-pending-connection", "pending-1");
try {
const root = createRoot(container);
const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } });
await act(async () => {
root.render(
<QueryClientProvider client={queryClient}>
<CloudUpstream />
</QueryClientProvider>,
);
});
await flushReact();
await flushReact();
await flushReact();
expect(mockCloudUpstreamsApi.finishConnect).toHaveBeenCalledTimes(1);
expect(container.textContent).toContain("state expired");
await act(async () => {
root.unmount();
});
} finally {
window.localStorage.removeItem("paperclip-cloud-upstream-pending-connection");
}
});
it("keeps retry only for failed or cancelled runs", async () => {
mockCloudUpstreamsApi.list.mockResolvedValue(stateWithRun(buildRun({ status: "failed" })));
const root = createRoot(container);
const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } });
await act(async () => {
root.render(
<QueryClientProvider client={queryClient}>
<CloudUpstream />
</QueryClientProvider>,
);
});
await flushReact();
await flushReact();
expect(container.textContent).toContain("Retry");
expect(container.textContent).not.toContain("Re-run");
expect(container.textContent).not.toContain("Activation checklist");
await act(async () => {
root.unmount();
});
});
});
describe("buildActivationRows", () => {
it("reads activation decisions from the run report", () => {
const rows = buildActivationRows(buildRun({
status: "succeeded",
report: {
activationChecklist: {
agents: {
entityType: "agents",
count: 2,
status: "activated",
activatedAt: "2026-05-18T19:00:00.000Z",
},
},
},
}));
expect(rows[0]).toMatchObject({ key: "agents", count: 2, status: "activated", statusLabel: "2 activated" });
expect(rows[2]).toMatchObject({ key: "monitors", count: 0, status: "paused", statusLabel: "0 imported" });
});
});
function stateWithRun(run: CloudUpstreamRun): CloudUpstreamsState {
return {
connections: [
{
id: "connection-1",
companyId: "company-1",
remoteUrl: "https://paperclip.example/PAP",
target: {
stackId: "stack-1",
stackSlug: "stack",
stackDisplayName: "Paperclip Cloud",
companyId: "cloud-company-1",
primaryHost: "paperclip.example",
origin: "https://paperclip.example",
product: "Paperclip Cloud",
schemaMajor: 1,
maxChunkBytes: 1024,
},
tokenStatus: "connected",
scopes: ["upstream_import:write"],
authorizedGlobalUserId: "user-1",
expiresAt: null,
createdAt: "2026-05-18T18:00:00.000Z",
updatedAt: "2026-05-18T18:00:00.000Z",
lastRunId: run.id,
},
],
runs: [run],
};
}
function buildRun(input: {
status: CloudUpstreamRun["status"];
report?: Record<string, unknown>;
}): CloudUpstreamRun {
return {
id: "run-1",
connectionId: "connection-1",
companyId: "company-1",
status: input.status,
activeStep: input.status === "succeeded" ? "activate" : "push",
progressPercent: input.status === "running" ? 70 : 100,
dryRun: false,
summary: [
{ key: "agents", label: "Agents", count: 2 },
{ key: "routines", label: "Routines", count: 1 },
{ key: "issues", label: "Issues", count: 7 },
],
warnings: [],
conflicts: [],
events: [
{
id: "event-1",
at: "2026-05-18T18:30:00.000Z",
phase: input.status === "succeeded" ? "activate" : "push",
type: input.status === "failed" ? "failed" : "completed",
message: input.status === "failed" ? "Push failed." : "Activation checklist is ready.",
},
],
targetUrl: "https://paperclip.example",
report: input.report ?? {},
retryOfRunId: null,
createdAt: "2026-05-18T18:00:00.000Z",
updatedAt: "2026-05-18T18:30:00.000Z",
completedAt: input.status === "running" ? null : "2026-05-18T18:30:00.000Z",
};
}
+646
View File
@@ -0,0 +1,646 @@
import { useEffect, useMemo, useState } from "react";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import {
AlertTriangle,
CheckCircle2,
CloudUpload,
ExternalLink,
FileJson,
History,
Loader2,
RefreshCcw,
ShieldAlert,
} from "lucide-react";
import type {
CloudUpstreamActivationDecision,
CloudUpstreamActivationEntityType,
CloudUpstreamPreview,
CloudUpstreamRun,
CloudUpstreamStep,
} from "@paperclipai/shared";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import { cloudUpstreamsApi } from "@/api/cloudUpstreams";
import { instanceSettingsApi } from "@/api/instanceSettings";
import { useBreadcrumbs } from "@/context/BreadcrumbContext";
import { useCompany } from "@/context/CompanyContext";
import { applyCompanyPrefix, extractCompanyPrefixFromPath } from "@/lib/company-routes";
import { Link, useLocation } from "@/lib/router";
import { queryKeys } from "@/lib/queryKeys";
const PENDING_CONNECTION_KEY = "paperclip-cloud-upstream-pending-connection";
const STEPS: Array<{ key: CloudUpstreamStep; label: string }> = [
{ key: "connect", label: "Connect" },
{ key: "scan", label: "Scan" },
{ key: "preview", label: "Preview" },
{ key: "push", label: "Push" },
{ key: "verify", label: "Verify" },
{ key: "activate", label: "Activate" },
];
const ACTIVATION_CATEGORIES: Array<{
key: CloudUpstreamActivationEntityType;
label: string;
singular: string;
detail: string;
}> = [
{
key: "agents",
label: "Agents",
singular: "agent",
detail: "Confirm cloud secrets and adapter credentials before unpausing imported agents.",
},
{
key: "routines",
label: "Routines",
singular: "routine",
detail: "Review schedules and trigger settings before enabling imported routines.",
},
{
key: "monitors",
label: "Monitors",
singular: "monitor",
detail: "Activate after the target stack has been smoke tested.",
},
];
export function CloudUpstream() {
const { selectedCompany, selectedCompanyId } = useCompany();
const { setBreadcrumbs } = useBreadcrumbs();
const queryClient = useQueryClient();
const location = useLocation();
const [remoteUrl, setRemoteUrl] = useState("");
const [preview, setPreview] = useState<CloudUpstreamPreview | null>(null);
const [activeRun, setActiveRun] = useState<CloudUpstreamRun | null>(null);
const [notice, setNotice] = useState<string | null>(null);
const [actionError, setActionError] = useState<string | null>(null);
useEffect(() => {
setBreadcrumbs([
{ label: selectedCompany?.name ?? "Company", href: "/dashboard" },
{ label: "Settings", href: "/company/settings" },
{ label: "Cloud upstream" },
]);
}, [selectedCompany?.name, setBreadcrumbs]);
const experimentalQuery = useQuery({
queryKey: queryKeys.instance.experimentalSettings,
queryFn: () => instanceSettingsApi.getExperimental(),
});
const cloudSyncEnabled = experimentalQuery.data?.enableCloudSync === true;
const upstreamQuery = useQuery({
queryKey: selectedCompanyId ? queryKeys.cloudUpstreams(selectedCompanyId) : ["cloud-upstreams", "__disabled__"],
queryFn: () => cloudUpstreamsApi.list(selectedCompanyId!),
enabled: !!selectedCompanyId && cloudSyncEnabled,
});
const connection = upstreamQuery.data?.connections[0] ?? null;
const latestRun = activeRun ?? upstreamQuery.data?.runs[0] ?? null;
const callbackParams = useMemo(() => new URLSearchParams(location.search), [location.search]);
const code = callbackParams.get("code");
const state = callbackParams.get("state");
const callbackError = callbackParams.get("error");
const settingsPath = useMemo(() => {
const pathPrefix = extractCompanyPrefixFromPath(location.pathname);
return applyCompanyPrefix("/company/settings/cloud-upstream", pathPrefix ?? selectedCompany?.issuePrefix ?? null);
}, [location.pathname, selectedCompany?.issuePrefix]);
const finishMutation = useMutation({
mutationFn: (input: { pendingConnectionId: string; code: string; state: string }) =>
cloudUpstreamsApi.finishConnect(input),
onSuccess: async () => {
localStorage.removeItem(PENDING_CONNECTION_KEY);
setNotice("Cloud upstream connection approved.");
setActionError(null);
await invalidateUpstreams();
window.history.replaceState(null, "", settingsPath);
},
onError: (error) => setActionError(error instanceof Error ? error.message : "Failed to finish connection."),
});
const {
mutate: finishConnect,
isError: finishConnectFailed,
isPending: finishConnectPending,
isSuccess: finishConnectSucceeded,
} = finishMutation;
useEffect(() => {
if (!cloudSyncEnabled || !code || !state || finishConnectPending || finishConnectSucceeded || finishConnectFailed) return;
const pendingConnectionId = localStorage.getItem(PENDING_CONNECTION_KEY);
if (!pendingConnectionId) {
setActionError("No pending cloud upstream connection was found. Start the connection again.");
return;
}
finishConnect({ pendingConnectionId, code, state });
}, [cloudSyncEnabled, code, finishConnect, finishConnectFailed, finishConnectPending, finishConnectSucceeded, state]);
useEffect(() => {
if (callbackError) {
setActionError(`Cloud upstream connection was not approved: ${callbackError}`);
}
}, [callbackError]);
const startMutation = useMutation({
mutationFn: () =>
cloudUpstreamsApi.startConnect({
companyId: selectedCompanyId!,
remoteUrl,
redirectUri: `${window.location.origin}${settingsPath}`,
}),
onSuccess: (result) => {
localStorage.setItem(PENDING_CONNECTION_KEY, result.pendingConnectionId);
setActionError(null);
window.location.assign(result.authorizationUrl);
},
onError: (error) => setActionError(error instanceof Error ? error.message : "Failed to start connection."),
});
const previewMutation = useMutation({
mutationFn: (input: { connectionId: string; companyId: string }) =>
cloudUpstreamsApi.preview(input.connectionId, { companyId: input.companyId }),
onSuccess: (nextPreview) => {
setPreview(nextPreview);
setActionError(null);
},
onError: (error) => setActionError(previewErrorMessage(error)),
});
const runMutation = useMutation({
mutationFn: (input: { connectionId: string; companyId: string; retryOfRunId?: string | null }) =>
cloudUpstreamsApi.createRun(input.connectionId, {
companyId: input.companyId,
retryOfRunId: input.retryOfRunId ?? null,
}),
onSuccess: async (run) => {
setActiveRun(run);
setNotice(run.status === "succeeded"
? "Push run completed. Review activation before unpausing automations."
: "Push run failed. Review the run events and retry after correcting the issue.");
setActionError(null);
await invalidateUpstreams();
},
onError: (error) => setActionError(error instanceof Error ? error.message : "Failed to run push."),
});
const activationMutation = useMutation({
mutationFn: (input: { run: CloudUpstreamRun; entityType: CloudUpstreamActivationEntityType }) =>
cloudUpstreamsApi.activateEntities(input.run.connectionId, input.run.id, {
companyId: input.run.companyId,
entityType: input.entityType,
}),
onSuccess: async (run) => {
setActiveRun(run);
setNotice("Activation checklist updated.");
setActionError(null);
await invalidateUpstreams();
},
onError: (error) => setActionError(error instanceof Error ? error.message : "Failed to activate imported entities."),
});
async function invalidateUpstreams() {
if (!selectedCompanyId) return;
await queryClient.invalidateQueries({ queryKey: queryKeys.cloudUpstreams(selectedCompanyId) });
}
if (!selectedCompanyId || !selectedCompany) {
return <div className="text-sm text-muted-foreground">Select a company to configure cloud upstream.</div>;
}
if (experimentalQuery.isLoading) {
return <div className="text-sm text-muted-foreground">Loading experimental settings...</div>;
}
if (!cloudSyncEnabled) {
return (
<div className="max-w-2xl space-y-4">
<div className="flex items-center gap-2">
<CloudUpload className="h-5 w-5 text-muted-foreground" />
<h1 className="text-lg font-semibold">Cloud upstream</h1>
</div>
<div className="rounded-md border border-border px-4 py-4 text-sm text-muted-foreground">
Cloud sync is disabled. Enable it in{" "}
<Link className="text-primary underline-offset-2 hover:underline" to="/instance/settings/experimental">
Instance Settings
</Link>{" "}
to show upstream connection and push tools.
</div>
</div>
);
}
return (
<div className="max-w-6xl space-y-6">
<div className="flex flex-wrap items-start justify-between gap-3">
<div className="space-y-1">
<div className="flex items-center gap-2">
<CloudUpload className="h-5 w-5 text-muted-foreground" />
<h1 className="text-lg font-semibold">Cloud upstream</h1>
</div>
<p className="max-w-2xl text-sm text-muted-foreground">
Push {selectedCompany.name} into a Paperclip Cloud stack. Automations stay paused until activation.
</p>
</div>
{connection?.target.origin ? (
<Button variant="outline" size="sm" asChild>
<a href={connection.target.origin} target="_blank" rel="noreferrer">
<ExternalLink className="h-4 w-4" />
Open cloud
</a>
</Button>
) : null}
</div>
{notice ? (
<div className="rounded-md border border-emerald-500/30 bg-emerald-500/5 px-3 py-2 text-sm text-emerald-700 dark:text-emerald-300">
{notice}
</div>
) : null}
{actionError ? (
<div className="rounded-md border border-destructive/40 bg-destructive/5 px-3 py-2 text-sm text-destructive">
{actionError}
</div>
) : null}
<Stepper activeStep={latestRun?.activeStep ?? (preview ? "preview" : connection?.tokenStatus === "connected" ? "scan" : "connect")} />
<section className="space-y-3">
<div className="text-xs font-medium uppercase tracking-wide text-muted-foreground">Connection</div>
<div className="rounded-md border border-border px-4 py-4">
{connection ? (
<div className="grid gap-3 lg:grid-cols-[1fr_auto] lg:items-start">
<div>
<div className="text-sm font-medium">
{connection.target.stackDisplayName ?? connection.target.stackSlug ?? connection.target.stackId}
</div>
<div className="mt-1 text-xs text-muted-foreground">
{connection.target.product} · {connection.target.origin} · token {connection.tokenStatus}
</div>
<div className="mt-2 text-xs text-muted-foreground">
Schema {connection.target.schemaMajor}. Max chunk {formatBytes(connection.target.maxChunkBytes)}.
</div>
</div>
<div className="flex flex-col items-end gap-1">
<Button
variant="outline"
size="sm"
onClick={() => previewMutation.mutate({ connectionId: connection.id, companyId: connection.companyId })}
disabled={previewMutation.isPending || connection.tokenStatus !== "connected"}
>
{previewMutation.isPending ? <Loader2 className="h-4 w-4 animate-spin" /> : <RefreshCcw className="h-4 w-4" />}
Preview push
</Button>
{previewMutation.isPending ? <PreviewProgressHint /> : null}
</div>
</div>
) : (
<div className="grid gap-3 md:grid-cols-[1fr_auto]">
<Input
value={remoteUrl}
onChange={(event) => setRemoteUrl(event.target.value)}
placeholder="https://paperclip.paperclip.app/PC521D/dashboard"
aria-label="Paperclip Cloud stack URL"
/>
<Button onClick={() => startMutation.mutate()} disabled={startMutation.isPending || !remoteUrl.trim()}>
{startMutation.isPending ? <Loader2 className="h-4 w-4 animate-spin" /> : <CloudUpload className="h-4 w-4" />}
Connect
</Button>
</div>
)}
</div>
</section>
{preview ? (
<section className="space-y-3">
<div className="flex items-center justify-between gap-3">
<div className="text-xs font-medium uppercase tracking-wide text-muted-foreground">Preview</div>
<Button
onClick={() => runMutation.mutate({ connectionId: preview.connectionId, companyId: preview.sourceCompanyId })}
disabled={runMutation.isPending || !preview.schemaCompatible}
>
{runMutation.isPending ? <Loader2 className="h-4 w-4 animate-spin" /> : <CloudUpload className="h-4 w-4" />}
Push to cloud
</Button>
</div>
<SummaryGrid summary={preview.summary} />
<WarningsPanel warnings={preview.warnings} />
<ConflictTable conflicts={preview.conflicts} />
</section>
) : null}
{latestRun ? (
<section className="space-y-3">
<div className="flex flex-wrap items-center justify-between gap-2">
<div className="text-xs font-medium uppercase tracking-wide text-muted-foreground">Progress and finish</div>
<div className="flex flex-wrap gap-2">
<Button variant="outline" size="sm" onClick={() => downloadRunReport(latestRun)}>
<FileJson className="h-4 w-4" />
Download report
</Button>
{latestRun.status === "failed" || latestRun.status === "cancelled" ? (
<Button
variant="outline"
size="sm"
onClick={() => runMutation.mutate({
connectionId: latestRun.connectionId,
companyId: latestRun.companyId,
retryOfRunId: latestRun.id,
})}
disabled={runMutation.isPending}
>
<RefreshCcw className="h-4 w-4" />
Retry
</Button>
) : latestRun.status === "succeeded" ? (
<Button
variant="outline"
size="sm"
onClick={() => runMutation.mutate({ connectionId: latestRun.connectionId, companyId: latestRun.companyId })}
disabled={runMutation.isPending}
>
<RefreshCcw className="h-4 w-4" />
Re-run
</Button>
) : null}
</div>
</div>
<div className="rounded-md border border-border px-4 py-4">
<div className="flex items-center justify-between gap-3">
<div>
<div className="text-sm font-medium capitalize">{latestRun.status}</div>
<div className="mt-1 text-xs text-muted-foreground">
Run {latestRun.id.slice(0, 8)} · {latestRun.completedAt ? `completed ${formatDate(latestRun.completedAt)}` : "in progress"}
</div>
</div>
<div className="text-sm tabular-nums">{latestRun.progressPercent}%</div>
</div>
<div className="mt-3 h-2 rounded-full bg-muted">
<div className="h-2 rounded-full bg-primary" style={{ width: `${latestRun.progressPercent}%` }} />
</div>
<div className="mt-4 divide-y divide-border">
{latestRun.events.map((event) => (
<div key={event.id} className="grid gap-2 py-2 text-sm sm:grid-cols-[7rem_8rem_1fr]">
<span className="text-xs text-muted-foreground">{formatDate(event.at)}</span>
<span className="text-xs capitalize text-muted-foreground">{event.phase}</span>
<span>{event.message}</span>
</div>
))}
</div>
</div>
{latestRun.status === "succeeded" ? (
<ActivationChecklist
run={latestRun}
pendingEntityType={activationMutation.variables?.entityType ?? null}
isPending={activationMutation.isPending}
onActivate={(entityType) => activationMutation.mutate({ run: latestRun, entityType })}
/>
) : null}
</section>
) : null}
{upstreamQuery.data?.runs.length ? (
<section className="space-y-3">
<div className="flex items-center gap-2 text-xs font-medium uppercase tracking-wide text-muted-foreground">
<History className="h-3.5 w-3.5" />
History
</div>
<div className="divide-y divide-border rounded-md border border-border">
{upstreamQuery.data.runs.map((run) => (
<button
key={run.id}
type="button"
className="grid w-full gap-1 px-4 py-3 text-left text-sm hover:bg-accent/40 sm:grid-cols-[1fr_auto]"
onClick={() => setActiveRun(run)}
>
<span>Run {run.id.slice(0, 8)} · {run.status}</span>
<span className="text-xs text-muted-foreground">{formatDate(run.createdAt)}</span>
</button>
))}
</div>
</section>
) : null}
</div>
);
}
function PreviewProgressHint() {
const [elapsed, setElapsed] = useState(0);
useEffect(() => {
const startedAt = Date.now();
const interval = window.setInterval(() => setElapsed(Math.round((Date.now() - startedAt) / 1000)), 1000);
return () => window.clearInterval(interval);
}, []);
const message = elapsed < 15
? "Building manifest..."
: elapsed < 45
? `Building manifest... ${elapsed}s. Large companies can take up to a minute.`
: `Still building manifest... ${elapsed}s. PAP-scale companies routinely take ~60s.`;
return <div className="text-xs text-muted-foreground">{message}</div>;
}
function Stepper({ activeStep }: { activeStep: CloudUpstreamStep }) {
const activeIndex = STEPS.findIndex((step) => step.key === activeStep);
return (
<div className="grid gap-2 rounded-md border border-border px-3 py-3 sm:grid-cols-6">
{STEPS.map((step, index) => {
const complete = index < activeIndex;
const active = index === activeIndex;
return (
<div key={step.key} className="flex items-center gap-2 text-xs">
{complete ? (
<CheckCircle2 className="h-4 w-4 text-emerald-600" />
) : (
<span className={active ? "h-4 w-4 rounded-full border-2 border-primary" : "h-4 w-4 rounded-full border border-border"} />
)}
<span className={active ? "font-medium text-foreground" : "text-muted-foreground"}>{step.label}</span>
</div>
);
})}
</div>
);
}
function SummaryGrid({ summary }: { summary: CloudUpstreamPreview["summary"] }) {
return (
<div className="grid gap-2 sm:grid-cols-4">
{summary.map((item) => (
<div key={item.key} className="rounded-md border border-border px-3 py-2">
<div className="text-lg font-semibold tabular-nums">{item.count}</div>
<div className="text-xs text-muted-foreground">{item.label}</div>
</div>
))}
</div>
);
}
function WarningsPanel({ warnings }: { warnings: CloudUpstreamPreview["warnings"] }) {
return (
<div className="rounded-md border border-border px-4 py-3">
<div className="mb-2 flex items-center gap-2 text-sm font-medium">
<ShieldAlert className="h-4 w-4 text-muted-foreground" />
Warnings
</div>
<div className="divide-y divide-border">
{warnings.map((warning) => (
<div key={warning.code} className="grid gap-2 py-2 sm:grid-cols-[1.25rem_12rem_1fr]">
<AlertTriangle className={warning.severity === "blocker" ? "h-4 w-4 text-destructive" : "h-4 w-4 text-amber-600"} />
<div className="text-sm font-medium">{warning.title}</div>
<div className="text-sm text-muted-foreground">{warning.detail}</div>
</div>
))}
</div>
</div>
);
}
function ConflictTable({ conflicts }: { conflicts: CloudUpstreamPreview["conflicts"] }) {
return (
<div className="rounded-md border border-border px-4 py-3">
<div className="mb-2 text-sm font-medium">Conflicts</div>
{conflicts.length === 0 ? (
<div className="text-sm text-muted-foreground">No target conflicts detected for this preview.</div>
) : (
<div className="divide-y divide-border">
{conflicts.map((conflict) => (
<div key={conflict.id} className="grid gap-2 py-2 text-sm sm:grid-cols-[8rem_1fr_1fr_8rem]">
<span className="text-muted-foreground">{conflict.entityType}</span>
<span>{conflict.sourceLabel}</span>
<span>{conflict.targetLabel}</span>
<span className="capitalize">{conflict.plannedAction}</span>
</div>
))}
</div>
)}
</div>
);
}
function ActivationChecklist({
run,
pendingEntityType,
isPending,
onActivate,
}: {
run: CloudUpstreamRun;
pendingEntityType: CloudUpstreamActivationEntityType | null;
isPending: boolean;
onActivate: (entityType: CloudUpstreamActivationEntityType) => void;
}) {
const rows = buildActivationRows(run);
return (
<div className="rounded-md border border-border px-4 py-3">
<div className="mb-2 text-sm font-medium">Activation checklist</div>
<div className="divide-y divide-border">
{rows.map((row) => {
const pending = isPending && pendingEntityType === row.key;
const activated = row.status === "activated";
return (
<div key={row.key} className="grid gap-2 py-2 text-sm sm:grid-cols-[8rem_1fr_auto] sm:items-center">
<div>
<div className="font-medium">{row.label}</div>
<div className="text-xs text-muted-foreground">{row.statusLabel}</div>
</div>
<div className="text-muted-foreground">
{row.count === 0 ? `0 imported ${row.pluralLabel} in this run.` : row.detail}
</div>
<div className="flex flex-wrap gap-2 sm:justify-end">
<Button
variant={activated ? "secondary" : "default"}
size="sm"
onClick={() => onActivate(row.key)}
disabled={row.count === 0 || activated || isPending}
>
{pending ? <Loader2 className="h-4 w-4 animate-spin" /> : null}
{activated ? "Activated" : "Activate"}
</Button>
<Button variant="ghost" size="sm" disabled={activated || isPending}>
Keep paused
</Button>
</div>
</div>
);
})}
</div>
</div>
);
}
export function buildActivationRows(run: CloudUpstreamRun) {
const activationChecklist = activationChecklistFromReport(run.report);
return ACTIVATION_CATEGORIES.map((category) => {
const decision = activationChecklist[category.key];
const count = summaryCount(run.summary, category.key);
const status = decision?.status === "activated" ? "activated" : "paused";
const pluralLabel = `${category.singular}${count === 1 ? "" : "s"}`;
return {
...category,
count,
pluralLabel,
status,
detail: `${count} imported ${pluralLabel} are paused by default. ${category.detail}`,
statusLabel: status === "activated"
? `${count} activated`
: count === 0
? "0 imported"
: `${count} paused`,
};
});
}
function summaryCount(summary: CloudUpstreamRun["summary"], key: CloudUpstreamActivationEntityType): number {
return summary.find((item) => item.key === key)?.count ?? 0;
}
function activationChecklistFromReport(report: CloudUpstreamRun["report"]): Partial<Record<CloudUpstreamActivationEntityType, CloudUpstreamActivationDecision>> {
const value = optionalRecord(report.activationChecklist);
const decisions: Partial<Record<CloudUpstreamActivationEntityType, CloudUpstreamActivationDecision>> = {};
for (const key of ["agents", "routines", "monitors"] as const) {
const item = optionalRecord(value[key]);
if (!item) continue;
decisions[key] = {
entityType: key,
count: typeof item.count === "number" ? item.count : 0,
status: item.status === "activated" ? "activated" : "paused",
activatedAt: typeof item.activatedAt === "string" ? item.activatedAt : null,
};
}
return decisions;
}
function optionalRecord(value: unknown): Record<string, unknown> {
return value && typeof value === "object" && !Array.isArray(value) ? value as Record<string, unknown> : {};
}
function downloadRunReport(run: CloudUpstreamRun) {
const blob = new Blob([JSON.stringify(run.report, null, 2)], { type: "application/json" });
const url = URL.createObjectURL(blob);
const anchor = document.createElement("a");
anchor.href = url;
anchor.download = `cloud-upstream-run-${run.id}.json`;
anchor.click();
URL.revokeObjectURL(url);
}
function formatDate(value: string) {
return new Date(value).toLocaleString(undefined, {
month: "short",
day: "numeric",
hour: "2-digit",
minute: "2-digit",
});
}
function formatBytes(value: number) {
if (value >= 1024 * 1024) return `${Math.round(value / (1024 * 1024))} MiB`;
if (value >= 1024) return `${Math.round(value / 1024)} KiB`;
return `${value} B`;
}
function previewErrorMessage(error: unknown): string {
const code = error instanceof Error ? error.message : null;
if (code === "payload_too_large" || code === "bad_request") {
return "Local company is too large to preview as a single request. Click Push to continue (the Push step uploads in chunks), or see the docs for chunked-preview options.";
}
return code ?? "Failed to preview push.";
}
+822
View File
@@ -0,0 +1,822 @@
import { useMemo } from "react";
import {
AlertTriangle,
CheckCircle2,
CloudUpload,
ExternalLink,
FileJson,
History,
Loader2,
RefreshCcw,
ShieldAlert,
} from "lucide-react";
import type {
CloudUpstreamActivationDecision,
CloudUpstreamActivationEntityType,
CloudUpstreamConflict,
CloudUpstreamConnection,
CloudUpstreamPreview,
CloudUpstreamRun,
CloudUpstreamStep,
CloudUpstreamSummaryCount,
CloudUpstreamWarning,
} from "@paperclipai/shared";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import { useLocation } from "@/lib/router";
type FixtureStateKey =
| "settings-pane"
| "connect-wizard"
| "schema-mismatch"
| "preview"
| "preview-clean"
| "progress"
| "retry"
| "finish";
const STEPS: Array<{ key: CloudUpstreamStep; label: string }> = [
{ key: "connect", label: "Connect" },
{ key: "scan", label: "Scan" },
{ key: "preview", label: "Preview" },
{ key: "push", label: "Push" },
{ key: "verify", label: "Verify" },
{ key: "activate", label: "Activate" },
];
const ACTIVATION_CATEGORIES: Array<{
key: CloudUpstreamActivationEntityType;
label: string;
singular: string;
detail: string;
}> = [
{
key: "agents",
label: "Agents",
singular: "agent",
detail: "Keep paused until cloud secrets and adapter credentials are verified.",
},
{
key: "routines",
label: "Routines",
singular: "routine",
detail: "Review schedules before enabling triggers.",
},
{
key: "monitors",
label: "Monitors",
singular: "monitor",
detail: "Activate after the target instance has been smoke tested.",
},
];
const FIXTURE_LABELS: Record<FixtureStateKey, string> = {
"settings-pane": "1 · Settings → Cloud upstream pane (enabled)",
"connect-wizard": "2 · Connect wizard — remote URL entry + PKCE launch",
"schema-mismatch": "3 · Connect wizard — schema-mismatch hard block",
preview: "4 · Preview — conflicts, warnings, planned actions",
"preview-clean": "5 · Preview — clean run with no conflicts",
progress: "6 · Durable progress — mid-run from run events",
retry: "7 · Retry without duplicating ledger entries",
finish: "8 · Finish / activation checklist with run report",
};
const PARSE_ORDER: FixtureStateKey[] = [
"settings-pane",
"connect-wizard",
"schema-mismatch",
"preview",
"preview-clean",
"progress",
"retry",
"finish",
];
export function CloudUpstreamUxLab() {
const location = useLocation();
const { state, showChrome } = useMemo(() => {
const params = new URLSearchParams(location.search);
const raw = (params.get("state") ?? "settings-pane") as FixtureStateKey;
return {
state: PARSE_ORDER.includes(raw) ? raw : "settings-pane",
showChrome: params.get("chrome") === "on",
};
}, [location.search]);
const fixture = useMemo(() => buildFixture(state), [state]);
return (
<div className="mx-auto max-w-6xl space-y-6 p-6">
{showChrome ? <FixtureNav active={state} /> : null}
<CloudUpstreamRender fixture={fixture} />
</div>
);
}
function FixtureNav({ active }: { active: FixtureStateKey }) {
return (
<div className="rounded-md border border-dashed border-border/70 bg-muted/30 px-3 py-2 text-xs text-muted-foreground">
<div className="mb-1 font-semibold uppercase tracking-wide">UX lab · cloud upstream</div>
<div className="flex flex-wrap gap-x-3 gap-y-1">
{PARSE_ORDER.map((key) => (
<a
key={key}
href={`?state=${key}`}
className={
active === key
? "rounded bg-primary/10 px-2 py-0.5 font-medium text-primary"
: "rounded px-2 py-0.5 hover:bg-accent/40"
}
>
{FIXTURE_LABELS[key]}
</a>
))}
</div>
</div>
);
}
interface Fixture {
selectedCompanyName: string;
connection: CloudUpstreamConnection | null;
preview: CloudUpstreamPreview | null;
latestRun: CloudUpstreamRun | null;
history: CloudUpstreamRun[];
notice: string | null;
actionError: string | null;
}
function CloudUpstreamRender({ fixture }: { fixture: Fixture }) {
const { connection, preview, latestRun, history, notice, actionError, selectedCompanyName } = fixture;
const activeStep: CloudUpstreamStep = latestRun?.activeStep
?? (preview ? "preview" : connection?.tokenStatus === "connected" ? "scan" : "connect");
return (
<div className="space-y-6">
<div className="flex flex-wrap items-start justify-between gap-3">
<div className="space-y-1">
<div className="flex items-center gap-2">
<CloudUpload className="h-5 w-5 text-muted-foreground" />
<h1 className="text-lg font-semibold">Cloud upstream</h1>
</div>
<p className="max-w-2xl text-sm text-muted-foreground">
Push {selectedCompanyName} into a Paperclip Cloud stack. Automations stay paused until activation.
</p>
</div>
{connection?.target.origin ? (
<Button variant="outline" size="sm" asChild>
<a href={connection.target.origin} target="_blank" rel="noreferrer">
<ExternalLink className="h-4 w-4" />
Open cloud
</a>
</Button>
) : null}
</div>
{notice ? (
<div className="rounded-md border border-emerald-500/30 bg-emerald-500/5 px-3 py-2 text-sm text-emerald-700 dark:text-emerald-300">
{notice}
</div>
) : null}
{actionError ? (
<div className="rounded-md border border-destructive/40 bg-destructive/5 px-3 py-2 text-sm text-destructive">
{actionError}
</div>
) : null}
<Stepper activeStep={activeStep} />
<section className="space-y-3">
<div className="text-xs font-medium uppercase tracking-wide text-muted-foreground">Connection</div>
<div className="rounded-md border border-border px-4 py-4">
{connection ? (
<div className="grid gap-3 lg:grid-cols-[1fr_auto] lg:items-start">
<div>
<div className="text-sm font-medium">
{connection.target.stackDisplayName ?? connection.target.stackSlug ?? connection.target.stackId}
</div>
<div className="mt-1 text-xs text-muted-foreground">
{connection.target.product} · {connection.target.origin} · token {connection.tokenStatus}
</div>
<div className="mt-2 text-xs text-muted-foreground">
Schema {connection.target.schemaMajor}. Max chunk {formatBytes(connection.target.maxChunkBytes)}.
</div>
</div>
<Button variant="outline" size="sm">
<RefreshCcw className="h-4 w-4" />
Preview push
</Button>
</div>
) : (
<div className="grid gap-3 md:grid-cols-[1fr_auto]">
<Input
defaultValue="https://paperclip.paperclip.app/PC521D/dashboard"
placeholder="https://paperclip.paperclip.app/PC521D/dashboard"
aria-label="Paperclip Cloud stack URL"
autoFocus
/>
<Button disabled>
<Loader2 className="h-4 w-4 animate-spin" />
Discovering
</Button>
</div>
)}
</div>
</section>
{preview ? (
<section className="space-y-3">
<div className="flex items-center justify-between gap-3">
<div className="text-xs font-medium uppercase tracking-wide text-muted-foreground">Preview</div>
<Button disabled={!preview.schemaCompatible}>
<CloudUpload className="h-4 w-4" />
Push to cloud
</Button>
</div>
<SummaryGrid summary={preview.summary} />
<WarningsPanel warnings={preview.warnings} />
<ConflictTable conflicts={preview.conflicts} />
</section>
) : null}
{latestRun ? (
<section className="space-y-3">
<div className="flex flex-wrap items-center justify-between gap-2">
<div className="text-xs font-medium uppercase tracking-wide text-muted-foreground">Progress and finish</div>
<div className="flex flex-wrap gap-2">
<Button variant="outline" size="sm">
<FileJson className="h-4 w-4" />
Download report
</Button>
{latestRun.status === "failed" || latestRun.status === "cancelled" ? (
<Button variant="outline" size="sm">
<RefreshCcw className="h-4 w-4" />
Retry
</Button>
) : latestRun.status === "succeeded" ? (
<Button variant="outline" size="sm">
<RefreshCcw className="h-4 w-4" />
Re-run
</Button>
) : null}
</div>
</div>
<div className="rounded-md border border-border px-4 py-4">
<div className="flex items-center justify-between gap-3">
<div>
<div className="text-sm font-medium capitalize">{latestRun.status}</div>
<div className="mt-1 text-xs text-muted-foreground">
Run {latestRun.id.slice(0, 8)} · {latestRun.completedAt
? `completed ${formatDate(latestRun.completedAt)}`
: latestRun.status === "running"
? "in progress"
: "in progress"}
</div>
</div>
<div className="text-sm tabular-nums">{latestRun.progressPercent}%</div>
</div>
<div className="mt-3 h-2 rounded-full bg-muted">
<div className="h-2 rounded-full bg-primary" style={{ width: `${latestRun.progressPercent}%` }} />
</div>
<div className="mt-4 divide-y divide-border">
{latestRun.events.map((event) => (
<div key={event.id} className="grid gap-2 py-2 text-sm sm:grid-cols-[7rem_8rem_1fr]">
<span className="text-xs text-muted-foreground">{formatDate(event.at)}</span>
<span className="text-xs capitalize text-muted-foreground">{event.phase}</span>
<span>{event.message}</span>
</div>
))}
</div>
</div>
{latestRun.status === "succeeded" ? <ActivationChecklist run={latestRun} /> : null}
</section>
) : null}
{history.length ? (
<section className="space-y-3">
<div className="flex items-center gap-2 text-xs font-medium uppercase tracking-wide text-muted-foreground">
<History className="h-3.5 w-3.5" />
History
</div>
<div className="divide-y divide-border rounded-md border border-border">
{history.map((run) => (
<div
key={run.id}
className="grid w-full gap-1 px-4 py-3 text-left text-sm hover:bg-accent/40 sm:grid-cols-[1fr_auto]"
>
<span>Run {run.id.slice(0, 8)} · {run.status}</span>
<span className="text-xs text-muted-foreground">{formatDate(run.createdAt)}</span>
</div>
))}
</div>
</section>
) : null}
</div>
);
}
function Stepper({ activeStep }: { activeStep: CloudUpstreamStep }) {
const activeIndex = STEPS.findIndex((step) => step.key === activeStep);
return (
<div className="grid gap-2 rounded-md border border-border px-3 py-3 sm:grid-cols-6">
{STEPS.map((step, index) => {
const complete = index < activeIndex;
const active = index === activeIndex;
return (
<div key={step.key} className="flex items-center gap-2 text-xs">
{complete ? (
<CheckCircle2 className="h-4 w-4 text-emerald-600" />
) : (
<span className={active ? "h-4 w-4 rounded-full border-2 border-primary" : "h-4 w-4 rounded-full border border-border"} />
)}
<span className={active ? "font-medium text-foreground" : "text-muted-foreground"}>{step.label}</span>
</div>
);
})}
</div>
);
}
function SummaryGrid({ summary }: { summary: CloudUpstreamSummaryCount[] }) {
return (
<div className="grid gap-2 sm:grid-cols-4">
{summary.map((item) => (
<div key={item.key} className="rounded-md border border-border px-3 py-2">
<div className="text-lg font-semibold tabular-nums">{item.count}</div>
<div className="text-xs text-muted-foreground">{item.label}</div>
</div>
))}
</div>
);
}
function WarningsPanel({ warnings }: { warnings: CloudUpstreamWarning[] }) {
return (
<div className="rounded-md border border-border px-4 py-3">
<div className="mb-2 flex items-center gap-2 text-sm font-medium">
<ShieldAlert className="h-4 w-4 text-muted-foreground" />
Warnings
</div>
<div className="divide-y divide-border">
{warnings.map((warning) => (
<div key={warning.code} className="grid gap-2 py-2 sm:grid-cols-[1.25rem_12rem_1fr]">
<AlertTriangle className={warning.severity === "blocker" ? "h-4 w-4 text-destructive" : "h-4 w-4 text-amber-600"} />
<div className="text-sm font-medium">{warning.title}</div>
<div className="text-sm text-muted-foreground">{warning.detail}</div>
</div>
))}
</div>
</div>
);
}
function ConflictTable({ conflicts }: { conflicts: CloudUpstreamConflict[] }) {
return (
<div className="rounded-md border border-border px-4 py-3">
<div className="mb-2 text-sm font-medium">Conflicts</div>
{conflicts.length === 0 ? (
<div className="text-sm text-muted-foreground">No target conflicts detected for this preview.</div>
) : (
<div className="divide-y divide-border">
{conflicts.map((conflict) => (
<div key={conflict.id} className="grid gap-2 py-2 text-sm sm:grid-cols-[8rem_1fr_1fr_8rem]">
<span className="text-muted-foreground">{conflict.entityType}</span>
<span>{conflict.sourceLabel}</span>
<span>{conflict.targetLabel}</span>
<span className="capitalize">{conflict.plannedAction}</span>
</div>
))}
</div>
)}
</div>
);
}
function ActivationChecklist({ run }: { run: CloudUpstreamRun }) {
const rows = buildActivationRows(run);
return (
<div className="rounded-md border border-border px-4 py-3">
<div className="mb-2 text-sm font-medium">Activation checklist</div>
<div className="divide-y divide-border">
{rows.map((row) => {
const activated = row.status === "activated";
return (
<div key={row.key} className="grid gap-2 py-2 text-sm sm:grid-cols-[8rem_1fr_auto] sm:items-center">
<div>
<div className="font-medium">{row.label}</div>
<div className="text-xs text-muted-foreground">{row.statusLabel}</div>
</div>
<div className="text-muted-foreground">
{row.count === 0 ? `0 imported ${row.pluralLabel} in this run.` : row.detail}
</div>
<div className="flex flex-wrap gap-2 sm:justify-end">
<Button variant={activated ? "secondary" : "default"} size="sm" disabled={row.count === 0 || activated}>
{activated ? "Activated" : "Activate"}
</Button>
<Button variant="ghost" size="sm" disabled={activated}>
Keep paused
</Button>
</div>
</div>
);
})}
</div>
</div>
);
}
function buildActivationRows(run: CloudUpstreamRun) {
const decisions = decisionsFromReport(run.report);
return ACTIVATION_CATEGORIES.map((category) => {
const decision = decisions[category.key];
const count = summaryCount(run.summary, category.key);
const status = decision?.status === "activated" ? "activated" : "paused";
const pluralLabel = `${category.singular}${count === 1 ? "" : "s"}`;
return {
...category,
count,
pluralLabel,
status,
detail: `${count} imported ${pluralLabel} are paused by default. ${category.detail}`,
statusLabel: status === "activated"
? `${count} activated`
: count === 0
? "0 imported"
: `${count} paused`,
};
});
}
function decisionsFromReport(report: Record<string, unknown>): Partial<Record<CloudUpstreamActivationEntityType, CloudUpstreamActivationDecision>> {
const value = optionalRecord(report.activationChecklist);
const decisions: Partial<Record<CloudUpstreamActivationEntityType, CloudUpstreamActivationDecision>> = {};
for (const key of ["agents", "routines", "monitors"] as const) {
const item = optionalRecord(value[key]);
if (!item) continue;
decisions[key] = {
entityType: key,
count: typeof item.count === "number" ? item.count : 0,
status: item.status === "activated" ? "activated" : "paused",
activatedAt: typeof item.activatedAt === "string" ? item.activatedAt : null,
};
}
return decisions;
}
function summaryCount(summary: CloudUpstreamSummaryCount[], key: CloudUpstreamActivationEntityType): number {
return summary.find((item) => item.key === key)?.count ?? 0;
}
function optionalRecord(value: unknown): Record<string, unknown> {
return value && typeof value === "object" && !Array.isArray(value) ? value as Record<string, unknown> : {};
}
function formatDate(value: string) {
return new Date(value).toLocaleString(undefined, {
month: "short",
day: "numeric",
hour: "2-digit",
minute: "2-digit",
timeZone: "UTC",
});
}
function formatBytes(value: number) {
if (value >= 1024 * 1024) return `${Math.round(value / (1024 * 1024))} MiB`;
if (value >= 1024) return `${Math.round(value / 1024)} KiB`;
return `${value} B`;
}
const STACK_TARGET = {
stackId: "stk_2vKqz9D8mNFqQ7Rp",
stackSlug: "paperclip-prod",
stackDisplayName: "Paperclip Prod",
companyId: "co_4hT2yX",
primaryHost: "paperclip.paperclip.app",
origin: "https://paperclip.paperclip.app",
product: "paperclip-cloud",
schemaMajor: 7,
maxChunkBytes: 5 * 1024 * 1024,
};
const STACK_TARGET_SCHEMA_BEHIND = {
...STACK_TARGET,
schemaMajor: 5,
};
function connectedConnection(target = STACK_TARGET): CloudUpstreamConnection {
return {
id: "cu_conn_8d3f1b6a",
companyId: "co_4hT2yX",
remoteUrl: "https://paperclip.paperclip.app/PC521D/dashboard",
target,
tokenStatus: "connected",
scopes: ["upstream.push", "upstream.preview"],
authorizedGlobalUserId: "user_9pXqYzAbCdEf",
expiresAt: "2026-08-18T19:00:00.000Z",
createdAt: "2026-05-18T18:45:00.000Z",
updatedAt: "2026-05-18T19:02:18.000Z",
lastRunId: null,
};
}
const PREVIEW_SUMMARY: CloudUpstreamSummaryCount[] = [
{ key: "users", label: "Users", count: 14 },
{ key: "agents", label: "Agents", count: 6 },
{ key: "routines", label: "Routines", count: 4 },
{ key: "monitors", label: "Monitors", count: 2 },
];
const PREVIEW_WARNINGS_NORMAL: CloudUpstreamWarning[] = [
{
code: "imported_automations_paused",
severity: "warning",
title: "Automations stay paused",
detail: "Imported agents, routines, and monitors require explicit activation after the push.",
},
{
code: "unmatched_users_import_as_historical_authors",
severity: "warning",
title: "Unmatched users become historical authors",
detail: "Invite now remains a secondary action after the transfer is complete.",
},
{
code: "secret_values_redacted",
severity: "warning",
title: "Secret values are not transferred",
detail: "The push carries secret requirements only. Configure cloud secrets before activating automations.",
},
];
const PREVIEW_WARNINGS_SCHEMA: CloudUpstreamWarning[] = [
{
code: "schema_mismatch",
severity: "blocker",
title: "Cloud stack upgrade required",
detail: "This local build uses upstream schema 7, but the cloud stack reports schema 5.",
},
...PREVIEW_WARNINGS_NORMAL,
];
const PREVIEW_CONFLICTS: CloudUpstreamConflict[] = [
{
id: "conflict_user_serena",
entityType: "user",
sourceLabel: "serena@magicmachine.co (unmatched)",
targetLabel: "→ historical author Serena R.",
plannedAction: "create",
reason: "Target stack has no matching identity. Will arrive as historical author; invite available after push.",
},
{
id: "conflict_user_dotta",
entityType: "user",
sourceLabel: "dotta@magicmachine.co",
targetLabel: "↦ dotta@magicmachine.co (cloud)",
plannedAction: "update",
reason: "Existing cloud identity matches local user; will be merged.",
},
{
id: "conflict_agent_qa",
entityType: "agent",
sourceLabel: "QA · qa-bot",
targetLabel: "↦ QA · qa-bot (cloud)",
plannedAction: "update",
reason: "Mapped to existing cloud agent. Imported run history will be appended.",
},
{
id: "conflict_routine_nightly_reports",
entityType: "routine",
sourceLabel: "Nightly status report",
targetLabel: "(new in cloud)",
plannedAction: "create",
reason: "Routine does not exist in the target stack and will be created in paused state.",
},
];
function basePreview(): CloudUpstreamPreview {
return {
connectionId: "cu_conn_8d3f1b6a",
sourceCompanyId: "co_local_pc521d",
target: STACK_TARGET,
schemaCompatible: true,
summary: PREVIEW_SUMMARY,
warnings: PREVIEW_WARNINGS_NORMAL,
conflicts: PREVIEW_CONFLICTS,
generatedAt: "2026-05-18T19:03:14.000Z",
};
}
function schemaMismatchPreview(): CloudUpstreamPreview {
return {
...basePreview(),
target: STACK_TARGET_SCHEMA_BEHIND,
schemaCompatible: false,
summary: [],
conflicts: [],
warnings: PREVIEW_WARNINGS_SCHEMA,
};
}
function cleanPreview(): CloudUpstreamPreview {
return {
...basePreview(),
conflicts: [],
warnings: PREVIEW_WARNINGS_NORMAL.slice(0, 1),
};
}
const PROGRESS_EVENTS = [
{ id: "evt_01", at: "2026-05-18T19:10:02.000Z", phase: "scan" as CloudUpstreamStep, type: "completed" as const, message: "Scanned 14 users, 6 agents, 4 routines, 2 monitors." },
{ id: "evt_02", at: "2026-05-18T19:10:11.000Z", phase: "preview" as CloudUpstreamStep, type: "completed" as const, message: "Preview generated with 4 conflicts and 3 warnings." },
{ id: "evt_03", at: "2026-05-18T19:10:31.000Z", phase: "push" as CloudUpstreamStep, type: "created" as const, message: "users · 8 created, 6 mapped to existing identities." },
{ id: "evt_04", at: "2026-05-18T19:10:48.000Z", phase: "push" as CloudUpstreamStep, type: "updated" as const, message: "agents · 4 created paused, 2 updated paused." },
{ id: "evt_05", at: "2026-05-18T19:10:58.000Z", phase: "push" as CloudUpstreamStep, type: "updated" as const, message: "routines · 3 created paused, 1 updated." },
{ id: "evt_06", at: "2026-05-18T19:11:09.000Z", phase: "push" as CloudUpstreamStep, type: "created" as const, message: "monitors · 2 created paused." },
{ id: "evt_07", at: "2026-05-18T19:11:18.000Z", phase: "verify" as CloudUpstreamStep, type: "updated" as const, message: "Verifying transferred ledger checksums…" },
];
function runningRun(): CloudUpstreamRun {
return {
id: "run_3kQ8mNpW9bX2zL4Y",
connectionId: "cu_conn_8d3f1b6a",
companyId: "co_local_pc521d",
status: "running",
activeStep: "push",
progressPercent: 62,
dryRun: false,
summary: PREVIEW_SUMMARY,
warnings: PREVIEW_WARNINGS_NORMAL,
conflicts: PREVIEW_CONFLICTS,
events: PROGRESS_EVENTS,
targetUrl: "https://paperclip.paperclip.app/PC521D/dashboard",
report: {},
retryOfRunId: null,
createdAt: "2026-05-18T19:10:01.000Z",
updatedAt: "2026-05-18T19:11:18.000Z",
completedAt: null,
};
}
function failedRun(): CloudUpstreamRun {
return {
id: "run_5fXqR2bT7aD8zP1K",
connectionId: "cu_conn_8d3f1b6a",
companyId: "co_local_pc521d",
status: "failed",
activeStep: "push",
progressPercent: 78,
dryRun: false,
summary: PREVIEW_SUMMARY,
warnings: PREVIEW_WARNINGS_NORMAL,
conflicts: PREVIEW_CONFLICTS,
events: [
...PROGRESS_EVENTS,
{
id: "evt_08",
at: "2026-05-18T19:11:30.000Z",
phase: "push",
type: "failed",
message: "Apply rejected: cloud rejected chunk 4 of 6 (HTTP 502). Ledger entries from chunks 13 retained; chunk 4 not committed.",
},
],
targetUrl: "https://paperclip.paperclip.app/PC521D/dashboard",
report: { ledgerCheckpoint: "chunk-3" },
retryOfRunId: null,
createdAt: "2026-05-18T19:10:01.000Z",
updatedAt: "2026-05-18T19:11:30.000Z",
completedAt: null,
};
}
function succeededRun(): CloudUpstreamRun {
return {
id: "run_7aBcD9eFgH2iJ3kL",
connectionId: "cu_conn_8d3f1b6a",
companyId: "co_local_pc521d",
status: "succeeded",
activeStep: "activate",
progressPercent: 100,
dryRun: false,
summary: PREVIEW_SUMMARY,
warnings: PREVIEW_WARNINGS_NORMAL,
conflicts: PREVIEW_CONFLICTS,
events: [
...PROGRESS_EVENTS,
{
id: "evt_08",
at: "2026-05-18T19:11:25.000Z",
phase: "verify",
type: "completed",
message: "Ledger checksums match. Push committed.",
},
{
id: "evt_09",
at: "2026-05-18T19:11:31.000Z",
phase: "activate",
type: "completed",
message: "Activation checklist pending operator approval — automations remain paused.",
},
],
targetUrl: "https://paperclip.paperclip.app/PC521D/dashboard",
report: {
activationChecklist: {
agents: { count: 6, status: "paused", activatedAt: null },
routines: { count: 4, status: "paused", activatedAt: null },
monitors: { count: 2, status: "paused", activatedAt: null },
},
},
retryOfRunId: null,
createdAt: "2026-05-18T19:10:01.000Z",
updatedAt: "2026-05-18T19:11:31.000Z",
completedAt: "2026-05-18T19:11:31.000Z",
};
}
function buildFixture(state: FixtureStateKey): Fixture {
switch (state) {
case "settings-pane":
return {
selectedCompanyName: "Paperclip · PC521D",
connection: connectedConnection(),
preview: null,
latestRun: null,
history: [],
notice: "Cloud upstream connection approved.",
actionError: null,
};
case "connect-wizard":
return {
selectedCompanyName: "Paperclip · PC521D",
connection: null,
preview: null,
latestRun: null,
history: [],
notice: null,
actionError: null,
};
case "schema-mismatch":
return {
selectedCompanyName: "Paperclip · PC521D",
connection: connectedConnection(STACK_TARGET_SCHEMA_BEHIND),
preview: schemaMismatchPreview(),
latestRun: null,
history: [],
notice: null,
actionError: "Cloud stack is on schema 5 but this local build pushes schema 7. Upgrade the cloud stack to continue.",
};
case "preview":
return {
selectedCompanyName: "Paperclip · PC521D",
connection: connectedConnection(),
preview: basePreview(),
latestRun: null,
history: [],
notice: null,
actionError: null,
};
case "preview-clean":
return {
selectedCompanyName: "Paperclip · PC521D",
connection: connectedConnection(),
preview: cleanPreview(),
latestRun: null,
history: [],
notice: "Preview completed. No target conflicts detected.",
actionError: null,
};
case "progress":
return {
selectedCompanyName: "Paperclip · PC521D",
connection: connectedConnection(),
preview: null,
latestRun: runningRun(),
history: [],
notice: null,
actionError: null,
};
case "retry":
return {
selectedCompanyName: "Paperclip · PC521D",
connection: connectedConnection(),
preview: null,
latestRun: failedRun(),
history: [
{ ...failedRun(), id: "run_9pYqXwVtSrQ" },
],
notice: null,
actionError: "Push run failed. Review the events. Retry resumes from ledger checkpoint chunk-3 — chunks 13 will not be re-applied.",
};
case "finish":
return {
selectedCompanyName: "Paperclip · PC521D",
connection: connectedConnection(),
preview: null,
latestRun: succeededRun(),
history: [
{ ...succeededRun(), id: "run_aZcXvBnMqWeR" },
],
notice: "Push run completed. Review activation before unpausing automations.",
actionError: null,
};
}
}
+17 -3
View File
@@ -1,5 +1,5 @@
import { ChangeEvent, useEffect, useState } from "react";
import { useMutation, useQueryClient } from "@tanstack/react-query";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import {
DEFAULT_COMPANY_ATTACHMENT_MAX_BYTES,
MAX_COMPANY_ATTACHMENT_MAX_BYTES,
@@ -9,9 +9,10 @@ import { useBreadcrumbs } from "../context/BreadcrumbContext";
import { companiesApi } from "../api/companies";
import { accessApi } from "../api/access";
import { assetsApi } from "../api/assets";
import { instanceSettingsApi } from "../api/instanceSettings";
import { queryKeys } from "../lib/queryKeys";
import { Button } from "@/components/ui/button";
import { Settings, Check, Download, Upload } from "lucide-react";
import { Settings, Check, CloudUpload, Download, Upload } from "lucide-react";
import { CompanyPatternIcon } from "../components/CompanyPatternIcon";
import {
Field,
@@ -37,6 +38,10 @@ export function CompanySettings() {
} = useCompany();
const { setBreadcrumbs } = useBreadcrumbs();
const queryClient = useQueryClient();
const { data: experimentalSettings } = useQuery({
queryKey: queryKeys.instance.experimentalSettings,
queryFn: () => instanceSettingsApi.getExperimental(),
});
// General settings local state
const [companyName, setCompanyName] = useState("");
const [description, setDescription] = useState("");
@@ -65,6 +70,7 @@ export function CompanySettings() {
Number.isInteger(attachmentMaxBytes)
&& attachmentMaxBytes >= BYTES_PER_MIB
&& attachmentMaxBytes <= MAX_COMPANY_ATTACHMENT_MAX_BYTES;
const cloudSyncEnabled = experimentalSettings?.enableCloudSync === true;
const generalDirty =
!!selectedCompany &&
@@ -520,7 +526,15 @@ export function CompanySettings() {
Import and export have moved to dedicated pages accessible from the{" "}
<a href="/org" className="underline hover:text-foreground">Org Chart</a> header.
</p>
<div className="mt-3 flex items-center gap-2">
<div className="mt-3 flex flex-wrap items-center gap-2">
{cloudSyncEnabled ? (
<Button size="sm" asChild>
<a href="/company/settings/cloud-upstream">
<CloudUpload className="mr-1.5 h-3.5 w-3.5" />
Send to Paperclip Cloud
</a>
</Button>
) : null}
<Button size="sm" variant="outline" asChild>
<a href="/company/export">
<Download className="mr-1.5 h-3.5 w-3.5" />
@@ -205,6 +205,7 @@ export function InstanceExperimentalSettings() {
const enableEnvironments = experimentalQuery.data?.enableEnvironments === true;
const enableIsolatedWorkspaces = experimentalQuery.data?.enableIsolatedWorkspaces === true;
const enableCloudSync = experimentalQuery.data?.enableCloudSync === true;
const autoRestartDevServerWhenIdle = experimentalQuery.data?.autoRestartDevServerWhenIdle === true;
const enableIssueGraphLivenessAutoRecovery =
experimentalQuery.data?.enableIssueGraphLivenessAutoRecovery === true;
@@ -298,6 +299,24 @@ export function InstanceExperimentalSettings() {
</div>
</section>
<section className="rounded-xl border border-border bg-card p-5">
<div className="flex items-start justify-between gap-4">
<div className="space-y-1.5">
<h2 className="text-sm font-semibold">Cloud Sync</h2>
<p className="max-w-2xl text-sm text-muted-foreground">
Show local Paperclip Cloud upstream connection, preview, push, retry, and activation review surfaces.
Saved connections and run history are preserved when this is disabled.
</p>
</div>
<ToggleSwitch
checked={enableCloudSync}
onCheckedChange={() => toggleMutation.mutate({ enableCloudSync: !enableCloudSync })}
disabled={toggleMutation.isPending}
aria-label="Toggle cloud sync experimental setting"
/>
</div>
</section>
<section className="rounded-xl border border-border bg-card p-5">
<div className="flex items-start justify-between gap-4">
<div className="space-y-1.5">