Commit Graph

34 Commits

Author SHA1 Message Date
Dotta 778e775c35 Add secrets provider vaults and remote import (#5429)
## Thinking Path

> - Paperclip orchestrates AI-agent companies and needs secrets handling
to work across local development, hosted operators, and governed agent
execution.
> - The affected subsystem is the company-scoped secrets control plane:
database schema, server services/routes, CLI workflows, and the Secrets
settings UI.
> - The gap was that secrets were local-only and operators could not
manage provider vaults or import existing remote references without
exposing plaintext.
> - This branch adds provider vault configuration plus an AWS Secrets
Manager remote-import path while preserving company boundaries, binding
context, and audit trails.
> - I kept the PR to a single branch PR, removed unrelated
lockfile/package drift, rebased the full branch onto the current
`public-gh/master`, and addressed fresh Greptile findings.
> - The benefit is a reviewable implementation of provider-backed
secrets with focused tests covering provider selection, import
conflicts, deleted secret reuse, rotation guards, and AWS signing
behavior.

## What Changed

- Added provider vault support for company secrets, including provider
config storage, default vault handling, health checks, binding usage,
access events, and remote import preview/commit.
- Added an AWS Secrets Manager provider using SigV4 request signing,
bounded request timeouts, namespace guardrails, cached runtime
credential resolution, and external-reference linking without plaintext
reads.
- Added Secrets UI surfaces for vault management and remote import, plus
CLI/API documentation for setup and operations.
- Stabilized routine webhook secret binding paths and SSH
environment-driver fixture bindings discovered during verification.
- Addressed Greptile and CI findings: no lockfile/package drift,
monotonic migration metadata, disabled-vault default races, soft-deleted
secret hiding/recreate behavior, remove behavior with disabled vaults,
soft-deleted external-reference re-import, non-active rotation guards,
managed-secret soft deletion through PATCH, and per-call AWS SDK
credential client churn.
- Rebased this branch onto `public-gh/master` at `0e1a5828` and
force-pushed with lease to keep this as the single PR for the branch.

## Verification

- `git fetch public-gh master`
- `git rebase public-gh/master`
- `git diff --name-only public-gh/master...HEAD | grep
'^pnpm-lock\.yaml$' || true` confirmed `pnpm-lock.yaml` is not in the PR
diff.
- Confirmed migration ordering: master ends at `0081_optimal_dormammu`;
this PR adds `0082_dry_vision` and
`0083_company_secret_provider_configs`.
- Inspected migrations for repeat safety: new tables/indexes use `IF NOT
EXISTS`; foreign keys are guarded by `DO $$ ... IF NOT EXISTS`; column
additions use `ADD COLUMN IF NOT EXISTS`.
- `pnpm -r typecheck` passed before the Greptile follow-up commits.
- `pnpm test:run` ran the full stable Vitest path before the Greptile
follow-up commits; it completed with 3 timing-related failures under
parallel load: `codex-local-execute.test.ts`,
`cursor-local-execute.test.ts`, and `environment-service.test.ts`.
- `pnpm --filter @paperclipai/server exec vitest run
src/__tests__/codex-local-execute.test.ts
src/__tests__/cursor-local-execute.test.ts
src/__tests__/environment-service.test.ts` passed on targeted rerun
(`24/24`).
- `pnpm build` passed before the Greptile follow-up commits. Vite
reported existing chunk-size/dynamic-import warnings.
- After Greptile follow-up commits: `pnpm --filter @paperclipai/server
exec vitest run src/__tests__/secrets-service.test.ts` passed (`26/26`).
- After Greptile follow-up commits: `pnpm --filter @paperclipai/server
exec vitest run src/__tests__/aws-secrets-manager-provider.test.ts
src/__tests__/secrets-service.test.ts` passed (`39/39`).
- After Greptile follow-up commits: `pnpm --filter @paperclipai/server
typecheck` passed.
- Captured Storybook screenshots from `ui/storybook-static` for visual
review.
- Latest PR checks on `5ca3a5cf`: `policy`, serialized server suites
1/4-4/4, `Canary Dry Run`, `e2e`, `security/snyk`, and `Greptile Review`
pass; aggregate `verify` is still registering the completed child
checks.
- Greptile review loop continued through the latest requested pass; all
Greptile review threads are resolved and the latest `Greptile Review`
check on `5ca3a5cf` passed with 0 comments added.

## Screenshots

Before: the provider-vault and remote-import surfaces did not exist on
`master`; these are after-state screenshots from the Storybook fixtures.

![Secrets
inventory](https://raw.githubusercontent.com/paperclipai/paperclip/PAP-2339-secrets-make-a-plan/doc/pr/5429/secrets-inventory.png)

![Secret binding
picker](https://raw.githubusercontent.com/paperclipai/paperclip/PAP-2339-secrets-make-a-plan/doc/pr/5429/secret-binding-picker.png)

![Environment editor with
secrets](https://raw.githubusercontent.com/paperclipai/paperclip/PAP-2339-secrets-make-a-plan/doc/pr/5429/env-editor-with-secrets.png)

## Risks

- Migration risk: this adds new secret provider tables and extends
existing secret rows. The migrations were checked for monotonic ordering
and idempotent guards, but reviewers should still inspect upgrade
behavior carefully.
- Provider risk: AWS support uses direct SigV4 requests. Automated tests
cover signing, request timeouts, vault-config selection, namespace
guardrails, pending-version archival, sanitized provider errors, and
service-level cleanup paths. A real-vault AWS smoke test remains
deployment validation for an operator with AWS credentials rather than
an unverified merge blocker in this local branch.
- UI risk: the Secrets page and import dialog are large new surfaces;
screenshots are included above for reviewer inspection.
- Verification risk: the full local stable test command hit
parallel-load timing failures, although the exact failed files passed
when rerun directly.
- Operational risk: remote import intentionally avoids plaintext reads;
operators must understand that imported external references resolve at
runtime and may fail if AWS permissions change.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, GPT-5 coding agent with local shell/tool use in the
Paperclip worktree. Exact context-window size was not exposed by the
runtime.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 18:22:17 -05:00
Devin Foley e4995bbb1c Add SSH environment support (#4358)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies
> - The environments subsystem already models execution environments,
but before this branch there was no end-to-end SSH-backed runtime path
for agents to actually run work against a remote box
> - That meant agents could be configured around environment concepts
without a reliable way to execute adapter sessions remotely, sync
workspace state, and preserve run context across supported adapters
> - We also need environment selection to participate in normal
Paperclip control-plane behavior: agent defaults, project/issue
selection, route validation, and environment probing
> - Because this capability is still experimental, the UI surface should
be easy to hide and easy to remove later without undoing the underlying
implementation
> - This pull request adds SSH environment execution support across the
runtime, adapters, routes, schema, and tests, then puts the visible
environment-management UI behind an experimental flag
> - The benefit is that we can validate real SSH-backed agent execution
now while keeping the user-facing controls safely gated until the
feature is ready to come out of experimentation

## What Changed

- Added SSH-backed execution target support in the shared adapter
runtime, including remote workspace preparation, skill/runtime asset
sync, remote session handling, and workspace restore behavior after
runs.
- Added SSH execution coverage for supported local adapters, plus remote
execution tests across Claude, Codex, Cursor, Gemini, OpenCode, and Pi.
- Added environment selection and environment-management backend support
needed for SSH execution, including route/service work, validation,
probing, and agent default environment persistence.
- Added CLI support for SSH environment lab verification and updated
related docs/tests.
- Added the `enableEnvironments` experimental flag and gated the
environment UI behind it on company settings, agent configuration, and
project configuration surfaces.

## Verification

- `pnpm exec vitest run
packages/adapters/claude-local/src/server/execute.remote.test.ts
packages/adapters/cursor-local/src/server/execute.remote.test.ts
packages/adapters/gemini-local/src/server/execute.remote.test.ts
packages/adapters/opencode-local/src/server/execute.remote.test.ts
packages/adapters/pi-local/src/server/execute.remote.test.ts`
- `pnpm exec vitest run server/src/__tests__/environment-routes.test.ts`
- `pnpm exec vitest run
server/src/__tests__/instance-settings-routes.test.ts`
- `pnpm exec vitest run ui/src/lib/new-agent-hire-payload.test.ts
ui/src/lib/new-agent-runtime-config.test.ts`
- `pnpm -r typecheck`
- `pnpm build`
- Manual verification on a branch-local dev server:
  - enabled the experimental flag
  - created an SSH environment
  - created a Linux Claude agent using that environment
- confirmed a run executed on the Linux box and synced workspace changes
back

## Risks

- Medium: this touches runtime execution flow across multiple adapters,
so regressions would likely show up in remote session setup, workspace
sync, or environment selection precedence.
- The UI flag reduces exposure, but the underlying runtime and route
changes are still substantial and rely on migration correctness.
- The change set is broad across adapters, control-plane services,
migrations, and UI gating, so review should pay close attention to
environment-selection precedence and remote workspace lifecycle
behavior.

## Model Used

- OpenAI Codex via Paperclip's local Codex adapter, GPT-5-class coding
model with tool use and code execution in the local repo workspace. The
local adapter does not surface a more specific public model version
string in this branch workflow.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-04-23 19:15:22 -07:00
dotta 2a84e53c1b Introduce bind presets for deployment setup
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-11 07:09:07 -05:00
dotta 909e8cd4c8 feat(routines): add workspace-aware routine runs
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-02 11:38:57 -05:00
dotta 34044cdfce feat: implement app-side telemetry sender
Add the shared telemetry sender, wire the CLI/server emit points,
and cover the config and completion behavior with tests.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-02 10:47:29 -05:00
dotta c0d0d03bce Add feedback voting and thumbs capture flow
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-02 09:11:49 -05:00
dotta 37c2c4acc4 Add browser-based board CLI auth flow
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-23 08:46:05 -05:00
gsxdsm 0f831e09c1 Add plugin cli commands 2026-03-14 13:58:43 -07:00
Dotta 0704854926 Add worktree init CLI for isolated development instances 2026-03-10 10:13:05 -05:00
Dotta ec0b7daca2 Add paperclipai db:backup CLI command 2026-03-05 06:02:12 -06:00
Dotta 358467a506 chore: release v0.2.7 2026-03-04 14:51:33 -06:00
Dotta 5dffdbb382 chore: release v0.2.6 2026-03-04 10:24:03 -06:00
Dotta 6141d5c3f2 chore: release v0.2.5
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 16:14:33 -06:00
Dotta ab3b9ab19f chore: bump all packages to 0.2.4
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 16:10:52 -06:00
Dotta 28bf5e9e9b chore: release v0.2.3 2026-03-03 15:39:13 -06:00
Dotta b2737b9571 chore: release v0.2.2 2026-03-03 15:10:25 -06:00
Dotta defccdd4d9 feat: integrate Changesets for multi-package npm publishing
Migrate from single-bundle CLI publishing to publishing all @paperclipai/*
packages individually via Changesets. This fixes the "Cannot find package
@paperclipai/server" error when installing from npm.

Changes:
- Add @changesets/cli with fixed versioning (all packages share version)
- Make 7 packages publishable (shared, adapter-utils, db, 3 adapters, server)
- Add build scripts, publishConfig, and files fields to all packages
- Mark @paperclipai/server as external in CLI esbuild config
- Simplify CLI importServerEntry() to use string-literal dynamic import
- Add generate-npm-package-json support for external workspace packages
- Create scripts/release.sh for one-command releases
- Remove old bump-and-publish.sh and version-bump.sh
- All packages start at version 0.2.0

Usage: ./scripts/release.sh patch|minor|major [--dry-run]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 14:46:16 -06:00
Dotta 6e7f948314 chore: bump version to 0.1.2 2026-03-03 14:01:03 -06:00
Dotta 6a8b3e921b fix(cli): prevent duplicate shebang in npm bundle 2026-03-03 13:54:40 -06:00
Dotta 28a33d1359 chore: bump version to 0.1.1 2026-03-03 13:41:02 -06:00
Dotta b5e2433822 feat(cli): add --yes flag to onboard for non-interactive quickstart
Skip the setup path prompt and auto-start the server when --yes is
passed, enabling fully non-interactive onboarding with local defaults.
Opens the browser automatically on server listen.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 12:18:25 -06:00
Dotta 3248974435 cli: add quickstart onboarding flow and config-scoped jwt env handling 2026-03-03 10:24:34 -06:00
Dotta f60c1001ec refactor: rename packages to @paperclipai and CLI binary to paperclipai
Rename all workspace packages from @paperclip/* to @paperclipai/* and
the CLI binary from `paperclip` to `paperclipai` in preparation for
npm publishing. Bump CLI version to 0.1.0 and add package metadata
(description, keywords, license, repository, files). Update all
imports, documentation, user-facing messages, and tests accordingly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 08:45:26 -06:00
Dotta cabf09e7b1 feat(cli): add --data-dir flag to isolate local state
Add --data-dir option to all CLI commands, allowing users to override
the default ~/.paperclip root for config, context, database, logs, and
storage. Includes preAction hook to auto-derive --config and --context
paths when --data-dir is set. Add unit tests and doc updates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 14:20:37 -06:00
Forgotten 85c0b9a3dc feat: private hostname guard for authenticated/private mode
Reject requests from unrecognised Host headers when running
authenticated/private. Adds server middleware, CLI `allowed-hostname`
command, config-schema field, and prompt support for configuring
allowed hostnames during onboard/configure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 19:43:52 -06:00
Forgotten 5b983ca4d3 feat(cli): add deployment mode prompts, auth bootstrap-ceo command, and doctor check
Extend server setup prompts with deployment mode (local_trusted vs
authenticated), exposure (private vs public), bind host, and auth config.
Add auth bootstrap-ceo command that creates a one-time invite URL for the
initial instance admin. Add deployment-auth-check to doctor diagnostics.
Register the new command in the CLI entry point.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 14:40:59 -06:00
Forgotten fdd2ea6157 feat: add storage system with local disk and S3 providers
Introduces a provider-agnostic storage subsystem for file attachments.
Includes local disk and S3 backends, asset/attachment DB schemas, issue
attachment CRUD routes with multer upload, CLI configure/doctor/env
integration, and enriched issue ancestors with project/goal resolution.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 10:31:56 -06:00
Forgotten 8f3fc077fa feat(cli): add client commands and home-based local runtime defaults 2026-02-20 07:10:58 -06:00
Forgotten f1b558dcfb CLI: add secrets configuration, doctor check, and path resolver extraction
Add secrets section to onboard, configure, and doctor commands. Doctor
validates local encrypted provider key file and can auto-repair missing
keys. Extract shared path resolution into path-resolver module used by
database and log checks. Show secrets env vars in `paperclip env`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 15:43:59 -06:00
Forgotten fe6a8687c1 Implement local agent JWT authentication for adapters
Add HS256 JWT-based authentication for local adapters (claude_local, codex_local)
so agents authenticate automatically without manual API key configuration. The
server mints short-lived JWTs per heartbeat run and injects them as PAPERCLIP_API_KEY.
The auth middleware verifies JWTs alongside existing static API keys.

Includes: CLI onboard/doctor JWT secret management, env command for deployment,
config path resolution from ancestor directories, dotenv loading on server startup,
event payload secret redaction, multi-status issue filtering, and adapter transcript
parsing for thinking/user message kinds.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 16:46:45 -06:00
Forgotten 7e4a20645c Improve CLI: config store, heartbeat-run, and onboarding
Rework config store with better file handling. Expand heartbeat-run
command with richer output and error reporting. Improve configure
and onboard commands. Update doctor checks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 13:02:12 -06:00
Forgotten af44f45c31 Improve heartbeat-run CLI and agent detail UI
Rework heartbeat-run command with better error handling and output
formatting. Improve AgentConfigForm field layout. Add CSS for agent
run timeline. Enhance AgentDetail page with runtime status section.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 21:12:31 -06:00
Forgotten 6dbbf1bbec Add CLI heartbeat-run command for manual agent invocation
Add heartbeat-run command that triggers a single agent heartbeat from
the CLI. Register it in the CLI entrypoint alongside existing commands.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 20:46:07 -06:00
Forgotten 5306142542 Add CLI package, config file support, and workspace setup
Add cli/ package with initial scaffolding. Add config-schema to shared
package for typed configuration. Add server config-file loader for
paperclip.config.ts support. Register cli in pnpm workspace. Add
.paperclip/ and .pnpm-store/ to gitignore. Minor Companies page fix.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 13:39:47 -06:00