Commit Graph

40 Commits

Author SHA1 Message Date
Dotta 16b2b84d84 [codex] Improve agent runtime recovery and governance (#4086)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies.
> - The heartbeat runtime, agent import path, and agent configuration
defaults determine whether work is dispatched safely and predictably.
> - Several accumulated fixes all touched agent execution recovery, wake
routing, import behavior, and runtime concurrency defaults.
> - Those changes need to land together so the heartbeat service and
agent creation defaults stay internally consistent.
> - This pull request groups the runtime/governance changes from the
split branch into one standalone branch.
> - The benefit is safer recovery for stranded runs, bounded high-volume
reads, imported-agent approval correctness, skill-template support, and
a clearer default concurrency policy.

## What Changed

- Fixed stranded continuation recovery so successful automatic retries
are requeued instead of incorrectly blocking the issue.
- Bounded high-volume issue/log reads across issue, heartbeat, agent,
project, and workspace paths.
- Fixed imported-agent approval and instruction-path permission
handling.
- Quarantined seeded worktree execution state during worktree
provisioning.
- Queued approval follow-up wakes and hardened SQL_ASCII heartbeat
output handling.
- Added reusable agent instruction templates for hiring flows.
- Set the default max concurrent agent runs to five and updated related
UI/tests/docs.

## Verification

- `pnpm install --frozen-lockfile`
- `pnpm exec vitest run server/src/__tests__/company-portability.test.ts
server/src/__tests__/heartbeat-process-recovery.test.ts
server/src/__tests__/heartbeat-comment-wake-batching.test.ts
server/src/__tests__/heartbeat-list.test.ts
server/src/__tests__/issues-service.test.ts
server/src/__tests__/agent-permissions-routes.test.ts
packages/adapter-utils/src/server-utils.test.ts
ui/src/lib/new-agent-runtime-config.test.ts`
- Split integration check: merged this branch first, followed by the
other [PAP-1614](/PAP/issues/PAP-1614) branches, with no merge
conflicts.
- Confirmed this branch does not include `pnpm-lock.yaml`.

## Risks

- Medium risk: touches heartbeat recovery, queueing, and issue list
bounds in central runtime paths.
- Imported-agent and concurrency default behavior changes may affect
existing automation that assumes one-at-a-time default runs.
- No database migrations are included.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, GPT-5.4 tool-enabled coding model, agentic
code-editing/runtime with local shell and GitHub CLI access; exact
context window and reasoning mode are not exposed by the Paperclip
harness.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-20 06:19:48 -05:00
Dotta 7f893ac4ec [codex] Harden execution reliability and heartbeat tooling (#3679)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies
> - Reliable execution depends on heartbeat routing, issue lifecycle
semantics, telemetry, and a fast enough local verification loop to keep
regressions visible
> - The remaining commits on this branch were mostly server/runtime
correctness fixes plus test and documentation follow-ups in that area
> - Those changes are logically separate from the UI-focused
issue-detail and workspace/navigation branches even when they touch
overlapping issue APIs
> - This pull request groups the execution reliability, heartbeat,
telemetry, and tooling changes into one standalone branch
> - The benefit is a focused review of the control-plane correctness
work, including the follow-up fix that restored the implicit
comment-reopen helpers after branch splitting

## What Changed

- Hardened issue/heartbeat execution behavior, including self-review
stage skipping, deferred mention wakes during active execution, stranded
execution recovery, active-run scoping, assignee resolution, and
blocked-to-todo wake resumption
- Reduced noisy polling/logging overhead by trimming issue run payloads,
compacting persisted run logs, silencing high-volume request logs, and
capping heartbeat-run queries in dashboard/inbox surfaces
- Expanded telemetry and status semantics with adapter/model fields on
task completion plus clearer status guidance in docs/onboarding material
- Updated test infrastructure and verification defaults with faster
route-test module isolation, cheaper default `pnpm test`, e2e isolation
from local state, and repo verification follow-ups
- Included docs/release housekeeping from the branch and added a small
follow-up commit restoring the implicit comment-reopen helpers that were
dropped during branch reconstruction

## Verification

- `pnpm vitest run
server/src/__tests__/issue-comment-reopen-routes.test.ts
server/src/__tests__/issue-telemetry-routes.test.ts`
- `pnpm vitest run server/src/__tests__/http-log-policy.test.ts
server/src/__tests__/heartbeat-run-log.test.ts
server/src/__tests__/health.test.ts`
- `server/src/__tests__/activity-service.test.ts`,
`server/src/__tests__/heartbeat-comment-wake-batching.test.ts`, and
`server/src/__tests__/heartbeat-process-recovery.test.ts` were attempted
on this host but the embedded Postgres harness reported
init-script/data-dir problems and skipped or failed to start, so they
are noted as environment-limited

## Risks

- Medium: this branch changes core issue/heartbeat routing and
reopen/wakeup behavior, so regressions would affect agent execution flow
rather than isolated UI polish
- Because it also updates verification infrastructure, reviewers should
pay attention to whether the new tests are asserting the right failure
modes and not just reshaping harness behavior

## Model Used

- OpenAI Codex coding agent (GPT-5-class runtime in Codex CLI; exact
deployed model ID is not exposed in this environment), reasoning
enabled, tool use and local code execution enabled

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [ ] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-04-14 13:34:52 -05:00
Dotta 8bdf4081ee chore: improve worktree tooling and security docs 2026-04-10 22:26:30 -05:00
Dotta b00d52c5b6 Merge pull request #3015 from aronprins/feature/backups-configuration
feat(backups): gzip compression and tiered retention with UI controls
2026-04-10 11:56:12 -05:00
dotta 8e88577371 chore(dev): preflight workspace links and simplify worktree helpers 2026-04-09 06:14:12 -05:00
dotta 46892ded18 Add worktree reseed command
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-09 06:14:12 -05:00
Aron Prins fcbae62baf feat(backups): tiered daily/weekly/monthly retention with UI controls
Replace single retentionDays with a three-tier BackupRetentionPolicy:
- Daily: keep all backups (presets: 3, 7, 14 days; default 7)
- Weekly: keep one per calendar week (presets: 1, 2, 4 weeks; default 4)
- Monthly: keep one per calendar month (presets: 1, 3, 6 months; default 1)

Pruning sorts backups newest-first and applies each tier's cutoff,
keeping only the newest entry per ISO week/month bucket. The Instance
Settings General page now shows three preset selectors (no icon, matches
existing page design). Remove Database icon import.
2026-04-08 14:40:05 +02:00
dotta b5e177df7e Address greptile review feedback 2026-04-08 06:02:34 -05:00
dotta 34589ad457 Add worktree reseed command 2026-04-08 06:02:34 -05:00
dotta 76a692c260 Improve embedded Postgres bootstrap errors
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-26 12:35:19 -05:00
dotta 555f026c24 Avoid sibling worktree port collisions
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-26 11:12:39 -05:00
dotta 5602576ae1 Fix embedded Postgres initdb failure in Docker slim containers
The embedded-postgres library hardcodes --lc-messages=en_US.UTF-8 and
strips the parent process environment when spawning initdb/postgres.
In slim Docker images (e.g. node:20-bookworm-slim), the en_US.UTF-8
locale isn't installed, causing initdb to exit with code 1.

Two fixes applied:
1. Add --lc-messages=C to all initdbFlags arrays (overrides the
   library's hardcoded locale since our flags come after in the spread)
2. pnpm patch on embedded-postgres to preserve process.env in spawn
   calls, preventing loss of PATH, LD_LIBRARY_PATH, and other vars

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-24 11:48:59 -05:00
dotta 5dfdbe91bb Add merge-history project import option
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-23 08:14:51 -05:00
dotta d47ec56eca fix: renumber worktree merge history migrations 2026-03-20 17:23:45 -05:00
dotta 54b99d5096 Search sibling storage roots for attachments
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-20 17:01:52 -05:00
dotta fb63d61ae5 Skip missing worktree attachment objects
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-20 17:01:52 -05:00
dotta 73ada45037 Import worktree documents and attachments
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-20 17:01:52 -05:00
dotta cff06c9a54 Add issue titles to worktree merge preview
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-20 17:01:52 -05:00
dotta ad011fbf1e Clarify worktree import source and target flags
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-20 17:01:52 -05:00
dotta 28a5f858b7 Add worktree source discovery commands
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-20 17:01:52 -05:00
dotta 220a5ec5dd Add project mapping prompts for worktree imports
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-20 17:01:52 -05:00
dotta 0ec79d4295 Add worktree history merge command
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-20 17:01:52 -05:00
Dotta aa799bba4c Fix worktree seed source selection
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-13 15:07:42 -05:00
Dotta cce9941464 Add worktree UI branding 2026-03-13 11:12:43 -05:00
Dotta 24d6e3a543 fix: align embedded postgres ctor types with initdbFlags usage 2026-03-13 09:25:04 -05:00
Dotta 77af1ae544 feat(worktree): add worktree:cleanup command, env var defaults, and auto-prefix
- Add `worktree:cleanup <name>` command that safely removes a worktree,
  its branch, and instance data. Idempotent and safe by default — refuses
  to delete branches with unique unmerged commits or worktrees with
  uncommitted changes unless --force is passed.
- Support PAPERCLIP_WORKTREES_DIR env var as default for --home across
  all worktree commands.
- Support PAPERCLIP_WORKTREE_START_POINT env var as default for
  --start-point on worktree:make.
- Auto-prefix worktree names with "paperclip-" if not already present,
  so `worktree:make subissues` creates ~/paperclip-subissues.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 07:24:39 -05:00
Dotta d14e656ec1 Merge pull request #645 from vkartaviy/fix/embedded-postgres-utf8
fix: ensure embedded PostgreSQL databases use UTF-8 encoding
2026-03-12 14:15:58 -05:00
Dotta 4435e14838 Merge branch 'master' of github.com-dotta:paperclipai/paperclip
* 'master' of github.com-dotta:paperclipai/paperclip:
  Tighten transcript label styling
  Fix env-sensitive worktree and runtime config tests
  Refine executed command row centering
  Tighten live run transcript streaming and stdout
  Center collapsed command group rows
  Refine collapsed command failure styling
  Tighten command transcript rows and dashboard card
  Polish transcript event widgets
  Refine transcript chrome and labels
  fix: remove paperclip property from OpenClaw Gateway agent params
  Add a run transcript UX fixture lab
  Humanize run transcripts across run detail and live surfaces
  fix(adapters/gemini-local): address PR review feedback
  fix(adapters/gemini-local): inject skills into ~/.gemini/ instead of tmpdir
  fix(adapters/gemini-local): downgrade missing API key to info level
  feat(adapters/gemini-local): add auth detection, turn-limit handling, sandbox, and approval modes
  fix(adapters/gemini-local): address PR review feedback for skills and formatting
  feat(adapters): add Gemini CLI local adapter support

# Conflicts:
#	cli/src/__tests__/worktree.test.ts
2026-03-11 17:04:30 -05:00
Dotta f7cc292742 Fix env-sensitive worktree and runtime config tests
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-11 16:38:31 -05:00
Dotta 2efc3a3ef6 Fix worktree JWT env persistence
Ensure worktree init writes PAPERCLIP_AGENT_JWT_SECRET into the new .paperclip/.env when the source instance already has a usable secret loaded or configured. Also harden the affected integration tests against shell env leakage and full-suite timeout pressure.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-11 16:38:16 -05:00
Volodymyr Kartavyi 057e3a494c fix: ensure embedded PostgreSQL databases use UTF-8 encoding
On macOS, `initdb` defaults to SQL_ASCII encoding because it infers
locale from the system environment. When `ensurePostgresDatabase()`
creates a database without specifying encoding, the new database
inherits SQL_ASCII from the cluster. This causes string functions like
`left()` to operate on bytes instead of characters, producing invalid
UTF-8 when multi-byte characters are truncated.

Two-part fix:
1. Pass `--encoding=UTF8 --locale=C` via `initdbFlags` to all
   EmbeddedPostgres constructors so the cluster defaults to UTF-8.
2. Explicitly set `encoding 'UTF8'` in the CREATE DATABASE statement
   with `template template0` (required because template1 may already
   have a different encoding) and `C` locale for portability.

Existing databases created with SQL_ASCII are NOT automatically fixed;
users must delete their local `data/db` directory and restart to
re-initialize the cluster.

Relates to #636

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 21:59:02 +01:00
Dotta 472322de24 Install dependencies after creating worktree
Run pnpm install in the new worktree directory before initializing the
Paperclip instance so that node_modules are ready when the init step runs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 09:31:41 -05:00
Dotta d9492f02d6 Add worktree start-point support 2026-03-11 09:15:27 -05:00
Dotta 80d87d3b4e Add paperclipai worktree:make command 2026-03-10 16:52:26 -05:00
Dotta 93a8b55ff8 Copy git hooks during worktree init 2026-03-10 14:55:35 -05:00
Dotta 12216b5cc6 Rebind seeded project workspaces to the current worktree 2026-03-10 13:50:29 -05:00
Dotta deec68ab16 Copy seeded secrets key into worktree instances 2026-03-10 12:57:53 -05:00
Dotta 83738b45cd Fix worktree minimal clone startup 2026-03-10 10:13:05 -05:00
Dotta 4a67db6a4d Add minimal worktree seed mode 2026-03-10 10:13:05 -05:00
Dotta 0704854926 Add worktree init CLI for isolated development instances 2026-03-10 10:13:05 -05:00