Files
paperclip/packages/db/src/schema/routines.ts
T
Dotta 705c1b8d81 [codex] Add routine env secrets support (#6212)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies.
> - Scheduled routines are the control-plane path for recurring agent
work.
> - Routines already had dispatch/history, but their runtime environment
did not carry routine-owned secret bindings through execution.
> - Operators need routine-specific secrets that can override
project/agent env without exposing secret values in history, logs, or
access events.
> - This pull request adds the routine env runtime contract, wires it
into execution, and makes the routine UI/history surfaces show safe
secret metadata.
> - The benefit is that routine executions can use scoped secret refs
predictably while preserving company boundaries and auditability.

## What Changed

- Added routine env persistence/runtime support, including
`routines.env`, `routine_runs.routine_revision_id`, revision snapshots,
and idempotent migration `0086_routine_env_runtime_contract`.
- Resolved routine env during heartbeat adapter config assembly with
precedence `agent < project < routine` and secret access events recorded
against the routine consumer.
- Added secret binding synchronization for routine create/update/restore
flows and guarded cross-company, missing, disabled, and deleted secret
cases.
- Added a Secrets tab to routine detail, env/secret history diff
rendering, and Storybook coverage for the new UI states.
- Added server/UI regression tests, including an embedded-Postgres QA
path for routine secret execution and restore behavior.
- Updated implementation/database docs for routine env and
secret-binding behavior.

## Verification

- `pnpm install --frozen-lockfile` after rebasing onto
`public-gh/master` to refresh workspace links for the newly-added
upstream Grok adapter package.
- `pnpm exec vitest run
server/src/__tests__/heartbeat-project-env.test.ts
server/src/__tests__/routines-service.test.ts
server/src/__tests__/secrets-service.test.ts
server/src/__tests__/qa-routine-secrets-e2e.test.ts
ui/src/components/RoutineHistoryTab.test.tsx` passed: 5 files, 92 tests.
- `pnpm -r typecheck` passed across the workspace.
- `pnpm build` passed. Vite emitted the existing
large-chunk/dynamic-import warnings.
- UI screenshots were captured locally during QA in
`artifacts/pap-9521/` and `artifacts/pap-9522/`; generated screenshots
are not committed to avoid adding binary artifacts to the repo.

## Risks

- Migration risk is limited by `IF NOT EXISTS` guards for the new
columns, FK, and index, and the migration is ordered as `0086`
immediately after upstream `0085`.
- Runtime behavior changes env precedence for routine executions by
adding routine env as the highest-precedence layer; tests cover
agent/project/routine precedence.
- Secret handling is security-sensitive; tests cover value-free
manifests/events/errors, disabled/missing/deleted secrets, and
cross-company rejection.
- UI history now renders routine env/secret diffs; tests and Storybook
stories cover the main rendering paths.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex coding agent based on GPT-5, with shell/tool use and
medium reasoning effort.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-05-17 16:30:34 -05:00

155 lines
7.8 KiB
TypeScript

import {
type AnyPgColumn,
boolean,
index,
integer,
jsonb,
pgTable,
text,
timestamp,
uniqueIndex,
uuid,
} from "drizzle-orm/pg-core";
import { agents } from "./agents.js";
import { companies } from "./companies.js";
import { companySecrets } from "./company_secrets.js";
import { issues } from "./issues.js";
import { projects } from "./projects.js";
import { goals } from "./goals.js";
import { heartbeatRuns } from "./heartbeat_runs.js";
import type { RoutineEnvConfig, RoutineRevisionSnapshotV1, RoutineVariable } from "@paperclipai/shared";
export const routines = pgTable(
"routines",
{
id: uuid("id").primaryKey().defaultRandom(),
companyId: uuid("company_id").notNull().references(() => companies.id, { onDelete: "cascade" }),
projectId: uuid("project_id").references(() => projects.id, { onDelete: "cascade" }),
goalId: uuid("goal_id").references(() => goals.id, { onDelete: "set null" }),
parentIssueId: uuid("parent_issue_id").references(() => issues.id, { onDelete: "set null" }),
title: text("title").notNull(),
description: text("description"),
assigneeAgentId: uuid("assignee_agent_id").references(() => agents.id),
priority: text("priority").notNull().default("medium"),
status: text("status").notNull().default("active"),
concurrencyPolicy: text("concurrency_policy").notNull().default("coalesce_if_active"),
catchUpPolicy: text("catch_up_policy").notNull().default("skip_missed"),
variables: jsonb("variables").$type<RoutineVariable[]>().notNull().default([]),
env: jsonb("env").$type<RoutineEnvConfig>(),
latestRevisionId: uuid("latest_revision_id"),
latestRevisionNumber: integer("latest_revision_number").notNull().default(1),
createdByAgentId: uuid("created_by_agent_id").references(() => agents.id, { onDelete: "set null" }),
createdByUserId: text("created_by_user_id"),
updatedByAgentId: uuid("updated_by_agent_id").references(() => agents.id, { onDelete: "set null" }),
updatedByUserId: text("updated_by_user_id"),
lastTriggeredAt: timestamp("last_triggered_at", { withTimezone: true }),
lastEnqueuedAt: timestamp("last_enqueued_at", { withTimezone: true }),
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
updatedAt: timestamp("updated_at", { withTimezone: true }).notNull().defaultNow(),
},
(table) => ({
companyStatusIdx: index("routines_company_status_idx").on(table.companyId, table.status),
companyAssigneeIdx: index("routines_company_assignee_idx").on(table.companyId, table.assigneeAgentId),
companyProjectIdx: index("routines_company_project_idx").on(table.companyId, table.projectId),
}),
);
export const routineRevisions = pgTable(
"routine_revisions",
{
id: uuid("id").primaryKey().defaultRandom(),
companyId: uuid("company_id").notNull().references(() => companies.id, { onDelete: "cascade" }),
routineId: uuid("routine_id").notNull().references(() => routines.id, { onDelete: "cascade" }),
revisionNumber: integer("revision_number").notNull(),
title: text("title").notNull(),
description: text("description"),
snapshot: jsonb("snapshot").$type<RoutineRevisionSnapshotV1>().notNull(),
changeSummary: text("change_summary"),
restoredFromRevisionId: uuid("restored_from_revision_id").references(
(): AnyPgColumn => routineRevisions.id,
{ onDelete: "set null" },
),
createdByAgentId: uuid("created_by_agent_id").references(() => agents.id, { onDelete: "set null" }),
createdByUserId: text("created_by_user_id"),
createdByRunId: uuid("created_by_run_id").references(() => heartbeatRuns.id, { onDelete: "set null" }),
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
},
(table) => ({
routineRevisionUq: uniqueIndex("routine_revisions_routine_revision_uq").on(
table.routineId,
table.revisionNumber,
),
companyRoutineCreatedIdx: index("routine_revisions_company_routine_created_idx").on(
table.companyId,
table.routineId,
table.createdAt,
),
}),
);
export const routineTriggers = pgTable(
"routine_triggers",
{
id: uuid("id").primaryKey().defaultRandom(),
companyId: uuid("company_id").notNull().references(() => companies.id, { onDelete: "cascade" }),
routineId: uuid("routine_id").notNull().references(() => routines.id, { onDelete: "cascade" }),
kind: text("kind").notNull(),
label: text("label"),
enabled: boolean("enabled").notNull().default(true),
cronExpression: text("cron_expression"),
timezone: text("timezone"),
nextRunAt: timestamp("next_run_at", { withTimezone: true }),
lastFiredAt: timestamp("last_fired_at", { withTimezone: true }),
publicId: text("public_id"),
secretId: uuid("secret_id").references(() => companySecrets.id, { onDelete: "set null" }),
signingMode: text("signing_mode"),
replayWindowSec: integer("replay_window_sec"),
lastRotatedAt: timestamp("last_rotated_at", { withTimezone: true }),
lastResult: text("last_result"),
createdByAgentId: uuid("created_by_agent_id").references(() => agents.id, { onDelete: "set null" }),
createdByUserId: text("created_by_user_id"),
updatedByAgentId: uuid("updated_by_agent_id").references(() => agents.id, { onDelete: "set null" }),
updatedByUserId: text("updated_by_user_id"),
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
updatedAt: timestamp("updated_at", { withTimezone: true }).notNull().defaultNow(),
},
(table) => ({
companyRoutineIdx: index("routine_triggers_company_routine_idx").on(table.companyId, table.routineId),
companyKindIdx: index("routine_triggers_company_kind_idx").on(table.companyId, table.kind),
nextRunIdx: index("routine_triggers_next_run_idx").on(table.nextRunAt),
publicIdIdx: index("routine_triggers_public_id_idx").on(table.publicId),
publicIdUq: uniqueIndex("routine_triggers_public_id_uq").on(table.publicId),
}),
);
export const routineRuns = pgTable(
"routine_runs",
{
id: uuid("id").primaryKey().defaultRandom(),
companyId: uuid("company_id").notNull().references(() => companies.id, { onDelete: "cascade" }),
routineId: uuid("routine_id").notNull().references(() => routines.id, { onDelete: "cascade" }),
triggerId: uuid("trigger_id").references(() => routineTriggers.id, { onDelete: "set null" }),
source: text("source").notNull(),
status: text("status").notNull().default("received"),
triggeredAt: timestamp("triggered_at", { withTimezone: true }).notNull().defaultNow(),
routineRevisionId: uuid("routine_revision_id").references(() => routineRevisions.id, { onDelete: "set null" }),
idempotencyKey: text("idempotency_key"),
triggerPayload: jsonb("trigger_payload").$type<Record<string, unknown>>(),
dispatchFingerprint: text("dispatch_fingerprint"),
linkedIssueId: uuid("linked_issue_id").references(() => issues.id, { onDelete: "set null" }),
coalescedIntoRunId: uuid("coalesced_into_run_id"),
failureReason: text("failure_reason"),
completedAt: timestamp("completed_at", { withTimezone: true }),
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
updatedAt: timestamp("updated_at", { withTimezone: true }).notNull().defaultNow(),
},
(table) => ({
companyRoutineIdx: index("routine_runs_company_routine_idx").on(table.companyId, table.routineId, table.createdAt),
routineRevisionIdx: index("routine_runs_revision_idx").on(table.routineRevisionId),
triggerIdx: index("routine_runs_trigger_idx").on(table.triggerId, table.createdAt),
dispatchFingerprintIdx: index("routine_runs_dispatch_fingerprint_idx").on(table.routineId, table.dispatchFingerprint),
linkedIssueIdx: index("routine_runs_linked_issue_idx").on(table.linkedIssueId),
idempotencyIdx: index("routine_runs_trigger_idempotency_idx").on(table.triggerId, table.idempotencyKey),
}),
);