Compare commits

..

27 Commits

Author SHA1 Message Date
Barcode Betty 01ed6dac00 fix(deps): pin safe versions of audit-flagged transitive deps (CAR-1162 audit)
The CI's npm audit (10.8.2) flagged three transitive vulnerabilities
that local newer-npm runs (11.x) miss due to advisory-DB divergence:

- @babel/plugin-transform-modules-systemjs: 7.29.0 -> ^7.29.4
  (CVE-2026-44728: arbitrary code generation, fixed in 7.29.4)
- fast-uri: 3.1.0 -> ^3.1.2
  (path traversal / host confusion via percent-encoded segments)
- brace-expansion: 5.0.5 -> >=5.0.6
  (DoS via large numeric range defeating max protection)

These are non-breaking transitive updates within the same major
version. The previous override for brace-expansion (>=1.1.13) was
too loose to exclude 5.0.2-5.0.5; tightening it to >=5.0.6.

Ref CAR-1162, CAR-1122, CAR-1078

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-06-03 15:53:46 +00:00
Barcode Betty a7a55bbf79 fix(ci): unblock dev PR #271 CI
- Remove .mcp.json (scope creep, unrelated to CAR-1078)
- Bump vitest to ^4.1.8 (fixes GHSA-5xrq-8626-4rwp critical)
- Run npm audit fix for non-breaking vulns
- Pin actions/checkout and actions/setup-node to commit SHAs
  in .gitea/workflows/ci.yml to force a clean cache fetch on
  the act runner (workaround for corrupted /root/.cache/act cache)

Refs CAR-1162, CAR-1122, CAR-1078
2026-06-03 11:41:19 +00:00
Flea Flicker fb0bb0102c fix(receiptwitness): pool DB engine and Redis client to prevent connection exhaustion
email_worker calls get_async_session_factory() inside every resolve_user()
call, which creates a brand-new async engine (and thus a brand-new
connection pool) on every message.  In a tight consumer loop processing
5 messages per batch, this rapidly exhausts DragonflyDB/Postgres
connection limits and manifests as ConnectionResetError.

Fix: cache the async engine in a module-level dict keyed by URL in
cartsnitch_common.database:get_async_engine(), matching the pattern
already used in receiptwitness:events.py for the Redis connection pool.
Also add pool_size=10, max_overflow=20, pool_pre_ping=True for
健壮连接管理.

Similarly, receiptwitness/queue/email.py:get_redis() was creating a new
Redis connection on every call with no pooling.  Share a
ConnectionPool (max_connections=30) across all get_redis() callers.

Fixes CAR-1078
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-28 18:53:05 +00:00
Chris Farhood d90b00d7ac Add .mcp.json 2026-05-25 21:47:10 +00:00
Savannah Savings 8983fe5d8f Merge pull request 'Promote to Production: CAR-894 Gitea workflows migration' (#270) from uat into main 2026-05-24 18:51:41 +00:00
Savannah Savings a26082d099 Merge pull request 'Promote dev → uat: Fix API crash (dispose_engine import)' (#268) from dev into uat
Merge PR #268: Promote dev → uat — Fix API crash (dispose_engine import)

Promotes fix for ImportError/CrashLoopBackOff to UAT environment.

Approved-by: Savannah Savings (CTO)
2026-05-23 15:52:56 +00:00
Savannah Savings f8b8f4feef Merge pull request 'Fix API crash: remove dead dispose_engine import' (#266) from fix/dispose-engine-import into dev
Merge PR #266: Fix API crash — remove dead dispose_engine import

Removes non-existent dispose_engine import from main.py that caused ImportError and CrashLoopBackOff on API pods.

Reviewed-by: Checkout Charlie (QA PASS)
Approved-by: Savannah Savings (CTO)
2026-05-23 15:52:33 +00:00
Flea Flicker 5464e1a671 Fix API crash: remove dead dispose_engine import
ImportError: cannot import name 'dispose_engine' from 'cartsnitch_api.database'
The function does not exist and is unused - lifespan already calls close_db() directly.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-23 15:11:00 +00:00
Savannah Savings c39b26050b Merge pull request 'Promote dev → uat: CI registry migration [CAR-933]' (#265) from dev into uat
Promote dev → uat: CI registry migration [CAR-933] (#265)
2026-05-23 14:39:41 +00:00
Savannah Savings 9cc1e49d86 Merge pull request 'ci(CAR-933): migrate image registry from ghcr.io to Gitea' (#264) from carl/car-933-gitea-registry into dev
Merge PR #264: Migrate CI from ghcr.io to Gitea package registry (CAR-955)

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-23 14:35:15 +00:00
Coupon Carl 2c4e9985b1 ci: rename GITEA_TOKEN -> REGISTRY_TOKEN to match configured secret name
cpfarhood confirmed the Gitea registry token is configured as REGISTRY_TOKEN
(not GITEA_TOKEN). This applies to both the registry docker login steps
and the infra repo checkout steps in deploy-dev/deploy-uat.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-23 14:29:45 +00:00
cartsnitch-ci[bot] 821f1d20b3 fix(ci): replace hardcoded cs_carl username and fix kustomize image rename syntax
- Replace hardcoded 'cs_carl' Gitea registry username with '${{ github.actor }}' in all 5 login steps
- Use kustomize 'OLD=NEW:TAG' rename syntax so existing ghcr.io image entries are updated instead of duplicated

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-23 14:27:24 +00:00
Coupon Carl 555ced4fdc ci: migrate image registry from ghcr.io to git.farh.net
Replace GitHub Container Registry with Gitea's built-in container registry.
- REGISTRY env var: ghcr.io -> git.farh.net
- All 4 build-and-push jobs: replace GHCR login with Gitea registry login
  using cs_carl + GITEA_TOKEN (token already required for infra checkout)
- deploy-dev/deploy-uat: update kustomize image refs to git.farh.net/*
- Also fix legacy api/.gitea/workflows/ci.yml (non-executing nested file)

Required secrets drop from 5 to 3: DOCKERHUB_USERNAME, DOCKERHUB_TOKEN,
GITEA_TOKEN. GHCR_USERNAME and GHCR_TOKEN no longer needed.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-23 14:20:38 +00:00
Savannah Savings 6b6a50b9ec Merge pull request 'Promote dev → uat: .gitea/workflows migration [CAR-934]' (#261) from dev into uat
Promote dev → uat: .gitea/workflows migration [CAR-934]

cc @cpfarhood
2026-05-21 19:19:40 +00:00
Savannah Savings 4797f07af9 Merge pull request 'ci: move .github/workflows to .gitea/workflows [CAR-900]' (#259) from barcode-betty/move-workflows-to-gitea into dev
Merge PR #259: ci: move .github/workflows to .gitea/workflows [CAR-897]

QA-approved. CTO merge to dev.

cc @cpfarhood
2026-05-21 19:19:20 +00:00
Flea Flicker 96331c9fa7 Move .github/workflows to .gitea/workflows
- Relocate all CI workflows from .github/workflows/ to .gitea/workflows/
- Root: .github/workflows/ci.yml -> .gitea/workflows/ci.yml
- api/: api/.github/workflows/ci.yml -> api/.gitea/workflows/ci.yml
- common/: common/.github/workflows/ci.yml -> common/.gitea/workflows/ci.yml
- Gitea uses .gitea/workflows/ for CI configuration

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-21 11:59:35 +00:00
Savannah Savings a4e0b664e1 Merge pull request 'fix: use GHCR_USERNAME and GHCR_TOKEN for GHCR login in Gitea Actions' (#258) from betty/fix-gitea-ci-secrets into dev 2026-05-21 06:21:55 +00:00
Flea Flicker f4bbddd0dd fix: use GHCR_USERNAME and GHCR_TOKEN for GHCR login in Gitea Actions
Gitea's GITHUB_TOKEN authenticates against git.farh.net, not ghcr.io.
Use explicit GHCR_USERNAME and GHCR_TOKEN secrets instead.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-21 05:25:36 +00:00
Savannah Savings 7c021c4eb5 Merge pull request 'chore: promote dev to uat - Gitea Actions workflow conversion' (#254) from dev into uat 2026-05-21 04:23:11 +00:00
Savannah Savings 5a97290356 Merge pull request 'ci: convert GitHub Actions to Gitea Actions (ubuntu-latest)' (#253) from betty/car-869-gitea-actions-cartsnitch into dev 2026-05-21 04:22:43 +00:00
Flea Flicker 32495b150b ci: convert GitHub Actions to Gitea Actions (ubuntu-latest)
- Replace all runs-on: runners-cartsnitch with ubuntu-latest
- Remove SARIF upload steps (no Gitea Security tab)
- Replace GitHub App token with secrets.GITEA_TOKEN in deploy-dev and deploy-uat

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-21 03:54:18 +00:00
savannah-savings-cto[bot] a5404dc824 promote: dev → uat (fix auth tsc build) (#252)
promote: dev → uat (fix auth tsc build)
2026-05-05 11:19:44 +00:00
savannah-savings-cto[bot] b39280ee2a fix(auth): exclude test files from tsc build (#251)
fix(auth): exclude test files from tsc build
2026-05-05 11:19:19 +00:00
Chris Farhood 752d7ed3d0 fix(auth): exclude test files from tsc compilation
Exclude src/__tests__ from tsconfig to prevent test files from being
compiled during Docker build. Fixes build-and-push-auth CI failure.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-05-05 11:11:53 +00:00
savannah-savings-cto[bot] 618da593a6 Merge pull request #250 from cartsnitch/dev
ci: promote dev → uat (auth CI pipeline)
2026-05-05 10:56:35 +00:00
savannah-savings-cto[bot] 1f317a0616 Merge pull request #249 from cartsnitch/betty/car-843-auth-ci-pipeline
ci: add auth service build/deploy to CI pipeline
2026-05-05 10:56:19 +00:00
Chris Farhood 912239a97b ci: add auth service build/deploy to CI pipeline
Add build-and-push-auth job and update deploy-dev/uat to include auth image.

- Add AUTH_IMAGE_NAME env var
- Add build-and-push-auth job (modeled on build-and-push-api)
- Add build-and-push-auth to deploy-dev and deploy-uat needs
- Add auth image tag determination and update steps in both deploy jobs
- Update commit messages to include auth
2026-05-05 06:44:15 +00:00
9 changed files with 513 additions and 480 deletions
@@ -16,17 +16,18 @@ permissions:
security-events: write
env:
REGISTRY: ghcr.io
REGISTRY: git.farh.net
IMAGE_NAME: cartsnitch/cartsnitch
RECEIPTWITNESS_IMAGE_NAME: cartsnitch/receiptwitness
API_IMAGE_NAME: cartsnitch/api
AUTH_IMAGE_NAME: cartsnitch/auth
jobs:
lint:
runs-on: runners-cartsnitch
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with:
node-version: "20"
cache: npm
@@ -37,10 +38,10 @@ jobs:
run: npx tsc --noEmit
test:
runs-on: runners-cartsnitch
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with:
node-version: "20"
cache: npm
@@ -49,10 +50,10 @@ jobs:
run: npx vitest run
audit:
runs-on: runners-cartsnitch
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with:
node-version: "20"
cache: npm
@@ -61,10 +62,10 @@ jobs:
run: npm audit --audit-level=high
e2e:
runs-on: runners-cartsnitch
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with:
node-version: "20"
cache: npm
@@ -73,11 +74,11 @@ jobs:
- run: npx playwright test
lighthouse:
runs-on: runners-cartsnitch
runs-on: ubuntu-latest
needs: [test]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
with:
node-version: "20"
cache: npm
@@ -98,14 +99,14 @@ jobs:
CHROME_PATH="$CHROME_PATH" lhci autorun --chrome-flags="--headless=new --no-sandbox --disable-gpu --disable-dev-shm-usage"
build-and-push:
runs-on: runners-cartsnitch
runs-on: ubuntu-latest
if: github.event_name == 'push'
needs: [lint, test, e2e]
outputs:
calver_tag: ${{ steps.calver.outputs.version }}
sha_tag: sha-${{ github.sha }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
@@ -133,13 +134,13 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Log in to GHCR
- name: Log in to Gitea registry
if: github.event_name == 'push'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Extract metadata
id: meta
@@ -174,11 +175,7 @@ jobs:
only-fixed: "true"
output-format: sarif
- name: Upload frontend scan results to GitHub Security
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: ${{ steps.scan.outputs.sarif }}
- name: Push Docker image
if: github.event_name == 'push'
@@ -198,14 +195,14 @@ jobs:
git push origin "v${{ steps.calver.outputs.version }}"
build-and-push-receiptwitness:
runs-on: runners-cartsnitch
runs-on: ubuntu-latest
if: github.event_name == 'push'
needs: [lint, test]
outputs:
calver_tag: ${{ steps.calver.outputs.version }}
sha_tag: sha-${{ github.sha }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
@@ -227,13 +224,13 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Log in to GHCR
- name: Log in to Gitea registry
if: github.event_name == 'push'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Extract metadata
id: meta
@@ -270,11 +267,7 @@ jobs:
only-fixed: "true"
output-format: sarif
- name: Upload receiptwitness scan results to GitHub Security
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: ${{ steps.scan.outputs.sarif }}
- name: Push Docker image
if: github.event_name == 'push'
@@ -290,14 +283,14 @@ jobs:
cache-from: type=gha
build-and-push-api:
runs-on: runners-cartsnitch
runs-on: ubuntu-latest
if: github.event_name == 'push'
needs: [lint, test]
outputs:
calver_tag: ${{ steps.calver.outputs.version }}
sha_tag: sha-${{ github.sha }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
@@ -319,13 +312,13 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Log in to GHCR
- name: Log in to Gitea registry
if: github.event_name == 'push'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Extract metadata (API)
id: meta
@@ -362,11 +355,7 @@ jobs:
only-fixed: "true"
output-format: sarif
- name: Upload api scan results to GitHub Security
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: ${{ steps.scan.outputs.sarif }}
- name: Push Docker image
if: github.event_name == 'push'
@@ -381,25 +370,104 @@ jobs:
APT_CACHE_BUST=${{ github.run_id }}
cache-from: type=gha
build-and-push-auth:
runs-on: ubuntu-latest
if: github.event_name == 'push'
needs: [lint, test]
outputs:
calver_tag: ${{ steps.calver.outputs.version }}
sha_tag: sha-${{ github.sha }}
steps:
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
fetch-depth: 0
- name: Generate CalVer tag
id: calver
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
run: |
DATE_TAG=$(date -u +%Y.%m.%d)
EXISTING=$(git tag -l "v${DATE_TAG}*" | sort -V | tail -1)
if [ -z "$EXISTING" ]; then VERSION="$DATE_TAG"
elif [ "$EXISTING" = "v${DATE_TAG}" ]; then VERSION="${DATE_TAG}.2"
else BUILD_NUM=$(echo "$EXISTING" | sed "s/v${DATE_TAG}\.//"); VERSION="${DATE_TAG}.$((BUILD_NUM + 1))"; fi
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
- name: Log in to Docker Hub
if: github.event_name == 'push'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Log in to Gitea registry
if: github.event_name == 'push'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Extract metadata (auth)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.AUTH_IMAGE_NAME }}
tags: |
type=sha,prefix=sha-,format=long
type=raw,value=${{ steps.calver.outputs.version }},enable=${{ github.ref == 'refs/heads/main' }}
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' }}
- name: Build Docker image
uses: docker/build-push-action@v6
with:
context: ./auth
file: ./auth/Dockerfile
load: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-args: |
APT_CACHE_BUST=${{ github.run_id }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Scan auth image for vulnerabilities
uses: anchore/scan-action@v5
id: scan
env:
GRYPE_CONFIG: .grype.yaml
with:
image: "${{ env.REGISTRY }}/${{ env.AUTH_IMAGE_NAME }}:sha-${{ github.sha }}"
fail-build: true
severity-cutoff: high
only-fixed: "true"
output-format: sarif
- name: Push Docker image
if: github.event_name == 'push'
uses: docker/build-push-action@v6
with:
context: ./auth
file: ./auth/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-args: |
APT_CACHE_BUST=${{ github.run_id }}
cache-from: type=gha
deploy-dev:
runs-on: runners-cartsnitch
needs: [build-and-push, build-and-push-receiptwitness, build-and-push-api]
runs-on: ubuntu-latest
needs: [build-and-push, build-and-push-receiptwitness, build-and-push-api, build-and-push-auth]
if: always() && !cancelled() && github.event_name == 'push' && (github.ref == 'refs/heads/dev' || github.ref == 'refs/heads/main')
steps:
- name: Generate GitHub App token
id: app-token
uses: actions/create-github-app-token@v1
with:
app-id: ${{ secrets.CARTSNITCH_APP_ID }}
private-key: ${{ secrets.CARTSNITCH_APP_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: infra
- name: Checkout infra repo
uses: actions/checkout@v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
repository: cartsnitch/infra
token: ${{ steps.app-token.outputs.token }}
token: ${{ secrets.REGISTRY_TOKEN }}
ref: main
path: infra
@@ -422,7 +490,7 @@ jobs:
if: needs.build-and-push.result == 'success'
run: |
cd infra/apps/overlays/dev
kustomize edit set image ghcr.io/cartsnitch/cartsnitch:${{ steps.frontend_tag.outputs.tag }}
kustomize edit set image ghcr.io/cartsnitch/cartsnitch=git.farh.net/cartsnitch/cartsnitch:${{ steps.frontend_tag.outputs.tag }}
- name: Determine image tag for receiptwitness
id: receiptwitness_tag
@@ -437,7 +505,7 @@ jobs:
if: needs.build-and-push-receiptwitness.result == 'success'
run: |
cd infra/apps/overlays/dev
kustomize edit set image ghcr.io/cartsnitch/receiptwitness:${{ steps.receiptwitness_tag.outputs.tag }}
kustomize edit set image ghcr.io/cartsnitch/receiptwitness=git.farh.net/cartsnitch/receiptwitness:${{ steps.receiptwitness_tag.outputs.tag }}
- name: Determine image tag for api
id: api_tag
@@ -452,38 +520,44 @@ jobs:
if: needs.build-and-push-api.result == 'success'
run: |
cd infra/apps/overlays/dev
kustomize edit set image ghcr.io/cartsnitch/api:${{ steps.api_tag.outputs.tag }}
kustomize edit set image ghcr.io/cartsnitch/api=git.farh.net/cartsnitch/api:${{ steps.api_tag.outputs.tag }}
- name: Determine image tag for auth
id: auth_tag
run: |
if [ "${{ github.ref }}" == "refs/heads/main" ]; then
echo "tag=${{ needs.build-and-push-auth.outputs.calver_tag }}" >> "$GITHUB_OUTPUT"
else
echo "tag=${{ needs.build-and-push-auth.outputs.sha_tag }}" >> "$GITHUB_OUTPUT"
fi
- name: Update auth image tag
if: needs.build-and-push-auth.result == 'success'
run: |
cd infra/apps/overlays/dev
kustomize edit set image ghcr.io/cartsnitch/auth=git.farh.net/cartsnitch/auth:${{ steps.auth_tag.outputs.tag }}
- name: Commit and push to infra
run: |
cd infra
git config user.name "cartsnitch-ci[bot]"
git config user.email "cartsnitch-ci[bot]@users.noreply.github.com"
git config user.email "cartsnitch-ci[bot]@users.noreply.git.farh.net"
git add apps/overlays/dev/kustomization.yaml
git diff --cached --quiet && echo "No image changes to deploy" && exit 0
git commit -m "ci(dev): update cartsnitch, receiptwitness, and api images"
git commit -m "ci(dev): update cartsnitch, receiptwitness, api, and auth images"
git pull --rebase origin main
git push origin main
deploy-uat:
runs-on: runners-cartsnitch
needs: [build-and-push, build-and-push-receiptwitness, build-and-push-api]
runs-on: ubuntu-latest
needs: [build-and-push, build-and-push-receiptwitness, build-and-push-api, build-and-push-auth]
if: always() && !cancelled() && github.event_name == 'push' && (github.ref == 'refs/heads/uat' || github.ref == 'refs/heads/main')
steps:
- name: Generate GitHub App token
id: app-token
uses: actions/create-github-app-token@v1
with:
app-id: ${{ secrets.CARTSNITCH_APP_ID }}
private-key: ${{ secrets.CARTSNITCH_APP_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: infra
- name: Checkout infra repo
uses: actions/checkout@v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
with:
repository: cartsnitch/infra
token: ${{ steps.app-token.outputs.token }}
token: ${{ secrets.REGISTRY_TOKEN }}
ref: main
path: infra
@@ -506,7 +580,7 @@ jobs:
if: needs.build-and-push.result == 'success'
run: |
cd infra/apps/overlays/uat
kustomize edit set image ghcr.io/cartsnitch/cartsnitch:${{ steps.frontend_tag.outputs.tag }}
kustomize edit set image ghcr.io/cartsnitch/cartsnitch=git.farh.net/cartsnitch/cartsnitch:${{ steps.frontend_tag.outputs.tag }}
- name: Determine image tag for receiptwitness
id: receiptwitness_tag
@@ -521,7 +595,7 @@ jobs:
if: needs.build-and-push-receiptwitness.result == 'success'
run: |
cd infra/apps/overlays/uat
kustomize edit set image ghcr.io/cartsnitch/receiptwitness:${{ steps.receiptwitness_tag.outputs.tag }}
kustomize edit set image ghcr.io/cartsnitch/receiptwitness=git.farh.net/cartsnitch/receiptwitness:${{ steps.receiptwitness_tag.outputs.tag }}
- name: Determine image tag for api
id: api_tag
@@ -536,15 +610,30 @@ jobs:
if: needs.build-and-push-api.result == 'success'
run: |
cd infra/apps/overlays/uat
kustomize edit set image ghcr.io/cartsnitch/api:${{ steps.api_tag.outputs.tag }}
kustomize edit set image ghcr.io/cartsnitch/api=git.farh.net/cartsnitch/api:${{ steps.api_tag.outputs.tag }}
- name: Determine image tag for auth
id: auth_tag
run: |
if [ "${{ github.ref }}" == "refs/heads/main" ]; then
echo "tag=${{ needs.build-and-push-auth.outputs.calver_tag }}" >> "$GITHUB_OUTPUT"
else
echo "tag=${{ needs.build-and-push-auth.outputs.sha_tag }}" >> "$GITHUB_OUTPUT"
fi
- name: Update auth image tag
if: needs.build-and-push-auth.result == 'success'
run: |
cd infra/apps/overlays/uat
kustomize edit set image ghcr.io/cartsnitch/auth=git.farh.net/cartsnitch/auth:${{ steps.auth_tag.outputs.tag }}
- name: Commit and push to infra
run: |
cd infra
git config user.name "cartsnitch-ci[bot]"
git config user.email "cartsnitch-ci[bot]@users.noreply.github.com"
git config user.email "cartsnitch-ci[bot]@users.noreply.git.farh.net"
git add apps/overlays/uat/kustomization.yaml
git diff --cached --quiet && echo "No image changes to deploy" && exit 0
git commit -m "ci(uat): update cartsnitch, receiptwitness, and api images"
git commit -m "ci(uat): update cartsnitch, receiptwitness, api, and auth images"
git pull --rebase origin main
git push origin main
@@ -15,7 +15,7 @@ permissions:
packages: write
env:
REGISTRY: ghcr.io
REGISTRY: git.farh.net
IMAGE_NAME: cartsnitch/api
jobs:
@@ -130,13 +130,13 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Log in to GHCR
- name: Log in to Gitea registry
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Extract metadata
id: meta
-1
View File
@@ -6,7 +6,6 @@ from fastapi import APIRouter, FastAPI
from cartsnitch_api.auth.routes import router as auth_router
from cartsnitch_api.cache import cache_client
from cartsnitch_api.database import dispose_engine
from cartsnitch_api.middleware.cors import add_cors_middleware
from cartsnitch_api.middleware.error_handler import add_error_handlers, add_error_monitor_middleware
from cartsnitch_api.middleware.rate_limit import add_rate_limit_middleware
+1 -1
View File
@@ -12,5 +12,5 @@
"resolveJsonModule": true
},
"include": ["src"],
"exclude": ["node_modules", "dist"]
"exclude": ["node_modules", "dist", "src/__tests__"]
}
+23 -4
View File
@@ -1,17 +1,36 @@
"""Database engine and session factories for sync and async usage."""
from collections.abc import AsyncGenerator, Generator
from typing import TYPE_CHECKING
from sqlalchemy import create_engine
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from sqlalchemy.ext.asyncio import AsyncEngine, AsyncSession, async_sessionmaker, create_async_engine
from sqlalchemy.orm import Session, sessionmaker
from cartsnitch_common.config import settings
if TYPE_CHECKING:
from sqlalchemy.engine import Engine
def get_async_engine(url: str | None = None):
"""Create an async SQLAlchemy engine."""
return create_async_engine(url or settings.database_url, echo=settings.debug)
# Module-level async engine cache — one engine per unique URL, shared across all callers.
# This prevents pool exhaustion in high-throughput workers (e.g. email-worker hitting
# DragonflyDB/Postgres repeatedly per message). pool_size=10, max_overflow=20 gives
# headroom for bursts while capping max connections at 30 per URL.
_async_engine_cache: dict[str, "AsyncEngine"] = {}
def get_async_engine(url: str | None = None) -> "AsyncEngine":
"""Get or create a cached async engine for the given URL."""
target = url or settings.database_url
if target not in _async_engine_cache:
_async_engine_cache[target] = create_async_engine(
target,
echo=settings.debug,
pool_size=10,
max_overflow=20,
pool_pre_ping=True,
)
return _async_engine_cache[target]
def get_sync_engine(url: str | None = None):
+290 -384
View File
File diff suppressed because it is too large Load Diff
+5 -3
View File
@@ -45,14 +45,16 @@
"typescript-eslint": "^8.56.1",
"vite": "^6.4.2",
"vite-plugin-pwa": "^0.21.2",
"vitest": "^3.2.4"
"vitest": "^4.1.8"
},
"overrides": {
"@rollup/pluginutils": "5.3.0",
"flatted": "^3.4.2",
"serialize-javascript": "7.0.5",
"brace-expansion": ">=1.1.13",
"brace-expansion": ">=5.0.6",
"lodash": ">=4.17.24",
"minimatch": "^10.2.4"
"minimatch": "^10.2.4",
"@babel/plugin-transform-modules-systemjs": "^7.29.4",
"fast-uri": "^3.1.2"
}
}
@@ -16,6 +16,29 @@ logger = logging.getLogger(__name__)
STREAM_KEY = "email:receipts"
CONSUMER_GROUP = "email-workers"
# Module-level Redis/DragonflyDB connection pool — shared across all worker calls.
# Without pooling, each call to get_redis() opens a new TCP connection. In a tight
# consumer loop this causes ConnectionResetError when DragonflyDB's connection limit
# is hit under load. max_connections=30 (10 base + 20 overflow) mirrors the engine pool.
_redis_pool: aioredis.ConnectionPool | None = None
def _get_redis_pool() -> aioredis.ConnectionPool:
"""Get or create the shared DragonflyDB connection pool."""
global _redis_pool
if _redis_pool is None:
_redis_pool = aioredis.ConnectionPool.from_url(
settings.redis_url,
decode_responses=True,
max_connections=30,
)
return _redis_pool
async def get_redis() -> aioredis.Redis:
"""Get async Redis/DragonflyDB client backed by a shared connection pool."""
return aioredis.Redis(connection_pool=_get_redis_pool())
@dataclass
class EmailJob:
@@ -31,11 +54,6 @@ class EmailJob:
message_id: str # from email provider, for dedup
async def get_redis() -> aioredis.Redis:
"""Get async Redis/DragonflyDB client."""
return cast(aioredis.Redis, aioredis.from_url(settings.redis_url, decode_responses=True))
async def ensure_consumer_group(client: aioredis.Redis) -> None:
"""Create consumer group if it does not exist."""
try: