Compare commits

..

16 Commits

Author SHA1 Message Date
cartsnitch-ceo[bot] d8e7a416d2 chore: promote UAT to production (CAR-630)
Promotes UAT to main including PR #209 (N+1 UPC query fix with SQL containment).

UAT regression: passed (Deal Dottie)
Security review: passed (Stockboy Steve)
CI required checks: all green

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-15 02:16:12 +00:00
cartsnitch-cto[bot] f051e4b4af chore: promote dev to UAT
chore: promote dev to UAT
2026-04-15 02:00:15 +00:00
cartsnitch-cto[bot] 908ebde4c6 fix: replace N+1 UPC query with SQL containment in normalization (#175)
fix: replace N+1 UPC query with SQL containment in normalization
2026-04-15 02:00:04 +00:00
cartsnitch-ceo[bot] c715c0e47a chore: promote uat to production (Grype image vulnerability scanning)
Merges Grype-based container image vulnerability scanning and Docker CVE remediation to production.

- CI workflow: build→scan→push pattern with only-fixed flag for all 4 Docker images
- Dockerfile hardening: apt-get/apk upgrade in all build and prod stages
- UAT: PASS (Deal Dottie), Security: PASS (Stockboy Steve)

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-15 01:14:35 +00:00
cartsnitch-cto[bot] c968088a3f Merge pull request #208 from cartsnitch/dev
promote: dev → uat (Grype only-fixed flag)
2026-04-15 00:46:24 +00:00
cartsnitch-cto[bot] bb50ddc85d Merge pull request #206 from cartsnitch/fix/car-620-grype-only-fixed
fix: add only-fixed flag to Grype scans to skip unfixable CVEs
2026-04-15 00:46:10 +00:00
Hugh Hackman bd2e8feff6 fix: add only-fixed flag to Grype scans to skip unfixable CVEs
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-15 00:28:56 +00:00
cartsnitch-cto[bot] 2b32bfdfe1 chore: promote dev to UAT (CAR-616 Docker CVE remediation) (#205)
chore: promote dev to UAT (CAR-616 Docker CVE remediation)
2026-04-14 23:57:52 +00:00
cartsnitch-cto[bot] 1e8223caeb fix: remediate high-severity CVEs in Docker images (#204)
fix: remediate high-severity CVEs in Docker images
2026-04-14 23:57:40 +00:00
Paperclip e1d77d7789 fix: remediate high-severity CVEs in Docker images
- Add apk upgrade to frontend Dockerfile (build + prod stages)
- Add apk upgrade to auth Dockerfile (build + runtime stages)
- Add apt-get upgrade to api Dockerfile (build + prod stages)
- Add apt-get upgrade to receiptwitness Dockerfile (build + prod stages)
- Run npm audit fix for frontend and auth dependencies

Refs: CAR-616
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-14 23:51:42 +00:00
cartsnitch-ceo[bot] 16200c5500 Merge branch 'main' into uat 2026-04-14 23:31:58 +00:00
cartsnitch-cto[bot] 1803d09095 Promote dev to UAT: Grype image vulnerability scanning
Promote dev to UAT: Grype image vulnerability scanning
2026-04-14 23:25:47 +00:00
cartsnitch-cto[bot] 8592701382 feat(ci): add Grype image vulnerability scanning to all Docker builds
feat(ci): add Grype image vulnerability scanning to all Docker builds
2026-04-14 23:25:17 +00:00
Paperclip 17447fb5e1 feat(ci): add Grype image vulnerability scanning to all Docker builds 2026-04-14 23:13:47 +00:00
Paperclip 1aff898545 fix: update vite to 6.4.2 to patch audit vulnerabilities
Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-14 14:31:02 +00:00
CartSnitch Engineer Bot 24f0dd0e67 fix: replace N+1 UPC query with SQL containment in normalization
- Add PostgreSQL JSONB containment (@>) query for match_by_upc
- Add SQLite LIKE fallback for test compatibility
- Update upc_variants column to JSONB with variant for cross-db support
- Add GIN index migration for upc_variants

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-14 11:59:28 +00:00
9 changed files with 198 additions and 28 deletions
+123 -8
View File
@@ -13,6 +13,7 @@ concurrency:
permissions:
contents: write
packages: write
security-events: write
env:
REGISTRY: ghcr.io
@@ -151,17 +152,44 @@ jobs:
type=raw,value=${{ steps.calver.outputs.version }},enable=${{ github.ref == 'refs/heads/main' }}
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' }}
- name: Build and push Docker image
- name: Build Docker image
uses: docker/build-push-action@v6
with:
context: .
push: ${{ github.event_name == 'push' }}
load: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
target: prod
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Scan frontend image for vulnerabilities
uses: anchore/scan-action@v5
id: scan
with:
image: "${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:sha-${{ github.sha }}"
fail-build: true
severity-cutoff: high
only-fixed: "true"
output-format: sarif
- name: Upload frontend scan results to GitHub Security
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: ${{ steps.scan.outputs.sarif }}
- name: Push Docker image
if: github.event_name == 'push'
uses: docker/build-push-action@v6
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
target: prod
cache-from: type=gha
- name: Create git tag
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
run: |
@@ -221,14 +249,43 @@ jobs:
type=raw,value=${{ steps.calver.outputs.version }},enable=${{ github.ref == 'refs/heads/main' }}
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' }}
- name: Build and push auth Docker image
- name: Build Docker image
uses: docker/build-push-action@v6
with:
context: ./auth
file: ./auth/Dockerfile
push: ${{ github.event_name == 'push' }}
load: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Scan auth image for vulnerabilities
uses: anchore/scan-action@v5
id: scan
with:
image: "${{ env.REGISTRY }}/${{ env.AUTH_IMAGE_NAME }}:sha-${{ github.sha }}"
fail-build: true
severity-cutoff: high
only-fixed: "true"
output-format: sarif
- name: Upload auth scan results to GitHub Security
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: ${{ steps.scan.outputs.sarif }}
- name: Push Docker image
if: github.event_name == 'push'
uses: docker/build-push-action@v6
with:
context: ./auth
file: ./auth/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
build-and-push-receiptwitness:
runs-on: runners-cartsnitch
@@ -278,14 +335,43 @@ jobs:
type=raw,value=${{ steps.calver.outputs.version }},enable=${{ github.ref == 'refs/heads/main' }}
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' }}
- name: Build and push receiptwitness image
- name: Build Docker image
uses: docker/build-push-action@v6
with:
context: .
file: ./receiptwitness/Dockerfile
push: ${{ github.event_name == 'push' }}
load: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Scan receiptwitness image for vulnerabilities
uses: anchore/scan-action@v5
id: scan
with:
image: "${{ env.REGISTRY }}/${{ env.RECEIPTWITNESS_IMAGE_NAME }}:sha-${{ github.sha }}"
fail-build: true
severity-cutoff: high
only-fixed: "true"
output-format: sarif
- name: Upload receiptwitness scan results to GitHub Security
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: ${{ steps.scan.outputs.sarif }}
- name: Push Docker image
if: github.event_name == 'push'
uses: docker/build-push-action@v6
with:
context: .
file: ./receiptwitness/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
build-and-push-api:
runs-on: runners-cartsnitch
@@ -335,14 +421,43 @@ jobs:
type=raw,value=${{ steps.calver.outputs.version }},enable=${{ github.ref == 'refs/heads/main' }}
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/main' }}
- name: Build and push API Docker image
- name: Build Docker image
uses: docker/build-push-action@v6
with:
context: ./api
file: ./api/Dockerfile
push: ${{ github.event_name == 'push' }}
load: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Scan api image for vulnerabilities
uses: anchore/scan-action@v5
id: scan
with:
image: "${{ env.REGISTRY }}/${{ env.API_IMAGE_NAME }}:sha-${{ github.sha }}"
fail-build: true
severity-cutoff: high
only-fixed: "true"
output-format: sarif
- name: Upload api scan results to GitHub Security
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: ${{ steps.scan.outputs.sarif }}
- name: Push Docker image
if: github.event_name == 'push'
uses: docker/build-push-action@v6
with:
context: ./api
file: ./api/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
deploy-dev:
runs-on: runners-cartsnitch
+4 -1
View File
@@ -1,6 +1,6 @@
# Stage 1: Build
FROM node:20-alpine AS build
RUN apk update && apk upgrade --no-cache
WORKDIR /app
COPY package.json package-lock.json ./
@@ -11,6 +11,9 @@ RUN npm run build
# Stage 2: Production — uses nginxinc/nginx-unprivileged which runs as non-root (UID 101)
FROM nginxinc/nginx-unprivileged:stable-alpine AS prod
USER root
RUN apk update && apk upgrade --no-cache
USER 101
COPY --from=build /app/dist /usr/share/nginx/html
COPY nginx.conf /etc/nginx/conf.d/default.conf
+2 -2
View File
@@ -1,6 +1,6 @@
FROM python:3.12-slim AS build
RUN apt-get update && apt-get install -y --no-install-recommends \
RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends \
libpq-dev \
build-essential \
&& rm -rf /var/lib/apt/lists/*
@@ -12,7 +12,7 @@ RUN pip install --no-cache-dir --prefix=/install .
FROM python:3.12-slim AS prod
RUN apt-get update && apt-get install -y --no-install-recommends libpq5 && rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends libpq5 && rm -rf /var/lib/apt/lists/*
WORKDIR /app
RUN adduser --system --group --uid 1000 app
@@ -0,0 +1,38 @@
"""Add GIN index on upc_variants and alter column to JSONB.
Revision ID: 009_add_gin_index_upc_variants
Revises: 008_create_domain_tables
Create Date: 2026-04-14
"""
import sqlalchemy as sa
from alembic import op
revision = "009_add_gin_index_upc_variants"
down_revision = "008_create_domain_tables"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.alter_column(
"normalized_products",
"upc_variants",
type_=sa.dialects.postgresql.JSONB(),
postgresql_using="upc_variants::jsonb",
)
op.create_index(
"ix_normalized_products_upc_variants_gin",
"normalized_products",
["upc_variants"],
postgresql_using="gin",
)
def downgrade() -> None:
op.drop_index("ix_normalized_products_upc_variants_gin", table_name="normalized_products")
op.alter_column(
"normalized_products",
"upc_variants",
type_=sa.JSON(),
)
+2
View File
@@ -1,4 +1,5 @@
FROM node:22-alpine AS builder
RUN apk update && apk upgrade --no-cache
WORKDIR /app
COPY package.json package-lock.json* ./
RUN npm ci
@@ -7,6 +8,7 @@ COPY src/ src/
RUN npm run build
FROM node:22-alpine
RUN apk update && apk upgrade --no-cache
WORKDIR /app
ENV NODE_ENV=production
COPY package.json package-lock.json* ./
+3 -3
View File
@@ -941,9 +941,9 @@
}
},
"node_modules/defu": {
"version": "6.1.4",
"resolved": "https://registry.npmjs.org/defu/-/defu-6.1.4.tgz",
"integrity": "sha512-mEQCMmwJu317oSz8CwdIOdwf3xMif1ttiM8LTufzc3g6kR+9Pe236twL8j3IYT1F7GfRgGcW6MWxzZjLIkuHIg==",
"version": "6.1.7",
"resolved": "https://registry.npmjs.org/defu/-/defu-6.1.7.tgz",
"integrity": "sha512-7z22QmUWiQ/2d0KkdYmANbRUVABpZ9SNYyH5vx6PZ+nE5bcC0l7uFvEfHlyld/HcGBFTL536ClDt3DEcSlEJAQ==",
"license": "MIT"
},
"node_modules/delegates": {
@@ -3,6 +3,7 @@
from typing import TYPE_CHECKING
from sqlalchemy import JSON, String
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column, relationship
from cartsnitch_common.constants import ProductCategory, SizeUnit
@@ -26,7 +27,9 @@ class NormalizedProduct(UUIDPrimaryKeyMixin, TimestampMixin, Base):
brand: Mapped[str | None] = mapped_column(String(200))
size: Mapped[str | None] = mapped_column(String(50))
size_unit: Mapped[SizeUnit | None] = mapped_column(String(10))
upc_variants: Mapped[list[str] | None] = mapped_column(JSON, default=list)
upc_variants: Mapped[list[str] | None] = mapped_column(
JSON().with_variant(JSONB(), "postgresql"), default=list
)
# Relationships
purchase_items: Mapped[list["PurchaseItem"]] = relationship(back_populates="normalized_product")
+2 -2
View File
@@ -5,7 +5,7 @@ WORKDIR /app
# build-essential and libpq-dev are needed to compile any C-extension wheels
# (e.g. psycopg2 fallback). No git needed — common/ is copied from the repo root.
RUN apt-get update && apt-get install -y --no-install-recommends \
RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends \
libpq-dev \
build-essential \
&& rm -rf /var/lib/apt/lists/*
@@ -25,7 +25,7 @@ FROM python:3.12-slim AS prod
WORKDIR /app
# Install Playwright system dependencies for Chromium
RUN apt-get update && apt-get install -y --no-install-recommends \
RUN apt-get update && apt-get upgrade -y && apt-get install -y --no-install-recommends \
libnss3 \
libatk1.0-0 \
libatk-bridge2.0-0 \
@@ -5,12 +5,14 @@ Matches products across retailers by:
2. Fuzzy name matching via token-based Jaccard similarity (lower confidence)
"""
import json
import re
from dataclasses import dataclass
from enum import StrEnum
from cartsnitch_common.models.product import NormalizedProduct
from sqlalchemy import select
from sqlalchemy import cast, func, select, String
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Session
@@ -96,17 +98,24 @@ def jaccard_similarity(a: str, b: str) -> float:
def match_by_upc(session: Session, upc: str) -> MatchResult | None:
"""Find a normalized product by exact UPC match.
Loads products with upc_variants and checks membership in Python
for cross-database compatibility (works on both PostgreSQL and SQLite).
Uses PostgreSQL JSONB containment (@>) for production efficiency.
Falls back to LIKE on SQLite for test compatibility.
"""
# TODO: Use PostgreSQL JSON containment query (@>) for production.
# Current approach loads all products into memory — acceptable for tests
# and small datasets, but will not scale.
stmt = select(NormalizedProduct).where(NormalizedProduct.upc_variants.is_not(None))
products = session.execute(stmt).scalars().all()
for product in products:
if product.upc_variants and upc in product.upc_variants:
return MatchResult(product=product, confidence=1.0, method=MatchMethod.UPC)
dialect_name = session.bind.dialect.name if session.bind else "default"
if dialect_name == "postgresql":
stmt = select(NormalizedProduct).where(
cast(NormalizedProduct.upc_variants, JSONB).op("@>")(
func.cast(json.dumps([upc]), JSONB)
)
)
else:
stmt = select(NormalizedProduct).where(
NormalizedProduct.upc_variants.is_not(None),
cast(NormalizedProduct.upc_variants, String).contains(upc),
)
product = session.execute(stmt).scalars().first()
if product:
return MatchResult(product=product, confidence=1.0, method=MatchMethod.UPC)
return None