3101b43079
- Add deploy-dev and deploy-uat jobs to update infra overlays - Add Grype vulnerability scan step with APT_CACHE_BUST - Remove cartsnitch-common install from typecheck and test jobs - Fix CLAUDE.md: API has its own local models, no cartsnitch-common dep - Add .grype.yaml from monorepo root Co-Authored-By: Paperclip <noreply@paperclip.ing>
8.3 KiB
8.3 KiB
CartSnitch API Gateway
Project Context
CartSnitch is a self-hosted grocery price intelligence platform built as a polyrepo microservices architecture. This repo (cartsnitch/api) is the public-facing API gateway that serves the frontend and proxies requests to internal services.
GitHub org: github.com/cartsnitch Domain: cartsnitch.com
CartSnitch Services
| Repo | Service | Purpose |
|---|---|---|
cartsnitch/common |
— | Shared models, schemas, utilities |
cartsnitch/receiptwitness |
ReceiptWitness | Purchase data ingestion via retailer scrapers |
cartsnitch/api |
API Gateway | Frontend-facing REST API (this repo) |
cartsnitch/cartsnitch |
Frontend | React PWA (mobile-first) |
cartsnitch/stickershock |
StickerShock | Price increase detection & CPI comparison |
cartsnitch/shrinkray |
ShrinkRay | Shrinkflation monitoring |
cartsnitch/clipartist |
ClipArtist | Coupon/deal watching & shopping optimization |
cartsnitch/infra |
— | K8s manifests, Flux kustomizations |
Architecture Decisions
- Polyrepo: Each service has its own repo, Dockerfile, CI/CD pipeline.
- Shared DB: One PostgreSQL cluster. This service reads from all tables for serving frontend queries. The API has its own local SQLAlchemy models — it does NOT import from
cartsnitch-common. - Inter-service comms: REST to internal services, Redis pub/sub for event subscriptions.
- Target scale: 500–1,000 users initially.
What This Service Does
The API Gateway is the single entry point for the frontend PWA and any external consumers. It:
- Handles user authentication — registration, login, JWT token management
- Serves purchase/product/price data — reads from the shared DB
- Proxies scraping operations — forwards scrape triggers to ReceiptWitness
- Serves coupon/deal data — reads from shared DB (written by ClipArtist)
- Serves alerts — price increase alerts (StickerShock), shrinkflation alerts (ShrinkRay)
- Provides public data endpoints — aggregate price trends for the transparency/shaming features
Tech Stack
- Python 3.12+
- FastAPI (async)
- SQLAlchemy 2.0 (async, read-heavy)
- Pydantic v2 (request/response validation)
- python-jose or PyJWT (JWT auth)
- passlib + bcrypt (password hashing)
- httpx (async HTTP client for proxying to internal services)
- Redis (subscribe to events for websocket push, caching)
- uvicorn (ASGI server)
Repo Structure
api/
├── CLAUDE.md
├── README.md
├── pyproject.toml
├── Dockerfile
├── docker-compose.yml
├── src/
│ └── cartsnitch_api/
│ ├── __init__.py
│ ├── config.py # Service-specific settings
│ ├── main.py # FastAPI app factory, lifespan, middleware
│ ├── auth/
│ │ ├── __init__.py
│ │ ├── jwt.py # JWT creation/validation
│ │ ├── passwords.py # Hashing, verification
│ │ ├── dependencies.py # FastAPI dependency injection (get_current_user)
│ │ └── routes.py # /auth/register, /auth/login, /auth/refresh
│ ├── routes/
│ │ ├── __init__.py
│ │ ├── purchases.py # Purchase history endpoints
│ │ ├── products.py # Normalized product catalog
│ │ ├── prices.py # Price history and trends
│ │ ├── coupons.py # Active coupons and deals
│ │ ├── alerts.py # Price increase / shrinkflation alerts
│ │ ├── stores.py # Store info, user store account management
│ │ ├── scraping.py # Proxy to ReceiptWitness (trigger scrape, status)
│ │ ├── shopping.py # Optimized shopping list (proxy to ClipArtist)
│ │ ├── public.py # Public price transparency endpoints (no auth)
│ │ └── health.py
│ ├── services/
│ │ ├── __init__.py
│ │ ├── receiptwitness.py # HTTP client for ReceiptWitness internal API
│ │ ├── stickershock.py # HTTP client for StickerShock internal API
│ │ ├── clipartist.py # HTTP client for ClipArtist internal API
│ │ └── shrinkray.py # HTTP client for ShrinkRay internal API
│ ├── middleware/
│ │ ├── __init__.py
│ │ ├── cors.py
│ │ └── rate_limit.py
│ └── cache.py # Redis caching helpers
└── tests/
├── conftest.py
├── test_auth/
├── test_routes/
└── test_services/
API Endpoint Design
Auth
POST /auth/register— create accountPOST /auth/login— get JWT access + refresh tokensPOST /auth/refresh— refresh access tokenGET /auth/me— current user profile
Store Accounts
GET /stores— list supported storesGET /me/stores— list user's connected store accounts + sync statusPOST /me/stores/{store_slug}/connect— initiate store connection flowDELETE /me/stores/{store_slug}— disconnect store account
Purchases
GET /purchases— list user's purchases (paginated, filterable by store/date)GET /purchases/{id}— purchase detail with line itemsGET /purchases/stats— spending summary (by store, by category, by period)
Products
GET /products— normalized product catalog (search, filter)GET /products/{id}— product detail with cross-store price comparisonGET /products/{id}/prices— price history for a product across stores
Prices
GET /prices/trends— aggregate price trends (public-capable)GET /prices/increases— recent significant price increasesGET /prices/comparison— compare specific items across stores
Coupons
GET /coupons— active coupons/deals (filterable by store)GET /coupons/relevant— coupons relevant to user's purchase history
Shopping
POST /shopping/optimize— input: shopping list → output: store-split + couponsGET /shopping/lists— user's saved shopping lists
Alerts
GET /alerts— user's price increase and shrinkflation alertsPUT /alerts/settings— configure alert thresholds
Public (No Auth)
GET /public/trends/{product_id}— public price trend for a productGET /public/store-comparison— public store-vs-store price comparisonGET /public/inflation— price changes vs CPI baseline
Scraping (Proxy to ReceiptWitness)
POST /scraping/{store_slug}/sync— trigger a sync for the current userGET /scraping/status— sync status across all stores
Authentication
- JWT-based auth with short-lived access tokens (15 min) and longer refresh tokens (7 days).
- Passwords hashed with bcrypt via passlib.
- All user-specific endpoints require a valid JWT in the
Authorization: Bearerheader. - Public endpoints under
/public/do not require auth. - Internal service-to-service calls (ReceiptWitness, etc.) use a shared API key in the
X-Service-Keyheader — not user JWTs.
Development Workflow
- Never push directly to main. Always create feature branches and open PRs.
- Branch naming:
feature/<description>orfix/<description> - Use conventional commits:
feat:,fix:,refactor:,docs:,chore: - OpenAPI docs auto-generated at
/docs(Swagger) and/redoc. - Write tests for all routes. Use httpx.AsyncClient with FastAPI's TestClient pattern.
Important Notes
- This service is read-heavy on the shared DB. Use async SQLAlchemy sessions.
- Consider Redis caching for expensive queries (price trends, product comparisons). Cache invalidation via Redis pub/sub events from other services.
- Rate limiting on public endpoints is important — these could get hammered if the price transparency features get attention.
- CORS must allow the frontend origin (cartsnitch.com and localhost for dev).
- The store connection flow is the trickiest UX challenge: the user needs to authenticate with each retailer, and we need to capture the resulting session. This likely involves a controlled Playwright browser session that the user can see/interact with, or an OAuth-like redirect flow if the retailer supports it (Kroger does for its public API, but not for purchase history access).