forked from cartsnitch/cartsnitch
Compare commits
24 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| e3ed19f98c | |||
| e54736d900 | |||
| 59850c0cb4 | |||
| 40abf64888 | |||
| 4e72e61f6d | |||
| 04965eb89d | |||
| 3615a78f0e | |||
| d785606bd1 | |||
| 48eaf45121 | |||
| 48a999d569 | |||
| 4bf5cd3826 | |||
| ea2fddc5cb | |||
| 44d9502673 | |||
| a3fca65ea1 | |||
| 3ac61908f5 | |||
| 2a7f1921b0 | |||
| 25c27d08fe | |||
| aaf645fbe9 | |||
| 80aa58b37a | |||
| 062f6be8ea | |||
| 8d7e0b44ee | |||
| 9c7cd7454c | |||
| 60beb2d89e | |||
| 9120c834e4 |
@@ -1 +1,315 @@
|
|||||||
# CartSnitch
|
# CartSnitch
|
||||||
|
|
||||||
|
**Grocery price intelligence — know what you're paying, every time.**
|
||||||
|
|
||||||
|
CartSnitch is a self-hosted grocery price intelligence platform that connects to your store loyalty accounts, tracks prices across retailers, monitors shrinkflation, and helps you find the best deals.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Project Overview
|
||||||
|
|
||||||
|
CartSnitch solves the problem of **grocery price opacity**. Most shoppers don't know if they're getting a good deal, whether prices have spiked since their last visit, or if the "sale" is actually a worse price than a competitor. CartSnitch makes prices transparent.
|
||||||
|
|
||||||
|
**Core features:**
|
||||||
|
- Connect Meijer, Kroger, Target loyalty accounts
|
||||||
|
- View purchase history across all stores in one timeline
|
||||||
|
- Track per-item price charts across stores over time
|
||||||
|
- Receive shrinkflation and price increase alerts
|
||||||
|
- Browse active coupons and deals
|
||||||
|
- Generate optimized shopping lists with store-split plans
|
||||||
|
- Public price transparency dashboards
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
CartSnitch is a polyglot microservices platform. The monorepo contains the frontend PWA and core services.
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────────┐
|
||||||
|
│ CartSnitch PWA │
|
||||||
|
│ (React, mobile-first PWA) │
|
||||||
|
└──────────┬────────────────────┬────────────────────┬───────────┘
|
||||||
|
│ │ │
|
||||||
|
▼ ▼ ▼
|
||||||
|
┌──────────────────┐ ┌─────────────────┐ ┌─────────────────────┐
|
||||||
|
│ Auth Service │ │ API Gateway │ │ ReceiptWitness │
|
||||||
|
│ (Better-Auth) │ │ (Python/FastAPI)│ │ (Python/Scrapers) │
|
||||||
|
│ Session mgmt │ │ REST + proxy │ │ Purchase ingestion │
|
||||||
|
└────────┬─────────┘ └────────┬────────┘ └──────────┬──────────┘
|
||||||
|
│ │ │
|
||||||
|
└──────────────────────┼────────────────────────┘
|
||||||
|
▼
|
||||||
|
┌────────────────────────┐
|
||||||
|
│ CloudNativePG (PGSQL) │
|
||||||
|
│ Shared database │
|
||||||
|
└────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
### Services in This Repo
|
||||||
|
|
||||||
|
| Directory | Service | Description |
|
||||||
|
|-----------|---------|-------------|
|
||||||
|
| `/` (root) | Frontend | React PWA, mobile-first |
|
||||||
|
| `auth/` | Auth | Better-Auth service — session management, email/password, OAuth |
|
||||||
|
| `api/` | API Gateway | Frontend-facing REST API, Python/FastAPI |
|
||||||
|
| `common/` | Common | Shared Python models, Pydantic schemas, Alembic migrations |
|
||||||
|
| `receiptwitness/` | ReceiptWitness | Purchase data ingestion via retailer scrapers |
|
||||||
|
|
||||||
|
### Other CartSnitch Repos
|
||||||
|
|
||||||
|
| Repo | Service |
|
||||||
|
|------|---------|
|
||||||
|
| `cartsnitch/stickershock` | Price increase detection & CPI comparison |
|
||||||
|
| `cartsnitch/shrinkray` | Shrinkflation monitoring |
|
||||||
|
| `cartsnitch/clipartist` | Coupon/deal watching |
|
||||||
|
| `cartsnitch/infra` | Kubernetes manifests, Flux kustomizations |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Tech Stack
|
||||||
|
|
||||||
|
### Frontend
|
||||||
|
- **React 18+** with TypeScript
|
||||||
|
- **Vite** — build tool
|
||||||
|
- **Tailwind CSS v4** — mobile-first responsive design
|
||||||
|
- **Workbox** — service worker, offline caching, PWA manifest
|
||||||
|
- **Recharts** — price trend visualizations
|
||||||
|
- **TanStack Query** — data fetching and caching
|
||||||
|
- **React Router v7** — client-side routing
|
||||||
|
- **Zustand** — lightweight state management
|
||||||
|
|
||||||
|
### Backend Services
|
||||||
|
- **Better-Auth** — authentication (session management, email/password, OAuth)
|
||||||
|
- **Node.js** (API Gateway)
|
||||||
|
- **Python/FastAPI** (API Gateway, ReceiptWitness)
|
||||||
|
- **PostgreSQL** via CloudNativePG
|
||||||
|
- **DragonflyDB** for caching
|
||||||
|
|
||||||
|
### Infrastructure
|
||||||
|
- **Kubernetes** (k3s-compatible)
|
||||||
|
- **Flux CD** — GitOps deployment
|
||||||
|
- **GitHub Actions** — CI/CD
|
||||||
|
- **CalVer** (`YYYY.MM.DD[.N]`) — image tagging
|
||||||
|
- **Bitnami Sealed Secrets** — secret management
|
||||||
|
- **Authentik** — OIDC/OAuth2 provider
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Getting Started
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
|
||||||
|
- Node.js 20+
|
||||||
|
- npm or pnpm
|
||||||
|
- PostgreSQL (local or containerized)
|
||||||
|
- Docker (for running services locally)
|
||||||
|
|
||||||
|
### Local Development
|
||||||
|
|
||||||
|
1. **Clone the repo**
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/cartsnitch/cartsnitch.git
|
||||||
|
cd cartsnitch
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Install dependencies**
|
||||||
|
```bash
|
||||||
|
npm install
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Set up environment variables**
|
||||||
|
```bash
|
||||||
|
cp .env.example .env
|
||||||
|
# Edit .env with your local settings
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Start the frontend dev server**
|
||||||
|
```bash
|
||||||
|
npm run dev
|
||||||
|
```
|
||||||
|
The PWA will be available at `http://localhost:5173`.
|
||||||
|
|
||||||
|
5. **Run tests**
|
||||||
|
```bash
|
||||||
|
npm test
|
||||||
|
```
|
||||||
|
|
||||||
|
6. **Build for production**
|
||||||
|
```bash
|
||||||
|
npm run build
|
||||||
|
```
|
||||||
|
|
||||||
|
### Running Backend Services Locally
|
||||||
|
|
||||||
|
The frontend PWA communicates with three backend services. For full local development, you'll need to run each service:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Auth service (Better-Auth)
|
||||||
|
cd auth
|
||||||
|
npm install
|
||||||
|
npm run dev
|
||||||
|
|
||||||
|
# API Gateway (separate repo: cartsnitch/api)
|
||||||
|
# See api/README.md
|
||||||
|
|
||||||
|
# ReceiptWitness (separate repo: cartsnitch/receiptwitness)
|
||||||
|
# See receiptwitness/README.md
|
||||||
|
```
|
||||||
|
|
||||||
|
### Environment Variables
|
||||||
|
|
||||||
|
| Variable | Description | Default |
|
||||||
|
|----------|-------------|---------|
|
||||||
|
| `VITE_API_URL` | API Gateway base URL | `http://localhost:3000` |
|
||||||
|
| `VITE_AUTH_URL` | Auth service base URL | `http://localhost:3001` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Contributing
|
||||||
|
|
||||||
|
We welcome contributions. Please follow the workflow below.
|
||||||
|
|
||||||
|
### Branching Strategy
|
||||||
|
|
||||||
|
- Branch from `dev`
|
||||||
|
- Use prefix: `feature/`, `fix/`, `docs/`, `chore/`
|
||||||
|
- Examples: `feature/shopping-list-optimization`, `fix/price-chart-zoom`
|
||||||
|
|
||||||
|
### Commit Convention
|
||||||
|
|
||||||
|
We use [Conventional Commits](https://www.conventionalcommits.org/):
|
||||||
|
|
||||||
|
```
|
||||||
|
feat: add shopping list export
|
||||||
|
fix: correct price chart date formatting
|
||||||
|
docs: update API documentation
|
||||||
|
chore: update dependencies
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pull Request Workflow
|
||||||
|
|
||||||
|
1. Open a PR against `dev`
|
||||||
|
2. CI must pass (lint, type check, tests, e2e)
|
||||||
|
3. QA reviews and approves
|
||||||
|
4. CTO merges to `dev`
|
||||||
|
5. Dev deploys automatically
|
||||||
|
6. CTO promotes `dev → uat`
|
||||||
|
7. UAT and security review
|
||||||
|
8. CEO merges `uat → main`
|
||||||
|
9. Production deploys automatically
|
||||||
|
|
||||||
|
**Never push directly to `main`, `dev`, or `uat`.**
|
||||||
|
|
||||||
|
### Code Standards
|
||||||
|
|
||||||
|
- ESLint for linting
|
||||||
|
- TypeScript strict mode
|
||||||
|
- Mobile-first responsive design
|
||||||
|
- Accessibility (WCAG 2.1 AA)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
### Unit Tests
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm test
|
||||||
|
```
|
||||||
|
|
||||||
|
### E2E Tests (Playwright)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm run test:e2e
|
||||||
|
```
|
||||||
|
|
||||||
|
Tests run headless by default. For headed mode:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm run test:e2e:headed
|
||||||
|
```
|
||||||
|
|
||||||
|
### Lighthouse CI
|
||||||
|
|
||||||
|
Performance audits run automatically in CI. To run locally:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npm run build
|
||||||
|
npm run preview
|
||||||
|
# In another terminal:
|
||||||
|
npx lighthouse http://localhost:4173 --output=html --output-path=./report/lighthouse.html
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## CI/CD Pipeline
|
||||||
|
|
||||||
|
All branches (`main`, `dev`, `uat`) run through GitHub Actions on every push.
|
||||||
|
|
||||||
|
### Pipeline Stages
|
||||||
|
|
||||||
|
| Job | Trigger | Purpose |
|
||||||
|
|-----|---------|---------|
|
||||||
|
| `lint` | Every push | ESLint + TypeScript type check |
|
||||||
|
| `test` | Every push | Unit tests via Vitest |
|
||||||
|
| `audit` | Every push | Security vulnerability scan |
|
||||||
|
| `e2e` | Every push | Playwright end-to-end tests |
|
||||||
|
| `lighthouse` | After test | Performance budget check |
|
||||||
|
| `build-and-push` | On push to main/dev/uat | Build and push Docker images to GHCR |
|
||||||
|
| `deploy-dev` | On push to dev or main | Update `cartsnitch/infra` → auto-deploy to dev |
|
||||||
|
| `deploy-uat` | On push to uat or main | Update `cartsnitch/infra` → auto-deploy to uat |
|
||||||
|
|
||||||
|
### Image Tagging
|
||||||
|
|
||||||
|
- **Production (`main`):** CalVer tag (`YYYY.MM.DD[.N]`) + `latest`
|
||||||
|
- **Development (`dev`):** SHA tag (`sha-<short-sha>`)
|
||||||
|
|
||||||
|
### Deployment Environments
|
||||||
|
|
||||||
|
| Environment | Namespace | URL | Trigger |
|
||||||
|
|-------------|-----------|-----|---------|
|
||||||
|
| Dev | `cartsnitch-dev` | `cartsnitch.dev.farh.net` | Push to `dev` branch |
|
||||||
|
| UAT | `cartsnitch-uat` | `cartsnitch.uat.farh.net` | Push to `uat` branch |
|
||||||
|
| Production | `cartsnitch` | `cartsnitch.farh.net` | Push to `main` branch |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
### Infrastructure
|
||||||
|
|
||||||
|
The infrastructure repository ([cartsnitch/infra](https://github.com/cartsnitch/infra)) contains Kubernetes manifests and Flux Kustomize overlays.
|
||||||
|
|
||||||
|
### Flux GitOps Flow
|
||||||
|
|
||||||
|
1. CI builds and pushes a new Docker image
|
||||||
|
2. CI opens a PR to `cartsnitch/infra` updating the image tag
|
||||||
|
3. On merge, Flux reconciles the manifests and rolls out the new image
|
||||||
|
|
||||||
|
### Forcing a Rollout
|
||||||
|
|
||||||
|
To force pods to pick up a new `:latest` image:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl rollout restart deployment/<name> -n <namespace>
|
||||||
|
```
|
||||||
|
|
||||||
|
### Secrets
|
||||||
|
|
||||||
|
Secrets are managed via **Bitnami Sealed Secrets**. No plain Kubernetes secrets are used.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Related Projects
|
||||||
|
|
||||||
|
- [StickerShock](https://github.com/cartsnitch/stickershock) — Price increase detection
|
||||||
|
- [ShrinkRay](https://github.com/cartsnitch/shrinkray) — Shrinkflation monitoring
|
||||||
|
- [ClipArtist](https://github.com/cartsnitch/clipartist) — Coupon/deal optimization
|
||||||
|
- [Infra](https://github.com/cartsnitch/infra) — Kubernetes infrastructure
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
MIT © 2025 CartSnitch
|
||||||
|
|||||||
+2
-1
@@ -7,7 +7,8 @@
|
|||||||
"dev": "tsx watch src/index.ts",
|
"dev": "tsx watch src/index.ts",
|
||||||
"build": "tsc",
|
"build": "tsc",
|
||||||
"start": "node dist/index.js",
|
"start": "node dist/index.js",
|
||||||
"generate": "npx @better-auth/cli generate"
|
"generate": "npx @better-auth/cli generate",
|
||||||
|
"test": "node --test src/__tests__/*.test.ts"
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"bcrypt": "^6.0.0",
|
"bcrypt": "^6.0.0",
|
||||||
|
|||||||
@@ -0,0 +1,117 @@
|
|||||||
|
import { describe, it } from 'node:test';
|
||||||
|
import { equal } from 'node:assert';
|
||||||
|
import http from 'node:http';
|
||||||
|
|
||||||
|
describe('Auth health endpoint', () => {
|
||||||
|
const startHealthServer = (poolMock) => {
|
||||||
|
return new Promise((resolve) => {
|
||||||
|
const server = http.createServer(async (req, res) => {
|
||||||
|
if (req.url === '/health' && req.method === 'GET') {
|
||||||
|
try {
|
||||||
|
const client = await poolMock.connect();
|
||||||
|
try {
|
||||||
|
await Promise.race([
|
||||||
|
client.query('SELECT 1'),
|
||||||
|
new Promise((_, reject) => setTimeout(() => reject(new Error('DB timeout')), 2000)),
|
||||||
|
]);
|
||||||
|
} finally {
|
||||||
|
client.release();
|
||||||
|
}
|
||||||
|
res.writeHead(200, { 'Content-Type': 'application/json' });
|
||||||
|
res.end(JSON.stringify({ status: 'ok', db: 'reachable' }));
|
||||||
|
} catch {
|
||||||
|
res.writeHead(503, { 'Content-Type': 'application/json' });
|
||||||
|
res.end(JSON.stringify({ status: 'error', db: 'unreachable' }));
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
res.writeHead(404);
|
||||||
|
res.end();
|
||||||
|
});
|
||||||
|
server.listen(0, '0.0.0.0', () => {
|
||||||
|
const addr = server.address();
|
||||||
|
const port = typeof addr === 'object' && addr ? addr.port : 0;
|
||||||
|
resolve({ port, close: () => server.close() });
|
||||||
|
});
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
const makeRequest = (port) => {
|
||||||
|
return new Promise((resolve) => {
|
||||||
|
const req = http.get(`http://localhost:${port}/health`, (res) => {
|
||||||
|
let body = '';
|
||||||
|
res.on('data', (chunk) => { body += chunk; });
|
||||||
|
res.on('end', () => {
|
||||||
|
resolve({ status: res.statusCode, body });
|
||||||
|
});
|
||||||
|
});
|
||||||
|
req.on('error', () => resolve({ status: 0, body: '' }));
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
it('returns 200 with db=reachable when pool.connect succeeds', async () => {
|
||||||
|
const mockClient = {
|
||||||
|
query: async () => ({ rows: [{ 1: 1 }] }),
|
||||||
|
release: () => {},
|
||||||
|
};
|
||||||
|
const poolMock = {
|
||||||
|
connect: async () => mockClient,
|
||||||
|
};
|
||||||
|
|
||||||
|
const { port, close } = await startHealthServer(poolMock);
|
||||||
|
const { status, body } = await makeRequest(port);
|
||||||
|
close();
|
||||||
|
|
||||||
|
equal(status, 200);
|
||||||
|
equal(body, '{"status":"ok","db":"reachable"}');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns 503 with db=unreachable when pool.connect throws', async () => {
|
||||||
|
const poolMock = {
|
||||||
|
connect: async () => { throw new Error('connection refused'); },
|
||||||
|
};
|
||||||
|
|
||||||
|
const { port, close } = await startHealthServer(poolMock);
|
||||||
|
const { status, body } = await makeRequest(port);
|
||||||
|
close();
|
||||||
|
|
||||||
|
equal(status, 503);
|
||||||
|
equal(body, '{"status":"error","db":"unreachable"}');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns 503 with db=unreachable when query times out', async () => {
|
||||||
|
const mockClient = {
|
||||||
|
query: async () => {
|
||||||
|
await new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), 3000));
|
||||||
|
},
|
||||||
|
release: () => {},
|
||||||
|
};
|
||||||
|
const poolMock = {
|
||||||
|
connect: async () => mockClient,
|
||||||
|
};
|
||||||
|
|
||||||
|
const { port, close } = await startHealthServer(poolMock);
|
||||||
|
const { status, body } = await makeRequest(port);
|
||||||
|
close();
|
||||||
|
|
||||||
|
equal(status, 503);
|
||||||
|
equal(body, '{"status":"error","db":"unreachable"}');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns a terminal response for unknown paths (no hang)', async () => {
|
||||||
|
const poolMock = { connect: async () => ({ query: async () => {}, release: () => {} }) };
|
||||||
|
const { port, close } = await startHealthServer(poolMock);
|
||||||
|
|
||||||
|
const result = await new Promise<{ status: number }>((resolve) => {
|
||||||
|
const req = http.get(`http://localhost:${port}/`, (res) => {
|
||||||
|
res.resume();
|
||||||
|
res.on('end', () => resolve({ status: res.statusCode ?? 0 }));
|
||||||
|
});
|
||||||
|
req.on('error', () => resolve({ status: 0 }));
|
||||||
|
setTimeout(() => resolve({ status: -1 }), 1000);
|
||||||
|
});
|
||||||
|
close();
|
||||||
|
|
||||||
|
equal(result.status !== -1, true, 'Unknown path must return a terminal response within 1s');
|
||||||
|
});
|
||||||
|
});
|
||||||
+3
-3
@@ -8,7 +8,7 @@ const handler = toNodeHandler(auth);
|
|||||||
|
|
||||||
const server = createServer(async (req, res) => {
|
const server = createServer(async (req, res) => {
|
||||||
// Health check
|
// Health check
|
||||||
if (req.url === "/health" && req.method === "GET") {
|
if ((req.url === "/health" || req.url === "/auth/health") && req.method === "GET") {
|
||||||
try {
|
try {
|
||||||
const client = await pool.connect();
|
const client = await pool.connect();
|
||||||
try {
|
try {
|
||||||
@@ -20,7 +20,7 @@ const server = createServer(async (req, res) => {
|
|||||||
client.release();
|
client.release();
|
||||||
}
|
}
|
||||||
res.writeHead(200, { "Content-Type": "application/json" });
|
res.writeHead(200, { "Content-Type": "application/json" });
|
||||||
res.end(JSON.stringify({ status: "ok", db: "connected" }));
|
res.end(JSON.stringify({ status: "ok", db: "reachable" }));
|
||||||
} catch {
|
} catch {
|
||||||
res.writeHead(503, { "Content-Type": "application/json" });
|
res.writeHead(503, { "Content-Type": "application/json" });
|
||||||
res.end(JSON.stringify({ status: "error", db: "unreachable" }));
|
res.end(JSON.stringify({ status: "error", db: "unreachable" }));
|
||||||
@@ -28,7 +28,7 @@ const server = createServer(async (req, res) => {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
// All /auth/* routes handled by Better-Auth
|
// All other routes handled by Better-Auth (returns 404 for unknown paths)
|
||||||
await handler(req, res);
|
await handler(req, res);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -7,6 +7,6 @@ export default defineConfig({
|
|||||||
environment: 'jsdom',
|
environment: 'jsdom',
|
||||||
globals: true,
|
globals: true,
|
||||||
setupFiles: ['./src/test/setup.ts'],
|
setupFiles: ['./src/test/setup.ts'],
|
||||||
exclude: ['e2e/**', 'node_modules/**'],
|
exclude: ['e2e/**', 'auth/**', 'node_modules/**'],
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
|
|||||||
Reference in New Issue
Block a user