cs_betty/cartsnitch-fork-test
1
0
Fork 0
forked from cartsnitch/cartsnitch
Code Pull Requests Activity
Files
757444e58200d0e0c23ab12908c05ecf15b41e5e
cartsnitch-fork-test/docs/uat-receipt-submission.md
T

8.7 KiB
Raw Blame History

UAT Receipt Submission Path

Issue: CAR-812 Author: Barcode Betty Date: 2026-05-04

Overview

The UAT environment supports receipt submission via inbound email. This is the only supported submission method in UAT — there is no public REST API surface for receipt ingestion.

How It Works

Architecture

User composes email
    ↓
Email sent to <user_token>@cartsnitch.<env>.farh.net
    ↓
Mailgun webhook receives the email
    ↓
Email job enqueued to DragonflyDB stream: email:receipts
    ↓
email-worker (ReceiptWitness) consumes the job
    ↓
Worker resolves user via email_inbound_token lookup in DB
    ↓
Retailer detected from email content (meijer / kroger / target)
    ↓
Email parsed into Purchase + PurchaseItem records
    ↓
receipt.ingested event published to Redis
    ↓
MatchResult created with method=upc, confidence=1.0 for known UPCs

Key Components

Component Location Role
users.email_inbound_token DB (migration 001_add_email_inbound_token) 22-char unique token per user; used as email routing identifier
email:receipts stream DragonflyDB Queue holding pending email jobs
email-worker receiptwitness/src/receiptwitness/worker/email_worker.py Async worker consuming the stream
BaseEmailParser receiptwitness/src/receiptwitness/parsers/email/base.py Abstract parser; subclasses for meijer/kroger/target
Retailer detectors receiptwitness/src/receiptwitness/parsers/email/detector.py Sifts sender/subject to pick the right parser

Email Address Format

Each user is assigned a unique inbound token. The receipt submission email address is shown in Settings → Receipt Email on the UI:

Address: receipts+<email_inbound_token>@receipts.cartsnitch.com

To find a user's token in the UAT database (requires kubectl access to cartsnitch-uat):

kubectl exec -n cartsnitch-uat deployment/cartsnitch-api -- \\
  python -c "from cartsnitch_common.database import get_sync_session; \\
  from cartsnitch_common.models.user import User; \\
  from sqlalchemy import select; \\
  s = get_sync_session('postgresql://cartsnitch:cartsnitch@cartsnitch-pg-rw:5432/cartsnitch'); \\
  u = s.execute(select(User).where(User.email=='dottie@example.com')).scalar_one(); \\
  print(u.email_inbound_token)"

Submitting a Test Receipt (Step-by-Step)

Prerequisites

  • A test user account in UAT with a known email_inbound_token
  • A sample receipt email with a known UPC from the seeded normalized_products table

Steps

  1. Obtain the test user's inbound token. Use the UAT Settings → Receipt Email page in the UI to see the full address receipts+<token>@receipts.cartsnitch.com, or query the DB directly (see above).

  2. Compose the email. Send to: the address shown in Settings → Receipt Email Subject: anything Body: plain-text or HTML receipt content

  3. Expected behavior after email is processed:

    • A Receipt row is created in purchases
    • PurchaseItem rows are created with upc matching the seeded product UPC
    • A MatchResult is created with method='upc' and confidence=1.0

Known UPC for Dottie (from UAT seed)

NOTE: kubectl is not available in this execution environment. The UAT seed and DB query could not be executed. The sample receipt below uses a plausible placeholder UPC. Before Dottie runs the regression:

  1. Run bash scripts/seed-env.sh uat from a machine with UAT kubecontext
  2. Query: SELECT id, canonical_name, upc_variants->0->>'upc' AS sample_upc FROM normalized_products WHERE jsonb_array_length(upc_variants) > 0 LIMIT 1;
  3. Replace the placeholder values below with the real captured row
  • id: TBD — run seed and query UAT DB
  • name: TBD — run seed and query UAT DB
  • sample UPC: TBD — run seed and query UAT DB

Meijer Sample Receipt (plain text)

Meijer
===================================
Purchase Date: 03/15/2026
Store: Meijer #127 - Ann Arbor, MI
-----------------------------------
 1 x Organic Whole Milk 1gal      $4.99
 1 x Whole Wheat Bread             $3.29
 1 x Bananas (2 lb)                $0.67
 1 x Chicken Breast (3 lb)        $12.47
 1 x Cheddar Cheese Block 8oz      $5.99
-----------------------------------
Subtotal:                        $27.41
Tax:                              $1.93
Total:                           $29.34
===================================
THANK YOU FOR SHOPPING MEIJER
===================================

Meijer

Purchase Date: 03/15/2026 Store: Meijer #127 - Ann Arbor, MI

1 x Organic Whole Milk 1gal $4.99 1 x Whole Wheat Bread $3.29 1 x Bananas (2 lb) $0.67 1 x Chicken Breast (3 lb) $12.47 1 x Cheddar Cheese Block 8oz $5.99

Subtotal: $27.41 Tax: $1.93 Total: $29.34

THANK YOU FOR SHOPPING MEIJER


> **Note:** The `email-worker` parses the email body and extracts line items by retailer. The exact format and field mapping depends on the retailer parser. For Meijer, the parser looks for item lines matching `(\d+) x (.+?)\s+\$([\d.]+)`. UPCs in the `upc_variants` JSONB of seeded products will be matched during the normalization step.

### Kroger Sample Receipt (plain text)

KROGER

Purchase Date: 03/15/2026 Store: KROGER #412 - Ann Arbor MI

1 Organic Whole Milk 1gal $5.29 1 Whole Wheat Bread $3.49 1 Bananas (2 lb) $0.69 1 Chicken Breast (3 lb) $11.99 1 Sharp Cheddar Cheese 8oz $4.99

Subtotal: $26.45 Tax: $1.85 Total: $28.30


### Target Sample Receipt (plain text)

TARGET

03/15/2026 14:32 Store: 0874 Ann Arbor, MI

1 Organic Whole Milk 1G $5.49 1 Whole Wheat Bread $3.29 1 Bananas LB 2 $0.68 1 Chicken Breast 3# $12.99 1 Cheddar Cheese 8OZ $5.79

Subtotal: $28.24 Tax (6%): $1.69 Total: $29.93


---

## Troubleshooting

### Email not processed

1. Check the `email:receipts` stream has messages:
   ```bash
   kubectl exec -n cartsnitch-uat deploy/email-worker -- python -c \\
     "import asyncio; from receiptwitness.queue.email import get_redis; \\
     async def chk(): c = await get_redis(); info = await c.xinfo_stream('email:receipts'); print(info); \\
     asyncio.run(chk())"

  1. Check email-worker logs for retailer detection failures:

    kubectl logs -n cartsnitch-uat deploy/email-worker -f

  2. Verify the token resolves to a user in the DB:

    kubectl exec -n cartsnitch-uat deploy/cartsnitch-api -- \\
  python -c "from cartsnitch_common.database import get_sync_session; \\
  from cartsnitch_common.models.user import User; \\
  from sqlalchemy import select; \\
  s = get_sync_session('postgresql://...'); \\
  r = s.execute(select(User.email_inbound_token).limit(5)).all(); \\
  print(r)"

No MatchResult created

The normalization pipeline requires a normalized_product row with the submitted UPC in upc_variants. If the seed was run, the product should be found. Check the match_results table after submission:

SELECT mr.*, np.canonical_name
FROM match_results mr
JOIN normalized_products np ON np.id = mr.normalized_product_id
WHERE mr.match_method = 'upc'
ORDER BY mr.created_at DESC
LIMIT 10;

Related Files

File Role
common/alembic/versions/001_add_email_inbound_token.py Adds email_inbound_token column
receiptwitness/src/receiptwitness/worker/email_worker.py Consumes email jobs from stream
receiptwitness/src/receiptwitness/queue/email.py DragonflyDB stream consumer group
receiptwitness/src/receiptwitness/parsers/email/detector.py Retailer detection
receiptwitness/src/receiptwitness/parsers/email/meijer.py Meijer email parser
receiptwitness/src/receiptwitness/parsers/email/kroger.py Kroger email parser
receiptwitness/src/receiptwitness/parsers/email/target.py Target email parser
docs/uat-runbook.md UAT runbook (defect classification, entry/exit criteria)
View Git Blame Copy Permalink