Compare commits

..

6 Commits

Author SHA1 Message Date
Barcode Betty 76781ed238 style: fix ruff format in conftest.py
CI / lint (pull_request) Successful in 5s
CI / typecheck (pull_request) Successful in 29s
CI / test (pull_request) Failing after 1m0s
CI / build-and-push (pull_request) Has been skipped
Add missing blank line between the _set_timestamp_defaults helper
and the next top-level constant so `ruff format --check .` passes.
Pre-existing on dev's HEAD; surfaced after rebasing PR #39 onto dev
in 2b20946.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-06-02 14:58:18 +00:00
Barcode Betty 2b20946ad7 fix: /health returns 503 on DB failure, pool_timeout=30, CI typecheck fixes
CI / lint (pull_request) Failing after 4s
CI / typecheck (pull_request) Failing after 25s
CI / test (pull_request) Failing after 1m5s
CI / build-and-push (pull_request) Has been skipped
QA review of PR #39 (CAR-1121) identified three blocking issues; this
commit addresses all three plus the typecheck errors flagged as CI RED.

CAR-1077 (PR #39) changes:
- database.py: add pool_timeout=30 so the engine fails fast when the
  connection pool is exhausted (defends against the "server closed
  connection unexpectedly" pod failures).
- routes/health.py: /health now calls SELECT 1 through Depends(get_db)
  and raises HTTPException(503) when the database is unreachable, so
  Kubernetes readiness probes can correctly mark the pod unhealthy and
  stop routing traffic to it.  Logs the failure at exception level for
  observability.
- Drop .mcp.json from this PR (root-level MCP server config, not
  related to the pool fix; tracked separately).

CI typecheck fixes (pre-existing on dev, were failing mypy on PR #39):
- auth/passwords.py: cast bcrypt return values so mypy doesn't widen
  to Any.
- config.py: silence the false-positive call-arg on Settings() — the
  three required fields are populated from the environment by
  pydantic-settings at runtime.
- cache.py: coerce the bytes/str union returned by the redis client
  to the documented str | None return type.
- middleware/rate_limit.py: annotate the three module-level limiters
  with the RateLimitBackend protocol, cast the redis zrange score to
  float before arithmetic, and add max_requests/window_seconds to the
  protocol so the response-header builder can read them.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-06-02 14:53:16 +00:00
Flea Flicker bd6b137c68 Fix SQLite timestamp and UUID server_defaults in test fixtures
CI / lint (push) Failing after 5s
CI / typecheck (push) Failing after 32s
CI / test (push) Failing after 1m7s
CI / build-and-push (push) Has been skipped
Add _set_timestamp_defaults event listener to populate created_at/updated_at
before insert when using SQLite, since func.now() server_default is stripped.

Extended server_default stripping to include "now()" expressions for
timestamp columns (created_at, updated_at) that were failing with
NOT NULL constraint errors.

Fixes remaining CI test failures after PR #35:
- NOT NULL constraint failed: stores.created_at
- NOT NULL constraint failed: normalized_products.created_at

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-06-02 02:53:46 +00:00
Flea Flicker f18df8a40c fix: rename loop variable to avoid shadowing SQLAlchemy table import (F402) 2026-06-01 12:38:46 +00:00
Barcode Betty ebf69976d4 Fix SQLite server_default AttributeError and pool_size errors (#35)
CI / lint (push) Failing after 6s
CI / typecheck (push) Failing after 30s
CI / test (push) Failing after 1m34s
CI / build-and-push (push) Has been skipped
Fix SQLite server_default AttributeError and pool_size errors

Co-authored-by: Barcode Betty <32+cs_betty@noreply.git.farh.net>
Co-committed-by: Barcode Betty <32+cs_betty@noreply.git.farh.net>
2026-06-01 12:38:21 +00:00
Barcode Betty 84c143c4e7 Remove deploy-dev/deploy-uat CI jobs (CAR-1069) (#37)
CI / lint (pull_request) Failing after 4s
CI / typecheck (push) Failing after 19s
CI / test (push) Failing after 30s
CI / build-and-push (push) Has been skipped
CI / typecheck (pull_request) Failing after 18s
CI / test (pull_request) Failing after 29s
CI / lint (push) Failing after 3s
CI / build-and-push (pull_request) Has been skipped
Co-authored-by: Barcode Betty <32+cs_betty@noreply.git.farh.net>
Co-committed-by: Barcode Betty <32+cs_betty@noreply.git.farh.net>
2026-05-27 01:56:53 +00:00
13 changed files with 87 additions and 169 deletions
-87
View File
@@ -175,90 +175,3 @@ jobs:
git tag "v${{ steps.calver.outputs.version }}" git tag "v${{ steps.calver.outputs.version }}"
git push origin "v${{ steps.calver.outputs.version }}" git push origin "v${{ steps.calver.outputs.version }}"
deploy-dev:
runs-on: ubuntu-latest
needs: [build-and-push]
if: always() && !cancelled() && github.event_name == 'push' && (github.ref == 'refs/heads/dev' || github.ref == 'refs/heads/main')
steps:
- name: Checkout infra repo
uses: actions/checkout@v4
with:
repository: cartsnitch/infra
token: ${{ secrets.GITEA_TOKEN }}
ref: main
path: infra
- name: Install kubectl
uses: azure/setup-kubectl@v4
- name: Install kustomize
uses: imranismail/setup-kustomize@v2
- name: Determine image tag
id: api_tag
run: |
if [ "${{ github.ref }}" == "refs/heads/main" ]; then
echo "tag=${{ needs.build-and-push.outputs.calver_tag }}" >> "$GITHUB_OUTPUT"
else
echo "tag=${{ needs.build-and-push.outputs.sha_tag }}" >> "$GITHUB_OUTPUT"
fi
- name: Update api image tag
if: needs.build-and-push.result == 'success'
run: |
cd infra/apps/overlays/dev
kustomize edit set image ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.api_tag.outputs.tag }}
- name: Commit and push to infra
run: |
cd infra
git config user.name "cartsnitch-ci[bot]"
git config user.email "cartsnitch-ci[bot]@users.noreply.github.com"
git add apps/overlays/dev/kustomization.yaml
git commit -m "ci(dev): update api image"
git pull --rebase origin main
git push origin main
deploy-uat:
runs-on: ubuntu-latest
needs: [build-and-push]
if: always() && !cancelled() && github.event_name == 'push' && (github.ref == 'refs/heads/uat' || github.ref == 'refs/heads/main')
steps:
- name: Checkout infra repo
uses: actions/checkout@v4
with:
repository: cartsnitch/infra
token: ${{ secrets.GITEA_TOKEN }}
ref: main
path: infra
- name: Install kubectl
uses: azure/setup-kubectl@v4
- name: Install kustomize
uses: imranismail/setup-kustomize@v2
- name: Determine image tag
id: api_tag
run: |
if [ "${{ github.ref }}" == "refs/heads/main" ]; then
echo "tag=${{ needs.build-and-push.outputs.calver_tag }}" >> "$GITHUB_OUTPUT"
else
echo "tag=${{ needs.build-and-push.outputs.sha_tag }}" >> "$GITHUB_OUTPUT"
fi
- name: Update api image tag
if: needs.build-and-push.result == 'success'
run: |
cd infra/apps/overlays/uat
kustomize edit set image ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.api_tag.outputs.tag }}
- name: Commit and push to infra
run: |
cd infra
git config user.name "cartsnitch-ci[bot]"
git config user.email "cartsnitch-ci[bot]@users.noreply.github.com"
git add apps/overlays/uat/kustomization.yaml
git commit -m "ci(uat): update api image"
git pull --rebase origin main
git push origin main
+2 -2
View File
@@ -4,8 +4,8 @@ import bcrypt
def hash_password(password: str) -> str: def hash_password(password: str) -> str:
return bcrypt.hashpw(password.encode(), bcrypt.gensalt()).decode() return str(bcrypt.hashpw(password.encode(), bcrypt.gensalt()).decode())
def verify_password(plain_password: str, hashed_password: str) -> bool: def verify_password(plain_password: str, hashed_password: str) -> bool:
return bcrypt.checkpw(plain_password.encode(), hashed_password.encode()) return bool(bcrypt.checkpw(plain_password.encode(), hashed_password.encode()))
+6 -1
View File
@@ -35,7 +35,12 @@ class CacheClient:
async def get(self, key: str) -> str | None: async def get(self, key: str) -> str | None:
if not self._client: if not self._client:
return None return None
return await self._client.get(key) value = await self._client.get(key)
if value is None:
return None
if isinstance(value, bytes):
return value.decode("utf-8", errors="replace")
return value
async def set(self, key: str, value: str, ttl_seconds: int = 300) -> None: async def set(self, key: str, value: str, ttl_seconds: int = 300) -> None:
if not self._client: if not self._client:
+1 -1
View File
@@ -86,4 +86,4 @@ class Settings(BaseSettings):
return self return self
settings = Settings() settings = Settings() # type: ignore[call-arg]
+1
View File
@@ -14,6 +14,7 @@ def _build_engine_kwargs() -> dict:
kwargs.update( kwargs.update(
pool_size=10, pool_size=10,
max_overflow=20, max_overflow=20,
pool_timeout=30,
pool_pre_ping=True, pool_pre_ping=True,
pool_recycle=3600, pool_recycle=3600,
) )
+9 -1
View File
@@ -25,6 +25,9 @@ logger = logging.getLogger(__name__)
class RateLimitBackend(Protocol): class RateLimitBackend(Protocol):
"""Protocol for rate limit backends.""" """Protocol for rate limit backends."""
max_requests: int
window_seconds: int
async def is_allowed(self, key: str) -> tuple[bool, int, int]: async def is_allowed(self, key: str) -> tuple[bool, int, int]:
"""Check if request is allowed. Returns (allowed, remaining, retry_after).""" """Check if request is allowed. Returns (allowed, remaining, retry_after)."""
@@ -82,7 +85,8 @@ class RedisSlidingWindow:
if current_count >= self.max_requests: if current_count >= self.max_requests:
oldest = await self.redis.zrange(key, 0, 0, withscores=True) oldest = await self.redis.zrange(key, 0, 0, withscores=True)
if oldest: if oldest:
retry_after = int((oldest[0][1] - cutoff) / 1000) + 1 oldest_score = float(oldest[0][1])
retry_after = int((oldest_score - cutoff) / 1000) + 1
else: else:
retry_after = self.window_seconds retry_after = self.window_seconds
return False, 0, retry_after return False, 0, retry_after
@@ -114,6 +118,10 @@ if settings.rate_limit_redis_enabled:
logger.warning("Failed to connect to Redis for rate limiting, using in-memory: %s", e) logger.warning("Failed to connect to Redis for rate limiting, using in-memory: %s", e)
_use_redis = False _use_redis = False
_public_limiter: RateLimitBackend
_auth_limiter: RateLimitBackend
_auth_strict_limiter: RateLimitBackend
if _use_redis and _redis_client: if _use_redis and _redis_client:
_public_limiter = RedisSlidingWindow( _public_limiter = RedisSlidingWindow(
_redis_client, settings.rate_limit_requests, settings.rate_limit_window_seconds _redis_client, settings.rate_limit_requests, settings.rate_limit_window_seconds
+5 -18
View File
@@ -1,43 +1,30 @@
"""Base model and mixins for all CartSnitch ORM models.""" """Base model and mixins for all CartSnitch ORM models."""
import uuid import uuid
from datetime import UTC, datetime from datetime import datetime
from sqlalchemy import DateTime, func from sqlalchemy import DateTime, func
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column
from cartsnitch_api.types import GuidType
class Base(DeclarativeBase): class Base(DeclarativeBase):
"""Base class for all CartSnitch models.""" """Base class for all CartSnitch models."""
def _utcnow():
return datetime.now(UTC)
class TimestampMixin: class TimestampMixin:
"""Mixin providing created_at / updated_at columns.""" """Mixin providing created_at / updated_at columns."""
created_at: Mapped[datetime] = mapped_column( created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), DateTime(timezone=True), server_default=func.now(), nullable=False
server_default=func.now(),
default=_utcnow,
nullable=False,
) )
updated_at: Mapped[datetime] = mapped_column( updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), DateTime(timezone=True), server_default=func.now(), onupdate=func.now(), nullable=False
server_default=func.now(),
onupdate=_utcnow,
default=_utcnow,
nullable=False,
) )
class UUIDPrimaryKeyMixin: class UUIDPrimaryKeyMixin:
"""Mixin providing a UUID primary key using GuidType for cross-DB compatibility.""" """Mixin providing a UUID primary key."""
id: Mapped[uuid.UUID] = mapped_column( id: Mapped[uuid.UUID] = mapped_column(
GuidType(), primary_key=True, default=uuid.uuid4 primary_key=True, default=uuid.uuid4, server_default=func.gen_random_uuid()
) )
+1 -2
View File
@@ -18,7 +18,7 @@ from sqlalchemy import (
) )
from sqlalchemy.orm import Mapped, mapped_column, relationship from sqlalchemy.orm import Mapped, mapped_column, relationship
from cartsnitch_api.models.base import Base, TimestampMixin, UUIDPrimaryKeyMixin, _utcnow from cartsnitch_api.models.base import Base, TimestampMixin, UUIDPrimaryKeyMixin
if TYPE_CHECKING: if TYPE_CHECKING:
from cartsnitch_api.models.price import PriceHistory from cartsnitch_api.models.price import PriceHistory
@@ -46,7 +46,6 @@ class Purchase(UUIDPrimaryKeyMixin, TimestampMixin, Base):
ingested_at: Mapped[datetime] = mapped_column( ingested_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), DateTime(timezone=True),
server_default=func.now(), server_default=func.now(),
default=_utcnow,
nullable=False, nullable=False,
) )
+3 -6
View File
@@ -1,7 +1,6 @@
"""User and UserStoreAccount models.""" """User and UserStoreAccount models."""
import secrets import secrets
import uuid
from datetime import datetime from datetime import datetime
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
@@ -11,7 +10,7 @@ from sqlalchemy.orm import Mapped, mapped_column, relationship
from cartsnitch_api.constants import AccountStatus from cartsnitch_api.constants import AccountStatus
from cartsnitch_api.models.base import Base, TimestampMixin, UUIDPrimaryKeyMixin from cartsnitch_api.models.base import Base, TimestampMixin, UUIDPrimaryKeyMixin
from cartsnitch_api.types import EncryptedJSON, GuidType from cartsnitch_api.types import EncryptedJSON
if TYPE_CHECKING: if TYPE_CHECKING:
from cartsnitch_api.models.purchase import Purchase from cartsnitch_api.models.purchase import Purchase
@@ -23,13 +22,11 @@ class User(TimestampMixin, Base):
__tablename__ = "users" __tablename__ = "users"
id: Mapped[uuid.UUID] = mapped_column(GuidType(), primary_key=True, default=uuid.uuid4) id: Mapped[str] = mapped_column(Text, primary_key=True)
email: Mapped[str] = mapped_column(String(255), nullable=False, unique=True) email: Mapped[str] = mapped_column(String(255), nullable=False, unique=True)
hashed_password: Mapped[str | None] = mapped_column(String(255), nullable=True) hashed_password: Mapped[str | None] = mapped_column(String(255), nullable=True)
display_name: Mapped[str | None] = mapped_column(String(100)) display_name: Mapped[str | None] = mapped_column(String(100))
email_verified: Mapped[bool] = mapped_column( email_verified: Mapped[bool] = mapped_column(Boolean, nullable=False, server_default="false")
Boolean, nullable=False, default=False, server_default="false"
)
image: Mapped[str | None] = mapped_column(Text, nullable=True) image: Mapped[str | None] = mapped_column(Text, nullable=True)
email_inbound_token: Mapped[str] = mapped_column( email_inbound_token: Mapped[str] = mapped_column(
String(22), String(22),
+27 -3
View File
@@ -1,16 +1,40 @@
"""Health check and error metrics endpoints.""" """Health check and error metrics endpoints."""
from fastapi import APIRouter, Depends import logging
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession
from cartsnitch_api.auth.dependencies import verify_service_key from cartsnitch_api.auth.dependencies import verify_service_key
from cartsnitch_api.database import get_db
from cartsnitch_api.middleware.error_handler import get_error_monitor from cartsnitch_api.middleware.error_handler import get_error_monitor
logger = logging.getLogger(__name__)
router = APIRouter(tags=["health"]) router = APIRouter(tags=["health"])
@router.get("/health") @router.get("/health")
async def health(): async def health(db: AsyncSession = Depends(get_db)):
return {"status": "ok"} """Liveness + DB connectivity probe.
Returns HTTP 200 when the API process is responsive *and* the database
is reachable, so Kubernetes readiness probes can correctly route traffic
away from pods that have lost their database connection.
Returns HTTP 503 when the database is unreachable so K8s marks the pod
unhealthy and stops sending traffic to it.
"""
try:
await db.execute(text("SELECT 1"))
except Exception as exc:
logger.exception("Health check failed: database unreachable")
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail={"status": "unavailable", "database": "disconnected"},
) from exc
return {"status": "ok", "database": "connected"}
@router.get("/internal/error-stats", dependencies=[Depends(verify_service_key)]) @router.get("/internal/error-stats", dependencies=[Depends(verify_service_key)])
+1 -26
View File
@@ -1,10 +1,9 @@
"""Custom SQLAlchemy column types.""" """Custom SQLAlchemy column types."""
import json import json
import uuid as uuid_lib
from cryptography.fernet import Fernet from cryptography.fernet import Fernet
from sqlalchemy import String, Text from sqlalchemy import Text
from sqlalchemy.types import TypeDecorator from sqlalchemy.types import TypeDecorator
from cartsnitch_api.config import settings from cartsnitch_api.config import settings
@@ -35,27 +34,3 @@ class EncryptedJSON(TypeDecorator):
return None return None
decrypted = _get_fernet().decrypt(value.encode()) decrypted = _get_fernet().decrypt(value.encode())
return json.loads(decrypted) return json.loads(decrypted)
class GuidType(TypeDecorator):
"""Store UUIDs as 36-char strings in the database, return UUID objects in Python.
Uses PostgreSQL UUID type when available, String(36) otherwise (SQLite).
"""
impl = String(36)
cache_ok = True
def process_bind_param(self, value, dialect):
if value is None:
return None
if isinstance(value, uuid_lib.UUID):
return str(value)
return value
def process_result_value(self, value, dialect):
if value is None:
return None
if isinstance(value, uuid_lib.UUID):
return value
return uuid_lib.UUID(value)
+28 -18
View File
@@ -8,7 +8,6 @@ import secrets
import uuid import uuid
from datetime import UTC, datetime, timedelta from datetime import UTC, datetime, timedelta
import aiosqlite
import pytest import pytest
from httpx import ASGITransport, AsyncClient from httpx import ASGITransport, AsyncClient
from sqlalchemy import create_engine, event, text from sqlalchemy import create_engine, event, text
@@ -20,7 +19,14 @@ from cartsnitch_api.database import get_db
from cartsnitch_api.main import create_app from cartsnitch_api.main import create_app
from cartsnitch_api.models import Base from cartsnitch_api.models import Base
aiosqlite.register_adapter(uuid.UUID, lambda u: str(u))
def _set_timestamp_defaults(mapper, connection, target):
"""Populate created_at/updated_at before insert for SQLite compatibility."""
now = datetime.now(UTC)
for col in [c for c in mapper.columns if c.key in ("created_at", "updated_at")]:
if getattr(target, col.key, None) is None:
setattr(target, col.key, now)
TEST_JWT_SECRET = secrets.token_urlsafe(32) TEST_JWT_SECRET = secrets.token_urlsafe(32)
TEST_SERVICE_KEY = secrets.token_urlsafe(32) TEST_SERVICE_KEY = secrets.token_urlsafe(32)
@@ -56,29 +62,28 @@ def disable_rate_limiting():
def engine(): def engine():
"""Sync in-memory SQLite engine for model unit tests. """Sync in-memory SQLite engine for model unit tests.
Strips ALL PostgreSQL-specific server_default expressions so SQLite can Strips PostgreSQL-specific server_default expressions and provides
handle all column inserts without missing-function errors. Python-side defaults for SQLite compatibility.
""" """
eng = create_engine("sqlite:///:memory:") eng = create_engine("sqlite:///:memory:")
@event.listens_for(eng, "connect") for tbl in Base.metadata.tables.values():
def set_sqlite_pragma(dbapi_connection, connection_record): for col in tbl.columns.values():
cursor = dbapi_connection.cursor()
cursor.execute("PRAGMA foreign_keys=ON")
cursor.close()
for metadata_table in Base.metadata.tables.values():
for col in metadata_table.columns.values():
sd = col.server_default sd = col.server_default
if sd is not None: if sd is not None:
if not hasattr(sd, "expression"): if not hasattr(sd, "expression"):
col.server_default = None col.server_default = None
continue continue
expr_str = str(sd.expression).lower() expr_str = str(sd.expression).lower()
_pg_fns = ("gen_random_uuid", "gen_random_bytes", "now()") # Strip PostgreSQL-specific defaults
if any(pg_fn in expr_str for pg_fn in _pg_fns): if any(x in expr_str for x in ["gen_random_uuid", "gen_random_bytes", "now()"]):
col.server_default = None col.server_default = None
# Register event listener to populate timestamps on insert
for cls in Base.registry._class_registry.values():
if hasattr(cls, "__mapper__"):
event.listen(cls, "before_insert", _set_timestamp_defaults)
Base.metadata.create_all(eng) Base.metadata.create_all(eng)
yield eng yield eng
eng.dispose() eng.dispose()
@@ -102,18 +107,23 @@ async def db_engine():
cursor.execute("PRAGMA foreign_keys=ON") cursor.execute("PRAGMA foreign_keys=ON")
cursor.close() cursor.close()
for table in Base.metadata.tables.values(): for tbl in Base.metadata.tables.values():
for col in table.columns.values(): for col in tbl.columns.values():
sd = col.server_default sd = col.server_default
if sd is not None: if sd is not None:
if not hasattr(sd, "expression"): if not hasattr(sd, "expression"):
col.server_default = None col.server_default = None
continue continue
expr_str = str(sd.expression).lower() expr_str = str(sd.expression).lower()
_pg_fns = ("gen_random_uuid", "gen_random_bytes", "now()") # Strip PostgreSQL-specific defaults
if any(pg_fn in expr_str for pg_fn in _pg_fns): if any(x in expr_str for x in ["gen_random_uuid", "gen_random_bytes", "now()"]):
col.server_default = None col.server_default = None
# Register event listener to populate timestamps on insert
for cls in Base.registry._class_registry.values():
if hasattr(cls, "__mapper__"):
event.listen(cls, "before_insert", _set_timestamp_defaults)
async with engine.begin() as conn: async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all) await conn.run_sync(Base.metadata.create_all)
await conn.execute( await conn.execute(
+3 -4
View File
@@ -18,16 +18,15 @@ from cartsnitch_api.models.user import User, UserStoreAccount
def engine(): def engine():
eng = create_engine("sqlite:///:memory:") eng = create_engine("sqlite:///:memory:")
for metadata_table in Base.metadata.tables.values(): for tbl in Base.metadata.tables.values():
for col in metadata_table.columns.values(): for col in tbl.columns.values():
sd = col.server_default sd = col.server_default
if sd is not None: if sd is not None:
if not hasattr(sd, "expression"): if not hasattr(sd, "expression"):
col.server_default = None col.server_default = None
continue continue
expr_str = str(sd.expression).lower() expr_str = str(sd.expression).lower()
_pg_fns = ("gen_random_uuid", "gen_random_bytes", "now()") if "gen_random_uuid" in expr_str or "gen_random_bytes" in expr_str:
if any(pg_fn in expr_str for pg_fn in _pg_fns):
col.server_default = None col.server_default = None
Base.metadata.create_all(eng) Base.metadata.create_all(eng)