Add stale-scrape health alerting (email + in-app banner)

Silent scraper breakage (200-with-empty-body) went unnoticed for 16 days
because nothing flagged it. Now, after each daily direct scrape, a health
check compares every enabled competitor's freshest captured rate against a
staleness threshold (default 5 days, config: scrape_health_stale_days).

State lives in a new direct_scrape_health table so we alert on transitions
only — one email when a competitor goes stale, one when it recovers, never a
daily repeat. Email uses the stack's shared SMTP integration via
central_settings (recipient: scrape_health_alert_email, else SMTP reply_to/
from) — no new mail secrets in this app. The current stale set also drives an
in-app warning banner on the Direct Rates page (GET /direct/health), with an
on-demand re-check endpoint (POST /direct/health/check).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
jtricerolph 2026-08-13 15:44:03 +00:00
parent b916f1a09b
commit ea240b3f96
6 changed files with 261 additions and 1 deletions

View file

@ -0,0 +1,52 @@
"""
Minimal SMTP sender for the rates app.
Reuses the stack's shared SMTP integration (managed once in the Settings
app, same as the tasks app's mailer) via central_settings — no per-app mail
secrets. Fire-and-forget: a mail failure must never break the scheduler.
"""
import logging
import os
import smtplib
from email.message import EmailMessage
from services.central_settings import get_integration_sync
logger = logging.getLogger(__name__)
HOTEL_NAME = os.getenv("VITE_HOTEL_NAME", "Hotel")
def send_mail(to: str, subject: str, body: str) -> bool:
"""Send a plain-text email through the central SMTP integration.
Returns True on success. Never raises logs and returns False instead."""
if not to:
return False
cfg = get_integration_sync("smtp")
if not cfg or not cfg.get("host"):
logger.warning("SMTP not configured in central settings — email skipped")
return False
try:
msg = EmailMessage()
msg["From"] = cfg.get("from") or f"{HOTEL_NAME} Rates <noreply@localhost>"
msg["To"] = to
msg["Subject"] = subject
if cfg.get("reply_to"):
msg["Reply-To"] = cfg["reply_to"]
msg.set_content(body)
port = int(cfg.get("port") or 587)
with smtplib.SMTP(cfg["host"], port, timeout=15) as s:
if port != 465:
try:
s.starttls()
except smtplib.SMTPException:
pass # server without STARTTLS (e.g. local relay)
if cfg.get("user"):
s.login(cfg["user"], cfg.get("pass") or "")
s.send_message(msg)
return True
except Exception as e:
logger.error(f"Email send failed: {e}")
return False