hotel-manage-stack/rates
1
0
Fork 0
Code Issues 3 Pull requests Projects Releases Packages Wiki Activity Actions

Auto-recheck parity after scrape + manual Recheck button + faster badge

Browse source 
Parity logic:
- Extract _fetch_latest_alerts() and _upsert_alerts() helpers so the alert
  upsert loop is no longer duplicated between the daily job and per-date runs
- Add run_parity_check_for_date(date) which runs the full comparison +
  alert upsert for a single date

Scraper integration:
- _safe_parity_recheck(date) wrapper (never raises) called after each
  successful date in both search-results and hotel-page workers; hotel-page
  mode waits until all hotels for the date are done before rechecking

API:
- POST /competitors/parity/check-date?rate_date=YYYY-MM-DD for manual recheck

Frontend:
- Recheck button on every parity alert row (all statuses); invalidates
  alerts list and badge count on success
- parity-alert-count badge polls every 15s (was 60s) so new alerts from
  the scheduled job or post-scrape rechecks appear quickly

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
jtricerolph 2026-07-15 09:33:33 +00:00
parent 8a80c66a3b
commit cab1a39503
5 changed files with 146 additions and 66 deletions
Download patch file Download diff file Expand all files Collapse all files

15
backend/services/booking_scraper.py
View file

@ -613,6 +613,7 @@ async def _scrape_hotels_concurrent(
backend = PlaywrightHotelPageBackend(proxy_config=proxy_util.load_config(wdb))
try:
for rate_date in date_shard:
date_rates = 0
for hotel in hotels_seen:
try:
result = await scrape_hotel_date(wdb, hotel, rate_date, backend, batch_id, adults)
@ -632,12 +633,16 @@ async def _scrape_hotels_concurrent(
elif result['success']:
acc['rates'] += result['rates_count']
acc['completed'] += 1
date_rates += result['rates_count']
_increment_batch_progress(wdb, batch_id,
delta_rates=result['rates_count'],
delta_completed=1)
else:
acc['failed'] += 1
_increment_batch_progress(wdb, batch_id, delta_failed=1)
if date_rates > 0:
_safe_parity_recheck(rate_date)
finally:
try:
await backend.close()
@ -687,6 +692,15 @@ def _increment_batch_progress(
pass
def _safe_parity_recheck(rate_date: date):
"""Re-run parity check for one date after it has been successfully scraped. Never raises."""
try:
from jobs.check_rate_parity import run_parity_check_for_date
run_parity_check_for_date(rate_date)
except Exception as e:
logger.warning(f"Post-scrape parity recheck for {rate_date} failed: {e}")
def _safe_mark_queue(db: Session, queue_id: Optional[int], status: str, error: str = None):
"""mark_queue_item that never raises — a marking failure shouldn't kill a worker."""
if queue_id is None:
@ -749,6 +763,7 @@ async def _scrape_dates_concurrent(
delta_hotels=result['hotels_count'],
delta_rates=result['rates_count'],
delta_completed=1)
_safe_parity_recheck(rate_date)
else:
acc['failed'] += 1
_safe_mark_queue(wdb, queue_id, 'failed', result.get('error'))