Add hotel-page scraper backend with full room/rate plan extraction

Switches booking.com scraping from search-results page (Cloudflare-targeted)
to individual hotel property pages (not CF-protected). Each page load returns
all room types, all rate plan variants (room-only/B&B × refundable/non-ref ×
1-2 adults), and availability counts.

Key changes:
- New PlaywrightHotelPageBackend: proxy reuse until block, rotate on CF/WAF
- booking_scraper.py: _run_hotel_page_scrape(), scrape_hotel_date(),
  _scrape_hotels_concurrent() — sharded by hotel so one proxy session covers
  all dates for one hotel (looks human)
- schema.sql: ADD COLUMN rate_plan_id, max_persons on booking_com_rates
- competitors API: filter max_persons=2, order by rate_gross ASC as tiebreaker
  so DISTINCT ON returns cheapest 2-adult rate from latest batch

Enable via Settings → Scraper Backend → playwright_hotel_page

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
jtricerolph 2026-07-09 23:08:03 +00:00
parent b10b2f5990
commit bb37fcf501
6 changed files with 608 additions and 8 deletions

View file

@ -36,6 +36,8 @@ class RateData:
no_prepayment: Optional[bool] = None
rooms_left: Optional[int] = None # "Only X rooms left"
available_qty: Optional[int] = None # Future: from hotel page dropdown
rate_plan_id: Optional[str] = None # block_id from hotel page (identifies rate variant)
max_persons: Optional[int] = None # Occupancy this rate applies to
@dataclass