Add hotel-page scraper backend with full room/rate plan extraction
Switches booking.com scraping from search-results page (Cloudflare-targeted) to individual hotel property pages (not CF-protected). Each page load returns all room types, all rate plan variants (room-only/B&B × refundable/non-ref × 1-2 adults), and availability counts. Key changes: - New PlaywrightHotelPageBackend: proxy reuse until block, rotate on CF/WAF - booking_scraper.py: _run_hotel_page_scrape(), scrape_hotel_date(), _scrape_hotels_concurrent() — sharded by hotel so one proxy session covers all dates for one hotel (looks human) - schema.sql: ADD COLUMN rate_plan_id, max_persons on booking_com_rates - competitors API: filter max_persons=2, order by rate_gross ASC as tiebreaker so DISTINCT ON returns cheapest 2-adult rate from latest batch Enable via Settings → Scraper Backend → playwright_hotel_page Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
b10b2f5990
commit
bb37fcf501
6 changed files with 608 additions and 8 deletions
|
|
@ -36,6 +36,8 @@ class RateData:
|
|||
no_prepayment: Optional[bool] = None
|
||||
rooms_left: Optional[int] = None # "Only X rooms left"
|
||||
available_qty: Optional[int] = None # Future: from hotel page dropdown
|
||||
rate_plan_id: Optional[str] = None # block_id from hotel page (identifies rate variant)
|
||||
max_persons: Optional[int] = None # Occupancy this rate applies to
|
||||
|
||||
|
||||
@dataclass
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue