hotel-manage-stack/rates
1
0
Fork 0
Code Issues 3 Pull requests Projects Releases Packages Wiki Activity Actions
rates/backend/services
History
jtricerolph 11b995739c Add residential proxy support to Booking.com scraper 
- Route through DataImpulse residential proxy when BOOKING_PROXY_* env
  vars are set (empty = direct connection, unchanged behaviour)
- Sticky one IP per session via DataImpulse sessid; rotate_session()
  burns the IP for a fresh one
- Rotate-on-block: retry a date on a new IP when the page is challenged
  or page 1 renders zero hotels (up to 3 IPs; single attempt sans proxy)
- Persistent, pre-warmed context across dates so cache stays hot
  (~5 MB first page, ~0.3 MB per date after) instead of per-date cold loads
- Block images/media/fonts and third-party ad/consent/analytics hosts to
  cut bandwidth; never touch the anti-bot challenge (awswaf)
- Faster inter-page pacing now that a burned IP is cheap

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 20:30:15 +00:00
..
direct_profiles Add Rate Monitor app — Booking.com + direct booking engine competitor rates 2026-07-05 12:06:30 +00:00
scraper_backends Add residential proxy support to Booking.com scraper 2026-07-05 20:30:15 +00:00
__init__.py Add Rate Monitor app — Booking.com + direct booking engine competitor rates 2026-07-05 12:06:30 +00:00
booking_scraper.py Scraper: process-wide lock, one scrape at a time 2026-07-05 16:14:55 +00:00
central_settings.py Add Rate Monitor app — Booking.com + direct booking engine competitor rates 2026-07-05 12:06:30 +00:00
direct_scraper.py Add Rate Monitor app — Booking.com + direct booking engine competitor rates 2026-07-05 12:06:30 +00:00
newbook_rates_client.py Port occupancy sync + category ordering from the forecasting version 2026-07-05 14:18:03 +00:00