jtricerolph
11b995739c
- Route through DataImpulse residential proxy when BOOKING_PROXY_* env vars are set (empty = direct connection, unchanged behaviour) - Sticky one IP per session via DataImpulse sessid; rotate_session() burns the IP for a fresh one - Rotate-on-block: retry a date on a new IP when the page is challenged or page 1 renders zero hotels (up to 3 IPs; single attempt sans proxy) - Persistent, pre-warmed context across dates so cache stays hot (~5 MB first page, ~0.3 MB per date after) instead of per-date cold loads - Block images/media/fonts and third-party ad/consent/analytics hosts to cut bandwidth; never touch the anti-bot challenge (awswaf) - Faster inter-page pacing now that a burned IP is cheap Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| api | ||
| jobs | ||
| services | ||
| auth.py | ||
| database.py | ||
| Dockerfile | ||
| main.py | ||
| migrate_direct_data.py | ||
| requirements.txt | ||
| scheduler.py | ||
| schema.sql | ||