jtricerolph
1b985e5a16
The proxy-off → serial guard was correct for search-results (all workers hit the same URL), but hotel-page workers each scrape a different hotel's property page — parallel requests look like multi-tab browsing, not a hammered aggregation endpoint. Added require_proxy param to _effective_concurrency; hotel-page path passes require_proxy=False. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| direct_profiles | ||
| scraper_backends | ||
| __init__.py | ||
| booking_scraper.py | ||
| central_settings.py | ||
| direct_scraper.py | ||
| newbook_rates_client.py | ||
| proxy.py | ||