The proxy-off → serial guard was correct for search-results (all workers hit the same URL), but hotel-page workers each scrape a different hotel's property page — parallel requests look like multi-tab browsing, not a hammered aggregation endpoint. Added require_proxy param to _effective_concurrency; hotel-page path passes require_proxy=False. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| api | ||
| jobs | ||
| services | ||
| auth.py | ||
| database.py | ||
| Dockerfile | ||
| main.py | ||
| migrate_direct_data.py | ||
| requirements.txt | ||
| scheduler.py | ||
| schema.sql | ||