No description
jtricerolph
b10b2f5990
When page.goto() times out while Cloudflare's JS challenge redirect is in progress, page.content() silently blocks until that navigation completes before raising — adding ~1m46s of hidden delay per blocked-page attempt. Fix: skip page.content() entirely when page_loaded=False (goto already timed out), setting content="" so block detection treats it as no signal. Also break out of the page loop early when page 1 failed to load with 0 hotels — no point spending another 60s on the offset-URL page 2 when the browser is in a Cloudflare challenge state. Per-blocked-attempt time: ~4 min → ~1 min (30s goto + 30s selectors). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|---|---|---|
| backend | ||
| frontend | ||
| .gitignore | ||
| docker-compose.yml | ||