Retry page load timeouts: treat as blocks, bump max retries to 4
DataImpulse endpoints are intermittently broken — some return EOF after CONNECT immediately, others work fine. DNS round-robins between them so the same request can fail 3× then succeed on the 4th. Timeouts were returning blocked=False and silently failing with no retry. Now treated as retryable blocks (with context rotation) so a fresh endpoint is tried. Max retries 2→4 (5 total attempts) to give enough chances to land a working DataImpulse endpoint. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
28c4dd9f54
commit
b44e688b3e
2 changed files with 6 additions and 2 deletions
|
|
@ -318,7 +318,7 @@ async def scrape_hotel_date(
|
|||
backend: 'PlaywrightHotelPageBackend',
|
||||
batch_id: uuid.UUID,
|
||||
adults: int = 2,
|
||||
max_retries: int = 2,
|
||||
max_retries: int = 4,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Scrape all rate plans for one hotel on one date via the hotel page backend.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue