DataImpulse has two broken gateway nodes (.89, .97) and one working (.105).
Production DNS round-robin consistently landed on broken nodes. Fix is to
pin the proxy host to 67.213.121.105 in system_config. Revert to Chromium.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Booking.com detects headless Chromium via TLS ClientHello fingerprint (JA3)
and immediately flags the residential proxy IP on first contact. Firefox has
a different TLS fingerprint and GREASE extensions that aren't yet flagged.
Image blocking via Firefox preference (permissions.default.image=2) replaces
the Chrome --blink-settings flag. Dockerfile updated to install both browsers.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Headless mode gets blocked by Cloudflare on the search endpoint. Headed
mode with Xvfb works on dev (Unraid). The missing piece on Proxmox LXC was
insufficient /dev/shm (Docker default 64 MB); setting shm_size: 256m gives
Xvfb enough shared memory to start.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Xvfb exits immediately on the production LXC (no framebuffer device
support), so headed mode was never working. Switch to headless=True
and remove Xvfb from the Dockerfile entirely.
Stealth coverage is unchanged: playwright-stealth patches webdriver,
plugins and chrome.runtime; the residential proxy + sticky session
provide the GB residential fingerprint; homepage warmup carries real
session cookies into the search. --enable-unsafe-swiftshader and
--disable-blink-features=AutomationControlled are kept.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
xvfb-run as container PID 1 hangs before exec'ing uvicorn — its Xvfb
readiness handshake (SIGUSR1) is never delivered to PID 1, so the backend
sat 'Up (unhealthy)' with no logs and the frontend never started.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tier 1+2 anti-detection to stop Booking.com throttling page 2:
- Run Chromium headful under xvfb (Dockerfile) — headless leaks SwiftShader
WebGL, empty plugins, missing chrome.runtime
- playwright-stealth patches navigator.webdriver/plugins/WebGL vendor
- Single coherent Chrome-121 identity: UA + matching sec-ch-ua client hints +
platform (dropped the Firefox/Safari UA strings — a mismatched UA on a
Chromium engine is a stronger tell than no rotation)
- Homepage warm-up so search requests carry real session cookies + cookie
consent dismiss
- Paginate by clicking next (offset= deep-link was the page-2 tell), offset
URL kept as fallback
- Viewport jitter, mouse movement, longer scroll, selector retry on lazy load
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
