rates/backend/requirements.txt
jtricerolph 349c795708 Scraper stealth: headful+xvfb, playwright-stealth, coherent UA, warm-up, click-paginate
Tier 1+2 anti-detection to stop Booking.com throttling page 2:
- Run Chromium headful under xvfb (Dockerfile) — headless leaks SwiftShader
  WebGL, empty plugins, missing chrome.runtime
- playwright-stealth patches navigator.webdriver/plugins/WebGL vendor
- Single coherent Chrome-121 identity: UA + matching sec-ch-ua client hints +
  platform (dropped the Firefox/Safari UA strings — a mismatched UA on a
  Chromium engine is a stronger tell than no rotation)
- Homepage warm-up so search requests carry real session cookies + cookie
  consent dismiss
- Paginate by clicking next (offset= deep-link was the page-2 tell), offset
  URL kept as fallback
- Viewport jitter, mouse movement, longer scroll, selector retry on lazy load

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 16:26:01 +00:00

15 lines
322 B
Text

fastapi==0.109.0
uvicorn[standard]==0.27.0
sqlalchemy==2.0.25
asyncpg==0.29.0
psycopg2-binary==2.9.9
python-jose[cryptography]==3.3.0
python-multipart==0.0.6
httpx==0.26.0
apscheduler==3.10.4
pydantic==2.5.3
pydantic-settings==2.1.0
python-dotenv==1.0.0
python-dateutil==2.8.2
playwright>=1.40.0
playwright-stealth==1.0.6