Both competitor booking engines changed their API contracts, silently
breaking their direct scrapes ~16 days ago (200 responses with empty
bodies, so no exception was ever raised):
- Guestline: the /enhanced availability endpoint now returns a bare []
for every date. Switch to the base /api/availabilities/{coll}/{hotel}
endpoint, which returns {"rooms":[...]} with the same room shape.
- Mews: getAvailability now rejects the old body with "Invalid
EnterpriseId" — it requires enterpriseId + serviceId. getPricing
additionally requires currencyCode. Capture the enterprise's
defaultCurrencyCode in _ensure_config and send all three.
Also stop last_scraped_at advancing when a run saves 0 rows, so the
overview no longer reports a phantom-fresh scrape while the per-date
data stays stale — the symptom that surfaced this.
Verified live: Three Ways now yields 41 rows/night, Old Stocks 10.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Proxy config, DataImpulse sticky-session username syntax, and Playwright/
httpx proxy builders now live in one place (services/proxy.py) instead of
being duplicated across the Booking.com backend and the /config/proxy test
endpoint. Both scrapers consume it.
- services/proxy.py: load_config/normalize (DB-authoritative, env fallback),
new_session_id, username, playwright_proxy, httpx_proxy_url
- PlaywrightLocalBackend delegates proxy building to the module
- get_scraper_backend factory uses proxy.load_config (one resolution path)
- test_proxy_config endpoint uses the shared URL builder; httpx proxies= ->
proxy= (forward-compatible, 0.28-safe)
- Direct booking-engine scraper (httpx) can now route through the same proxy,
gated by the direct_scraper_use_proxy flag (default off, plumbing ready)
Dead code removed: set_scraper_paused (never called — rotate-on-block
replaced pause-on-block), get_competitor_matrix / get_hotels_list /
update_hotel_tier (endpoints have their own SQL), unused PROXY_KEYS tuple.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
