Scraper: don't flag not_listed from partial scrapes

A page-load timeout was logged and skipped, so a scrape could 'succeed'
with only page 1 of results — and the not_listed flagging then marked
every page-2 hotel absent, suppressing their last known rates.

- ScraperResult now tracks pages_requested/pages_ok
- scrape_date skips not_listed flagging when pages failed or when the
  scrape saw <60% of the date's 7-day coverage baseline; scraped rates
  are still saved, unseen hotels keep last known rate + scrape time

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

backend/services/booking_scraper.py

@@ -357,10 +357,49 @@ async def scrape_date(
     # Flag known hotels absent from this successful scrape as 'not_listed'
     # (sold out or pushed off the search results). Without this their last
     # 'available' rate stays the latest row for the date and reads as a
-    # current price, skewing market averages. Skipped when the parse found
-    # nothing at all — that looks like a scraper fault, not real absence.
+    # current price, skewing market averages.
+    #
+    # Absence is only trustworthy when the scrape saw the WHOLE market:
+    # a partial scrape (a page timed out / never rendered) or one that saw
+    # far fewer hotels than this date normally lists would mark still-listed
+    # hotels not_listed and wrongly suppress their last known rates. In
+    # those cases keep the rates we did save but skip the flagging, so
+    # unseen hotels retain their last known rate and scrape time.
     seen_ids = [h.booking_com_id for h in result.hotels if h.booking_com_id]
-    if seen_ids:
+    flag_absent = bool(seen_ids)
+
+    if flag_absent and result.pages_ok < result.pages_requested:
+        logger.warning(
+            f"Partial scrape for {rate_date} ({result.pages_ok}/{result.pages_requested} "
+            f"pages ok, {len(seen_ids)} hotels) — skipping not_listed flagging"
+        )
+        flag_absent = False
+
+    if flag_absent:
+        try:
+            baseline = db.execute(
+                text("""
+                    SELECT COUNT(DISTINCT hotel_id)
+                    FROM booking_com_rates
+                    WHERE rate_date = :rate_date
+                      AND availability_status IN ('available', 'sold_out')
+                      AND scraped_at > NOW() - INTERVAL '7 days'
+                      AND scrape_batch_id != :batch_id
+                """),
+                {'rate_date': rate_date, 'batch_id': str(batch_id)}
+            ).scalar() or 0
+            if baseline and len(seen_ids) < baseline * 0.6:
+                logger.warning(
+                    f"Scrape for {rate_date} saw {len(seen_ids)} hotels vs recent "
+                    f"baseline {baseline} — skipping not_listed flagging"
+                )
+                flag_absent = False
+        except Exception as e:
+            logger.warning(f"Baseline check failed for {rate_date}: {e}")
+            db.rollback()
+            flag_absent = False
+
+    if flag_absent:
         try:
             db.execute(
                 text("""