Fix Booking.com scrape saves — widen room_type to TEXT + rollback per failure

Scraped room descriptions exceed VARCHAR(100) (StringDataRightTruncation),
and the failed insert poisoned the transaction so every subsequent save in
the batch died with InFailedSqlTransaction — scrapes reported success with
0 rows saved. Widen the column (with ALTER for existing tables) and roll
back after a failed save.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
jtricerolph 2026-07-05 14:32:20 +00:00
parent f51e1dcb76
commit 75c2adf803
2 changed files with 7 additions and 1 deletions

View file

@ -349,6 +349,9 @@ async def scrape_date(
rates_saved += 1
except Exception as e:
logger.warning(f"Error saving hotel/rate: {e}")
# A failed statement aborts the transaction — roll back so the
# remaining hotels in this batch can still be saved
db.rollback()
continue
db.commit()