Fix Booking.com scrape saves — widen room_type to TEXT + rollback per failure
Scraped room descriptions exceed VARCHAR(100) (StringDataRightTruncation), and the failed insert poisoned the transaction so every subsequent save in the batch died with InFailedSqlTransaction — scrapes reported success with 0 rows saved. Widen the column (with ALTER for existing tables) and roll back after a failed save. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
f51e1dcb76
commit
75c2adf803
2 changed files with 7 additions and 1 deletions
|
|
@ -349,6 +349,9 @@ async def scrape_date(
|
|||
rates_saved += 1
|
||||
except Exception as e:
|
||||
logger.warning(f"Error saving hotel/rate: {e}")
|
||||
# A failed statement aborts the transaction — roll back so the
|
||||
# remaining hotels in this batch can still be saved
|
||||
db.rollback()
|
||||
continue
|
||||
|
||||
db.commit()
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue