kitchen/docs/archive/LLM-INTEGRATION-PLAN.md

LLM Integration Plan — Review & Enhancements

Supersedes the original 3-feature plan. Reviewed and expanded to cover 11 features across 6 phases, with master kill switch, removal manifest, cost guardrails, and detailed UX specification.

Context

The kitchen-invoice-flash system has several manual data entry bottlenecks where an LLM can reduce friction. Using Claude Haiku via the Anthropic Python SDK. Estimated cost: ~$0.50-1/month for a single kitchen. Mixed trigger approach: auto for cheap label parsing, user-triggered button for invoice analysis and ingredient matching.

Scope: Original 3 features + Tier 1 (A, B, C) + Tier 2 (D, E, F, G) + new Feature H (line item reconciliation). Instance model: Single instance (not multi-kitchen). Settings are global, not per-kitchen. API key storage: Plaintext in DB (matching existing pattern for Azure, NewBook, etc.).

---

1. What the Plan Gets Right

• Model choice: Claude Haiku is correct — structured extraction, not creative generation.
• API key in Settings: Follows the established KitchenSettings pattern (same as azure_key, resos_api_key, etc.).
• Structured output via tool_use: Guarantees parseable JSON. Right approach over prompt-based JSON.
• Graceful degradation: LLM features are additive, never blocking. Existing regex/trigram still works without a key.
• Trigger strategy: Auto for cheap label parsing, user-triggered buttons for invoice assist and ingredient matching.

---

2. Infrastructure Enhancements

2a. Client Instantiation

Single shared client instance, re-initialized if API key changes in settings.

2b. Response Caching

New llm_analysis_cache table:

• Columns: id, feature, input_hash (SHA-256), result_json, model_used, prompt_version, created_at
• Unique constraint: (feature, input_hash, prompt_version)
• Feature 1 (labels): cache by text hash, TTL 30 days. Feature 3 (matching): TTL 7 days. Feature 2 (invoices): skip caching.

2c. Usage Tracking

New llm_usage_log table:

• Columns: id, feature, model, input_tokens, output_tokens, latency_ms, success, error_message, created_at
• GET /api/settings/llm-usage endpoint (aggregated last 30 days). Display on Settings page.

2d. Configurable Model

llm_model column in KitchenSettings (default "claude-haiku-4-5-latest"). Settings UI shows dropdown (Haiku / Sonnet).

2e. Prompt Versioning

Constants in llm_service.py. Cache lookup includes prompt_version — version bump auto-invalidates stale cache.

2f. Error UX

Three-state llm_status in API responses: "success", "unavailable" (no key), "error" (call failed + message).

2g. Rate Limiting

asyncio.Semaphore(5) caps concurrent LLM calls.

2h. Master Kill Switch

llm_enabled (Boolean, default False) in KitchenSettings. When disabled: zero AI footprint in frontend, no API calls, no logging. See LLM-MANIFEST.md for full details.

2i. Removal Manifest

LLM-MANIFEST.md in project root — updated each phase. All LLM code marked with breadcrumb comments: LLM FEATURE — see LLM-MANIFEST.md for removal instructions.

2j. Cost Guardrails (5 layers)

1. Per-feature toggles — llm_features_enabled JSONB column, individually disable features
2. Monthly token budget — llm_monthly_token_limit (default 500,000 tokens ~$1.25/month)
3. Auto-trigger throttle — per-entity cooldown via cache check
4. Single-call token cap — max_tokens on every API call
5. Cost visibility — Usage stats card on Settings page

2k. Prompt Injection Mitigation

tool_use structured output mitigates this. System messages note input is "untrusted product/invoice text".

---

3. Features

Original Features

Feature 1: Product Label Allergen Parsing (Auto)

• Trigger: automatic when product_ingredients text is populated
• analyse_product_label(ingredients_text, flag_categories) → [{flag_id, status: "contains"|"may_contain"|"suitable_for", reason}]
• ~550 tokens/call, ~$0.001

Feature 2: Invoice OCR Assist (User-triggered)

• "AI Assist" button on Review page
• assist_invoice_ocr(ocr_data, supplier_list, line_items) → supplier match, date correction, pack size extraction, OCR corrections
• Batches line items in groups of 15-20 for 50+ line invoices
• ~1,900 tokens/call, ~$0.003

Feature 3: Smart Ingredient Matching (User-triggered)

• "AI Match" button when trigram results have low confidence
• rank_ingredient_matches(description, candidates) → re-ranked list with confidence scores
• ~650 tokens/call, ~$0.001

Tier 1 — High Value

A. Recipe Text Allergen Scanning

Same analyse_product_label() function with recipe text as input. Catches contextual allergens regex misses.

B. Menu Description Generation

"Generate Description" button in PublishToMenuModal. Customer-facing descriptions with allergen callout.

C. Dispute Email Drafting

"Draft Email" button on DisputeDetailModal. Professional supplier email requesting credit.

Tier 2 — Medium Value

D. Smart Duplicate Detection

"AI Check" button when creating ingredients. Reuses rank_ingredient_matches().

E. OCR Field Extraction Fallback

Automatic when regex returns null. Mark LLM-extracted fields with source: "llm".

F. Supplier Alias Resolution

LLM fallback when identify_supplier() returns no match.

G. Ingredient Yield Estimation

Auto-hint on ingredient creation: "Typical yield: ~85%".

H. Invoice Line Item Reconciliation (Auto)

Matches unmatched line items against supplier's own historical naming from past 90 days.

---

4. Frontend UX

Core Principles

1. Suggestions only, never auto-change data (except allergen "Contains" following existing Brakes pattern)
2. Visible loading + notification for auto-triggered features
3. Sparkle icon as consistent AI indicator

Auto-triggered Features

Feature	Where	Visual
Label Parsing	IngredientModal	Spinner → toast → sparkle icon suggestions
Recipe Scanning	RecipeEditor	Spinner on flag matrix → sparkle suggestions
OCR Fallback	Invoice upload	Dashed amber border + "AI extracted" tooltip
Reconciliation	Review page	Amber "AI match" badge

User-triggered Features

Feature	Where	Trigger
AI Assist	Review top bar	Button → yellow-highlighted suggestions
AI Match	IngredientModal	Button → re-sorted dropdown
Menu Description	PublishToMenuModal	"Generate" button → pre-filled textarea
Dispute Email	DisputeDetailModal	"Draft Email" button → pre-filled text
Duplicate Detection	IngredientModal	"AI Check" button → warning panel
Supplier Alias	Review page	Auto suggestion banner
Yield Estimation	IngredientModal	Auto hint below field

---

5. OCR Correction Enhancements (Feature 2)

LLM significantly enhances 7 existing OCR correction scenarios:

1. Qty × Price ≠ Total — identifies which field Azure misread
2. Line items vs invoice total mismatch — identifies delivery charges, subtotal rows
3. Description content vs value — recommends which is the correct description
4. SKU in description — distinguishes product codes from descriptions
5. Weight-as-quantity — handles non-standard weight formats
6. Subtotal/discount row detection — catches "Goods Total", "Delivery Surcharge", etc.
7. Gross-to-net VAT — identifies VAT treatment from invoice context

---

6. Implementation Order

Phase 1: Infrastructure ✅

• backend/services/llm_service.py — client, caching, logging, rate limiting
• backend/requirements.txt — anthropic>=0.40.0
• backend/models/settings.py — LLM columns
• backend/models/llm.py — usage log + cache models
• backend/migrations/add_llm_infrastructure.py
• backend/api/settings.py — LLM fields, usage stats endpoint
• LLM-MANIFEST.md — removal manifest

Phase 2: Label Parsing + Recipe Text (Features 1 + A)

• analyse_product_label() with caching
• /analyse-label endpoint
• Frontend auto-trigger + sparkle suggestions

Phase 3: Invoice OCR Assist + Reconciliation (Features 2 + E + H)

• assist_invoice_ocr() with batching
• reconcile_line_items() for supplier history
• "AI Assist" button + suggestion UI
• LLM fallback for regex field extraction

Phase 4: Ingredient Matching + Supplier (Features 3 + D + F)

• rank_ingredient_matches(), supplier alias matching
• "AI Match" button, duplicate detection

Phase 5: Text Generation (Features B + C)

• Menu description generation
• Dispute email drafting

Phase 6: Polish + Yield (Feature G)

• Yield estimation hints
• Usage dashboard
• Prompt tuning

---

7. Verification Checklist

1. Add API key in Settings → saved, model dropdown works
2. llm_enabled = False (default) → zero AI footprint anywhere
3. Enable → full LLM settings section appears
4. Ingredient with "Contains: wheat flour, milk" → Gluten, Dairy suggestions
5. Upload invoice → "AI Assist" → corrections + suggestions
6. Invalid API key → toast error, regex/trigram still works
7. Budget exceeded → graceful degradation
8. All AI suggestions visually distinct with sparkle icon