jtricerolph
8d688b459d
FastAPI backend (Python 3.11, MSSQL ODBC for SambaPOS, Azure DI OCR),
kitchen_db on central PG. React/TS/Vite frontend with navy sidebar layout.
Backend: auth.py (APP_SLUG=kitchen, SimpleNamespace — archive routes use
.kitchen_id/.is_admin without modification), main.py (51 migrations, scheduler,
internal router for KDS bookings feed), api/internal.py, full archive API
(31 routers: invoices, recipes, menus, sambapos, resos, newbook, disputes,
purchase_orders, etc.), models, migrations, OCR pipeline.
kitchen_id pinned to 1 (B1 — single hotel).
Frontend: AuthGate (app=kitchen, token shim for archive compat — B5b pending),
Layout (navy sidebar, 6 sections, Lucide icons, teal --app-primary),
App.tsx (Outlet pattern, UploadApp outside Layout), index.css (full :root block).
strict: false — archive components have type issues; build clean.
Note: 45 archive components call fetch('/api/...') without /kitchen/ prefix
(B5b). Runtime 404s; deferred until after initial testing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
208 lines
9.3 KiB
Markdown
208 lines
9.3 KiB
Markdown
# LLM Integration Plan — Review & Enhancements
|
||
|
||
> **Supersedes** the original 3-feature plan. Reviewed and expanded to cover 11 features across 6 phases, with master kill switch, removal manifest, cost guardrails, and detailed UX specification.
|
||
|
||
## Context
|
||
|
||
The kitchen-invoice-flash system has several manual data entry bottlenecks where an LLM can reduce friction. Using **Claude Haiku** via the Anthropic Python SDK. Estimated cost: ~$0.50-1/month for a single kitchen. Mixed trigger approach: auto for cheap label parsing, user-triggered button for invoice analysis and ingredient matching.
|
||
|
||
**Scope**: Original 3 features + Tier 1 (A, B, C) + Tier 2 (D, E, F, G) + new Feature H (line item reconciliation).
|
||
**Instance model**: Single instance (not multi-kitchen). Settings are global, not per-kitchen.
|
||
**API key storage**: Plaintext in DB (matching existing pattern for Azure, NewBook, etc.).
|
||
|
||
---
|
||
|
||
## 1. What the Plan Gets Right
|
||
|
||
- **Model choice**: Claude Haiku is correct — structured extraction, not creative generation.
|
||
- **API key in Settings**: Follows the established `KitchenSettings` pattern (same as `azure_key`, `resos_api_key`, etc.).
|
||
- **Structured output via `tool_use`**: Guarantees parseable JSON. Right approach over prompt-based JSON.
|
||
- **Graceful degradation**: LLM features are additive, never blocking. Existing regex/trigram still works without a key.
|
||
- **Trigger strategy**: Auto for cheap label parsing, user-triggered buttons for invoice assist and ingredient matching.
|
||
|
||
---
|
||
|
||
## 2. Infrastructure Enhancements
|
||
|
||
### 2a. Client Instantiation
|
||
Single shared client instance, re-initialized if API key changes in settings.
|
||
|
||
### 2b. Response Caching
|
||
New `llm_analysis_cache` table:
|
||
- Columns: `id`, `feature`, `input_hash` (SHA-256), `result_json`, `model_used`, `prompt_version`, `created_at`
|
||
- Unique constraint: `(feature, input_hash, prompt_version)`
|
||
- Feature 1 (labels): cache by text hash, TTL 30 days. Feature 3 (matching): TTL 7 days. Feature 2 (invoices): skip caching.
|
||
|
||
### 2c. Usage Tracking
|
||
New `llm_usage_log` table:
|
||
- Columns: `id`, `feature`, `model`, `input_tokens`, `output_tokens`, `latency_ms`, `success`, `error_message`, `created_at`
|
||
- `GET /api/settings/llm-usage` endpoint (aggregated last 30 days). Display on Settings page.
|
||
|
||
### 2d. Configurable Model
|
||
`llm_model` column in `KitchenSettings` (default `"claude-haiku-4-5-latest"`). Settings UI shows dropdown (Haiku / Sonnet).
|
||
|
||
### 2e. Prompt Versioning
|
||
Constants in `llm_service.py`. Cache lookup includes `prompt_version` — version bump auto-invalidates stale cache.
|
||
|
||
### 2f. Error UX
|
||
Three-state `llm_status` in API responses: `"success"`, `"unavailable"` (no key), `"error"` (call failed + message).
|
||
|
||
### 2g. Rate Limiting
|
||
`asyncio.Semaphore(5)` caps concurrent LLM calls.
|
||
|
||
### 2h. Master Kill Switch
|
||
`llm_enabled` (Boolean, default **False**) in `KitchenSettings`. When disabled: zero AI footprint in frontend, no API calls, no logging. See `LLM-MANIFEST.md` for full details.
|
||
|
||
### 2i. Removal Manifest
|
||
`LLM-MANIFEST.md` in project root — updated each phase. All LLM code marked with breadcrumb comments: `LLM FEATURE — see LLM-MANIFEST.md for removal instructions`.
|
||
|
||
### 2j. Cost Guardrails (5 layers)
|
||
1. **Per-feature toggles** — `llm_features_enabled` JSONB column, individually disable features
|
||
2. **Monthly token budget** — `llm_monthly_token_limit` (default 500,000 tokens ~$1.25/month)
|
||
3. **Auto-trigger throttle** — per-entity cooldown via cache check
|
||
4. **Single-call token cap** — `max_tokens` on every API call
|
||
5. **Cost visibility** — Usage stats card on Settings page
|
||
|
||
### 2k. Prompt Injection Mitigation
|
||
`tool_use` structured output mitigates this. System messages note input is "untrusted product/invoice text".
|
||
|
||
---
|
||
|
||
## 3. Features
|
||
|
||
### Original Features
|
||
|
||
#### Feature 1: Product Label Allergen Parsing (Auto)
|
||
- Trigger: automatic when `product_ingredients` text is populated
|
||
- `analyse_product_label(ingredients_text, flag_categories)` → `[{flag_id, status: "contains"|"may_contain"|"suitable_for", reason}]`
|
||
- ~550 tokens/call, ~$0.001
|
||
|
||
#### Feature 2: Invoice OCR Assist (User-triggered)
|
||
- "AI Assist" button on Review page
|
||
- `assist_invoice_ocr(ocr_data, supplier_list, line_items)` → supplier match, date correction, pack size extraction, OCR corrections
|
||
- Batches line items in groups of 15-20 for 50+ line invoices
|
||
- ~1,900 tokens/call, ~$0.003
|
||
|
||
#### Feature 3: Smart Ingredient Matching (User-triggered)
|
||
- "AI Match" button when trigram results have low confidence
|
||
- `rank_ingredient_matches(description, candidates)` → re-ranked list with confidence scores
|
||
- ~650 tokens/call, ~$0.001
|
||
|
||
### Tier 1 — High Value
|
||
|
||
#### A. Recipe Text Allergen Scanning
|
||
Same `analyse_product_label()` function with recipe text as input. Catches contextual allergens regex misses.
|
||
|
||
#### B. Menu Description Generation
|
||
"Generate Description" button in PublishToMenuModal. Customer-facing descriptions with allergen callout.
|
||
|
||
#### C. Dispute Email Drafting
|
||
"Draft Email" button on DisputeDetailModal. Professional supplier email requesting credit.
|
||
|
||
### Tier 2 — Medium Value
|
||
|
||
#### D. Smart Duplicate Detection
|
||
"AI Check" button when creating ingredients. Reuses `rank_ingredient_matches()`.
|
||
|
||
#### E. OCR Field Extraction Fallback
|
||
Automatic when regex returns null. Mark LLM-extracted fields with `source: "llm"`.
|
||
|
||
#### F. Supplier Alias Resolution
|
||
LLM fallback when `identify_supplier()` returns no match.
|
||
|
||
#### G. Ingredient Yield Estimation
|
||
Auto-hint on ingredient creation: "Typical yield: ~85%".
|
||
|
||
#### H. Invoice Line Item Reconciliation (Auto)
|
||
Matches unmatched line items against supplier's own historical naming from past 90 days.
|
||
|
||
---
|
||
|
||
## 4. Frontend UX
|
||
|
||
### Core Principles
|
||
1. **Suggestions only, never auto-change data** (except allergen "Contains" following existing Brakes pattern)
|
||
2. **Visible loading + notification** for auto-triggered features
|
||
3. **Sparkle icon** as consistent AI indicator
|
||
|
||
### Auto-triggered Features
|
||
| Feature | Where | Visual |
|
||
|---------|-------|--------|
|
||
| Label Parsing | IngredientModal | Spinner → toast → sparkle icon suggestions |
|
||
| Recipe Scanning | RecipeEditor | Spinner on flag matrix → sparkle suggestions |
|
||
| OCR Fallback | Invoice upload | Dashed amber border + "AI extracted" tooltip |
|
||
| Reconciliation | Review page | Amber "AI match" badge |
|
||
|
||
### User-triggered Features
|
||
| Feature | Where | Trigger |
|
||
|---------|-------|---------|
|
||
| AI Assist | Review top bar | Button → yellow-highlighted suggestions |
|
||
| AI Match | IngredientModal | Button → re-sorted dropdown |
|
||
| Menu Description | PublishToMenuModal | "Generate" button → pre-filled textarea |
|
||
| Dispute Email | DisputeDetailModal | "Draft Email" button → pre-filled text |
|
||
| Duplicate Detection | IngredientModal | "AI Check" button → warning panel |
|
||
| Supplier Alias | Review page | Auto suggestion banner |
|
||
| Yield Estimation | IngredientModal | Auto hint below field |
|
||
|
||
---
|
||
|
||
## 5. OCR Correction Enhancements (Feature 2)
|
||
|
||
LLM significantly enhances 7 existing OCR correction scenarios:
|
||
1. **Qty × Price ≠ Total** — identifies which field Azure misread
|
||
2. **Line items vs invoice total mismatch** — identifies delivery charges, subtotal rows
|
||
3. **Description content vs value** — recommends which is the correct description
|
||
4. **SKU in description** — distinguishes product codes from descriptions
|
||
5. **Weight-as-quantity** — handles non-standard weight formats
|
||
6. **Subtotal/discount row detection** — catches "Goods Total", "Delivery Surcharge", etc.
|
||
7. **Gross-to-net VAT** — identifies VAT treatment from invoice context
|
||
|
||
---
|
||
|
||
## 6. Implementation Order
|
||
|
||
### Phase 1: Infrastructure ✅
|
||
- `backend/services/llm_service.py` — client, caching, logging, rate limiting
|
||
- `backend/requirements.txt` — `anthropic>=0.40.0`
|
||
- `backend/models/settings.py` — LLM columns
|
||
- `backend/models/llm.py` — usage log + cache models
|
||
- `backend/migrations/add_llm_infrastructure.py`
|
||
- `backend/api/settings.py` — LLM fields, usage stats endpoint
|
||
- `LLM-MANIFEST.md` — removal manifest
|
||
|
||
### Phase 2: Label Parsing + Recipe Text (Features 1 + A)
|
||
- `analyse_product_label()` with caching
|
||
- `/analyse-label` endpoint
|
||
- Frontend auto-trigger + sparkle suggestions
|
||
|
||
### Phase 3: Invoice OCR Assist + Reconciliation (Features 2 + E + H)
|
||
- `assist_invoice_ocr()` with batching
|
||
- `reconcile_line_items()` for supplier history
|
||
- "AI Assist" button + suggestion UI
|
||
- LLM fallback for regex field extraction
|
||
|
||
### Phase 4: Ingredient Matching + Supplier (Features 3 + D + F)
|
||
- `rank_ingredient_matches()`, supplier alias matching
|
||
- "AI Match" button, duplicate detection
|
||
|
||
### Phase 5: Text Generation (Features B + C)
|
||
- Menu description generation
|
||
- Dispute email drafting
|
||
|
||
### Phase 6: Polish + Yield (Feature G)
|
||
- Yield estimation hints
|
||
- Usage dashboard
|
||
- Prompt tuning
|
||
|
||
---
|
||
|
||
## 7. Verification Checklist
|
||
|
||
1. Add API key in Settings → saved, model dropdown works
|
||
2. `llm_enabled = False` (default) → zero AI footprint anywhere
|
||
3. Enable → full LLM settings section appears
|
||
4. Ingredient with "Contains: wheat flour, milk" → Gluten, Dairy suggestions
|
||
5. Upload invoice → "AI Assist" → corrections + suggestions
|
||
6. Invalid API key → toast error, regex/trigram still works
|
||
7. Budget exceeded → graceful degradation
|
||
8. All AI suggestions visually distinct with sparkle icon
|