hotel-manage-stack/kitchen
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Initial kitchen scaffold — Phase 1 kitchen port (build-verified 2026-07-11)

Browse source 
FastAPI backend (Python 3.11, MSSQL ODBC for SambaPOS, Azure DI OCR),
kitchen_db on central PG. React/TS/Vite frontend with navy sidebar layout.

Backend: auth.py (APP_SLUG=kitchen, SimpleNamespace — archive routes use
.kitchen_id/.is_admin without modification), main.py (51 migrations, scheduler,
internal router for KDS bookings feed), api/internal.py, full archive API
(31 routers: invoices, recipes, menus, sambapos, resos, newbook, disputes,
purchase_orders, etc.), models, migrations, OCR pipeline.
kitchen_id pinned to 1 (B1 — single hotel).

Frontend: AuthGate (app=kitchen, token shim for archive compat — B5b pending),
Layout (navy sidebar, 6 sections, Lucide icons, teal --app-primary),
App.tsx (Outlet pattern, UploadApp outside Layout), index.css (full :root block).
strict: false — archive components have type issues; build clean.

Note: 45 archive components call fetch('/api/...') without /kitchen/ prefix
(B5b). Runtime 404s; deferred until after initial testing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
jtricerolph 2026-07-12 12:15:39 +00:00
commit 8d688b459d
10003 changed files with 1928395 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

208
docs/archive/LLM-INTEGRATION-PLAN.md Normal file
View file

@ -0,0 +1,208 @@
# LLM Integration Plan — Review & Enhancements
> **Supersedes** the original 3-feature plan. Reviewed and expanded to cover 11 features across 6 phases, with master kill switch, removal manifest, cost guardrails, and detailed UX specification.
## Context
The kitchen-invoice-flash system has several manual data entry bottlenecks where an LLM can reduce friction. Using **Claude Haiku** via the Anthropic Python SDK. Estimated cost: ~$0.50-1/month for a single kitchen. Mixed trigger approach: auto for cheap label parsing, user-triggered button for invoice analysis and ingredient matching.
**Scope**: Original 3 features + Tier 1 (A, B, C) + Tier 2 (D, E, F, G) + new Feature H (line item reconciliation).
**Instance model**: Single instance (not multi-kitchen). Settings are global, not per-kitchen.
**API key storage**: Plaintext in DB (matching existing pattern for Azure, NewBook, etc.).
---
## 1. What the Plan Gets Right
- **Model choice**: Claude Haiku is correct — structured extraction, not creative generation.
- **API key in Settings**: Follows the established `KitchenSettings` pattern (same as `azure_key`, `resos_api_key`, etc.).
- **Structured output via `tool_use`**: Guarantees parseable JSON. Right approach over prompt-based JSON.
- **Graceful degradation**: LLM features are additive, never blocking. Existing regex/trigram still works without a key.
- **Trigger strategy**: Auto for cheap label parsing, user-triggered buttons for invoice assist and ingredient matching.
---
## 2. Infrastructure Enhancements
### 2a. Client Instantiation
Single shared client instance, re-initialized if API key changes in settings.
### 2b. Response Caching
New `llm_analysis_cache` table:
- Columns: `id`, `feature`, `input_hash` (SHA-256), `result_json`, `model_used`, `prompt_version`, `created_at`
- Unique constraint: `(feature, input_hash, prompt_version)`
- Feature 1 (labels): cache by text hash, TTL 30 days. Feature 3 (matching): TTL 7 days. Feature 2 (invoices): skip caching.
### 2c. Usage Tracking
New `llm_usage_log` table:
- Columns: `id`, `feature`, `model`, `input_tokens`, `output_tokens`, `latency_ms`, `success`, `error_message`, `created_at`
- `GET /api/settings/llm-usage` endpoint (aggregated last 30 days). Display on Settings page.
### 2d. Configurable Model
`llm_model` column in `KitchenSettings` (default `"claude-haiku-4-5-latest"`). Settings UI shows dropdown (Haiku / Sonnet).
### 2e. Prompt Versioning
Constants in `llm_service.py`. Cache lookup includes `prompt_version` — version bump auto-invalidates stale cache.
### 2f. Error UX
Three-state `llm_status` in API responses: `"success"`, `"unavailable"` (no key), `"error"` (call failed + message).
### 2g. Rate Limiting
`asyncio.Semaphore(5)` caps concurrent LLM calls.
### 2h. Master Kill Switch
`llm_enabled` (Boolean, default **False**) in `KitchenSettings`. When disabled: zero AI footprint in frontend, no API calls, no logging. See `LLM-MANIFEST.md` for full details.
### 2i. Removal Manifest
`LLM-MANIFEST.md` in project root — updated each phase. All LLM code marked with breadcrumb comments: `LLM FEATURE — see LLM-MANIFEST.md for removal instructions`.
### 2j. Cost Guardrails (5 layers)
1. **Per-feature toggles** — `llm_features_enabled` JSONB column, individually disable features
2. **Monthly token budget** — `llm_monthly_token_limit` (default 500,000 tokens ~$1.25/month)
3. **Auto-trigger throttle** — per-entity cooldown via cache check
4. **Single-call token cap** — `max_tokens` on every API call
5. **Cost visibility** — Usage stats card on Settings page
### 2k. Prompt Injection Mitigation
`tool_use` structured output mitigates this. System messages note input is "untrusted product/invoice text".
---
## 3. Features
### Original Features
#### Feature 1: Product Label Allergen Parsing (Auto)
- Trigger: automatic when `product_ingredients` text is populated
- `analyse_product_label(ingredients_text, flag_categories)` → `[{flag_id, status: "contains"|"may_contain"|"suitable_for", reason}]`
- ~550 tokens/call, ~$0.001
#### Feature 2: Invoice OCR Assist (User-triggered)
- "AI Assist" button on Review page
- `assist_invoice_ocr(ocr_data, supplier_list, line_items)` → supplier match, date correction, pack size extraction, OCR corrections
- Batches line items in groups of 15-20 for 50+ line invoices
- ~1,900 tokens/call, ~$0.003
#### Feature 3: Smart Ingredient Matching (User-triggered)
- "AI Match" button when trigram results have low confidence
- `rank_ingredient_matches(description, candidates)` → re-ranked list with confidence scores
- ~650 tokens/call, ~$0.001
### Tier 1 — High Value
#### A. Recipe Text Allergen Scanning
Same `analyse_product_label()` function with recipe text as input. Catches contextual allergens regex misses.
#### B. Menu Description Generation
"Generate Description" button in PublishToMenuModal. Customer-facing descriptions with allergen callout.
#### C. Dispute Email Drafting
"Draft Email" button on DisputeDetailModal. Professional supplier email requesting credit.
### Tier 2 — Medium Value
#### D. Smart Duplicate Detection
"AI Check" button when creating ingredients. Reuses `rank_ingredient_matches()`.
#### E. OCR Field Extraction Fallback
Automatic when regex returns null. Mark LLM-extracted fields with `source: "llm"`.
#### F. Supplier Alias Resolution
LLM fallback when `identify_supplier()` returns no match.
#### G. Ingredient Yield Estimation
Auto-hint on ingredient creation: "Typical yield: ~85%".
#### H. Invoice Line Item Reconciliation (Auto)
Matches unmatched line items against supplier's own historical naming from past 90 days.
---
## 4. Frontend UX
### Core Principles
1. **Suggestions only, never auto-change data** (except allergen "Contains" following existing Brakes pattern)
2. **Visible loading + notification** for auto-triggered features
3. **Sparkle icon** as consistent AI indicator
### Auto-triggered Features
| Feature | Where | Visual |
|---------|-------|--------|
| Label Parsing | IngredientModal | Spinner → toast → sparkle icon suggestions |
| Recipe Scanning | RecipeEditor | Spinner on flag matrix → sparkle suggestions |
| OCR Fallback | Invoice upload | Dashed amber border + "AI extracted" tooltip |
| Reconciliation | Review page | Amber "AI match" badge |
### User-triggered Features
| Feature | Where | Trigger |
|---------|-------|---------|
| AI Assist | Review top bar | Button → yellow-highlighted suggestions |
| AI Match | IngredientModal | Button → re-sorted dropdown |
| Menu Description | PublishToMenuModal | "Generate" button → pre-filled textarea |
| Dispute Email | DisputeDetailModal | "Draft Email" button → pre-filled text |
| Duplicate Detection | IngredientModal | "AI Check" button → warning panel |
| Supplier Alias | Review page | Auto suggestion banner |
| Yield Estimation | IngredientModal | Auto hint below field |
---
## 5. OCR Correction Enhancements (Feature 2)
LLM significantly enhances 7 existing OCR correction scenarios:
1. **Qty × Price ≠ Total** — identifies which field Azure misread
2. **Line items vs invoice total mismatch** — identifies delivery charges, subtotal rows
3. **Description content vs value** — recommends which is the correct description
4. **SKU in description** — distinguishes product codes from descriptions
5. **Weight-as-quantity** — handles non-standard weight formats
6. **Subtotal/discount row detection** — catches "Goods Total", "Delivery Surcharge", etc.
7. **Gross-to-net VAT** — identifies VAT treatment from invoice context
---
## 6. Implementation Order
### Phase 1: Infrastructure ✅
- `backend/services/llm_service.py` — client, caching, logging, rate limiting
- `backend/requirements.txt` — `anthropic>=0.40.0`
- `backend/models/settings.py` — LLM columns
- `backend/models/llm.py` — usage log + cache models
- `backend/migrations/add_llm_infrastructure.py`
- `backend/api/settings.py` — LLM fields, usage stats endpoint
- `LLM-MANIFEST.md` — removal manifest
### Phase 2: Label Parsing + Recipe Text (Features 1 + A)
- `analyse_product_label()` with caching
- `/analyse-label` endpoint
- Frontend auto-trigger + sparkle suggestions
### Phase 3: Invoice OCR Assist + Reconciliation (Features 2 + E + H)
- `assist_invoice_ocr()` with batching
- `reconcile_line_items()` for supplier history
- "AI Assist" button + suggestion UI
- LLM fallback for regex field extraction
### Phase 4: Ingredient Matching + Supplier (Features 3 + D + F)
- `rank_ingredient_matches()`, supplier alias matching
- "AI Match" button, duplicate detection
### Phase 5: Text Generation (Features B + C)
- Menu description generation
- Dispute email drafting
### Phase 6: Polish + Yield (Feature G)
- Yield estimation hints
- Usage dashboard
- Prompt tuning
---
## 7. Verification Checklist
1. Add API key in Settings → saved, model dropdown works
2. `llm_enabled = False` (default) → zero AI footprint anywhere
3. Enable → full LLM settings section appears
4. Ingredient with "Contains: wheat flour, milk" → Gluten, Dairy suggestions
5. Upload invoice → "AI Assist" → corrections + suggestions
6. Invalid API key → toast error, regex/trigram still works
7. Budget exceeded → graceful degradation
8. All AI suggestions visually distinct with sparkle icon

115
docs/archive/LLM-MANIFEST.md Normal file
View file

@ -0,0 +1,115 @@
# LLM Feature Manifest
Files that can be safely removed/reverted to fully remove LLM features.
Updated with each implementation phase. Keep this current when making LLM-related changes.
## How to Use This Manifest
1. **Quick disable**: Set `llm_enabled = False` in kitchen_settings — all LLM UI disappears, no API calls made.
2. **Full removal**: Follow the sections below to remove all LLM code and database objects.
3. **Code search**: All LLM code is marked with `LLM FEATURE — see LLM-MANIFEST.md` comments.
---
## New Files (delete entirely)
| File | Purpose | Phase |
|------|---------|-------|
| `backend/services/llm_service.py` | Central LLM wrapper — client, caching, logging, rate limiting | 1 |
| `backend/models/llm.py` | `LlmUsageLog` + `LlmAnalysisCache` models | 1 |
| `backend/migrations/add_llm_infrastructure.py` | Migration for all LLM tables + settings columns | 1 |
## Modified Files (sections to remove)
### Phase 1 — Infrastructure
| File | What to Remove | Search Pattern |
|------|---------------|----------------|
| `backend/requirements.txt` | `anthropic>=0.40.0` line | `anthropic` |
| `backend/models/settings.py` | 7 columns: `llm_enabled`, `anthropic_api_key`, `llm_model`, `llm_confidence_threshold`, `llm_monthly_token_limit`, `llm_features_enabled` | `llm_` or `anthropic_` |
| `backend/api/settings.py` | LLM fields in `SettingsResponse`, `SettingsUpdate`, `_build_settings_response()`, `LlmUsageStatsResponse`, `/llm-usage` endpoint, `/test-llm` endpoint | `llm` or `LLM` or `anthropic` |
| `backend/main.py` | Import + call of `run_llm_infrastructure_migration` | `llm_infrastructure` |
### Phase 2 — Label Parsing + Recipe Text (Features 1 + A)
| File | What to Remove | Search Pattern |
|------|---------------|----------------|
| `backend/services/llm_service.py` | `analyse_product_label()` function + `LABEL_ANALYSIS_SYSTEM_MSG` constant | `analyse_product_label` |
| `backend/api/food_flags.py` | `POST /analyse-label` endpoint, LLM integration in recipe text scanning (`GET /recipes/{id}/flags`) — LLM call + merge block | `llm_service` or `analyse_product_label` or `llm_recipe_suggestions` |
| `frontend/src/components/IngredientFlagEditor.tsx` | `llmSuggestions` + `llmAnalysing` props, LLM suggestion merge logic, AI spinner/indicator | `llmSuggestions` or `llmAnalysing` or `LLM FEATURE` |
| `frontend/src/components/IngredientModal.tsx` | `llmAnalysing` + `llmSuggestions` + `debouncedProductIngredients` state, settings query for `llm_enabled`, `/analyse-label` useEffect, LLM props passed to IngredientFlagEditor | `llmAnalysing` or `llmSuggestions` or `analyse-label` or `LLM FEATURE` |
| `frontend/src/pages/Settings.tsx` | `'llm'` in SettingsSection type, sidebar item, LLM fields in SettingsData interface, LLM state variables, LLM queries/mutations, `handleSaveLlmSettings`, entire `activeSection === 'llm'` block | `llm` or `LLM` |
### Phase 3 — Invoice OCR Assist + Reconciliation (Features 2 + E + H)
| File | What to Remove | Search Pattern |
|------|---------------|----------------|
| `backend/services/llm_service.py` | `assist_invoice_ocr()`, `reconcile_line_items()`, `extract_invoice_fields_llm()` functions | `assist_invoice_ocr` or `reconcile_line_items` or `extract_invoice_fields_llm` |
| `backend/api/invoices.py` | `POST /{invoice_id}/ai-assist` endpoint | `ai-assist` or `ai_assist` or `LLM FEATURE` |
| `backend/ocr/extractor.py` | LLM fallback block in `process_invoice_image()` — field extraction when Azure returns null | `extract_invoice_fields_llm` or `LLM FEATURE` |
| `frontend/src/components/Review.tsx` | `AiAssistSuggestions` + `AiReconciliationMatch` interfaces, `aiAssist*` state variables, `handleAiAssist` + `handleApplyAiCorrection` + `handleApplyAiSupplier` functions, AI Assist button, AI suggestions panel, pack size suggestions panel + pre-fill in `toggleCostBreakdown` | `aiAssist` or `aiReconciliation` or `AiAssist` or `pack_size_suggestions` or `LLM FEATURE` |
### Phase 4 — Ingredient Matching + Supplier (Features 3 + D + F)
| File | What to Remove | Search Pattern |
|------|---------------|----------------|
| `backend/services/llm_service.py` | `rank_ingredient_matches()`, `match_supplier_llm()`, `check_duplicate_ingredient_llm()` functions | `rank_ingredient_matches` or `match_supplier_llm` or `check_duplicate_ingredient_llm` |
| `backend/api/ingredients.py` | `GET /ai-match` endpoint, `GET /ai-check-duplicate` endpoint | `ai-match` or `ai-check-duplicate` or `LLM FEATURE` |
| `backend/ocr/parser.py` | LLM fallback block at end of `identify_supplier()` | `match_supplier_llm` or `LLM FEATURE` |
| `frontend/src/components/Review.tsx` | `aiMatchLoading` + `aiMatchResults` state, `handleAiMatch` function, AI Match button + results in cost breakdown modal | `aiMatch` or `ai-match` or `LLM FEATURE` |
### Phase 5 — Text Generation (Features B + C)
| File | What to Remove | Search Pattern |
|------|---------------|----------------|
| `backend/services/llm_service.py` | `generate_menu_description()`, `draft_dispute_email()` functions | `generate_menu_description` or `draft_dispute_email` |
| `backend/api/menus.py` | `GenerateDescriptionRequest` model, `POST /generate-description` endpoint | `generate-description` or `generate_menu_description` or `LLM FEATURE` |
| `backend/api/disputes.py` | `POST /{dispute_id}/draft-email` endpoint | `draft-email` or `draft_dispute_email` or `LLM FEATURE` |
| `frontend/src/components/PublishToMenuModal.tsx` | `aiDescLoading` state, `llmSettings` query, `handleGenerateDescription` function, Generate button next to Description label | `aiDescLoading` or `handleGenerateDescription` or `LLM FEATURE` |
| `frontend/src/components/DisputeDetailModal.tsx` | `aiEmail*` state variables, settings query, `handleDraftEmail` function, Draft Email section with subject/body/actions | `aiEmail` or `handleDraftEmail` or `LLM FEATURE` |
### Phase 6 — Polish + Yield (Feature G)
| File | What to Remove | Search Pattern |
|------|---------------|----------------|
| `backend/services/llm_service.py` | `estimate_yield()` function | `estimate_yield` |
| `backend/api/ingredients.py` | `GET /ai-estimate-yield` endpoint | `ai-estimate-yield` or `estimate_yield` or `LLM FEATURE` |
| `frontend/src/components/IngredientModal.tsx` | `yieldHint` + `yieldHintLoading` + `debouncedFormName` state, yield estimation useEffect, yield hint display below Yield % input | `yieldHint` or `yieldHintLoading` or `ai-estimate-yield` or `LLM FEATURE` |
---
## Database (migration to drop)
### Tables
- `llm_usage_log` — LLM API call tracking
- `llm_analysis_cache` — Response caching
### Columns on `kitchen_settings`
- `llm_enabled` (Boolean)
- `anthropic_api_key` (String)
- `llm_model` (String)
- `llm_confidence_threshold` (Numeric)
- `llm_monthly_token_limit` (Integer)
- `llm_features_enabled` (JSONB)
### Removal SQL
```sql
DROP TABLE IF EXISTS llm_usage_log;
DROP TABLE IF EXISTS llm_analysis_cache;
ALTER TABLE kitchen_settings DROP COLUMN IF EXISTS llm_enabled;
ALTER TABLE kitchen_settings DROP COLUMN IF EXISTS anthropic_api_key;
ALTER TABLE kitchen_settings DROP COLUMN IF EXISTS llm_model;
ALTER TABLE kitchen_settings DROP COLUMN IF EXISTS llm_confidence_threshold;
ALTER TABLE kitchen_settings DROP COLUMN IF EXISTS llm_monthly_token_limit;
ALTER TABLE kitchen_settings DROP COLUMN IF EXISTS llm_features_enabled;
```
---
## Design Principles for Clean Removal
1. **Conditional imports**: All LLM imports use `from services.llm_service import X` only inside functions, not at module top level (except main.py migration import).
2. **Kill switch**: All LLM UI elements gated behind `if (!llmEnabled)` — removing the settings column and defaulting to False effectively removes all UI.
3. **No existing signatures changed**: LLM features are additive (new endpoints, new UI elements), never modify existing function behaviour.
4. **Existing logic untouched**: Regex/trigram logic untouched — LLM runs alongside, not instead of.
5. **Breadcrumb comments**: Every LLM-related function and component includes `LLM FEATURE — see LLM-MANIFEST.md for removal instructions`.

280
docs/archive/MENUS-PLAN.md Normal file
View file

@ -0,0 +1,280 @@
# Plan: Menus Feature — Publish Dishes to Curated Menus
## Context
Dishes exist in the system with full allergen flag tracking, but there's no way to curate them into actual menus (e.g., "Evening Menu", "Lunch Menu"). The Menus feature lets chefs publish dishes to named menus with sections (Starters, Mains, Desserts, etc.), requiring allergen confirmation before publishing. Menus are served via the existing external API for website widgets. When a dish or its sub-recipes change after publishing, the menu item is flagged as stale, requiring republishing.
---
## Data Model
### New Tables
**`menus`** — Named menus (e.g., "Dinner Menu", "Sunday Lunch")
```sql
CREATE TABLE menus (
id SERIAL PRIMARY KEY,
kitchen_id INTEGER NOT NULL REFERENCES kitchens(id),
name VARCHAR(255) NOT NULL,
is_active BOOLEAN DEFAULT true,
sort_order INTEGER DEFAULT 0,
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW(),
UNIQUE(kitchen_id, name)
);
```
**`menu_item_sections`** — Sections within a specific menu (per-menu, not shared)
```sql
CREATE TABLE menu_item_sections (
id SERIAL PRIMARY KEY,
menu_id INTEGER NOT NULL REFERENCES menus(id) ON DELETE CASCADE,
name VARCHAR(100) NOT NULL,
sort_order INTEGER DEFAULT 0,
UNIQUE(menu_id, name)
);
```
**`menu_items`** — Published dishes on a menu
```sql
CREATE TABLE menu_items (
id SERIAL PRIMARY KEY,
menu_id INTEGER NOT NULL REFERENCES menus(id) ON DELETE CASCADE,
section_id INTEGER NOT NULL REFERENCES menu_item_sections(id) ON DELETE CASCADE,
recipe_id INTEGER NOT NULL REFERENCES recipes(id) ON DELETE CASCADE,
display_name VARCHAR(255) NOT NULL,
description TEXT,
price NUMERIC(10, 2),
sort_order INTEGER DEFAULT 0,
confirmed_flags_json TEXT, -- JSON snapshot of flags at publish time
confirmed_by_name VARCHAR(100), -- who confirmed allergens
published_at TIMESTAMP DEFAULT NOW(),
UNIQUE(menu_id, recipe_id) -- dish appears once per menu
);
```
### Staleness (computed, not stored)
At query time, compare `menu_item.published_at` against:
- `recipe.updated_at` — direct dish edit
- Sub-recipe `updated_at` — component recipe edit (reuse `_collect_recipe_ingredient_ids` pattern from `food_flags.py:582` to walk the sub-recipe tree)
No `is_stale` column needed — computed on each load.
---
## Backend
### Models (`backend/models/menu.py` — NEW)
- `Menu` — id, kitchen_id, name, is_active, sort_order, created_at, updated_at
- `MenuItemSection` — id, menu_id, name, sort_order
- `MenuItem` — id, menu_id, section_id, recipe_id, display_name, description, price, sort_order, confirmed_flags_json, confirmed_by_name, published_at
- Relationships: Menu → sections, Menu → items, MenuItem → recipe
### Migration (`backend/migrations/add_menus.py` — NEW)
Follow existing pattern (`add_ingredient_flag_dismissals.py`): `CREATE TABLE IF NOT EXISTS`, indexes, idempotent.
### API (`backend/api/menus.py` — NEW)
| Endpoint | Method | Purpose |
|----------|--------|---------|
| `/api/menus` | GET | List menus with section/item counts |
| `/api/menus` | POST | Create menu (name) |
| `/api/menus/{id}` | GET | Full menu: sections + items + staleness |
| `/api/menus/{id}` | PUT | Update menu name/is_active |
| `/api/menus/{id}` | DELETE | Delete menu |
| `/api/menus/{id}/sections` | POST | Add section |
| `/api/menus/{id}/sections/{sid}` | PUT | Rename section |
| `/api/menus/{id}/sections/{sid}` | DELETE | Delete section (cascade items) |
| `/api/menus/{id}/sections/reorder` | PUT | Reorder sections `[{id, sort_order}]` |
| `/api/menus/{id}/items` | POST | Publish dish to menu (allergen confirm) |
| `/api/menus/{id}/items/{iid}` | PUT | Edit display_name/description/price/section |
| `/api/menus/{id}/items/{iid}` | DELETE | Remove dish from menu |
| `/api/menus/{id}/items/{iid}/republish` | POST | Republish (re-confirm flags, update snapshot) |
| `/api/menus/{id}/items/reorder` | PUT | Reorder items within section |
| `/api/menus/{id}/flags` | GET | Consolidated flag matrix for all items (for print) |
**GET `/api/menus/{id}` response shape:**
```json
{
"id": 1, "name": "Evening Menu", "is_active": true,
"sections": [
{
"id": 10, "name": "Starters", "sort_order": 0,
"items": [
{
"id": 100, "recipe_id": 5, "display_name": "Beetroot Tartare",
"description": "...", "price": "12.50", "sort_order": 0,
"confirmed_flags": [...], "confirmed_by_name": "Chef James",
"published_at": "...",
"is_stale": true,
"stale_reason": "Dish edited after publishing"
}
]
}
]
}
```
**Staleness computation** (in the GET detail endpoint):
```python
# For each menu_item:
# 1. Check dish.updated_at > item.published_at
# 2. Walk sub-recipe tree, check any updated_at > item.published_at
# Reuse _collect_sub_recipe_ids() helper (similar to food_flags.py pattern)
```
**Publish endpoint validation** (`POST /items`):
1. Verify recipe_id is a dish (`recipe_type == 'dish'`), not archived
2. Run `compute_recipe_flags()` to get current flags
3. Check for unassessed ingredients (from flags endpoint pattern)
4. If unassessed exist → 400 error with details
5. Accept: `{ recipe_id, section_id, display_name, description, price, confirmed_by_name }`
6. Snapshot current flags as `confirmed_flags_json`
7. Create `MenuItem`
### External API (`backend/api/external.py` — MODIFY)
Add two endpoints using existing `get_kitchen_from_api_key` dependency:
| Endpoint | Purpose |
|----------|---------|
| `GET /api/external/menus` | List active menus with sections + items + flags |
| `GET /api/external/menus/{id}` | Single menu detail |
Response includes: menu name, sections with items (display_name, description, price, confirmed_flags). No cost data, no internal IDs beyond menu item IDs.
### Register in main.py
- Import `api.menus`
- Register router: `app.include_router(menus.router, prefix="/api/menus", tags=["menus"])`
- Import and run migration
---
## Frontend
### MenuList Page (`frontend/src/components/MenuList.tsx` — NEW)
Similar to DishList/RecipeList pattern:
- List of menus with name, section count, item count, active/inactive badge
- Create modal: menu name input
- Click menu → navigate to `/menus/{id}`
- Toggle active/inactive
- Delete with confirmation
- Sortable (drag handle or up/down buttons)
### MenuEditor Page (`frontend/src/components/MenuEditor.tsx` — NEW)
Full menu management page:
- **Header**: Menu name (editable), Active/Inactive toggle, Print Flag Matrix button
- **Sections**: Collapsible, sortable list
- Each section shows name (editable), sort handle
- Add Section button
- Delete section (with confirmation if items exist)
- **Items within each section**: Sortable list
- Each item shows: display_name, description preview, price, confirmed flag badges, staleness indicator
- Edit button → inline or modal edit of display_name, description, price, section assignment
- Remove button
- **Stale indicator**: amber/yellow badge "Needs Republishing" with tooltip showing reason
- **Republish button**: Re-runs allergen confirmation, snapshots new flags
- **Add Dish button** (per section or floating): Opens dish picker → then publish flow
### Publish Flow
Two entry points:
1. **From DishEditor** — "Publish to Menu" button in the dish header area
2. **From MenuEditor** — "Add Dish" button
Both use the same **PublishToMenuModal** component:
**Step 1 — Allergen Check:**
- Fetch `compute_recipe_flags` for the dish
- Show consolidated flags grouped by category
- If unassessed ingredients exist → show red warning, block publishing
- If open suggestions exist → show amber warning (advisory, not blocking)
**Step 2 — Confirm & Publish:**
- Display the consolidated flags for chef review
- "Confirmed by" name input (required, remembers last used)
- Menu picker dropdown (only active menus)
- Section picker dropdown (sections within selected menu)
- Display name (pre-filled from dish name)
- Description textarea
- Price input (pre-filled from dish `gross_sell_price` if set)
- **Publish** button
### MenuFlagMatrix (`frontend/src/components/MenuFlagMatrix.tsx` — NEW)
Print-optimized flag matrix:
- Rows: dishes grouped by section (section headers as group rows)
- Columns: all allergen flags
- Cells: flag status from `confirmed_flags_json`
- Uses the same visual style as RecipeFlagMatrix but read-only
- Print button triggers `window.print()` with print-optimized CSS
- Accessed from MenuEditor header
### Navigation Update (`frontend/src/App.tsx` — MODIFY)
- Add `import MenuList from './components/MenuList'`
- Add `import MenuEditor from './components/MenuEditor'`
- Add routes: `/menus` → MenuList, `/menus/:id` → MenuEditor (under `/recipes` access check)
- Add "Menus" link in Recipes dropdown (between "Dishes" and "Allergens")
### DishEditor Update (`frontend/src/components/DishEditor.tsx` — MODIFY)
- Add "Publish to Menu" button in the header area (next to Archive/Delete buttons)
- Button disabled with tooltip if dish has unassessed ingredients
- Opens PublishToMenuModal
---
## Files to Create/Modify
| File | Change |
|------|--------|
| `backend/migrations/add_menus.py` | **NEW** — migration for 3 tables |
| `backend/models/menu.py` | **NEW** — Menu, MenuItemSection, MenuItem models |
| `backend/api/menus.py` | **NEW** — Full CRUD + publish/republish API |
| `backend/api/external.py` | Add `GET /menus`, `GET /menus/{id}` public endpoints |
| `backend/main.py` | Register menus API + migration |
| `frontend/src/components/MenuList.tsx` | **NEW** — Menu list page |
| `frontend/src/components/MenuEditor.tsx` | **NEW** — Menu detail/editor page |
| `frontend/src/components/PublishToMenuModal.tsx` | **NEW** — Allergen confirm + publish flow |
| `frontend/src/components/MenuFlagMatrix.tsx` | **NEW** — Print-optimized flag matrix |
| `frontend/src/components/DishEditor.tsx` | Add "Publish to Menu" button |
| `frontend/src/App.tsx` | Routes + nav link |
## Key Patterns to Reuse
- `compute_recipe_flags()` — `backend/api/food_flags.py:582` for consolidated flag computation
- `_collect_recipe_ingredient_ids()` — `backend/api/food_flags.py` for walking sub-recipe tree (staleness check)
- `get_kitchen_from_api_key()` — `backend/api/external.py:25` for external API auth
- `RecipeFlagMatrix` — `frontend/src/components/RecipeFlagMatrix.tsx` visual patterns for MenuFlagMatrix
- `DishList` — `frontend/src/components/DishList.tsx` list page patterns
- `DishEditor` — `frontend/src/components/DishEditor.tsx` editor page patterns
- Migration pattern — `backend/migrations/add_ingredient_flag_dismissals.py`
## Implementation Order
1. Migration + models (backend/migrations + backend/models)
2. Backend menus API — CRUD + publish endpoints
3. Frontend MenuList + MenuEditor pages + routing
4. Frontend PublishToMenuModal + DishEditor "Publish" button
5. External API menu endpoints
6. MenuFlagMatrix print view
7. Docker rebuild + end-to-end test
## Verification
1. **Docker rebuild** — migration creates 3 tables
2. **Create menu** — name, add sections, reorder sections
3. **Publish dish** — from DishEditor, confirm allergens, pick menu+section, set display info → appears on menu
4. **Staleness** — edit a published dish (or its sub-recipe) → menu shows "Needs Republishing" indicator
5. **Republish** — click republish, re-confirm allergens → staleness clears, new flag snapshot saved
6. **External API** — `GET /api/external/menus` with X-API-Key returns active menus with dishes + flags
7. **Print matrix** — from MenuEditor, click Print → flag matrix shows consolidated allergens by section, renders cleanly in print
8. **Unassessed block** — try to publish dish with unassessed ingredients → blocked with error message
9. **Remove/edit** — edit display_name/price on published item, remove item from menu

750
docs/archive/PLAN-search-feature.md Normal file
View file

@ -0,0 +1,750 @@
# Search Feature Implementation Plan
## Overview
Add a new "Search" main menu dropdown with three search pages:
1. **Invoices Search** - Search/filter invoices (optionally including line item content)
2. **Line Items Search** - Consolidated unique line items with price change detection
3. **Unit/Portion Definitions Search** - Search ProductDefinition records
**Plus**: Reusable price history system with:
- Price change detection on new invoices
- Acknowledgement system to mark price changes as reviewed
- History modal with graphs for any line item
All pages share common patterns: live search, session persistence, date filtering, grouping, and links to source records.
---
## Data Models
### Existing Models
**Invoice**: `id`, `invoice_number`, `invoice_date`, `total`, `net_total`, `supplier_id`, `vendor_name`, `status`, `category`, `document_type`
**LineItem**: `id`, `invoice_id`, `product_code`, `description`, `unit`, `quantity`, `unit_price`, `amount`, `pack_quantity`, `unit_size`, `unit_size_type`, `portions_per_unit`
**ProductDefinition**: `id`, `kitchen_id`, `supplier_id`, `product_code`, `description_pattern`, `pack_quantity`, `unit_size`, `unit_size_type`, `portions_per_unit`, `portion_description`, `source_invoice_id`, `updated_at`
### New Model: AcknowledgedPrice
**File**: `backend/models/acknowledged_price.py` (NEW)
Tracks when a user acknowledges a price change so it doesn't keep flagging.
```python
class AcknowledgedPrice(Base):
__tablename__ = "acknowledged_prices"
id: Mapped[int] = mapped_column(primary_key=True)
kitchen_id: Mapped[int] = mapped_column(ForeignKey("kitchens.id"))
supplier_id: Mapped[int] = mapped_column(ForeignKey("suppliers.id"))
# Product identification (same logic as line item consolidation)
product_code: Mapped[str | None] = mapped_column(String(100), nullable=True)
description: Mapped[str | None] = mapped_column(Text, nullable=True)
# The acknowledged price point
acknowledged_price: Mapped[Decimal] = mapped_column(Numeric(10, 2))
acknowledged_at: Mapped[datetime] = mapped_column(DateTime, default=datetime.utcnow)
acknowledged_by_user_id: Mapped[int] = mapped_column(ForeignKey("users.id"))
# Which invoice triggered this acknowledgement
source_invoice_id: Mapped[int | None] = mapped_column(ForeignKey("invoices.id"))
source_line_item_id: Mapped[int | None] = mapped_column(ForeignKey("line_items.id"))
# Unique: one acknowledged price per product per supplier per kitchen
__table_args__ = (
UniqueConstraint('kitchen_id', 'supplier_id', 'product_code', 'description',
name='uix_acknowledged_price'),
)
```
### New Settings Fields
**File**: `backend/models/settings.py`
```python
# Price change detection settings
price_change_lookback_days: Mapped[int] = mapped_column(Integer, default=30)
price_change_amber_threshold: Mapped[int] = mapped_column(Integer, default=10) # %
price_change_red_threshold: Mapped[int] = mapped_column(Integer, default=20) # %
```
---
## Backend Changes
### 1. New Search API Router
**File**: `backend/api/search.py` (NEW)
```python
router = APIRouter(prefix="/api/search", tags=["search"])
# ============ Invoice Search ============
@router.get("/invoices")
async def search_invoices(
q: str = "", # Search term (invoice_number, vendor_name)
include_line_items: bool = False, # Also search line item product_code/description
supplier_id: int | None = None,
status: str | None = None,
date_from: date | None = None, # Default: 30 days ago
date_to: date | None = None, # Default: today
group_by: str | None = None, # "supplier", "month", or null
limit: int = 100,
offset: int = 0
) -> InvoiceSearchResponse
# ============ Line Items Search (Consolidated) ============
@router.get("/line-items")
async def search_line_items(
q: str = "", # Search term (product_code, description)
supplier_id: int | None = None,
date_from: date | None = None,
date_to: date | None = None,
group_by: str | None = None, # "supplier", "invoice", "month"
limit: int = 100,
offset: int = 0
) -> LineItemSearchResponse
# Returns DISTINCT line items by (product_code OR description + supplier)
# With: most_recent_price, price_change_status, total_qty, occurrence_count
# ============ Unit/Portion Definitions Search ============
@router.get("/definitions")
async def search_definitions(
q: str = "", # Search term (product_code, description_pattern)
supplier_id: int | None = None,
has_portions: bool | None = None, # Filter by portions_per_unit is set
limit: int = 100,
offset: int = 0
) -> DefinitionSearchResponse
# ============ Line Item History (Reusable) ============
@router.get("/line-items/history")
async def get_line_item_history(
product_code: str | None = None,
description: str | None = None,
supplier_id: int,
date_from: date | None = None, # Default: 12 months ago
date_to: date | None = None,
) -> LineItemHistoryResponse
# Returns price history, qty stats for a specific product
# ============ Price Acknowledgement ============
@router.post("/line-items/acknowledge-price")
async def acknowledge_price_change(
product_code: str | None,
description: str | None,
supplier_id: int,
new_price: Decimal,
source_invoice_id: int | None = None,
source_line_item_id: int | None = None,
) -> AcknowledgePriceResponse
# Creates/updates AcknowledgedPrice record
```
### 2. Price History Service (Reusable)
**File**: `backend/services/price_history.py` (NEW)
```python
class PriceHistoryService:
"""Reusable service for price change detection - used by search AND invoice review"""
async def get_price_status(
self,
db: AsyncSession,
kitchen_id: int,
supplier_id: int,
product_code: str | None,
description: str | None,
current_price: Decimal,
lookback_days: int = 30,
amber_threshold: int = 10,
red_threshold: int = 20,
) -> PriceStatus:
"""
Returns price status for a line item:
- "consistent": Price matches history (green tick)
- "no_history": First time seeing this item (no icon)
- "amber": Small price change within threshold
- "red": Large price change above threshold
- "acknowledged": Price was flagged but user acknowledged it
"""
async def get_history(
self,
db: AsyncSession,
kitchen_id: int,
supplier_id: int,
product_code: str | None,
description: str | None,
date_from: date,
date_to: date,
) -> LineItemHistory:
"""
Returns full history for a product:
- price_history: list of {date, price, invoice_id, invoice_number}
- total_occurrences: int
- total_quantity: Decimal
- avg_qty_per_invoice: Decimal
- avg_qty_per_week: Decimal
- avg_qty_per_month: Decimal
"""
async def acknowledge_price(
self,
db: AsyncSession,
kitchen_id: int,
user_id: int,
supplier_id: int,
product_code: str | None,
description: str | None,
new_price: Decimal,
source_invoice_id: int | None,
source_line_item_id: int | None,
) -> AcknowledgedPrice:
"""Creates or updates acknowledged price record"""
```
### 3. Response Models
```python
class InvoiceSearchItem(BaseModel):
id: int
invoice_number: str | None
invoice_date: date | None
total: Decimal | None
net_total: Decimal | None
supplier_id: int | None
supplier_name: str | None
vendor_name: str | None
status: str
document_type: str | None
class InvoiceSearchResponse(BaseModel):
items: list[InvoiceSearchItem]
total_count: int
grouped_by: str | None
groups: list[GroupSummary] | None # If grouped: [{name, count, total}]
class LineItemSearchItem(BaseModel):
product_code: str | None
description: str | None
supplier_id: int | None
supplier_name: str | None
unit: str | None
# Price info
most_recent_price: Decimal | None # Latest unit_price
earliest_price_in_period: Decimal | None
price_change_percent: float | None # % change from earliest to most recent
price_change_status: str # "consistent", "amber", "red", "no_history"
# Quantity info
total_quantity: Decimal | None
occurrence_count: int
# Links
most_recent_invoice_id: int
most_recent_invoice_number: str | None
most_recent_date: date | None
# Definition info
has_definition: bool
portions_per_unit: int | None
class LineItemSearchResponse(BaseModel):
items: list[LineItemSearchItem]
total_count: int
grouped_by: str | None
groups: list[GroupSummary] | None
class DefinitionSearchItem(BaseModel):
id: int
product_code: str | None
description_pattern: str | None
supplier_id: int | None
supplier_name: str | None
pack_quantity: int | None
unit_size: Decimal | None
unit_size_type: str | None
portions_per_unit: int | None
portion_description: str | None
source_invoice_id: int | None
source_invoice_number: str | None
updated_at: datetime
class DefinitionSearchResponse(BaseModel):
items: list[DefinitionSearchItem]
total_count: int
class GroupSummary(BaseModel):
name: str # Group name (supplier name, month "Jan 2024", etc.)
count: int # Number of items in group
total: Decimal | None # Sum of totals if applicable
# ============ History Modal Response ============
class PriceHistoryPoint(BaseModel):
date: date
price: Decimal
invoice_id: int
invoice_number: str | None
class LineItemHistoryResponse(BaseModel):
product_code: str | None
description: str | None
supplier_name: str | None
# Price history for chart
price_history: list[PriceHistoryPoint]
# Stats for period
total_occurrences: int
total_quantity: Decimal
avg_qty_per_invoice: Decimal
avg_qty_per_week: Decimal
avg_qty_per_month: Decimal
# Current status
current_price: Decimal | None
price_change_status: str
```
### 4. Register Router
**File**: `backend/main.py`
```python
from api import search
app.include_router(search.router)
```
### 5. Update Invoice Line Items Response
**File**: `backend/api/invoices.py`
Add price status to line items when fetching invoice details:
```python
# In get_invoice endpoint, for each line item:
price_status = await price_history_service.get_price_status(
db, kitchen_id, supplier_id, product_code, description, unit_price
)
# Return: price_status ("consistent", "amber", "red", "no_history", "acknowledged")
```
---
## Frontend Changes
### 1. Add Search Dropdown to Navigation
**File**: `frontend/src/App.tsx`
Add "Search" dropdown similar to "Reports" dropdown:
```tsx
// In Header component, add state:
const [searchOpen, setSearchOpen] = useState(false)
// Add showSearch check:
const showSearch = showNavItem('/search-invoices') ||
showNavItem('/search-line-items') ||
showNavItem('/search-definitions')
// Add dropdown after Invoices link:
{showSearch && (
<div style={styles.dropdownContainer}
onMouseEnter={() => setSearchOpen(true)}
onMouseLeave={() => setSearchOpen(false)}>
<span style={styles.navLink}>Search ▾</span>
{searchOpen && (
<div style={styles.dropdown}>
{showNavItem('/search-invoices') &&
<a href="/search/invoices" style={styles.dropdownLink}>Invoices</a>}
{showNavItem('/search-line-items') &&
<a href="/search/line-items" style={styles.dropdownLink}>Line Items</a>}
{showNavItem('/search-definitions') &&
<a href="/search/definitions" style={styles.dropdownLink}>Unit/Portion Definitions</a>}
</div>
)}
</div>
)}
```
Add routes:
```tsx
<Route path="/search/invoices" element={...} />
<Route path="/search/line-items" element={...} />
<Route path="/search/definitions" element={...} />
```
### 2. Create Search Components
#### 2a. SearchInvoices.tsx (NEW)
**File**: `frontend/src/components/SearchInvoices.tsx`
**Features**:
- Text search input (debounced 300ms for live search)
- **☑ Include line items** checkbox - when enabled, also searches line item product_code/description
- Supplier dropdown filter
- Status dropdown filter
- Date range (default: last 30 days)
- Group by dropdown: None / Supplier / Month
- Session storage persistence for all filters
- Results table with columns: Invoice #, Supplier, Date, Net Total, Status
- Invoice # links to `/invoice/{id}` (opens in new tab)
- When grouped: collapsible sections with group headers showing count/total
**Layout**:
```
┌───────────────────────────────────────────────────────────────────────────┐
│ Search Invoices │
├───────────────────────────────────────────────────────────────────────────┤
│ [Search input...] [☑ Include line items] [Supplier ▼] [Status ▼] [Group ▼]│
│ From: [____] To: [____] │
├───────────────────────────────────────────────────────────────────────────┤
│ Invoice # │ Supplier │ Date │ Net Total │ Status │
│ INV-001 ↗ │ Brakes │ 15/01/2026 │ £234.50 │ confirmed │
│ INV-002 ↗ │ Brakes │ 14/01/2026 │ £156.20 │ confirmed │
└───────────────────────────────────────────────────────────────────────────┘
```
#### 2b. SearchLineItems.tsx (NEW)
**File**: `frontend/src/components/SearchLineItems.tsx`
**Features**:
- Text search input (debounced 300ms)
- Supplier dropdown filter
- Date range (default: last 30 days)
- Group by dropdown: None / Supplier / Invoice / Month
- Session storage persistence
- Results show consolidated items with:
- Product Code, Description, Supplier
- **Most Recent Price** with price change indicator:
- 🟢 Green tick: Price consistent with history
- 🟡 Amber ?: Small change (≤ amber threshold %)
- 🔴 Red !: Large change (> red threshold %)
- No icon: First time seeing this item
- **📊 Price History button**: Opens history modal
- Total Qty with **📦 Qty History button**: Opens history modal
- # Occurrences
- Most Recent Invoice # (link to invoice)
- Portions icon if definition exists
**Consolidation Logic**:
- Group by: `COALESCE(product_code, '') || '||' || COALESCE(description, '') || '||' || supplier_id`
- Show most recent values for each unique item
- Aggregate: SUM(quantity), COUNT(*)
- Compare earliest vs latest price in period for change detection
**Layout**:
```
┌─────────────────────────────────────────────────────────────────────────────────┐
│ Search Line Items │
├─────────────────────────────────────────────────────────────────────────────────┤
│ [Search...] [Supplier ▼] [From] [To] [Group ▼] │
├─────────────────────────────────────────────────────────────────────────────────┤
│ Code │ Description │ Supplier │ Price 📊 │ Qty 📦 │ # │ Invoice │
│ ABC123 │ Chicken Breast │ Brakes │ £5.50 🟢 │ 45 │ 8 │ INV-001 ↗ │
│ XYZ789 │ Beef Mince 500g │ Brakes │ £6.20 🔴! │ 30 │ 5 │ INV-003 ↗ │
│ - │ Mixed Salad Bag │ Booker │ £3.20 🟡? │ 120 │12 │ INV-015 ↗ │
└─────────────────────────────────────────────────────────────────────────────────┘
```
#### 2c. SearchDefinitions.tsx (NEW)
**File**: `frontend/src/components/SearchDefinitions.tsx`
**Features**:
- Text search input (debounced 300ms)
- Supplier dropdown filter
- "Has Portions Defined" checkbox filter
- Session storage persistence
- Results show:
- Product Code, Description Pattern, Supplier
- Pack Info (e.g., "120x15g")
- Portions per Unit
- Last Updated, Source Invoice (link)
**Layout**:
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ Search Unit/Portion Definitions │
├─────────────────────────────────────────────────────────────────────────────┤
│ [Search...] [Supplier ▼] [☑ Has Portions Defined] │
├─────────────────────────────────────────────────────────────────────────────┤
│ Code │ Description │ Supplier │ Pack │ Portions │ Source │
│ ABC123 │ Chicken Breast │ Brakes │ 4x2.5kg │ 40 │ INV-001 ↗ │
│ XYZ789 │ Orange Juice 1L │ Booker │ 12x1L │ 48 │ INV-022 ↗ │
└─────────────────────────────────────────────────────────────────────────────┘
```
### 3. Shared Search Utilities
**File**: `frontend/src/utils/searchHelpers.ts` (NEW)
```typescript
// Session storage keys
export const SEARCH_STORAGE_KEYS = {
invoices: {
query: 'search-invoices-query',
includeLineItems: 'search-invoices-include-line-items',
supplier: 'search-invoices-supplier',
status: 'search-invoices-status',
dateFrom: 'search-invoices-from',
dateTo: 'search-invoices-to',
groupBy: 'search-invoices-group',
},
lineItems: { ... },
definitions: { ... },
}
// Debounce hook for live search
export function useDebounce<T>(value: T, delay: number): T
// Date helpers
export function getDefaultDateRange(): { from: string, to: string }
export function formatDateForDisplay(date: string): string
```
### 4. Line Item History Modal (Reusable)
**File**: `frontend/src/components/LineItemHistoryModal.tsx` (NEW)
A reusable modal component used by:
- SearchLineItems.tsx (📊 and 📦 buttons)
- Review.tsx (price indicator clicks)
**Props**:
```typescript
interface LineItemHistoryModalProps {
isOpen: boolean
onClose: () => void
productCode: string | null
description: string | null
supplierId: number
supplierName: string
currentPrice?: Decimal // For highlighting current price in chart
onAcknowledge?: (newPrice: Decimal) => void // Callback when price acknowledged
}
```
**Layout**:
```
┌─────────────────────────────────────────────────────────────────┐
│ Price History: Chicken Breast (ABC123) [X] │
│ Supplier: Brakes │
├─────────────────────────────────────────────────────────────────┤
│ Date Range: [From: ____] [To: ____] (default: last 12 months) │
├─────────────────────────────────────────────────────────────────┤
│ │
│ 📈 Price History Chart (Line chart with dates on X axis) │
│ £6.00 ─────────────────────• │
│ £5.50 ────•────────────────┘ │
│ £5.00 ────┘ │
│ Jan Feb Mar Apr May │
│ │
├─────────────────────────────────────────────────────────────────┤
│ Stats for Period: │
│ • Total Occurrences: 12 │
│ • Total Quantity: 45 │
│ • Avg Qty per Invoice: 3.75 │
│ • Avg Qty per Week: 1.2 │
│ • Avg Qty per Month: 5.0 │
├─────────────────────────────────────────────────────────────────┤
│ Current Price: £6.00 (🔴 +9% from previous £5.50) │
│ │
│ [Acknowledge Price Change] ← Only shown if price flagged │
└─────────────────────────────────────────────────────────────────┘
```
**Chart Library**: Use `recharts` (lightweight, React-native)
### 5. Invoice Review Price Indicators
**File**: `frontend/src/components/Review.tsx` (MODIFY)
Add price status indicators next to each line item's unit price:
**Per Line Item**:
- Fetch price_status from backend (included in line item response)
- Display icon next to unit_price:
- 🟢 ✓ (green): Price consistent with history
- 🟡 ? (amber): Small change ≤ threshold
- 🔴 ! (red): Large change > threshold
- No icon: No history (first purchase)
- Clicking icon opens LineItemHistoryModal
- In modal, "Acknowledge Price Change" button updates AcknowledgedPrice record
**Layout Change in Line Items Table**:
```
│ Code │ Description │ Qty │ Unit Price │ Amount │
│ ABC123 │ Chicken Breast │ 4 │ £5.50 🟢 │ £22.00 │
│ XYZ789 │ Beef Mince 500g │ 2 │ £6.20 🔴! [📊] │ £12.40 │
↑ Click to see history
```
### 6. Search Settings Section
**File**: `frontend/src/pages/Settings.tsx` (MODIFY)
Add new "Search Settings" section in Settings page:
```
┌─────────────────────────────────────────────────┐
│ Search Settings │
├─────────────────────────────────────────────────┤
│ Price Change Detection │
│ │
│ Lookback Period: [30] days │
│ (How far back to compare prices) │
│ │
│ Amber Threshold: [10] % │
│ (Highlight as warning if change ≤ this) │
│ │
│ Red Threshold: [20] % │
│ (Highlight as alert if change > amber) │
│ │
│ [Save] │
└─────────────────────────────────────────────────┘
```
---
## Files to Create/Modify
| File | Change |
|------|--------|
| **Backend** | |
| `backend/models/acknowledged_price.py` | NEW - AcknowledgedPrice model |
| `backend/models/settings.py` | Add price change threshold settings |
| `backend/services/price_history.py` | NEW - Reusable price history service |
| `backend/api/search.py` | NEW - Search endpoints + history + acknowledge |
| `backend/api/invoices.py` | Add price_status to line items response |
| `backend/main.py` | Register search router, run migration |
| `backend/migrations/add_price_settings.py` | NEW - Migration for new settings + acknowledged_prices table |
| **Frontend** | |
| `frontend/package.json` | Add recharts dependency |
| `frontend/src/App.tsx` | Add Search dropdown + routes |
| `frontend/src/utils/searchHelpers.ts` | NEW - Debounce hook, session storage keys |
| `frontend/src/components/SearchInvoices.tsx` | NEW - Invoice search page |
| `frontend/src/components/SearchLineItems.tsx` | NEW - Line items search with price flags |
| `frontend/src/components/SearchDefinitions.tsx` | NEW - Definitions search page |
| `frontend/src/components/LineItemHistoryModal.tsx` | NEW - Reusable history modal with chart |
| `frontend/src/components/Review.tsx` | Add price status icons to line items |
| `frontend/src/pages/Settings.tsx` | Add Search Settings section + Access Control paths |
---
## Implementation Steps
### Phase 1: Backend Foundation
1. Create `backend/models/acknowledged_price.py`
2. Add price change settings to `backend/models/settings.py`
3. Create migration `backend/migrations/add_price_settings.py`
4. Create `backend/services/price_history.py` service
5. Create `backend/api/search.py` with all endpoints
6. Update `backend/api/invoices.py` to include price_status
7. Register router in `backend/main.py`
### Phase 2: Frontend Search Pages
8. Install recharts: `npm install recharts`
9. Create `frontend/src/utils/searchHelpers.ts`
10. Create `SearchInvoices.tsx`
11. Create `SearchLineItems.tsx`
12. Create `SearchDefinitions.tsx`
### Phase 3: History Modal & Review Integration
13. Create `LineItemHistoryModal.tsx` with chart
14. Update `Review.tsx` with price status icons
15. Add "Search Settings" section to Settings.tsx
16. Add Search access control paths to Settings.tsx
17. Update `App.tsx` with Search dropdown and routes
### Phase 4: Test & Verify
18. Rebuild containers
19. Test all search pages
20. Test price change detection on invoice review
21. Test acknowledgement flow
---
## Access Control Paths
Add to Settings Access Control checkboxes:
- `/search-invoices` - Invoice Search
- `/search-line-items` - Line Items Search
- `/search-definitions` - Unit/Portion Definitions Search
---
## Session Storage Keys
All search state persists during browser session:
**Invoices**:
- `search-invoices-query`, `search-invoices-supplier`, `search-invoices-status`
- `search-invoices-from`, `search-invoices-to`, `search-invoices-group`
**Line Items**:
- `search-line-items-query`, `search-line-items-supplier`
- `search-line-items-from`, `search-line-items-to`, `search-line-items-group`
**Definitions**:
- `search-definitions-query`, `search-definitions-supplier`, `search-definitions-has-portions`
---
## Live Search Implementation
Use debounced input (300ms delay) with react-query:
```typescript
const [searchInput, setSearchInput] = useState('')
const debouncedSearch = useDebounce(searchInput, 300)
const { data, isLoading } = useQuery({
queryKey: ['search-invoices', debouncedSearch, supplier, status, dateFrom, dateTo, groupBy],
queryFn: () => fetchSearchResults(...)
})
```
Filter changes trigger immediate re-query (no debounce needed for dropdowns).
---
## Verification
### Search Pages
1. **Navigation**: Search dropdown appears between Invoices and Reports
2. **Invoice Search**:
- Type in search box, results filter after 300ms
- "Include line items" checkbox finds invoices by product names
- Filter by supplier/status works
- Date range defaults to last 30 days
- Group by Supplier shows collapsible sections
- Invoice # links open invoice in new tab
3. **Line Items Search**:
- Consolidated view shows unique items (not duplicates)
- Shows most recent price with change indicator (🟢/🟡/🔴)
- 📊 button opens history modal with price chart
- 📦 button opens history modal with qty stats
- Links to most recent invoice
4. **Definitions Search**:
- Shows all ProductDefinition records
- "Has Portions" filter works
- Links to source invoice
### Price Change Detection
5. **Invoice Review Page**:
- Line items show price status icon next to unit price
- 🟢 = consistent, 🟡 = small change, 🔴 = large change
- Clicking icon opens history modal
- "Acknowledge Price Change" button marks price as reviewed
- After acknowledgement, icon changes to 🟢 on future invoices
### History Modal
6. **Price History Chart**:
- Line chart shows price over time
- Default range: last 12 months
- Date range picker works
7. **Stats Display**:
- Total occurrences, total qty
- Avg qty per invoice/week/month
### Settings
8. **Search Settings Section**:
- Lookback period (default 30 days)
- Amber threshold % (default 10%)
- Red threshold % (default 20%)
- Changes apply to price detection
### Session Persistence
9. Enter search term, navigate away, come back - search term preserved
10. Close tab, reopen - state cleared (session storage)
### Access Control
11. Restrict search pages via Settings → Access Control
12. Non-admin users don't see restricted search options

390
docs/archive/PLAN_1_newbook_room_level_sync.md Normal file
View file

@ -0,0 +1,390 @@
# Plan 1: Newbook Room-Level Occupancy Sync
## Current State
The ResidentsTableChart feature has:
- ✅ Database schema with room-level fields (room_number, booking_id, guest_name, is_dbb, is_package)
- ✅ API endpoint to serve room-level data
- ✅ Frontend chart to display room occupancy
- ❌ Sync service only fetches aggregated daily totals, not individual room details
**Current database state:**
```sql
room_number | booking_id | guest_name | is_dbb
------------|------------|------------|-------
NULL | NULL | NULL | false
```
## Problem Statement
The Newbook API sync (`backend/services/newbook.py`) currently only fetches:
- Daily aggregated occupancy (total_rooms, occupied_rooms, occupancy_percentage)
- Meal allocations (breakfast/dinner counts and revenue)
- Arrival tracking (booking IDs and details for arrivals)
It does NOT fetch:
- Individual room numbers
- Which rooms are occupied on which dates
- Guest names per room
- Booking IDs per occupied room
- Meal plan details per booking (DBB, package deals)
## Goal
Update the Newbook sync service to fetch and store room-level occupancy data so the ResidentsTableChart displays actual rooms instead of "Unknown Room".
## Investigation Required
Before implementation, we need to verify what data is available from the Newbook API:
### 1. Review Newbook API Documentation
- What endpoint provides room-level occupancy?
- Does it return individual room statuses?
- Is guest name available (privacy concerns)?
- How are meal plans (DBB, packages) represented?
### 2. Examine Existing Sync Code
**File:** `backend/services/newbook.py`
Current occupancy sync likely uses:
- `/api/occupancy` or similar endpoint
- Returns aggregated daily statistics
- May need different endpoint for room-level details
### 3. Check Existing Arrival Tracking
The code already populates `arrival_booking_details` (JSONB) which might contain:
- Booking reference numbers
- Room assignments
- Guest names
- Meal plan flags
**SQL to check existing arrival data:**
```sql
SELECT date, arrival_booking_details
FROM newbook_daily_occupancy
WHERE arrival_booking_details IS NOT NULL
LIMIT 5;
```
## Potential Approaches
### Option A: Expand Arrival Tracking to All Stays
If `arrival_booking_details` contains room information, extend this to track:
- Not just arrivals, but all bookings staying on each date
- Store as `active_bookings_details` JSONB field
- Parse and populate room_number, booking_id, guest_name from this JSON
**Pros:**
- Minimal API changes if data already available
- Leverages existing JSON structure
**Cons:**
- JSONB storage plus denormalized columns (redundancy)
- May not scale well with many rooms
### Option B: New Endpoint for Room Status
Use a Newbook API endpoint that returns room-by-room status:
**Expected API response:**
```json
{
"date": "2026-01-22",
"rooms": [
{
"room_number": "101",
"status": "occupied",
"booking_id": "NB-12345",
"guest_name": "John Smith",
"check_in": "2026-01-22",
"check_out": "2026-01-25",
"meal_plan": "DBB",
"is_package": false
},
{
"room_number": "102",
"status": "vacant"
}
]
}
```
**Pros:**
- Clean, structured data
- All room details in one call
- Easier to maintain
**Cons:**
- May require different API endpoint
- More data to fetch/store
### Option C: Hybrid Approach (RECOMMENDED)
1. Use existing occupancy endpoint for aggregated stats
2. Fetch booking details separately (may already be happening for arrivals)
3. Match bookings to dates they span (check_in to check_out)
4. Create separate row per room per night
**Storage strategy:**
```
Current: 1 row per date (aggregated)
New: 1 row per room per night (denormalized)
Example for date 2026-01-22:
OLD:
- date: 2026-01-22, occupied_rooms: 6
NEW:
- date: 2026-01-22, room: 101, booking_id: NB-12345, guest: Smith
- date: 2026-01-22, room: 102, booking_id: NB-12346, guest: Jones
- date: 2026-01-22, room: 103, booking_id: NB-12347, guest: Brown
...
```
**Trade-off:** More database rows, but enables room-level reporting.
## Schema Considerations
### Current Schema Issue
`newbook_daily_occupancy` has unique constraint:
```sql
CONSTRAINT uq_newbook_occupancy_per_day
UNIQUE(kitchen_id, date)
```
This **prevents** multiple rows per date!
### Solution 1: Change Unique Constraint
```sql
-- Drop old constraint
ALTER TABLE newbook_daily_occupancy
DROP CONSTRAINT uq_newbook_occupancy_per_day;
-- Add new constraint for room-level uniqueness
ALTER TABLE newbook_daily_occupancy
ADD CONSTRAINT uq_newbook_occupancy_per_room_per_day
UNIQUE(kitchen_id, date, room_number);
```
**Impact:**
- Breaking change to schema
- Need migration to handle existing aggregated rows
- Backward compatibility: API endpoint must handle both aggregated and room-level rows
### Solution 2: Separate Table (Alternative)
Create `newbook_room_occupancy` table:
```sql
CREATE TABLE newbook_room_occupancy (
id SERIAL PRIMARY KEY,
kitchen_id INTEGER NOT NULL REFERENCES kitchens(id),
date DATE NOT NULL,
room_number VARCHAR(50) NOT NULL,
booking_id VARCHAR(100),
guest_name VARCHAR(255),
check_in DATE,
check_out DATE,
is_dbb BOOLEAN DEFAULT FALSE,
is_package BOOLEAN DEFAULT FALSE,
fetched_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT uq_room_occupancy_per_day
UNIQUE(kitchen_id, date, room_number)
);
```
Keep `newbook_daily_occupancy` for aggregated stats.
**Pros:**
- No breaking changes
- Clean separation of concerns
- Can keep both aggregated and room-level data
**Cons:**
- Two tables to maintain
- API endpoint needs to join data or use new table
## Recommended Implementation Plan
### Phase 1: Investigate Newbook API
1. Review Newbook API documentation for room-level endpoints
2. Test API calls to see what data is available
3. Check if current `arrival_booking_details` contains room info
4. Determine if guest names are available (privacy/GDPR)
### Phase 2: Database Migration
**Option A (if room data is sparse/optional):**
- Keep current schema
- Allow room_number, booking_id to be NULL
- Populate when available
- Current unique constraint stays
**Option B (if room data is always available):**
- Change unique constraint to (kitchen_id, date, room_number)
- Migrate existing aggregated rows (need strategy)
- Update sync to create multiple rows per date
**Option C (cleanest):**
- Create new `newbook_room_occupancy` table
- Keep existing table for aggregated stats
- Update API endpoint to use new table
### Phase 3: Update Sync Service
**File:** `backend/services/newbook.py`
**Current sync logic:**
```python
async def sync_occupancy(kitchen_id, date_from, date_to):
# Fetch aggregated occupancy
data = await newbook_api.get_occupancy(date_from, date_to)
# Store one row per date
for day in data:
occupancy = NewbookDailyOccupancy(
kitchen_id=kitchen_id,
date=day['date'],
total_rooms=day['total_rooms'],
occupied_rooms=day['occupied_rooms'],
# ...
)
```
**New sync logic (Option B - room-level):**
```python
async def sync_occupancy(kitchen_id, date_from, date_to):
# Fetch room-level occupancy
data = await newbook_api.get_room_status(date_from, date_to)
# Delete existing rows for this date range (full refresh)
await db.execute(
delete(NewbookDailyOccupancy).where(
and_(
NewbookDailyOccupancy.kitchen_id == kitchen_id,
NewbookDailyOccupancy.date >= date_from,
NewbookDailyOccupancy.date <= date_to
)
)
)
# Store one row per room per date
for day in data:
for room in day['rooms']:
if room['status'] == 'occupied':
occupancy = NewbookDailyOccupancy(
kitchen_id=kitchen_id,
date=day['date'],
room_number=room['room_number'],
booking_id=room['booking_id'],
guest_name=room.get('guest_name'), # May be NULL for privacy
is_dbb=room.get('meal_plan') == 'DBB',
is_package=room.get('is_package', False),
# Aggregated stats on each row (redundant but simple)
total_rooms=day['total_rooms'],
occupied_rooms=day['occupied_rooms'],
# ...
)
db.add(occupancy)
```
### Phase 4: Update API Endpoint
**File:** `backend/api/residents_table_chart.py`
**Current logic:**
- Groups by (room_number, booking_id)
- Expects rows already grouped
**Updated logic (if using room-level rows):**
- No changes needed! Already groups correctly
- Just needs room_number and booking_id to be populated
**Alternative (if keeping aggregated + separate table):**
- Query `newbook_room_occupancy` instead
- Join with `newbook_daily_occupancy` for aggregated stats
### Phase 5: Testing
1. Run sync for a test date range
2. Verify database populated:
```sql
SELECT date, room_number, booking_id, guest_name
FROM newbook_daily_occupancy
WHERE date = '2026-01-22'
ORDER BY room_number;
```
3. Test ResidentsTableChart page
4. Verify rooms display with actual names
5. Check restaurant booking linkage (hotel_booking_number)
## Privacy & GDPR Considerations
**Guest Names:**
- May need to be masked/hashed for privacy
- Consider: "Guest in Room 101" instead of actual name
- Add setting: "Show guest names" (admin only)
- Log access to guest PII
**Data Retention:**
- How long to keep room-level data?
- May need purge policy for old bookings
- Aggregate historical data, delete room details
## Performance Considerations
**Database Size:**
- Current: ~365 rows per year (1 per day)
- New: ~365 × 25 rooms = 9,125 rows per year
- 10 years: ~91,000 rows (still manageable)
**Indexing:**
- Existing: (kitchen_id, date)
- New: (kitchen_id, date, room_number) - already created
- Consider: (kitchen_id, booking_id) for linking to Resos
**Query Performance:**
- ResidentsTableChart queries 7-day window
- Fetching 7 × 25 = 175 rows max
- Indexes should handle this easily
## Migration Strategy
**For Existing Aggregated Rows:**
**Option 1:** Delete and re-sync
```sql
DELETE FROM newbook_daily_occupancy;
-- Then run sync to repopulate with room-level data
```
**Option 2:** Keep aggregated rows, add room-level
- Set room_number = NULL for aggregated rows
- Add new room-level rows alongside
- API filters WHERE room_number IS NOT NULL
**Option 3:** Backfill from booking history (if available)
- Query Newbook API for historical bookings
- Reconstruct room occupancy for past dates
- May be slow/rate-limited
## Next Steps
1. **Immediate:** Check Newbook API docs for room-level endpoint
2. **Investigate:** Review current `arrival_booking_details` structure
3. **Decide:** Choose schema approach (modify existing vs new table)
4. **Prototype:** Test API calls to fetch room data
5. **Implement:** Update sync service based on findings
6. **Test:** Verify ResidentsTableChart shows real rooms
## Unknown/Questions
- [ ] Does Newbook API provide room-level occupancy?
- [ ] What endpoint? (e.g., `/api/room_status`, `/api/bookings`)
- [ ] Are guest names available?
- [ ] How are meal plans represented in API?
- [ ] Is booking ID always available?
- [ ] Rate limits for room-level sync?
- [ ] Historical data availability?
## Success Criteria
✅ ResidentsTableChart shows actual room numbers (e.g., "101", "102") instead of "Unknown Room"
✅ Guest names populated (if available and permitted)
✅ Booking IDs linked correctly
✅ DBB/package flags set correctly
✅ Restaurant bookings link via hotel_booking_number
✅ Data refreshes daily (or more frequently)
✅ Performance remains acceptable (<500ms API response)

972
docs/archive/PLAN_2_newbook_frequent_updates.md Normal file
View file

@ -0,0 +1,972 @@
# Plan 2: Newbook Frequent Updates (15-Minute Intervals)
## Current State
### Existing Newbook Sync Mechanism
**File:** `backend/services/newbook.py`
Current sync behavior:
- Manual trigger via API endpoint
- Full historical sync on demand
- No automatic scheduling
- Fetches data from date range (date_from to date_to)
- Updates:
- Daily occupancy (aggregated)
- Meal allocations
- Arrival tracking
- GL account revenue
### Existing Resos Sync for Comparison
**Files:**
- `backend/services/resos.py` - Sync service
- `backend/models/resos.py` - `ResosUpcomingSyncSettings` model
- Database table: `resos_upcoming_sync_settings`
Resos already implements automatic frequent updates:
- Configurable sync interval (default 60 minutes)
- Automatic scheduling via background task
- Separate settings for upcoming bookings vs historical
- Settings page UI for interval configuration
## Problem Statement
Hotel managers need **near-real-time occupancy data** for the next 7 days to:
- Coordinate restaurant table assignments
- Plan staffing levels
- Monitor arrivals throughout the day
- React to last-minute bookings/cancellations
- Track meal plan changes
Current manual sync requires:
1. User remembers to trigger sync
2. User waits for completion
3. No automatic updates when bookings change
**Goal:** Implement automatic 15-minute sync for next 7 days, similar to Resos upcoming bookings feature.
## Requirements
### Functional Requirements
1. **Automatic Sync Scheduling**
- Default interval: 15 minutes
- Configurable via settings page
- Target date range: Today to Today + 7 days
- Run continuously while app is running
2. **Settings Management**
- Add "Newbook Frequent Update Interval" to settings page
- Dropdown options: 5, 10, 15, 30, 60 minutes
- Enable/disable toggle
- Per-kitchen configuration
3. **Background Task**
- Non-blocking execution
- Error handling and retry logic
- Logging for monitoring
- Graceful shutdown on app restart
4. **Scope Limitation**
- ONLY sync next 7 days (not full history)
- Keep existing manual full-sync functionality
- Frequent updates don't replace historical sync
### Non-Functional Requirements
1. **Performance**
- Sync completes within 30 seconds
- No impact on API response times
- Rate limit compliance with Newbook API
2. **Reliability**
- Failed sync doesn't stop scheduler
- Exponential backoff on API errors
- Alert on repeated failures
3. **Observability**
- Log each sync start/completion
- Track sync duration
- Record API error rates
- Dashboard widget showing last sync time
## Architecture
### Component Overview
```
┌─────────────────────────────────────────────────────┐
│ Frontend │
│ │
│ ┌──────────────────────────────────────────┐ │
│ │ Settings Page │ │
│ │ - Newbook Frequent Update Interval │ │
│ │ - Enable/Disable Toggle │ │
│ │ - Last Sync Time Display │ │
│ └──────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────┐
│ Backend API │
│ │
│ ┌──────────────────────────────────────────┐ │
│ │ POST /api/settings/newbook-sync │ │
│ │ GET /api/settings/newbook-sync │ │
│ └──────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────┐
│ Background Scheduler │
│ │
│ ┌──────────────────────────────────────────┐ │
│ │ newbook_frequent_sync_loop() │ │
│ │ - Runs every N minutes │ │
│ │ - Queries active kitchens │ │
│ │ - Calls sync_occupancy(today, today+7) │ │
│ └──────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────┐
│ Newbook Service │
│ │
│ ┌──────────────────────────────────────────┐ │
│ │ sync_occupancy(kitchen_id, from, to) │ │
│ │ - Existing sync logic │ │
│ │ - Fetches occupancy data │ │
│ │ - Updates database │ │
│ └──────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘
```
## Implementation Plan
### Phase 1: Database Schema
#### 1.1 Create Settings Table
**File:** `backend/models/newbook.py`
Add new model after `NewbookSyncLog`:
```python
class NewbookFrequentSyncSettings(Base):
"""Settings for automatic frequent Newbook occupancy sync"""
__tablename__ = "newbook_frequent_sync_settings"
id: Mapped[int] = mapped_column(primary_key=True, index=True)
kitchen_id: Mapped[int] = mapped_column(ForeignKey("kitchens.id"), nullable=False, unique=True)
# Sync configuration
is_enabled: Mapped[bool] = mapped_column(Boolean, default=True)
sync_interval_minutes: Mapped[int] = mapped_column(Integer, default=15) # Default 15 minutes
# Sync scope
days_ahead: Mapped[int] = mapped_column(Integer, default=7) # Sync next N days
# Status tracking
last_sync_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
last_sync_status: Mapped[str | None] = mapped_column(String(20), nullable=True) # success, failed, running
last_error_message: Mapped[str | None] = mapped_column(Text, nullable=True)
consecutive_failures: Mapped[int] = mapped_column(Integer, default=0)
created_at: Mapped[datetime] = mapped_column(DateTime, default=datetime.utcnow)
updated_at: Mapped[datetime] = mapped_column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
# Relationships
kitchen: Mapped["Kitchen"] = relationship("Kitchen", back_populates="newbook_frequent_sync_settings")
```
#### 1.2 Create Migration
**File:** `backend/migrations/add_newbook_frequent_sync.py` (NEW)
```python
import asyncio
import logging
from sqlalchemy import text
from database import engine
logger = logging.getLogger(__name__)
async def run_migration():
"""Create newbook_frequent_sync_settings table"""
create_table_sql = """
CREATE TABLE IF NOT EXISTS newbook_frequent_sync_settings (
id SERIAL PRIMARY KEY,
kitchen_id INTEGER NOT NULL REFERENCES kitchens(id) UNIQUE,
is_enabled BOOLEAN DEFAULT TRUE,
sync_interval_minutes INTEGER DEFAULT 15,
days_ahead INTEGER DEFAULT 7,
last_sync_at TIMESTAMP,
last_sync_status VARCHAR(20),
last_error_message TEXT,
consecutive_failures INTEGER DEFAULT 0,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
"""
create_index_sql = """
CREATE INDEX IF NOT EXISTS idx_newbook_frequent_sync_kitchen
ON newbook_frequent_sync_settings(kitchen_id);
"""
# Initialize settings for all existing kitchens
initialize_settings_sql = """
INSERT INTO newbook_frequent_sync_settings (kitchen_id, is_enabled, sync_interval_minutes, days_ahead)
SELECT id, TRUE, 15, 7
FROM kitchens
WHERE id NOT IN (SELECT kitchen_id FROM newbook_frequent_sync_settings)
ON CONFLICT (kitchen_id) DO NOTHING;
"""
try:
async with engine.begin() as conn:
await conn.execute(text(create_table_sql))
logger.info("Created newbook_frequent_sync_settings table")
await conn.execute(text(create_index_sql))
logger.info("Created newbook_frequent_sync_settings indexes")
await conn.execute(text(initialize_settings_sql))
logger.info("Initialized newbook frequent sync settings for existing kitchens")
except Exception as e:
if "already exists" not in str(e).lower():
raise
logger.warning(f"Newbook frequent sync migration: {e}")
if __name__ == "__main__":
asyncio.run(run_migration())
```
#### 1.3 Update Kitchen Model
**File:** `backend/models/user.py`
Add relationship to Kitchen class:
```python
# In Kitchen class, add to relationships section:
newbook_frequent_sync_settings: Mapped["NewbookFrequentSyncSettings"] = relationship(
"NewbookFrequentSyncSettings", back_populates="kitchen", uselist=False
)
```
### Phase 2: Background Scheduler
#### 2.1 Create Scheduler Service
**File:** `backend/services/newbook_scheduler.py` (NEW)
```python
import asyncio
import logging
from datetime import datetime, date, timedelta
from sqlalchemy import select, and_
from sqlalchemy.ext.asyncio import AsyncSession
from database import get_db_context
from models.newbook import NewbookFrequentSyncSettings
from models.user import Kitchen
from services.newbook import sync_occupancy_for_kitchen
logger = logging.getLogger(__name__)
class NewbookScheduler:
"""Background scheduler for frequent Newbook occupancy sync"""
def __init__(self):
self.is_running = False
self.task = None
async def start(self):
"""Start the background scheduler"""
if self.is_running:
logger.warning("Newbook scheduler already running")
return
self.is_running = True
self.task = asyncio.create_task(self._run_loop())
logger.info("Newbook frequent sync scheduler started")
async def stop(self):
"""Stop the background scheduler"""
self.is_running = False
if self.task:
self.task.cancel()
try:
await self.task
except asyncio.CancelledError:
pass
logger.info("Newbook frequent sync scheduler stopped")
async def _run_loop(self):
"""Main scheduler loop"""
while self.is_running:
try:
await self._sync_all_kitchens()
except Exception as e:
logger.error(f"Error in Newbook scheduler loop: {e}", exc_info=True)
# Wait for next cycle (check every minute, actual sync based on interval)
await asyncio.sleep(60)
async def _sync_all_kitchens(self):
"""Sync occupancy for all enabled kitchens"""
async with get_db_context() as db:
# Get all kitchens with frequent sync enabled
result = await db.execute(
select(NewbookFrequentSyncSettings, Kitchen).join(
Kitchen, NewbookFrequentSyncSettings.kitchen_id == Kitchen.id
).where(
and_(
NewbookFrequentSyncSettings.is_enabled == True,
Kitchen.newbook_api_key.isnot(None) # Only kitchens with Newbook configured
)
)
)
settings_and_kitchens = result.all()
for settings, kitchen in settings_and_kitchens:
# Check if enough time has passed since last sync
if settings.last_sync_at:
minutes_since_sync = (datetime.utcnow() - settings.last_sync_at).total_seconds() / 60
if minutes_since_sync < settings.sync_interval_minutes:
continue # Too soon, skip
# Perform sync
await self._sync_kitchen(db, settings, kitchen)
async def _sync_kitchen(self, db: AsyncSession, settings: NewbookFrequentSyncSettings, kitchen: Kitchen):
"""Sync occupancy for a single kitchen"""
logger.info(f"Starting Newbook frequent sync for kitchen {kitchen.id} ({kitchen.name})")
# Update status to running
settings.last_sync_status = "running"
await db.commit()
try:
# Calculate date range: today to today + N days
date_from = date.today()
date_to = date_from + timedelta(days=settings.days_ahead)
# Call existing sync service
await sync_occupancy_for_kitchen(kitchen.id, date_from, date_to, db)
# Update success status
settings.last_sync_at = datetime.utcnow()
settings.last_sync_status = "success"
settings.last_error_message = None
settings.consecutive_failures = 0
logger.info(f"Newbook frequent sync completed for kitchen {kitchen.id}")
except Exception as e:
# Update failure status
settings.last_sync_status = "failed"
settings.last_error_message = str(e)[:500] # Truncate long errors
settings.consecutive_failures += 1
logger.error(f"Newbook frequent sync failed for kitchen {kitchen.id}: {e}", exc_info=True)
# Alert if repeated failures
if settings.consecutive_failures >= 5:
logger.critical(f"Newbook sync has failed {settings.consecutive_failures} times for kitchen {kitchen.id}")
finally:
await db.commit()
# Global scheduler instance
newbook_scheduler = NewbookScheduler()
```
#### 2.2 Update Newbook Service
**File:** `backend/services/newbook.py`
Extract existing sync logic into a reusable function:
```python
async def sync_occupancy_for_kitchen(
kitchen_id: int,
date_from: date,
date_to: date,
db: AsyncSession
) -> dict:
"""
Sync occupancy data for a kitchen (reusable by scheduler and manual API)
Returns:
dict with sync stats (records_fetched, errors, etc.)
"""
# Extract existing logic from current sync endpoint
# This is the core sync logic that both manual and automatic sync will use
# Get kitchen and API credentials
kitchen = await db.get(Kitchen, kitchen_id)
if not kitchen or not kitchen.newbook_api_key:
raise ValueError(f"Kitchen {kitchen_id} not found or Newbook not configured")
# Create Newbook API client
client = NewbookAPIClient(
api_key=kitchen.newbook_api_key,
api_secret=kitchen.newbook_api_secret,
property_id=kitchen.newbook_property_id
)
# Fetch occupancy data
occupancy_data = await client.get_occupancy(date_from, date_to)
# Update database (existing logic)
records_fetched = 0
for day_data in occupancy_data:
# Existing upsert logic
records_fetched += 1
return {
"records_fetched": records_fetched,
"date_from": date_from.isoformat(),
"date_to": date_to.isoformat(),
"status": "success"
}
```
#### 2.3 Register Scheduler in main.py
**File:** `backend/main.py`
Update lifespan function to start/stop scheduler:
```python
from services.newbook_scheduler import newbook_scheduler
@asynccontextmanager
async def lifespan(app: FastAPI):
"""Application lifecycle manager"""
# ... existing startup code ...
# Run migrations
await run_migrations()
# Start background schedulers
await newbook_scheduler.start()
logger.info("Background schedulers started")
yield
# Shutdown
await newbook_scheduler.stop()
logger.info("Background schedulers stopped")
```
### Phase 3: API Endpoints
#### 3.1 Settings API
**File:** `backend/api/settings.py` (UPDATE)
Add endpoints for Newbook frequent sync settings:
```python
from models.newbook import NewbookFrequentSyncSettings
from pydantic import BaseModel
class NewbookFrequentSyncSettingsUpdate(BaseModel):
is_enabled: bool
sync_interval_minutes: int
days_ahead: int
class NewbookFrequentSyncSettingsResponse(BaseModel):
is_enabled: bool
sync_interval_minutes: int
days_ahead: int
last_sync_at: Optional[datetime]
last_sync_status: Optional[str]
last_error_message: Optional[str]
consecutive_failures: int
@router.get("/newbook-frequent-sync")
async def get_newbook_frequent_sync_settings(
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db)
) -> NewbookFrequentSyncSettingsResponse:
"""Get Newbook frequent sync settings for current kitchen"""
result = await db.execute(
select(NewbookFrequentSyncSettings).where(
NewbookFrequentSyncSettings.kitchen_id == current_user.kitchen_id
)
)
settings = result.scalar_one_or_none()
if not settings:
# Create default settings
settings = NewbookFrequentSyncSettings(
kitchen_id=current_user.kitchen_id,
is_enabled=True,
sync_interval_minutes=15,
days_ahead=7
)
db.add(settings)
await db.commit()
await db.refresh(settings)
return NewbookFrequentSyncSettingsResponse(
is_enabled=settings.is_enabled,
sync_interval_minutes=settings.sync_interval_minutes,
days_ahead=settings.days_ahead,
last_sync_at=settings.last_sync_at,
last_sync_status=settings.last_sync_status,
last_error_message=settings.last_error_message,
consecutive_failures=settings.consecutive_failures
)
@router.post("/newbook-frequent-sync")
async def update_newbook_frequent_sync_settings(
settings_update: NewbookFrequentSyncSettingsUpdate,
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db)
) -> NewbookFrequentSyncSettingsResponse:
"""Update Newbook frequent sync settings"""
# Validate interval
valid_intervals = [5, 10, 15, 30, 60]
if settings_update.sync_interval_minutes not in valid_intervals:
raise HTTPException(
status_code=400,
detail=f"Invalid sync interval. Must be one of: {valid_intervals}"
)
# Validate days_ahead
if settings_update.days_ahead < 1 or settings_update.days_ahead > 30:
raise HTTPException(
status_code=400,
detail="days_ahead must be between 1 and 30"
)
result = await db.execute(
select(NewbookFrequentSyncSettings).where(
NewbookFrequentSyncSettings.kitchen_id == current_user.kitchen_id
)
)
settings = result.scalar_one_or_none()
if not settings:
settings = NewbookFrequentSyncSettings(kitchen_id=current_user.kitchen_id)
db.add(settings)
settings.is_enabled = settings_update.is_enabled
settings.sync_interval_minutes = settings_update.sync_interval_minutes
settings.days_ahead = settings_update.days_ahead
settings.updated_at = datetime.utcnow()
await db.commit()
await db.refresh(settings)
return NewbookFrequentSyncSettingsResponse(
is_enabled=settings.is_enabled,
sync_interval_minutes=settings.sync_interval_minutes,
days_ahead=settings.days_ahead,
last_sync_at=settings.last_sync_at,
last_sync_status=settings.last_sync_status,
last_error_message=settings.last_error_message,
consecutive_failures=settings.consecutive_failures
)
```
### Phase 4: Frontend Implementation
#### 4.1 Settings Page Section
**File:** `frontend/src/pages/Settings.tsx` (UPDATE)
Add Newbook frequent sync settings section:
```typescript
interface NewbookFrequentSyncSettings {
is_enabled: boolean
sync_interval_minutes: number
days_ahead: number
last_sync_at: string | null
last_sync_status: string | null
last_error_message: string | null
consecutive_failures: number
}
// Add query hook
const { data: newbookSyncSettings, isLoading: isLoadingNewbookSync } = useQuery<NewbookFrequentSyncSettings>({
queryKey: ['newbook-frequent-sync-settings'],
queryFn: async () => {
const res = await fetch('/api/settings/newbook-frequent-sync', {
headers: { Authorization: `Bearer ${token}` }
})
if (!res.ok) throw new Error('Failed to fetch Newbook sync settings')
return res.json()
}
})
// Add mutation hook
const updateNewbookSyncMutation = useMutation({
mutationFn: async (settings: Partial<NewbookFrequentSyncSettings>) => {
const res = await fetch('/api/settings/newbook-frequent-sync', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${token}`
},
body: JSON.stringify(settings)
})
if (!res.ok) throw new Error('Failed to update settings')
return res.json()
},
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['newbook-frequent-sync-settings'] })
}
})
// Add UI section (after Resos settings):
<div style={styles.section}>
<h2>Newbook Frequent Updates</h2>
<p style={{ color: '#666', marginBottom: '1rem' }}>
Automatically sync hotel occupancy data for the next 7 days at regular intervals.
Keeps ResidentsTableChart and forecasts up-to-date with last-minute bookings.
</p>
{isLoadingNewbookSync ? (
<div>Loading...</div>
) : newbookSyncSettings ? (
<>
<div style={styles.formGroup}>
<label style={styles.label}>
<input
type="checkbox"
checked={newbookSyncSettings.is_enabled}
onChange={(e) => updateNewbookSyncMutation.mutate({
...newbookSyncSettings,
is_enabled: e.target.checked
})}
/>
Enable automatic frequent sync
</label>
</div>
{newbookSyncSettings.is_enabled && (
<>
<div style={styles.formGroup}>
<label style={styles.label}>Sync Interval</label>
<select
value={newbookSyncSettings.sync_interval_minutes}
onChange={(e) => updateNewbookSyncMutation.mutate({
...newbookSyncSettings,
sync_interval_minutes: parseInt(e.target.value)
})}
style={styles.input}
>
<option value={5}>Every 5 minutes</option>
<option value={10}>Every 10 minutes</option>
<option value={15}>Every 15 minutes (Recommended)</option>
<option value={30}>Every 30 minutes</option>
<option value={60}>Every 60 minutes</option>
</select>
</div>
<div style={styles.formGroup}>
<label style={styles.label}>Days Ahead</label>
<input
type="number"
min={1}
max={30}
value={newbookSyncSettings.days_ahead}
onChange={(e) => updateNewbookSyncMutation.mutate({
...newbookSyncSettings,
days_ahead: parseInt(e.target.value)
})}
style={styles.input}
/>
<small style={{ color: '#666' }}>
Number of days to sync ahead (default: 7)
</small>
</div>
{newbookSyncSettings.last_sync_at && (
<div style={styles.statusBox}>
<div style={styles.statusRow}>
<strong>Last Sync:</strong>
<span>{new Date(newbookSyncSettings.last_sync_at).toLocaleString()}</span>
</div>
<div style={styles.statusRow}>
<strong>Status:</strong>
<span style={{
color: newbookSyncSettings.last_sync_status === 'success' ? 'green' :
newbookSyncSettings.last_sync_status === 'failed' ? 'red' : 'orange'
}}>
{newbookSyncSettings.last_sync_status?.toUpperCase()}
</span>
</div>
{newbookSyncSettings.last_error_message && (
<div style={styles.errorMessage}>
<strong>Error:</strong> {newbookSyncSettings.last_error_message}
</div>
)}
{newbookSyncSettings.consecutive_failures > 0 && (
<div style={{ color: 'orange', marginTop: '0.5rem' }}>
⚠️ {newbookSyncSettings.consecutive_failures} consecutive failures
</div>
)}
</div>
)}
</>
)}
</>
) : null}
</div>
```
#### 4.2 Add Status Indicator to Dashboard
**File:** `frontend/src/pages/Dashboard.tsx` (UPDATE)
Add widget showing Newbook sync status:
```typescript
<div style={styles.widget}>
<h3>Newbook Sync Status</h3>
{newbookSyncSettings && (
<>
<div>
<strong>Interval:</strong> Every {newbookSyncSettings.sync_interval_minutes} minutes
</div>
{newbookSyncSettings.last_sync_at && (
<div>
<strong>Last Sync:</strong> {formatTimeAgo(newbookSyncSettings.last_sync_at)}
</div>
)}
<div style={{
padding: '0.5rem',
marginTop: '0.5rem',
borderRadius: '4px',
background: newbookSyncSettings.last_sync_status === 'success' ? '#e7f5e7' :
newbookSyncSettings.last_sync_status === 'failed' ? '#ffe7e7' : '#fff3cd',
color: newbookSyncSettings.last_sync_status === 'success' ? 'green' :
newbookSyncSettings.last_sync_status === 'failed' ? 'red' : 'orange'
}}>
{newbookSyncSettings.last_sync_status === 'success' ? '✓ Syncing' :
newbookSyncSettings.last_sync_status === 'failed' ? '✗ Sync Failed' : '⟳ Running'}
</div>
</>
)}
</div>
```
### Phase 5: Testing
#### 5.1 Unit Tests
**File:** `backend/tests/test_newbook_scheduler.py` (NEW)
```python
import pytest
from datetime import datetime, date, timedelta
from services.newbook_scheduler import NewbookScheduler
from models.newbook import NewbookFrequentSyncSettings
@pytest.mark.asyncio
async def test_scheduler_starts_and_stops():
scheduler = NewbookScheduler()
assert not scheduler.is_running
await scheduler.start()
assert scheduler.is_running
await scheduler.stop()
assert not scheduler.is_running
@pytest.mark.asyncio
async def test_sync_respects_interval(db_session):
# Create settings with 15-minute interval
settings = NewbookFrequentSyncSettings(
kitchen_id=1,
is_enabled=True,
sync_interval_minutes=15,
last_sync_at=datetime.utcnow() - timedelta(minutes=10) # 10 minutes ago
)
db_session.add(settings)
await db_session.commit()
scheduler = NewbookScheduler()
# Should skip sync (only 10 minutes passed, need 15)
await scheduler._sync_all_kitchens()
# Verify sync was not performed
await db_session.refresh(settings)
assert settings.last_sync_status != "running"
@pytest.mark.asyncio
async def test_sync_handles_errors(db_session, mock_newbook_api_error):
settings = NewbookFrequentSyncSettings(
kitchen_id=1,
is_enabled=True,
sync_interval_minutes=15,
consecutive_failures=0
)
db_session.add(settings)
await db_session.commit()
scheduler = NewbookScheduler()
await scheduler._sync_all_kitchens()
await db_session.refresh(settings)
assert settings.last_sync_status == "failed"
assert settings.consecutive_failures == 1
assert settings.last_error_message is not None
```
#### 5.2 Integration Tests
**Test Scenarios:**
1. **Enable sync via UI**
- Navigate to Settings page
- Enable Newbook frequent sync
- Set interval to 5 minutes (for faster testing)
- Verify settings saved
2. **Verify automatic sync**
- Wait 5 minutes
- Check backend logs for sync execution
- Verify database updated with recent dates
- Check settings show last_sync_at updated
3. **Test error handling**
- Temporarily break Newbook API credentials
- Wait for next sync
- Verify error message displayed in UI
- Fix credentials
- Verify sync recovers
4. **Test disable sync**
- Disable sync in Settings
- Wait past interval
- Verify no sync occurs
- Check logs confirm scheduler skips disabled kitchens
#### 5.3 Performance Tests
**Test:** Measure sync duration for 7-day window
- Expected: <30 seconds for typical hotel
- Alert if exceeds 60 seconds
**Test:** Verify no API response impact during sync
- Make API calls while sync running
- Measure latency
- Ensure <200ms response times
### Phase 6: Monitoring & Alerting
#### 6.1 Logging
Add structured logging:
```python
logger.info(
"Newbook frequent sync completed",
extra={
"kitchen_id": kitchen.id,
"records_fetched": result["records_fetched"],
"duration_seconds": duration,
"date_from": date_from.isoformat(),
"date_to": date_to.isoformat()
}
)
```
#### 6.2 Metrics
Track key metrics:
- Sync success rate (%)
- Average sync duration (seconds)
- API error rate
- Consecutive failure count per kitchen
#### 6.3 Alerts
Configure alerts for:
- **Critical:** 5+ consecutive failures
- **Warning:** Sync duration >60 seconds
- **Warning:** No sync in 2x expected interval
## Rollout Plan
### Phase 1: Backend Only (Week 1)
- Deploy database migration
- Deploy scheduler service
- Test with single kitchen
- Monitor logs and performance
### Phase 2: Settings UI (Week 2)
- Deploy settings API endpoints
- Deploy settings page UI
- Enable for pilot customers
- Gather feedback
### Phase 3: Dashboard Integration (Week 3)
- Add dashboard widget
- Add sync status indicators
- Document feature for users
- Enable for all customers
### Phase 4: Optimization (Week 4)
- Tune sync intervals based on usage
- Optimize API calls
- Add caching if needed
- Performance monitoring
## Configuration Options
### Recommended Intervals by Property Size
- **Small (<10 rooms):** 30 minutes
- **Medium (10-25 rooms):** 15 minutes (default)
- **Large (25+ rooms):** 10 minutes
- **High churn properties:** 5 minutes
### Advanced Settings (Future)
- **Smart intervals:** Increase frequency during check-in hours
- **Selective sync:** Only sync rooms with changes
- **Webhook integration:** Real-time updates on booking changes
- **Batch optimization:** Group multiple kitchens in single API call
## Success Criteria
✅ Scheduler runs continuously without crashes
✅ Sync completes within 30 seconds for 7-day window
✅ Settings UI allows enable/disable and interval configuration
✅ Dashboard shows last sync time and status
✅ Failed syncs logged with error details
✅ Consecutive failures trigger alerts
✅ No impact on API response times
✅ ResidentsTableChart shows up-to-date data
## Future Enhancements
1. **Predictive Sync**
- Increase frequency during peak booking hours
- Reduce frequency overnight
2. **Differential Sync**
- Only fetch changed bookings
- Reduce API load and sync time
3. **Multi-Property Optimization**
- Batch requests for properties with same owner
- Share rate limits across properties
4. **Webhook Integration**
- Real-time push updates from Newbook
- Eliminate polling entirely
5. **Sync History Dashboard**
- Chart showing sync frequency and success rate
- Identify patterns in failures
- Performance trends over time

66
docs/archive/PLAN_3_implementation_draft.md Normal file
View file

@ -0,0 +1,66 @@
# Plan 3: SambaPOS Database Replication - Implementation Plan (Draft)
**Status:** On hold - needs further consideration on approach
## Overview
Implement selective data replication from SambaPOS SQL Server to local PostgreSQL following Option B from PLAN_3. This reduces EPOS load, preserves historical data after clears, enables offline access, and includes SambaPOS data in Nextcloud backups.
## Current State
**Existing Implementation:** `backend/services/sambapos_api.py`
- `SambaPOSClient` class with direct MSSQL queries via aioodbc
- Real-time queries for categories, top sellers, GL codes, restaurant spend
- No local caching or replication
- Data lost when SambaPOS clears database periodically
**Connection Settings:** `backend/models/settings.py:76-88`
- `sambapos_db_host`, `sambapos_db_port`, `sambapos_db_name`, `sambapos_db_username`, `sambapos_db_password`
## Open Question
Should we:
1. Create new PostgreSQL tables with normalized schema (PLAN_3 Option B)
2. Mirror exact SambaPOS schema to reuse existing queries
3. Add local SQL Server container (zero query changes)
4. Just do periodic archives for backup (simplest)
## Implementation Phases (if proceeding with Option B)
### Phase 1: Database Models
**New File:** `backend/models/sambapos_replica.py`
Create 8 SQLAlchemy models:
1. `SambaposReplicationSettings` - Sync config and tracking
2. `SambaposTransaction` - Sales, refunds, voids
3. `SambaposPayment` - Payment types and amounts
4. `SambaposTicket` - Order headers
5. `SambaposTicketItem` - Line items with order tags
6. `SambaposMenuItem` - Product catalog
7. `SambaposAccount` - Customer accounts, hotel rooms
8. `SambaposArchive` - Full database snapshot metadata
### Phase 2: Replication Service
**New File:** `backend/services/sambapos_replicator.py`
### Phase 3: Background Scheduler
**New File:** `backend/services/sambapos_scheduler.py`
### Phase 4: Update SambaPOS API Service
### Phase 5: Backup Integration
### Phase 6: Frontend Settings UI
## Success Criteria
- [ ] SambaPOS data replicated to local PostgreSQL
- [ ] Automatic sync every 15 minutes (configurable)
- [ ] Reports can query replica instead of live database
- [ ] Historical data preserved after SambaPOS clears
- [ ] Backups include all replicated data
- [ ] Settings UI shows sync status and allows manual trigger
- [ ] Fallback to live database if replica empty

1136
docs/archive/PLAN_3_sambapos_database_replication.md Normal file
View file

File diff suppressed because it is too large Load diff

1288
docs/archive/PLAN_4_wastage_logbook.md Normal file
View file

File diff suppressed because it is too large Load diff

1048
docs/archive/PLAN_5_database_backup_verification.md Normal file
View file

File diff suppressed because it is too large Load diff

830
docs/archive/PLAN_6_pdf_highlighting_dext_emails.md Normal file
View file

@ -0,0 +1,830 @@
# Plan 6: PDF Highlighting for Non-Stock Items in Dext Emails
## Current State
### Existing Dext Integration
**Files:**
- `backend/services/dext.py` - Dext API client
- `backend/services/email_service.py` - Email sending
- Email templates for notifications
Current email functionality:
- Sends plain text email with invoice details
- Lists non-stock items in email body
- Provides link to PDF in Nextcloud
- No highlighting or annotation on PDF itself
**Example current email:**
```
Subject: Invoice Requires Attention - Non-Stock Items Found
Invoice #12345 from ACME Suppliers has 3 non-stock items:
- Garden Peas, Frozen (2.5 kg) - €15.50
- Tomato Sauce, Organic (1 L) - €8.25
- Cleaning Spray (500 ml) - €4.99
Please add these items to your product catalog or link them to existing products.
View Invoice: [Nextcloud Link]
```
## Problem Statement
**User Request:**
> "highlight or embed in the pdf itself not just the email body text"
**Current Limitation:**
- Non-stock items only highlighted in email text
- PDF remains unchanged from Dext
- User must manually find items in PDF
- Time-consuming when PDF has 50+ line items
**Goal:**
Modify the PDF to visually highlight non-stock items before sending email, so users can immediately identify problematic items in the document itself.
## Use Cases
### Use Case 1: Large Multi-Page Invoice
**Scenario:** 100-item invoice with 5 non-stock items scattered across 3 pages
**Current Experience:**
1. Read email listing 5 items
2. Open PDF
3. Manually scan all 100 items to find the 5 mentioned
4. Switch back and forth between email and PDF
**Desired Experience:**
1. Open PDF
2. **Immediately see yellow-highlighted items** on pages 1, 2, and 3
3. Quickly identify and resolve non-stock items
### Use Case 2: Similar Product Names
**Scenario:** Invoice has "Tomatoes, Vine" (in stock) and "Tomatoes, Cherry" (not in stock)
**Current Experience:**
- Email says "Tomatoes, Cherry" not in stock
- PDF has 3 different tomato products
- User must carefully read each tomato line to find "Cherry"
**Desired Experience:**
- PDF shows "Tomatoes, Cherry" with bright yellow highlight
- Instantly distinguishable from other tomato products
## Technical Approaches
### Option A: PDF Annotation (Recommended)
**Concept:** Add highlight annotations directly to PDF using PyPDF2 or reportlab
**Pros:**
- Native PDF feature (annotations)
- Works in all PDF viewers
- Non-destructive (original content preserved)
- Can add notes/comments
**Cons:**
- Complex to position highlights accurately
- Requires text coordinate detection
- May not work with scanned PDFs (OCR needed)
**Libraries:**
- `PyPDF2` / `pypdf` - PDF manipulation
- `pdfplumber` - Text extraction with coordinates
- `reportlab` - PDF generation/modification
### Option B: Render New PDF with Highlights
**Concept:** Extract content, render new PDF with highlighted sections
**Pros:**
- Full control over appearance
- Can add colored boxes, borders, icons
- Works with any PDF structure
**Cons:**
- More complex implementation
- May lose original formatting
- Larger file sizes
**Libraries:**
- `reportlab` - PDF rendering
- `pdf2image` + PIL - Image-based approach
### Option C: Embedded Annotations + Email Summary
**Concept:** Combine PDF annotations with enhanced email (current approach++)
**Pros:**
- Best of both worlds
- Fallback for annotation failures
- Accessible via email even without PDF viewer
**Cons:**
- Most development work
- Redundant information
## Recommended Approach: Option A (PDF Annotation)
Use pdfplumber to find text coordinates and PyPDF2 to add highlight annotations.
## Implementation Plan
### Phase 1: Text Coordinate Detection
#### 1.1 Install Dependencies
**File:** `backend/requirements.txt` (UPDATE)
```txt
# Existing dependencies...
# PDF processing
pdfplumber==0.10.3
pypdf==3.17.4
```
#### 1.2 Create PDF Highlighter Service
**File:** `backend/services/pdf_highlighter.py` (NEW)
```python
import pdfplumber
from pypdf import PdfReader, PdfWriter
from pypdf.generic import DictionaryObject, ArrayObject, FloatObject, NameObject
from pathlib import Path
import logging
from typing import List, Dict, Tuple
logger = logging.getLogger(__name__)
class PDFHighlighter:
"""Service for highlighting text in PDF files"""
def __init__(self, pdf_path: str):
self.pdf_path = pdf_path
self.reader = PdfReader(pdf_path)
self.writer = PdfWriter()
def highlight_text_items(self, items_to_highlight: List[str], output_path: str) -> str:
"""
Highlight specific text items in PDF
Args:
items_to_highlight: List of text strings to find and highlight
output_path: Path to save annotated PDF
Returns:
Path to annotated PDF
"""
logger.info(f"Highlighting {len(items_to_highlight)} items in {self.pdf_path}")
try:
# Extract text with coordinates using pdfplumber
text_coordinates = self._extract_text_coordinates(items_to_highlight)
# Add pages to writer with highlights
for page_num, page in enumerate(self.reader.pages):
# Add page to writer
self.writer.add_page(page)
# Get highlights for this page
page_highlights = text_coordinates.get(page_num, [])
if page_highlights:
# Add highlight annotations to page
for coords in page_highlights:
self._add_highlight_annotation(
page_num=page_num,
coordinates=coords
)
# Write annotated PDF
with open(output_path, "wb") as output_file:
self.writer.write(output_file)
logger.info(f"Annotated PDF saved to {output_path}")
return output_path
except Exception as e:
logger.error(f"PDF highlighting failed: {e}", exc_info=True)
# Return original PDF path if highlighting fails
return self.pdf_path
def _extract_text_coordinates(self, search_texts: List[str]) -> Dict[int, List[Dict]]:
"""
Find coordinates of text items in PDF
Returns:
Dict mapping page_num -> list of coordinate dicts
"""
coordinates = {}
with pdfplumber.open(self.pdf_path) as pdf:
for page_num, page in enumerate(pdf.pages):
page_coords = []
# Extract words with bounding boxes
words = page.extract_words(x_tolerance=3, y_tolerance=3)
# Search for each item
for search_text in search_texts:
# Normalize text for comparison
search_normalized = search_text.lower().strip()
# Find matching words
matches = []
for word in words:
word_text = word['text'].lower()
if search_normalized in word_text or word_text in search_normalized:
matches.append(word)
# Group consecutive words into phrases
if matches:
# Get bounding box for entire phrase
x0 = min(w['x0'] for w in matches)
x1 = max(w['x1'] for w in matches)
y0 = min(w['top'] for w in matches)
y1 = max(w['bottom'] for w in matches)
# Convert to PDF coordinates (bottom-left origin)
page_height = page.height
coord = {
'x0': x0,
'y0': page_height - y1, # Flip Y coordinate
'x1': x1,
'y1': page_height - y0,
'text': search_text
}
page_coords.append(coord)
logger.debug(f"Found '{search_text}' on page {page_num} at {coord}")
if page_coords:
coordinates[page_num] = page_coords
return coordinates
def _add_highlight_annotation(self, page_num: int, coordinates: Dict):
"""
Add highlight annotation to page
Args:
page_num: Page index
coordinates: Dict with x0, y0, x1, y1 coordinates
"""
page = self.writer.pages[page_num]
# Create highlight annotation
highlight = DictionaryObject()
highlight.update({
NameObject("/Type"): NameObject("/Annot"),
NameObject("/Subtype"): NameObject("/Highlight"),
NameObject("/Rect"): ArrayObject([
FloatObject(coordinates['x0'] - 2), # Add padding
FloatObject(coordinates['y0'] - 2),
FloatObject(coordinates['x1'] + 2),
FloatObject(coordinates['y1'] + 2)
]),
NameObject("/C"): ArrayObject([
FloatObject(1.0), # Red
FloatObject(1.0), # Green
FloatObject(0.0) # Blue -> Yellow
]),
NameObject("/CA"): FloatObject(0.5), # 50% transparency
NameObject("/T"): "Kitchen Invoice Flash", # Author
NameObject("/Contents"): "Non-stock item - requires product mapping"
})
# Add to page annotations
if "/Annots" in page:
page[NameObject("/Annots")].append(highlight)
else:
page[NameObject("/Annots")] = ArrayObject([highlight])
def highlight_non_stock_items_in_pdf(
original_pdf_path: str,
non_stock_items: List[Dict],
output_path: str
) -> str:
"""
Convenience function to highlight non-stock items in invoice PDF
Args:
original_pdf_path: Path to original PDF
non_stock_items: List of dicts with 'description' key
output_path: Path to save annotated PDF
Returns:
Path to annotated PDF (or original if highlighting fails)
Example:
non_stock_items = [
{"description": "Garden Peas, Frozen", "quantity": 2.5},
{"description": "Tomato Sauce, Organic", "quantity": 1.0}
]
highlighted_path = highlight_non_stock_items_in_pdf(
"/app/pdfs/invoice_123.pdf",
non_stock_items,
"/app/pdfs/invoice_123_highlighted.pdf"
)
"""
if not non_stock_items:
logger.info("No non-stock items to highlight")
return original_pdf_path
try:
highlighter = PDFHighlighter(original_pdf_path)
# Extract item descriptions
items_to_highlight = [item['description'] for item in non_stock_items]
# Create highlighted PDF
return highlighter.highlight_text_items(items_to_highlight, output_path)
except Exception as e:
logger.error(f"Failed to create highlighted PDF: {e}", exc_info=True)
return original_pdf_path
```
### Phase 2: Integration with Dext Processing
#### 2.1 Update Dext Service
**File:** `backend/services/dext.py` (UPDATE)
Modify invoice processing to generate highlighted PDF:
```python
from services.pdf_highlighter import highlight_non_stock_items_in_pdf
async def process_invoice_with_highlighting(invoice_data: dict, pdf_path: str) -> dict:
"""
Process invoice and create highlighted PDF if non-stock items found
Returns:
dict with processing results including highlighted_pdf_path
"""
# Existing invoice processing logic...
line_items = extract_line_items(invoice_data)
# Identify non-stock items
non_stock_items = []
for item in line_items:
product = await find_matching_product(item['description'], db)
if not product:
non_stock_items.append({
'description': item['description'],
'quantity': item['quantity'],
'total': item['total']
})
# Create highlighted PDF if non-stock items exist
highlighted_pdf_path = pdf_path # Default to original
if non_stock_items:
output_path = pdf_path.replace('.pdf', '_highlighted.pdf')
highlighted_pdf_path = highlight_non_stock_items_in_pdf(
original_pdf_path=pdf_path,
non_stock_items=non_stock_items,
output_path=output_path
)
logger.info(f"Created highlighted PDF with {len(non_stock_items)} items marked")
return {
'line_items': line_items,
'non_stock_items': non_stock_items,
'original_pdf_path': pdf_path,
'highlighted_pdf_path': highlighted_pdf_path,
'has_highlights': highlighted_pdf_path != pdf_path
}
```
### Phase 3: Update Email Service
#### 3.1 Attach Highlighted PDF to Email
**File:** `backend/services/email_service.py` (UPDATE)
```python
import os
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from email.mime.application import MIMEApplication
async def send_non_stock_items_alert(
recipient_email: str,
invoice_number: str,
supplier_name: str,
non_stock_items: List[Dict],
highlighted_pdf_path: str,
nextcloud_link: str
):
"""
Send email alert with highlighted PDF attached
Args:
recipient_email: Recipient email address
invoice_number: Invoice number
supplier_name: Supplier name
non_stock_items: List of non-stock items
highlighted_pdf_path: Path to PDF with highlights
nextcloud_link: Link to PDF in Nextcloud
"""
# Create multipart message
msg = MIMEMultipart()
msg['From'] = os.getenv('EMAIL_FROM', 'noreply@kitchen-invoice-flash.com')
msg['To'] = recipient_email
msg['Subject'] = f"⚠️ Invoice #{invoice_number} - Non-Stock Items Highlighted"
# Email body (HTML for better formatting)
html_body = f"""
<html>
<body style="font-family: Arial, sans-serif; line-height: 1.6;">
<h2 style="color: #d97706;">⚠️ Invoice Requires Attention</h2>
<p>
Invoice <strong>#{invoice_number}</strong> from <strong>{supplier_name}</strong>
contains <strong>{len(non_stock_items)}</strong> non-stock item(s).
</p>
<div style="background: #fef3c7; padding: 15px; border-left: 4px solid #f59e0b; margin: 20px 0;">
<p style="margin: 0; font-weight: bold; color: #92400e;">
📄 The attached PDF has been annotated with <span style="background: yellow; padding: 2px 6px;">yellow highlights</span>
to help you quickly locate these items.
</p>
</div>
<h3>Non-Stock Items:</h3>
<ul>
{"".join(f'<li><strong>{item["description"]}</strong> - {item["quantity"]} × €{item.get("unit_price", 0):.2f} = €{item["total"]:.2f}</li>' for item in non_stock_items)}
</ul>
<h3>Next Steps:</h3>
<ol>
<li>Open the attached PDF (highlights visible in any PDF viewer)</li>
<li>Review each highlighted item</li>
<li>Add new products to catalog OR link to existing products</li>
<li>Re-process invoice once products are mapped</li>
</ol>
<p style="margin-top: 30px;">
<a href="{nextcloud_link}" style="background: #667eea; color: white; padding: 10px 20px; text-decoration: none; border-radius: 5px;">
View Invoice in Nextcloud
</a>
</p>
<hr style="margin-top: 30px; border: none; border-top: 1px solid #e5e7eb;">
<p style="font-size: 0.9em; color: #6b7280;">
Generated by Kitchen Invoice Flash<br>
<em>Tip: Use Ctrl+F in the PDF to search for highlighted items if you have many pages.</em>
</p>
</body>
</html>
"""
msg.attach(MIMEText(html_body, 'html'))
# Attach highlighted PDF
if os.path.exists(highlighted_pdf_path):
with open(highlighted_pdf_path, 'rb') as pdf_file:
pdf_attachment = MIMEApplication(pdf_file.read(), _subtype="pdf")
pdf_attachment.add_header(
'Content-Disposition',
'attachment',
filename=f'invoice_{invoice_number}_highlighted.pdf'
)
msg.attach(pdf_attachment)
logger.info(f"Attached highlighted PDF: {highlighted_pdf_path}")
else:
logger.warning(f"Highlighted PDF not found: {highlighted_pdf_path}")
# Send email
await send_email(msg)
```
### Phase 4: Testing & Validation
#### 4.1 Unit Tests
**File:** `backend/tests/test_pdf_highlighter.py` (NEW)
```python
import pytest
from services.pdf_highlighter import PDFHighlighter, highlight_non_stock_items_in_pdf
import pdfplumber
def test_highlight_single_item(sample_pdf_path):
"""Test highlighting a single item"""
output_path = "/tmp/test_highlighted.pdf"
highlighter = PDFHighlighter(sample_pdf_path)
result_path = highlighter.highlight_text_items(
items_to_highlight=["Garden Peas"],
output_path=output_path
)
assert os.path.exists(result_path)
# Verify annotation exists
reader = PdfReader(result_path)
page = reader.pages[0]
assert "/Annots" in page
assert len(page["/Annots"]) > 0
# Check annotation type
annot = page["/Annots"][0].get_object()
assert annot["/Subtype"] == "/Highlight"
def test_highlight_multiple_items_across_pages(multi_page_pdf):
"""Test highlighting items on different pages"""
items = [
"Item on Page 1",
"Item on Page 2",
"Item on Page 3"
]
output_path = "/tmp/test_multi_page.pdf"
highlighter = PDFHighlighter(multi_page_pdf)
result_path = highlighter.highlight_text_items(items, output_path)
reader = PdfReader(result_path)
# Verify each page has annotations
for page_num in range(3):
page = reader.pages[page_num]
assert "/Annots" in page
def test_highlight_non_existent_text(sample_pdf_path):
"""Test highlighting text that doesn't exist in PDF"""
output_path = "/tmp/test_no_match.pdf"
highlighter = PDFHighlighter(sample_pdf_path)
result_path = highlighter.highlight_text_items(
items_to_highlight=["NonExistentItem12345"],
output_path=output_path
)
# Should still create output, just without highlights
assert os.path.exists(result_path)
def test_highlight_with_special_characters(sample_pdf_path):
"""Test highlighting items with special characters"""
items = [
"Tomato Sauce, Organic (1L)",
"Cheese - Parmesan, Grated"
]
output_path = "/tmp/test_special_chars.pdf"
result_path = highlight_non_stock_items_in_pdf(
original_pdf_path=sample_pdf_path,
non_stock_items=[
{"description": item, "quantity": 1.0}
for item in items
],
output_path=output_path
)
assert os.path.exists(result_path)
def test_highlight_fallback_on_error(corrupted_pdf_path):
"""Test that errors fall back to original PDF"""
output_path = "/tmp/test_fallback.pdf"
result_path = highlight_non_stock_items_in_pdf(
original_pdf_path=corrupted_pdf_path,
non_stock_items=[{"description": "Item", "quantity": 1}],
output_path=output_path
)
# Should return original path if highlighting fails
assert result_path == corrupted_pdf_path
```
#### 4.2 Integration Test
**Manual test procedure:**
1. **Prepare Test Invoice**
- Get sample invoice PDF with 10+ line items
- Identify 3-5 items to mark as "non-stock"
2. **Trigger Processing**
```bash
# Upload invoice via Dext
# Or manually trigger processing
curl -X POST http://localhost:8000/api/invoices/process \
-H "Authorization: Bearer $TOKEN" \
-d '{"invoice_id": 123}'
```
3. **Verify Highlighted PDF**
- Download highlighted PDF from email attachment
- Open in Adobe Acrobat, Preview, or Chrome
- Verify yellow highlights visible on non-stock items
- Check highlights are positioned correctly (not covering text)
4. **Test in Multiple PDF Viewers**
- Adobe Acrobat Reader
- macOS Preview
- Google Chrome (built-in PDF viewer)
- Firefox PDF viewer
- Mobile PDF viewers (iOS, Android)
5. **Test Edge Cases**
- Very long product names (>100 characters)
- Multi-line item descriptions
- Items with special characters (é, ñ, ü)
- Scanned PDFs (may not work - expected behavior)
### Phase 5: Error Handling & Fallbacks
#### 5.1 Graceful Degradation
**Scenarios where highlighting might fail:**
1. **Scanned PDF (Image-based)**
- No extractable text
- Solution: OCR first, or skip highlighting
2. **Encrypted/Protected PDF**
- Cannot modify
- Solution: Use original PDF, note in email
3. **Corrupted PDF**
- pdfplumber fails
- Solution: Fallback to original
4. **Text Not Found**
- Item description doesn't exactly match PDF text
- Solution: Try fuzzy matching, or skip specific item
**Implementation:**
```python
def highlight_with_fallback(pdf_path: str, items: List[str], output_path: str) -> Tuple[str, List[str]]:
"""
Highlight PDF with robust error handling
Returns:
(path_to_pdf, list_of_errors)
"""
errors = []
try:
# Attempt highlighting
result = highlight_non_stock_items_in_pdf(pdf_path, items, output_path)
if result == pdf_path:
errors.append("Highlighting failed, using original PDF")
return result, errors
except Exception as e:
logger.error(f"Highlighting error: {e}")
errors.append(f"Could not highlight PDF: {str(e)}")
return pdf_path, errors
```
### Phase 6: UI Enhancements
#### 6.1 Show Highlight Status in Invoice List
**File:** `frontend/src/pages/Invoices.tsx` (UPDATE)
Add indicator for invoices with highlights:
```typescript
{invoice.has_highlighted_pdf && (
<span style={styles.highlightBadge} title="PDF contains highlighted non-stock items">
🟡 Highlighted
</span>
)}
```
#### 6.2 Download Both Versions
Provide option to download original and highlighted versions:
```typescript
<div style={styles.downloadButtons}>
<a href={invoice.pdf_url} download>
📄 Download Original PDF
</a>
{invoice.highlighted_pdf_url && (
<a href={invoice.highlighted_pdf_url} download>
🟡 Download Highlighted PDF
</a>
)}
</div>
```
## Success Criteria
✅ Non-stock items highlighted in yellow in PDF
✅ Highlights visible in all major PDF viewers
✅ Email attaches highlighted PDF (not just link)
✅ Original PDF preserved (both versions available)
✅ Fallback to original PDF if highlighting fails
✅ Highlights positioned accurately on item descriptions
✅ Works with multi-page invoices
✅ Email explains highlights feature
✅ Processing time <5 seconds for highlighting
## Testing Checklist
- [ ] Single item highlighted on page 1
- [ ] Multiple items highlighted across multiple pages
- [ ] Very long product names (>80 chars)
- [ ] Special characters in item names (é, ñ, £, €)
- [ ] 50+ item invoice with 10 non-stock items
- [ ] Scanned PDF (should gracefully fail)
- [ ] PDF with complex layout (tables, multi-column)
- [ ] Email received with attachment
- [ ] Highlights visible in Adobe Acrobat
- [ ] Highlights visible in Chrome PDF viewer
- [ ] Highlights visible on mobile (iOS/Android)
## Performance Considerations
**Expected Processing Times:**
- Text extraction: 0.5-2 seconds per page
- Annotation creation: 0.1 seconds per item
- PDF writing: 0.5-1 second
**Total:** 2-5 seconds for typical invoice (3 pages, 5 non-stock items)
**Optimization:**
- Cache text extraction results
- Batch process annotations
- Async processing (don't block email)
## Limitations & Future Enhancements
### Current Limitations
1. **Text-Based PDFs Only**
- Scanned PDFs without OCR won't work
- Need OCR preprocessing for image-based invoices
2. **Exact Text Matching**
- Item description must match PDF text closely
- Punctuation/spacing differences may cause misses
3. **Fixed Highlight Color**
- Always yellow, not customizable
### Future Enhancements
1. **OCR Integration**
- Use Tesseract to OCR scanned PDFs
- Enable highlighting on image-based documents
2. **Smart Text Matching**
- Fuzzy matching with fuzzywuzzy
- Handle abbreviations and variations
3. **Customizable Highlights**
- Color coding by category (red=critical, yellow=review, green=optional)
- Different styles for different issue types
4. **Interactive PDF**
- Clickable highlights link to product catalog search
- Embedded forms for quick product mapping
5. **AI-Powered Annotation**
- Use LLM to suggest product matches
- Add sticky notes with mapping recommendations
6. **Highlight Dashboard**
- Track highlight accuracy
- Learn from user corrections
- Improve matching algorithm over time

1006
docs/archive/PLAN_7_invoice_dispute_tracking.md Normal file
View file

File diff suppressed because it is too large Load diff

424
docs/archive/PURCHASE-ORDER-PLAN.md Normal file
View file

@ -0,0 +1,424 @@
# Purchase Order System
## Context
Users want to pre-allocate budget for known upcoming orders on the Spend Budget page. Clicking on a budget table cell (supplier × date) opens a PO creation modal. POs appear on the budget table in blue/italic (distinct from green invoices). When a real invoice arrives, it can be linked to the PO, which replaces the PO value in the budget. This gives visibility into planned spend before invoices arrive.
## Phased Plan
### Phase 1: Core PO System (DB, API, Modal, Budget Integration, List Page)
### Phase 2: Supplier & Kitchen Settings (order_email, account_number, kitchen details)
### Phase 3: Preview & Email (print view, Save & Email using existing SMTP)
### Phase 4: Invoice Matching (auto-suggest, banner, linking)
---
## Phase 1: Core PO System
### 1.1 Database Tables
**`purchase_orders`** table:
```sql
CREATE TABLE IF NOT EXISTS purchase_orders (
id SERIAL PRIMARY KEY,
kitchen_id INTEGER NOT NULL REFERENCES kitchens(id),
supplier_id INTEGER NOT NULL REFERENCES suppliers(id),
order_date DATE NOT NULL, -- budget date this PO sits on
order_type VARCHAR(20) NOT NULL, -- 'itemised' or 'single_value'
status VARCHAR(20) NOT NULL DEFAULT 'DRAFT', -- DRAFT, PENDING, LINKED, CLOSED, CANCELLED
total_amount NUMERIC(12,2), -- for single_value orders
order_reference VARCHAR(200), -- external order number
notes TEXT,
attachment_path VARCHAR(500), -- uploaded photo/file
attachment_original_name VARCHAR(255),
linked_invoice_id INTEGER REFERENCES invoices(id) ON DELETE SET NULL,
created_by INTEGER NOT NULL REFERENCES users(id),
created_at TIMESTAMP DEFAULT NOW(),
updated_by INTEGER REFERENCES users(id),
updated_at TIMESTAMP DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_po_kitchen_date ON purchase_orders(kitchen_id, order_date);
CREATE INDEX IF NOT EXISTS idx_po_kitchen_supplier ON purchase_orders(kitchen_id, supplier_id);
CREATE INDEX IF NOT EXISTS idx_po_kitchen_status ON purchase_orders(kitchen_id, status);
```
**`purchase_order_line_items`** table:
```sql
CREATE TABLE IF NOT EXISTS purchase_order_line_items (
id SERIAL PRIMARY KEY,
purchase_order_id INTEGER NOT NULL REFERENCES purchase_orders(id) ON DELETE CASCADE,
kitchen_id INTEGER NOT NULL REFERENCES kitchens(id),
product_id INTEGER, -- nullable for manual entries
product_code VARCHAR(100),
description VARCHAR(500) NOT NULL,
unit VARCHAR(50),
unit_price NUMERIC(12,4) NOT NULL,
quantity NUMERIC(10,3) NOT NULL,
total NUMERIC(12,2) NOT NULL,
line_number INTEGER DEFAULT 0,
source VARCHAR(20) DEFAULT 'manual', -- 'search' or 'manual'
created_at TIMESTAMP DEFAULT NOW()
);
```
### 1.2 Files to Create
| File | Purpose |
|------|---------|
| `backend/models/purchase_order.py` | SQLAlchemy models: PurchaseOrder + PurchaseOrderLineItem |
| `backend/migrations/add_purchase_orders.py` | Migration creating both tables + indexes |
| `backend/api/purchase_orders.py` | Full CRUD API router |
| `frontend/src/components/PurchaseOrderModal.tsx` | Create/edit PO modal |
| `frontend/src/components/PurchaseOrderList.tsx` | PO list page with filters |
### 1.3 Files to Modify
| File | Changes |
|------|---------|
| `backend/models/__init__.py` | Import + register PurchaseOrder, PurchaseOrderLineItem |
| `backend/models/supplier.py` | Add `purchase_orders` relationship |
| `backend/main.py` | Register router + migration |
| `backend/api/budget.py` | Include POs in SupplierBudgetRow, add `purchase_orders_by_date` |
| `frontend/src/App.tsx` | Add route `/purchase-orders` + nav item in Invoices dropdown |
| `frontend/src/components/Budget.tsx` | Render POs in cells, add cell click → PO modal, PO styles |
### 1.4 API Endpoints
| Method | Path | Description |
|--------|------|-------------|
| `POST` | `/api/purchase-orders/` | Create PO (with line items) |
| `GET` | `/api/purchase-orders/` | List POs (query: status, supplier_id, date_from, date_to, limit, offset) |
| `GET` | `/api/purchase-orders/{po_id}` | Get PO detail with line items |
| `PUT` | `/api/purchase-orders/{po_id}` | Update PO (full replacement of line items) |
| `DELETE` | `/api/purchase-orders/{po_id}` | Delete PO (only DRAFT/CANCELLED) |
| `PUT` | `/api/purchase-orders/{po_id}/status` | Update status only (close, cancel) |
| `POST` | `/api/purchase-orders/{po_id}/attachment` | Upload attachment (multipart) |
| `DELETE` | `/api/purchase-orders/{po_id}/attachment` | Remove attachment |
| `GET` | `/api/purchase-orders/products/search` | Search products filtered by supplier_id |
| `GET` | `/api/purchase-orders/by-date` | POs for budget table (week_start, week_end) → `{supplier_id: {date: [PO]}}` |
### 1.5 PO Modal Structure (PurchaseOrderModal.tsx)
Follows WastageLogbook `CreateEntryModal` pattern (same modal overlay, header, CSS-in-JS):
```
┌──────────────────────────────────────────────────┐
│ Purchase Order [× Close]│
├──────────────────────────────────────────────────┤
│ Date: [2026-02-12] │ Notes: [optional textarea] │
│ Supplier: [name ▾] │ │
├──────────────────────────────────────────────────┤
│ [Itemised Order] | [Single Value] ← tab btns │
├──────────────────────────────────────────────────┤
│ IF Itemised: │
│ Line Items (3) │
│ ┌──────────────┬───────┬─────┬────────┬──┐ │
│ │ Product │ Price │ Qty │ Total │ ×│ │
│ ├──────────────┼───────┼─────┼────────┼──┤ │
│ │ Chicken 1kg │ 5.50 │ 10 │ 55.00 │ ×│ │
│ │ [manual] │ [inp] │[inp]│ [calc] │ ×│ │
│ └──────────────┴───────┴─────┴────────┴──┘ │
│ [+ Add Manual Item] │
│ │
│ Search Products (filtered to supplier): │
│ [🔍 Search by name or code...] │
│ ┌──────┬────────────┬──────┬───────┬─────┐ │
│ │ Code │ Product │ Unit │ Price │ Add │ │
│ └──────┴────────────┴──────┴───────┴─────┘ │
│ │
│ IF Single Value: │
│ Order Value: [£ ___.__] │
│ Order Ref: [optional] │
│ Attachment: [Upload] or [preview / remove] │
│ │
│ Total: £XX.XX │
├──────────────────────────────────────────────────┤
│ [Cancel] [Save Draft] [Save & Submit]│
└──────────────────────────────────────────────────┘
```
Key behaviors:
- Opened from budget cell click: supplier_id + order_date pre-populated
- Opened from PO list or budget PO button: loads existing PO for editing
- Product search filtered by supplier via `/api/purchase-orders/products/search?query=X&supplier_id=Y`
- Unit Price column before Qty (as requested)
- Auto-calc: `total = unit_price × quantity`
- Search result items show product_code when defined
- File upload via FormData to `/api/purchase-orders/{id}/attachment`
### 1.6 Budget Table Integration
**Backend** (`budget.py`): Add to `get_weekly_budget()`:
- Query `purchase_orders` where kitchen_id matches, status IN ('DRAFT','PENDING'), order_date in week range
- Group by supplier_id + order_date
- Add `purchase_orders_by_date` field to `SupplierBudgetRow`
- PO totals are shown visually but **not** added to `actual_spent` (they're planned, not actual)
- Ensure suppliers with POs but no invoices still appear in the table
**Frontend** (`Budget.tsx`): In the supplier row cell rendering (lines 952-981):
- After rendering invoices, also render POs from `supplier.purchase_orders_by_date[d]`
- PO buttons styled differently: blue text, dashed border, italic, "PO" suffix
- Empty future cells become clickable → open PO modal with that supplier+date
- Clicking existing PO button → open PO modal in edit mode
**PO button style** (distinct from invoiceBtn):
```typescript
poBtn: {
padding: '0.25rem 0.5rem',
background: '#e3f2fd',
border: '1px dashed #42a5f5',
borderRadius: '4px',
cursor: 'pointer',
fontSize: '0.8rem',
fontStyle: 'italic',
color: '#1565c0',
whiteSpace: 'nowrap',
}
```
**Invoice buttons changed to green** (user requested invoices = green, POs = blue):
```typescript
invoiceBtn: {
...existing,
background: '#d4edda', // was #e3f2fd (blue)
border: '1px solid #28a745', // was #90caf9
color: '#155724',
}
```
### 1.6b Budget Table Columns Update
Current columns: Supplier | days... | Budget | Spent | Remaining | Status
New columns: Supplier | days... | Budget | Spent | **Ordered** | Remaining | Status
- **Spent** = actual invoices only (unchanged)
- **Ordered** (NEW) = sum of pending PO totals (DRAFT + PENDING) for this supplier this week
- **Remaining** = Budget − Spent − **Ordered** (POs count as committed spend)
- Remove "OVER" text label from remaining column — red negative value is clear enough
- Status badge logic unchanged (uses remaining value which now factors in POs)
Backend `SupplierBudgetRow` additions:
```python
po_ordered: Decimal # sum of PO totals for this supplier this week (DRAFT + PENDING)
# remaining recalculated: allocated_budget - actual_spent - po_ordered
```
### 1.7 PO List Page (PurchaseOrderList.tsx)
Route: `/purchase-orders` (added to Invoices dropdown after "Disputes")
Layout similar to Disputes.tsx:
- Header: "Purchase Orders" + [+ New PO] button
- Filter bar: Status tabs (All | Draft | Pending | Linked | Closed), Supplier dropdown, Date range
- Default filter: Draft + Pending (open POs)
- Table: Date | Supplier | Type | Reference | Total | Status | Created
- Status badges: DRAFT=grey, PENDING=blue, LINKED=green, CLOSED=dark grey, CANCELLED=red
- Row click → open PO in edit modal
### 1.8 Verification (Phase 1)
1. Click empty future cell on budget table → PO modal opens with correct supplier+date
2. Create itemised PO with search items + manual items → appears on budget table in blue/italic/dashed
3. Create single-value PO with attachment → appears correctly
4. Click PO on budget table → edit modal opens with all data
5. PO list page shows all POs with working status filters
6. Edit PO, change line items → total recalculates
7. Delete PO (DRAFT only) → disappears from budget + list
8. Close PO → status changes, no longer on budget table, removed from "Ordered"
9. "Ordered" column shows PO totals; "Remaining" = Budget − Spent − Ordered
10. Invoice buttons now green, PO buttons blue/dashed/italic
11. No "OVER" text on remaining column — red negative value is sufficient
---
## Phase 2: Supplier & Kitchen Settings
### 2.1 Supplier Model Additions
Add to `backend/models/supplier.py`:
```python
order_email: Mapped[str | None] = mapped_column(String(255), nullable=True)
account_number: Mapped[str | None] = mapped_column(String(100), nullable=True)
```
**Migration**: `backend/migrations/add_supplier_po_fields.py`
- `ALTER TABLE suppliers ADD COLUMN order_email VARCHAR(255)`
- `ALTER TABLE suppliers ADD COLUMN account_number VARCHAR(100)`
**Modify**: `backend/api/suppliers.py` - add fields to create/update schemas
**Modify**: `frontend/src/components/Suppliers.tsx` - add form fields for order_email + account_number
### 2.2 Kitchen Details Settings Tab
Add to `backend/models/settings.py`:
```python
kitchen_display_name: Mapped[str | None] = mapped_column(String(255), nullable=True)
kitchen_address_line1: Mapped[str | None] = mapped_column(String(255), nullable=True)
kitchen_address_line2: Mapped[str | None] = mapped_column(String(255), nullable=True)
kitchen_city: Mapped[str | None] = mapped_column(String(100), nullable=True)
kitchen_postcode: Mapped[str | None] = mapped_column(String(20), nullable=True)
kitchen_phone: Mapped[str | None] = mapped_column(String(50), nullable=True)
kitchen_email: Mapped[str | None] = mapped_column(String(255), nullable=True)
```
**Migration**: `backend/migrations/add_kitchen_details.py`
**Modify**: `backend/api/settings.py` - add GET/PUT for kitchen details
**Modify**: `frontend/src/pages/Settings.tsx` - new "Kitchen Details" tab with form fields
### 2.3 SMTP Already Exists
SMTP settings already in `KitchenSettings` (lines 96-102). Email service at `backend/services/email_service.py` with `send_email()`. Settings UI already exposes SMTP fields. Test endpoint at `POST /api/settings/test-smtp`. **No new work needed for SMTP infrastructure.**
### 2.4 Verification (Phase 2)
1. Add order_email + account_number to a supplier → verify saved/displayed
2. Fill in Kitchen Details in Settings → verify persisted
3. Verify SMTP test still works from Settings
---
## Phase 3: Preview & Email
### 3.1 PO Preview/Print View
**Add endpoint**: `GET /api/purchase-orders/{po_id}/preview`
- Returns clean HTML page with:
- Kitchen letterhead (name, address, phone, email from KitchenSettings)
- "PURCHASE ORDER" title + PO number (PO-{id})
- Date, Supplier name, Supplier account number
- Items table (Code | Description | Unit | Price | Qty | Total) or single value
- Total
- Notes
- Print-friendly CSS (`@media print` styles)
**Frontend**: Add buttons to PO modal footer:
- "Save & Preview" → saves PO, opens `/api/purchase-orders/{id}/preview` in new tab
- "Preview" (when no unsaved changes) → opens preview directly
### 3.2 PO Email Sending
**Add endpoint**: `POST /api/purchase-orders/{po_id}/send-email`
- Loads PO + supplier → checks supplier.order_email exists
- Checks SMTP configured in settings
- Generates PO HTML (reuse preview template)
- Sends via existing `EmailService.send_email()` from `backend/services/email_service.py`
- Updates PO status to PENDING if currently DRAFT
**Frontend**: Add "Save & Email" button to PO modal (shown only when supplier has order_email AND SMTP configured)
- Saves PO, calls send-email endpoint, shows success/error message
### 3.3 Verification (Phase 3)
1. Click "Save & Preview" → new tab with clean formatted PO
2. Print PO from preview → verify layout
3. Set supplier order_email + SMTP config → "Save & Email" button appears
4. Send PO email → verify received with correct content
5. PO status changes to PENDING after email sent
---
## Phase 4: Invoice Matching
### 4.1 PO Matching Service
**File**: `backend/services/po_matching.py` (NEW)
```python
class POMatchingService:
async def find_matching_pos(db, kitchen_id, supplier_id, invoice_date=None):
"""Find pending POs for supplier, ordered by date proximity"""
# status IN ('DRAFT', 'PENDING'), order_date within ±7 days
def calculate_match_confidence(po, invoice) -> float:
"""Score 0-1: supplier match (+0.4), date proximity (+0.3), amount similarity (+0.3)"""
async def link_po_to_invoice(db, po_id, invoice_id, user_id):
"""Set PO status=LINKED, linked_invoice_id=invoice_id"""
async def unlink_po(db, po_id, user_id):
"""Reset PO status=PENDING, linked_invoice_id=None"""
```
### 4.2 API Endpoints
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/purchase-orders/matching?invoice_id={id}` | Find POs matching an invoice |
| `POST` | `/api/purchase-orders/{po_id}/link` | Link PO to invoice `{invoice_id}` |
| `POST` | `/api/purchase-orders/{po_id}/unlink` | Unlink PO from invoice |
### 4.3 Invoice Detail — PO Banner & Linked PO Display
**Modify**: `frontend/src/components/Review.tsx`
**A) Already-linked PO indicator** (shown when invoice has a linked PO):
- At top of invoice detail, show a compact info bar: "Linked to PO-42 (£125.00, Mon 10 Feb)" with a [View PO] button
- Clicking [View PO] opens the PurchaseOrderModal in read/edit mode for that PO
- Also show [Unlink] button to remove the link (returns PO to PENDING)
**B) Matching PO banner** (shown when invoice has NO linked PO but supplier has pending POs):
- Fetch matching POs via `/api/purchase-orders/matching?invoice_id={id}`
- Show info banner: "This supplier has N pending Purchase Orders"
- List each PO: `PO-42: £125.00 (Mon 10 Feb) [Link to this Invoice]`
- High-confidence matches (>0.8) highlighted with "Suggested match" label
- "Link" button calls POST link endpoint → PO becomes LINKED, banner switches to linked indicator (A)
### 4.4 Budget Table Behavior When Linked
When PO status = LINKED:
- PO **no longer** appears in `purchase_orders_by_date` on budget table
- The linked invoice naturally appears in `invoices_by_date` (it's a real invoice)
- Budget transitions seamlessly from showing planned PO → actual invoice
### 4.5 Verification (Phase 4)
1. Create PO for supplier + date, then upload invoice from same supplier
2. Invoice review page shows matching PO banner with pending POs
3. Link PO → status changes to LINKED, disappears from budget, invoice takes its place
4. Invoice review page now shows "Linked to PO-42" indicator with [View PO] button
5. Click [View PO] → PO modal opens with correct data
6. Unlink PO → returns to PENDING, reappears on budget, banner switches back to matching list
7. Auto-suggest works for high-confidence match (same supplier + close date + similar amount)
8. Banner always shows when supplier has any pending POs
---
## Key Existing Code to Reuse
| Existing Code | File | Reuse For |
|---------------|------|-----------|
| Modal overlay + header pattern | `WastageLogbook.tsx:541-997` | PO modal structure |
| Line item builder (add/update/remove) | `WastageLogbook.tsx` state handlers | PO itemised line items |
| Product search (debounced, deduped) | `WastageLogbook.tsx` + `/api/logbook/products/search` | PO product search (add supplier filter) |
| File upload (FormData + UUID naming) | `invoices.py` upload handler | PO attachment upload |
| Email sending (SMTP) | `backend/services/email_service.py` | PO email (Phase 3) |
| Invoice cell rendering on budget | `Budget.tsx:958-975` | PO cell rendering (same pattern, different style) |
| Dispute status badges | `Disputes.tsx` | PO status badges |
| Settings tab pattern | `Settings.tsx` | Kitchen Details tab (Phase 2) |
| Supplier form | `Suppliers.tsx` | Add order_email/account_number fields (Phase 2) |
## Complete File List (All Phases)
### New Files
1. `backend/models/purchase_order.py` — PO + line item models (Phase 1)
2. `backend/migrations/add_purchase_orders.py` — Create tables (Phase 1)
3. `backend/api/purchase_orders.py` — Full PO API (Phase 1, extended Phase 3-4)
4. `frontend/src/components/PurchaseOrderModal.tsx` — Create/edit modal (Phase 1)
5. `frontend/src/components/PurchaseOrderList.tsx` — PO list page (Phase 1)
6. `backend/migrations/add_supplier_po_fields.py` — Supplier order_email + account_number (Phase 2)
7. `backend/migrations/add_kitchen_details.py` — Kitchen detail columns (Phase 2)
8. `backend/services/po_matching.py` — PO-invoice matching service (Phase 4)
### Modified Files
1. `backend/models/__init__.py` — Register PO models (Phase 1)
2. `backend/models/supplier.py` — Add purchase_orders relationship + order_email + account_number (Phase 1+2)
3. `backend/main.py` — Register router + migrations (Phase 1+2)
4. `backend/api/budget.py` — Include POs in budget response (Phase 1)
5. `frontend/src/App.tsx` — Route + nav item (Phase 1)
6. `frontend/src/components/Budget.tsx` — PO cells + click handler + invoice color change (Phase 1)
7. `backend/api/suppliers.py` — Add new fields to schemas (Phase 2)
8. `frontend/src/components/Suppliers.tsx` — Add form fields (Phase 2)
9. `backend/models/settings.py` — Kitchen detail columns (Phase 2)
10. `backend/api/settings.py` — Kitchen details endpoints (Phase 2)
11. `frontend/src/pages/Settings.tsx` — Kitchen Details tab (Phase 2)
12. `frontend/src/components/Review.tsx` — PO matching banner (Phase 4)

900
docs/archive/RECIPE-SYSTEM-PLAN.md Normal file
View file

@ -0,0 +1,900 @@
# Recipe, Ingredient & Food Flag System — Implementation Plan
## Context
The kitchen-invoice-flash app currently tracks invoices, line items, and suppliers with a basic portioning feature (scales icon → cost breakdown modal). This plan introduces a full **Recipe & Ingredient Management System** that:
1. Creates a canonical **ingredient library** with yield tracking, duplicate detection, and multi-supplier price comparison
2. Builds a **hierarchical recipe system** with sub-recipes, batch portions, scaling, cost trending, and printable recipe cards
3. Replaces hardcoded allergens with a **configurable food flag system** — categories with different propagation logic ("contains" for allergens, "suitable_for" for dietary)
4. Adds flag tracking cascading from line items → ingredients → recipes → plated dishes, with audit trails for overrides
5. Introduces **event/function ordering** — select recipes × quantities to generate aggregated shopping lists and purchase orders
6. Provides an **internal API** (API key auth) for in-house apps (e.g., menu display plugin) and **KDS recipe linking** for kitchen display integration
The existing portioning inline expansion (scales icon) becomes an **ingredient-first mapping modal dialog** — pack/unit fields remain but now feed into ingredient unit conversion rather than standalone portioning.
---
## Phase 1: Database Schema & Backend Models
### New Tables
#### `ingredient_categories` — Configurable ingredient groupings
```sql
id SERIAL PK
kitchen_id INT FK → kitchens(id) NOT NULL
name VARCHAR(100) NOT NULL -- "Dairy", "Meat", "Produce", etc.
sort_order INT DEFAULT 0
created_at TIMESTAMP DEFAULT NOW()
UNIQUE(kitchen_id, name)
```
Pre-seeded: Dairy, Meat, Seafood, Produce, Dry Goods, Oils & Fats, Herbs & Spices, Bakery, Beverages, Condiments, Other
#### `ingredients` — Canonical ingredient library
```sql
id SERIAL PK
kitchen_id INT FK → kitchens(id) NOT NULL
name VARCHAR(255) NOT NULL -- "Butter", "Minced Beef 80/20", "Plain Flour"
category_id INT FK → ingredient_categories(id) ON DELETE SET NULL
standard_unit VARCHAR(20) NOT NULL -- "g", "kg", "ml", "ltr", "each"
yield_percent NUMERIC(5,2) DEFAULT 100.00 -- usable % after trim/peel/waste (e.g., 85 for carrots, 65 for whole chicken)
manual_price NUMERIC(12,6) -- placeholder price/std_unit for unmapped ingredients
notes TEXT
is_archived BOOL DEFAULT false
created_by INT FK → users(id)
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
UNIQUE(kitchen_id, name)
```
#### `ingredient_sources` — Maps supplier products → ingredients (with unit conversion)
```sql
id SERIAL PK
kitchen_id INT FK → kitchens(id) NOT NULL
ingredient_id INT FK → ingredients(id) ON DELETE CASCADE
supplier_id INT FK → suppliers(id) NOT NULL
product_code VARCHAR(100) -- matches line_items.product_code (NULL for no-SKU suppliers)
description_pattern VARCHAR(255) -- normalised substring match against line_item descriptions (used when product_code is NULL)
-- Pack/conversion data (persisted like product_definitions)
pack_quantity INT -- e.g., 10 (10 blocks of butter)
unit_size NUMERIC(10,3) -- e.g., 250 (250g each)
unit_size_type VARCHAR(10) -- "g", "kg", "ml", "ltr", "oz", "cl", "each"
-- Price tracking (auto-updated from most recent matched line item)
latest_unit_price NUMERIC(10,2)
latest_invoice_id INT FK → invoices(id) ON DELETE SET NULL
latest_invoice_date DATE
price_per_std_unit NUMERIC(12,6) -- auto-calc: latest_unit_price / total_in_standard_unit
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
-- Dual unique constraints for SKU and non-SKU suppliers
UNIQUE(kitchen_id, ingredient_id, supplier_id, product_code) -- for items WITH product_code
```
```sql
-- Partial unique index for no-SKU items (product_code IS NULL)
CREATE UNIQUE INDEX uix_ingredient_source_desc
ON ingredient_sources(kitchen_id, ingredient_id, supplier_id, description_pattern)
WHERE product_code IS NULL;
```
**Matching priority** (same as existing product_definitions pattern):
1. Try `supplier_id + product_code` exact match first
2. Fall back to `supplier_id + description_pattern` normalised contains-match (for no-SKU suppliers)
3. Longer patterns match before shorter ones (more specific wins)
**Validation rule**: When `product_code` is NULL, `description_pattern` is required (and vice versa — at least one must be set).
#### `food_flag_categories` — Configurable flag category types (Allergy, Dietary, etc.)
```sql
id SERIAL PK
kitchen_id INT FK → kitchens(id) NOT NULL
name VARCHAR(100) NOT NULL -- "Allergy", "Dietary", "Religious", etc.
propagation_type VARCHAR(20) NOT NULL -- "contains" (any-match, union) | "suitable_for" (all-must-match, intersection)
sort_order INT DEFAULT 0
created_at TIMESTAMP DEFAULT NOW()
UNIQUE(kitchen_id, name)
```
Pre-seeded:
- "Allergy" (propagation: "contains") — if ANY child ingredient has it, recipe has it
- "Dietary" (propagation: "suitable_for") — only applies if ALL children qualify
#### `food_flags` — Individual flags within categories
```sql
id SERIAL PK
category_id INT FK → food_flag_categories(id) ON DELETE CASCADE
kitchen_id INT FK → kitchens(id) NOT NULL
name VARCHAR(100) NOT NULL -- "Gluten", "Milk", "Vegetarian", "Vegan", etc.
code VARCHAR(10) -- short code: "Gl", "Mi", "V", "Ve" (for badges)
icon VARCHAR(10) -- optional emoji/symbol
sort_order INT DEFAULT 0
created_at TIMESTAMP DEFAULT NOW()
UNIQUE(kitchen_id, name)
```
Pre-seeded Allergy flags: Celery, Gluten, Crustaceans, Eggs, Fish, Lupin, Milk, Molluscs, Mustard, Tree Nuts, Peanuts, Sesame, Soya, Sulphites
Pre-seeded Dietary flags: Vegetarian, Vegan, Pescatarian, Gluten-Free (dietary, not allergy)
#### `ingredient_flags` — Canonical flag assignments on ingredients (latching)
```sql
id SERIAL PK
ingredient_id INT FK → ingredients(id) ON DELETE CASCADE
food_flag_id INT FK → food_flags(id) ON DELETE CASCADE
flagged_by INT FK → users(id)
source VARCHAR(20) DEFAULT 'manual' -- "manual" | "latched" (auto-set from line_item_flag)
created_at TIMESTAMP DEFAULT NOW()
UNIQUE(ingredient_id, food_flag_id)
```
**Latching behavior**: When a line item is flagged AND that line item is mapped to an ingredient (via `ingredient_id`), the system auto-creates an `ingredient_flag` with `source='latched'`. Flags latch on permanently — they never auto-remove. Only manual deletion by a user can remove an ingredient flag.
This table is the **canonical source of truth** for ingredient-level flags. Recipe flag propagation reads from here, not from line_item_flags.
#### `line_item_flags` — Flags on supplier line items (data entry mechanism)
```sql
id SERIAL PK
line_item_id INT FK → line_items(id) ON DELETE CASCADE
food_flag_id INT FK → food_flags(id) ON DELETE CASCADE
flagged_by INT FK → users(id)
created_at TIMESTAMP DEFAULT NOW()
UNIQUE(line_item_id, food_flag_id)
```
Line item flags serve as a data-entry point. When set, they trigger latching to the mapped ingredient (if `line_item.ingredient_id` is set). The ingredient_flags table holds the persistent truth.
#### `menu_sections` — Groupings for recipes (both plated and component)
```sql
id SERIAL PK
kitchen_id INT FK → kitchens(id) NOT NULL
name VARCHAR(100) NOT NULL -- Plated: "Starters", "Mains", "Desserts". Component: "Sauces", "Bases", "Preparations"
sort_order INT DEFAULT 0
created_at TIMESTAMP DEFAULT NOW()
UNIQUE(kitchen_id, name)
```
Sections work for both recipe types. The recipe list page filters sections by selected type (component/plated). No separate `section_type` needed — a section like "Sauces" naturally only has components assigned to it.
#### `recipes` — Component and plated recipes
```sql
id SERIAL PK
kitchen_id INT FK → kitchens(id) NOT NULL
name VARCHAR(255) NOT NULL
recipe_type VARCHAR(20) NOT NULL -- "component" | "plated"
menu_section_id INT FK → menu_sections(id) ON DELETE SET NULL -- optional grouping for either type
description TEXT
batch_portions INT NOT NULL DEFAULT 1 -- components only: how many portions this batch makes (plated always 1)
prep_time_minutes INT
cook_time_minutes INT
notes TEXT
is_archived BOOL DEFAULT false
kds_menu_item_name VARCHAR(255) -- Phase 7: matches KDS/SambaPOS menu item name for linking
created_by INT FK → users(id)
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
UNIQUE(kitchen_id, name)
```
#### `recipe_ingredients` — Ingredients used in a recipe
```sql
id SERIAL PK
recipe_id INT FK → recipes(id) ON DELETE CASCADE
ingredient_id INT FK → ingredients(id) ON DELETE RESTRICT
quantity NUMERIC(10,3) NOT NULL -- in ingredient's standard_unit
notes TEXT -- "finely diced", "room temperature"
sort_order INT DEFAULT 0
```
#### `recipe_sub_recipes` — Sub-recipes used in a recipe (max 5 levels deep)
```sql
id SERIAL PK
parent_recipe_id INT FK → recipes(id) ON DELETE CASCADE
child_recipe_id INT FK → recipes(id) ON DELETE RESTRICT
portions_needed NUMERIC(10,3) NOT NULL -- how many portions of the child batch we use
notes TEXT
sort_order INT DEFAULT 0
CHECK(parent_recipe_id != child_recipe_id)
```
#### `recipe_steps` — Cooking instructions
```sql
id SERIAL PK
recipe_id INT FK → recipes(id) ON DELETE CASCADE
step_number INT NOT NULL
instruction TEXT NOT NULL
image_path VARCHAR(500) -- optional step photo (local Docker volume)
duration_minutes INT
notes TEXT
```
#### `recipe_images` — General recipe/plating photos
```sql
id SERIAL PK
recipe_id INT FK → recipes(id) ON DELETE CASCADE
image_path VARCHAR(500) NOT NULL -- stored at /app/data/{kitchen_id}/recipes/{uuid}.{ext}
caption TEXT
image_type VARCHAR(20) DEFAULT 'general' -- "general" | "plating" | "method"
sort_order INT DEFAULT 0
uploaded_by INT FK → users(id)
created_at TIMESTAMP DEFAULT NOW()
```
#### `recipe_flags` — Flag state on recipes (manual additions + override state)
```sql
id SERIAL PK
recipe_id INT FK → recipes(id) ON DELETE CASCADE
food_flag_id INT FK → food_flags(id) ON DELETE CASCADE
source_type VARCHAR(20) NOT NULL -- "auto" | "manual"
is_active BOOL DEFAULT true -- false = overridden/deactivated
excludable_on_request BOOL DEFAULT false -- plated only: can prepare without on request
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
UNIQUE(recipe_id, food_flag_id)
```
#### `recipe_flag_overrides` — Audit log for flag changes (mandatory notes)
```sql
id SERIAL PK
recipe_id INT FK → recipes(id) ON DELETE CASCADE
food_flag_id INT FK → food_flags(id) ON DELETE CASCADE
action VARCHAR(20) NOT NULL -- "deactivated" | "reactivated" | "set_excludable" | "unset_excludable"
note TEXT NOT NULL -- mandatory reason
user_id INT FK → users(id)
created_at TIMESTAMP DEFAULT NOW()
```
#### `recipe_change_log` — Recipe change history
```sql
id SERIAL PK
recipe_id INT FK → recipes(id) ON DELETE CASCADE
change_summary TEXT NOT NULL -- "Butter quantity changed from 200g to 250g; Added Oregano 5g"
user_id INT FK → users(id)
created_at TIMESTAMP DEFAULT NOW()
```
#### `recipe_cost_snapshots` — Cost trending over time
```sql
id SERIAL PK
recipe_id INT FK → recipes(id) ON DELETE CASCADE
cost_per_portion NUMERIC(12,6) NOT NULL
total_cost NUMERIC(12,6) NOT NULL
snapshot_date DATE NOT NULL
trigger_source VARCHAR(100) -- "ingredient_price_update: Butter" or "manual_recalc"
created_at TIMESTAMP DEFAULT NOW()
UNIQUE(recipe_id, snapshot_date) -- one snapshot per recipe per day (upsert on conflict)
```
**Upsert behavior**: If a snapshot already exists for today, update it with the latest cost values. Multiple ingredient price changes on the same day result in one snapshot reflecting the final state.
#### `event_orders` — Function/event ordering (select recipes × quantities → generate shopping list)
```sql
id SERIAL PK
kitchen_id INT FK → kitchens(id) NOT NULL
name VARCHAR(255) NOT NULL -- "Wedding Reception 15th March", "Staff Party"
event_date DATE
notes TEXT
status VARCHAR(20) DEFAULT 'DRAFT' -- DRAFT | FINALISED | ORDERED
created_by INT FK → users(id)
created_at TIMESTAMP DEFAULT NOW()
updated_at TIMESTAMP DEFAULT NOW()
```
#### `event_order_items` — Recipes and quantities for an event
```sql
id SERIAL PK
event_order_id INT FK → event_orders(id) ON DELETE CASCADE
recipe_id INT FK → recipes(id) ON DELETE RESTRICT
quantity INT NOT NULL -- how many servings/batches of this recipe
notes TEXT
sort_order INT DEFAULT 0
```
Both plated and component recipes can be added to an event order. Components show batch_portions for context (e.g., "Burger Sauce — batch of 20 portions, qty: 3 batches = 60 portions").
### Existing Table Modifications
#### `line_items` — Add `ingredient_id` column
```sql
ALTER TABLE line_items ADD COLUMN ingredient_id INT REFERENCES ingredients(id) ON DELETE SET NULL;
CREATE INDEX idx_line_items_ingredient ON line_items(ingredient_id);
```
Direct link from line item to its mapped ingredient. Set when a user maps a line item to an ingredient via the ingredient mapping modal. Nullable — only populated for mapped items.
**Benefits**:
- Direct relationship for queries ("show all line items for Butter")
- Enables latching: when a line_item_flag is set, check `ingredient_id` and auto-create ingredient_flag
- Scales icon tooltip can show "→ Butter" without a lookup query
#### `kitchen_settings` — Add API key fields
```sql
ALTER TABLE kitchen_settings ADD COLUMN api_key VARCHAR(100);
ALTER TABLE kitchen_settings ADD COLUMN api_key_enabled BOOL DEFAULT false;
```
Used by external in-house apps (e.g., menu display plugin) to authenticate against the internal API endpoints.
### Database Extensions
#### `pg_trgm` — Trigram similarity for duplicate detection
```sql
CREATE EXTENSION IF NOT EXISTS pg_trgm;
```
Used for fuzzy ingredient name matching during creation. Provides `similarity()` function:
```sql
SELECT name, similarity(name, 'Butter') AS sim
FROM ingredients
WHERE kitchen_id = :kid AND similarity(name, 'Butter') > 0.3
ORDER BY sim DESC LIMIT 5;
```
Returns similar names like "Unsalted Butter" (0.47), "Salted Butter" (0.47) as warnings before creating a potential duplicate.
### Unit Conversion Constants
Standard units: `g`, `kg`, `ml`, `ltr`, `each`. Conversion factors in code:
```python
UNIT_CONVERSIONS = {
"g": {"g": 1, "kg": 0.001},
"kg": {"g": 1000, "kg": 1},
"oz": {"g": 28.3495, "kg": 0.0283495},
"lb": {"g": 453.592, "kg": 0.453592},
"ml": {"ml": 1, "ltr": 0.001},
"cl": {"ml": 10, "ltr": 0.01},
"ltr": {"ml": 1000, "ltr": 1},
"each": {"each": 1},
}
```
### Key Calculation Logic
**Ingredient cost from source:**
```
total_in_std_unit = pack_quantity × unit_size × conversion_factor(unit_size_type → standard_unit)
price_per_std_unit = latest_unit_price / total_in_std_unit
```
**Yield-adjusted effective price** (used in recipes):
```
raw_price = most recent source price_per_std_unit (or manual_price if unmapped)
effective_price = raw_price / (yield_percent / 100)
-- e.g., carrots at £1/kg with 85% yield → £1.18/kg usable
-- whole chicken at £3/kg with 65% yield → £4.62/kg usable
```
**Recipe cost:**
```
ingredient_cost = SUM(recipe_ingredient.quantity × ingredient.effective_price)
sub_recipe_cost = SUM((sub.portions_needed / child.batch_portions) × child.total_cost)
total_cost = ingredient_cost + sub_recipe_cost
cost_per_portion = total_cost / batch_portions
```
**Cost range on recipes:**
```
min_cost = use cheapest source for each ingredient
max_cost = use most expensive source for each ingredient
recent_cost = use most recent purchase for each ingredient (default)
```
**GP calculator on plated recipes:**
```
At target GP%: suggested_price = cost_per_portion / (1 - target_gp)
Show comparison at 60%, 65%, 70% GP targets
```
**Food flag propagation (computed on-read via ingredient_flags):**
```
For each food_flag_category:
if propagation_type == "contains":
Collect flags from ALL recipe_ingredients → ingredient → ingredient_flags
Union with flags from ALL sub-recipes (recursive)
→ Recipe has flag if ANY ingredient has it
if propagation_type == "suitable_for":
For each flag in category:
Check ALL recipe_ingredients → ingredient → ingredient_flags has this flag
AND ALL sub-recipes have this flag (recursive)
→ Recipe has flag only if ALL ingredients have it
→ Ingredients with NO flags assessed for this category count as "unknown" (not a match)
Merge with manual additions (source_type="manual" in recipe_flags)
Apply overrides (is_active=false entries from recipe_flags with audit log)
```
**Flag latching flow:**
```
1. User flags a line_item with "Contains: Milk" via Review.tsx flag button
2. System checks line_item.ingredient_id — if set (e.g., ingredient "Butter"):
a. Auto-create ingredient_flag(ingredient=Butter, flag=Milk, source='latched') if not exists
b. ingredient_flag persists permanently regardless of future line item changes
3. All recipes using "Butter" now auto-inherit "Contains: Milk" via propagation
```
### Files to Create
- `backend/models/ingredient.py` — Ingredient, IngredientCategory, IngredientSource, IngredientFlag
- `backend/models/recipe.py` — Recipe, MenuSection, RecipeIngredient, RecipeSubRecipe, RecipeStep, RecipeImage, RecipeChangeLog, RecipeCostSnapshot
- `backend/models/food_flag.py` — FoodFlagCategory, FoodFlag, LineItemFlag, RecipeFlag, RecipeFlagOverride
- `backend/models/event_order.py` — EventOrder, EventOrderItem
- `backend/api/ingredients.py` — Ingredient CRUD + source mapping + auto-price hook
- `backend/api/recipes.py` — Recipe CRUD + costing + sub-recipe cycle check + scaling + recipe card HTML + menu section CRUD + cost snapshot calculation
- `backend/api/food_flags.py` — Flag management + ingredient flagging + line item flagging + latching logic + recipe flag propagation + overrides
- `backend/api/event_orders.py` — Event ordering + aggregated shopping list generation
- `backend/api/external.py` — Internal API endpoints with API key auth for in-house apps
- `backend/migrations/add_recipe_system.py` — All new tables + pre-seeded data + pg_trgm extension
- `frontend/src/components/Ingredients.tsx` — Ingredient library page
- `frontend/src/components/RecipeList.tsx` — Recipe list page
- `frontend/src/components/RecipeEditor.tsx` — Recipe builder/editor page
- `frontend/src/components/RecipeFlagMatrix.tsx` — Flag breakdown matrix (ingredients × flags)
- `frontend/src/components/FoodFlagBadges.tsx` — Reusable flag badges component
- `frontend/src/components/EventOrders.tsx` — Event ordering page
- `frontend/src/components/EventOrderEditor.tsx` — Event order builder
### Files to Modify
- `backend/models/__init__.py` — Register new models (IngredientFlag added)
- `backend/models/line_item.py` — Add `ingredient_id` FK column
- `backend/main.py` — Register new routers (ingredients, recipes, food_flags, event_orders, external)
- `frontend/src/App.tsx` — Add routes + new **"Recipes" dropdown** in top header nav (matching existing Invoices/Bookings/Reports dropdown pattern) containing: Recipes, Ingredients, Event Orders
- `frontend/src/components/Review.tsx` — Replace inline scales expansion with ingredient mapping modal dialog + add flag button
- `frontend/src/components/Dashboard.tsx` — Add unmapped ingredients widget
- `backend/api/invoices.py` — Auto-price update hook when line items saved + flag latching trigger + cost snapshot trigger
- `frontend/src/pages/Settings.tsx` — Add Food Flag Categories/Flags management section + API Key management section
---
## Phase 2: Foundation — Schema, Models, Ingredient Library
### 2a: Migration & Models
- Create migration `add_recipe_system.py` with all tables above
- Enable `pg_trgm` extension
- Add `ingredient_id` column to `line_items` table
- Add `api_key` + `api_key_enabled` columns to `kitchen_settings` table
- Pre-seed `ingredient_categories` with defaults
- Pre-seed `food_flag_categories` (Allergy/contains, Dietary/suitable_for) and `food_flags` (EU 14 + dietary defaults). Pre-seeded categories and flags are user-editable and deletable — no special protection needed (kitchen can customise to their needs)
- Create all SQLAlchemy models
- Register in `__init__.py` and `main.py`
### 2b: Ingredient Backend
**Endpoints:**
- `GET /api/ingredient-categories` — List categories (for dropdowns)
- `POST /api/ingredient-categories` — Create category
- `PATCH /api/ingredient-categories/{id}` — Rename/reorder
- `DELETE /api/ingredient-categories/{id}` — Delete (sets ingredients to null category)
- `GET /api/ingredients` — List all (with source count, flag summary, effective price)
- `GET /api/ingredients?unmapped=true` — Filter to ingredients with no sources
- `POST /api/ingredients` — Create (name, category_id, standard_unit, yield_percent, optional manual_price). **Duplicate detection**: uses pg_trgm `similarity()` to fuzzy-match name against existing ingredients, returns warnings if similar names found (threshold > 0.3)
- `PATCH /api/ingredients/{id}` — Update (including yield_percent)
- `DELETE /api/ingredients/{id}` — Soft-archive (is_archived=true)
- `GET /api/ingredients/{id}/sources` — List all supplier sources with prices
- `POST /api/ingredients/{id}/sources` — Map a supplier product (requires product_code OR description_pattern)
- `PATCH /api/ingredient-sources/{id}` — Update pack/conversion data
- `DELETE /api/ingredient-sources/{id}` — Remove mapping
- `GET /api/ingredients/{id}/flags` — List ingredient's flags
- `PUT /api/ingredients/{id}/flags` — Set/update ingredient flags (manual)
- `GET /api/ingredients/suggest?description={text}` — Suggest existing ingredient matches for a line item description (uses pg_trgm similarity). Used by the ingredient mapping modal to auto-populate the ingredient dropdown
**Auto-price hook** (in `invoices.py` line item save/update):
- When a line item is saved/updated, get supplier_id via `line_item → invoice → supplier_id` (line_items don't have direct supplier_id)
- **Match priority**: Try `supplier_id + product_code` exact match against ingredient_sources first. If no product_code on the line item (or no match), try `supplier_id + description_pattern` normalised contains-match (lowercase, collapse whitespace, check if pattern is contained in description). Longer patterns match before shorter ones (more specific wins)
- If match found: update latest_unit_price, latest_invoice_id, latest_invoice_date, recalculate price_per_std_unit
- Also set `line_item.ingredient_id` to the matched ingredient (if not already set)
**Flag latching hook** (logic in `food_flags.py`, called from line item flag save endpoints):
- When a `line_item_flag` is created/updated AND `line_item.ingredient_id` is set:
- Auto-create `ingredient_flag` for that ingredient + flag if not already present (source='latched')
- Ingredient flags are permanent — latching only adds, never removes
### 2c: Ingredient Frontend — `/ingredients` page
- Searchable/filterable table of all ingredients
- Columns: name, category, standard unit, yield %, sources count, flags (FoodFlagBadges), effective price/unit (yield-adjusted)
- Expandable row: all sources with supplier name, product code/description pattern, pack info, price/std unit, last invoice date
- "Create Ingredient" modal with **duplicate detection** (on name blur, API call checks pg_trgm similarity, shows warning with similar existing names)
- Filter toggle: "Show unmapped only" (ingredients without any sources)
- Category management: "+Add" button in category filter dropdown (opens inline input)
### 2d: Ingredient Mapping Modal (replaces inline scales expansion in Review.tsx)
- **Keep scales icon** with existing colour scheme: red (no data), amber (partial/parsed), green (fully mapped to ingredient)
- Tooltip updates to show ingredient name + conversion info when mapped (e.g., "→ Butter (250g × 10 = 2.5kg @ £4.20/kg)")
- **Clicking scales icon opens a modal dialog** (replaces the current inline expandable row — the extra fields need more space):
1. **Auto-populate**: Parse line item description to suggest existing ingredient match (via pg_trgm similarity search). Pre-fill pack fields from line item's existing pack_quantity/unit_size/unit_size_type. Load product_definition if exists
2. **Ingredient mapping section**: Searchable dropdown of ingredients. If no match exists, "Create new ingredient" opens inline mini-form (name, category, standard unit) within the modal. Shows current mapping if already mapped
3. **Pack fields**: pack_quantity, unit_size, unit_size_type (editable, auto-filled from OCR/product_definition)
4. **Conversion display**: Shows calculated total_in_standard_unit and price_per_std_unit based on current pack fields + ingredient's standard unit
5. **Save**: Creates/updates the ingredient_source mapping. Sets `line_item.ingredient_id`. Checkbox to "Update saved definition" (existing product_definition behavior preserved)
- Existing `portions_per_unit` / `cost_per_portion` fields remain on line_item model for backward compatibility but are de-emphasised in the UI (shown in a collapsible "Legacy Portioning" section within the modal)
---
## Phase 3: Recipe Builder
### 3a: Recipe Backend
**Menu sections:**
- `GET /api/menu-sections` — List all sections (for dropdowns, filtered by recipe type in frontend)
- `POST /api/menu-sections` — Create section (name)
- `PATCH /api/menu-sections/{id}` — Rename/reorder
- `DELETE /api/menu-sections/{id}` — Delete (sets recipes to null section)
**Recipe endpoints:**
- `GET /api/recipes` — List all (filterable by type, menu section, search by name, flag include/exclude, ingredient contains)
- `GET /api/recipes/{id}` — Full recipe with ingredients, sub-recipes, steps, images, flags, costing
- `POST /api/recipes` — Create
- `PATCH /api/recipes/{id}` — Update metadata (logs change to recipe_change_log)
- `DELETE /api/recipes/{id}` — Soft-archive
- `POST /api/recipes/{id}/duplicate` — Clone recipe with rename prompt. Deep copies: ingredients, steps, images, flags, notes. Sub-recipe references are **linked** (not deep-copied) — the duplicate shares the same component recipes. User prompted to enter new name (pre-filled with "Original Name (Copy)")
**Recipe ingredients:**
- `POST /api/recipes/{id}/ingredients` — Add ingredient (logs change)
- `PATCH /api/recipe-ingredients/{id}` — Update quantity/notes (logs old→new)
- `DELETE /api/recipe-ingredients/{id}` — Remove (logs removal)
**Recipe sub-recipes:**
- `POST /api/recipes/{id}/sub-recipes` — Add (with circular dependency check via recursive CTE, max 5 levels)
- `PATCH /api/recipe-sub-recipes/{id}` — Update portions_needed
- `DELETE /api/recipe-sub-recipes/{id}` — Remove
**Circular dependency check:**
```sql
WITH RECURSIVE ancestors AS (
SELECT parent_recipe_id, child_recipe_id, 1 AS depth
FROM recipe_sub_recipes WHERE child_recipe_id = :new_parent_id
UNION ALL
SELECT rsr.parent_recipe_id, rsr.child_recipe_id, a.depth + 1
FROM recipe_sub_recipes rsr JOIN ancestors a ON rsr.child_recipe_id = a.parent_recipe_id
WHERE a.depth < 5
)
SELECT 1 FROM ancestors WHERE parent_recipe_id = :new_child_id LIMIT 1;
-- If returns a row → would create cycle → reject
```
**Recipe steps:**
- `POST /api/recipes/{id}/steps` — Add step
- `PATCH /api/recipe-steps/{id}` — Update
- `DELETE /api/recipe-steps/{id}` — Remove
- `PATCH /api/recipes/{id}/steps/reorder` — Bulk reorder
**Recipe images:**
- `POST /api/recipes/{id}/images` — Upload (multipart, stored at `/app/data/{kitchen_id}/recipes/{uuid}.{ext}`)
- `GET /api/recipes/{recipe_id}/images/{image_id}` — Serve image (authenticated, same pattern as invoice image endpoint)
- `DELETE /api/recipe-images/{id}` — Remove file + DB record
**Costing:**
- `GET /api/recipes/{id}/costing` — Full cost breakdown:
- Per ingredient: quantity, unit, yield %, effective price (yield-adjusted), all source prices, min/max
- Per sub-recipe: name, batch_portions, portions_needed, cost per portion, cost contribution
- Totals: recent cost, min cost, max cost, cost per portion
- GP calculator: suggested prices at 60%, 65%, 70% GP targets (plated only)
- `GET /api/recipes/{id}/costing?scale_to=50` — Same but with quantities scaled to target portions
- `GET /api/recipes/{id}/cost-trend` — Cost snapshot history for trend chart
- **Cost snapshot trigger**: Snapshot calculation function lives in `recipes.py`. Called by the auto-price hook in `invoices.py` when an ingredient source price updates — recalculates and snapshots all recipes that use that ingredient. Uses **upsert** (INSERT ... ON CONFLICT UPDATE) — if today's snapshot exists, update it; otherwise create new
**Recipe card (HTML + browser print):**
- `GET /api/recipes/{id}/print?format=full&token={jwt}` — Full recipe HTML page (all details, images, costs, flags). Print-optimised with `@media print` CSS, same pattern as PO preview (`_build_po_html()` in purchase_orders.py)
- `GET /api/recipes/{id}/print?format=kitchen&token={jwt}` — Kitchen card HTML (large font, ingredients, steps, plating photo, flags)
- User clicks "Print Recipe" → opens in new tab → browser print dialog (includes "Save as PDF" option)
### 3b: Recipe Frontend
**`/recipes` page (RecipeList.tsx):**
- Card/list view toggle
- **Basic filters** (always visible): type (component/plated), search by name, menu section dropdown
- **Expandable filter panel** ("Show Filters" toggle reveals):
- "Contains ingredient" searchable multi-select (find recipes using specific ingredients)
- Flag filters: every flag shown with three-state toggle — neutral (no filter) / must include / must exclude
- Flag filters grouped by category (Allergy section, Dietary section, etc.)
- Cost range filter (min/max cost per portion)
- All filters apply **live** with debounce on text fields — no "Apply" button needed
- Each card: name, type badge, menu section, batch portions (if component), cost/portion, flag badges
- Quick actions: edit, duplicate, archive
- Stats bar: total recipes, component count, plated count, unmapped ingredients count (links to /ingredients?unmapped=true)
- Menu section management: "+Add Section" (e.g., Starters, Mains, Desserts, Sauces, Bases)
**`/recipes/:id` page (RecipeEditor.tsx):**
- **Header**: Name, type (component/plated), menu section (dropdown, for either type), description, batch_portions (component only), prep/cook time
- **Ingredients section**:
- Table: ingredient name, quantity, unit, cost (recent/min/max), flag indicators
- "Add ingredient" searchable dropdown from library — or "Create new" inline modal (name, category, standard_unit, optional manual_price, optional first line item search+map)
- Drag-to-reorder via sort_order
- **Sub-recipes section**:
- Table: recipe name, type, batch size, portions needed, cost contribution
- "Add sub-recipe" dropdown (excludes self + descendants, filtered by recipe search)
- Shows "uses X of Y portions" with cost math inline
- **Steps section**:
- Ordered list with step number, instruction textarea, optional image upload, optional duration
- Drag-to-reorder, add/remove
- **Images section**:
- Grid gallery with upload, caption, type tag (method/plating/general)
- Plated recipes show plating photos prominently
- **Flags section**: Flag summary badges + notification block + expandable flag matrix (see Phase 4)
- **Cost summary panel** (sticky bottom bar):
- Recent cost | Min cost | Max cost | Cost per portion
- GP comparison table (plated only): suggested sell price at 60%, 65%, 70%
- Expandable ingredient-by-ingredient breakdown with source options
- **Scaling calculator** (in cost summary panel):
- Input: "Scale to X portions" → recalculates all ingredient quantities and sub-recipe portions for display
- Frontend-only calculation, no schema change — just multiplies quantities by (target_portions / batch_portions)
- Useful for event prep or varying batch sizes
- **Cost trend chart** (expandable in cost summary):
- Line chart showing cost_per_portion over time (from recipe_cost_snapshots)
- Highlights when/why cost changed (trigger_source label on hover)
- **Print recipe button**:
- Opens new tab with print-optimised HTML page from backend
- Dropdown: "Full Recipe" or "Kitchen Card"
- Browser print dialog (includes "Save as PDF")
- Includes flag badges and yield-adjusted costs
- **Change history** (expandable section at bottom):
- Scrollable log: timestamp, user, change summary
- Most recent first
---
## Phase 4: Food Flag System
### 4a: Flag Management Backend
**Settings endpoints:**
- `GET /api/food-flag-categories` — List categories with their flags
- `POST /api/food-flag-categories` — Create category (name, propagation_type)
- `PATCH /api/food-flag-categories/{id}` — Update name/propagation_type/sort_order
- `DELETE /api/food-flag-categories/{id}` — Delete (cascade deletes flags)
- `POST /api/food-flags` — Create flag within category
- `PATCH /api/food-flags/{id}` — Update name/code/icon/sort_order
- `DELETE /api/food-flags/{id}` — Delete flag
**Ingredient flags:**
- `GET /api/ingredients/{id}/flags` — Get all flags for an ingredient
- `PUT /api/ingredients/{id}/flags` — Set flags (full replacement: send array of food_flag_ids, source='manual')
- Flags are the canonical source for recipe propagation
**Line item flags (data entry + latching):**
- `GET /api/line-items/{id}/flags` — Get flags for a line item
- `PUT /api/line-items/{id}/flags` — Set flags (full replacement: send array of food_flag_ids)
- **Latching trigger**: For each flag being set, if `line_item.ingredient_id` is not null, auto-create `ingredient_flag` (source='latched') if not already present
- Icon button on line item row in Review.tsx
**Recipe flag propagation logic (computed on-read from ingredient_flags):**
```
For each food_flag_category:
if propagation_type == "contains":
For each recipe_ingredient → get ingredient → get ingredient_flags
Union with flags from ALL sub-recipes (recursive, same logic)
→ Recipe has flag if ANY ingredient has it
if propagation_type == "suitable_for":
For each flag in category:
Check ALL recipe_ingredients → ingredient → ingredient_flags has this flag
AND ALL sub-recipes have this flag (recursive)
→ Recipe has flag only if ALL ingredients have it
→ Ingredients with NO ingredient_flags for this category = "unassessed" (treated as unknown, NOT a match)
Merge with manual additions (source_type="manual" in recipe_flags)
Apply overrides (is_active=false entries from recipe_flags with audit log)
```
**Recipe flag endpoints:**
- `GET /api/recipes/{id}/flags` — Full flag state with source tracing per flag + unassessed ingredient list
- `POST /api/recipes/{id}/flags/{flag_id}/deactivate` — Override off (requires `note`, creates audit log)
- `POST /api/recipes/{id}/flags/{flag_id}/reactivate` — Undo override (creates audit log)
- `PATCH /api/recipes/{id}/flags/{flag_id}` — Toggle excludable_on_request (plated only, requires `note`)
- `POST /api/recipes/{id}/flags/manual` — Manually add a flag not auto-detected
- `GET /api/recipes/{id}/flags/audit-log` — Override history
- `GET /api/recipes/{id}/flags/matrix` — Full ingredient × flag matrix data for the flag breakdown table
### 4b: Flag Frontend
**Line item flag button (Review.tsx):**
- New icon button alongside scales icon (shield/warning icon)
- Opens modal with flags grouped by category (Allergy section, Dietary section, etc.)
- Checkboxes for each flag
- Icon color: grey = no flags, amber = has allergy flags, green = has dietary flags, both = combined indicator
- When saving: triggers latching to mapped ingredient (if ingredient_id set)
**Flag management in Settings page:**
- Section for "Food Flag Categories"
- Each category: name, propagation type display ("Contains" / "Suitable For"), expandable flag list
- "+Add Category" button
- Within each category: "+Add Flag" with name, code, icon fields
- Reorder via drag or arrows
**Recipe flag notification block (in RecipeEditor.tsx, above flag badges):**
- Appears when ingredients have incomplete flag coverage:
> ⚠️ **3 ingredients are missing allergen details** — Lettuce, Mustard, Salt
- Additional line when manual recipe-level flags exist AND there are still unassessed ingredients:
> ℹ️ Recipe-level flags have been manually set (may not reflect all ingredients)
- Links each ingredient name to the ingredient's flag editing interface
- Dismisses when all ingredients have been assessed
**Recipe flag summary badges (in RecipeEditor.tsx):**
- Compact FoodFlagBadges row showing the computed recipe-level flags
- Below the notification block (if present)
- Same badges as on recipe list cards
**Recipe flag matrix (RecipeFlagMatrix.tsx — expandable section in RecipeEditor.tsx):**
- Table layout: ingredients down the left, food flags as columns
- Columns grouped by category (Allergy columns, then Dietary columns, etc.)
- **Direct recipe ingredients** shown as regular rows
- **Sub-recipe ingredients** grouped under a bold header row with the component name:
```
| Ingredient | Crust. | Eggs | Milk | Gluten | ... | Veg | Vegan |
|---------------------|--------|------|------|--------|-----|------|-------|
| Brioche Bun | | | | 🔴✓ | | 🟢✓ | 🔴✗ |
| Lettuce | ❓ | ❓ | ❓ | ❓ | | ❓ | ❓ |
| ▸ Burger Patty | | | | | | | |
| ↳ Beef Mince | | | | | | 🔴✗ | 🔴✗ |
| ↳ Breadcrumbs | | | | 🔴✓ | | 🟢✓ | 🟢✓ |
| ↳ Egg | | 🔴✓ | | | | 🟢✓ | 🔴✗ |
| ▸ Burger Sauce | | | | | | | |
| ↳ Mayonnaise | | 🔴✓ | | | | 🟢✓ | 🔴✗ |
| ↳ Mustard | ❓ | ❓ | ❓ | ❓ | | ❓ | ❓ |
| ══ Recipe Total ══ | | 🔴✓ | | 🔴✓ | | 🔴✗ | 🔴✗ |
```
- **Colour logic per flag category**:
- **"Contains" flags (allergens)**: 🔴 red tick = contains (bad), empty = doesn't contain (good)
- **"Suitable for" flags (dietary)**: 🟢 green tick = qualifies (good), 🔴 red cross = doesn't qualify (bad)
- **❓ Amber question mark** = ingredient has NOT been assessed for ANY flags in this category (missing data)
- **Recipe total row** uses propagation logic:
- Allergens: union (any red tick in column → recipe total is red tick)
- Dietary: intersection (any red cross OR any amber ❓ in column → recipe total is red cross or ❓)
- Overrides shown with strikethrough + hover tooltip showing mandatory note
- "Excludable on request" flags shown with dashed border
- Click on any ingredient row to navigate to that ingredient's flag editor
**FoodFlagBadges.tsx (reusable component):**
- Compact row of colored badges using flag codes (Gl, Mi, Eg, V, Ve, etc.)
- Color by category (red for allergens, green for dietary, blue for other)
- Tooltip: full name + source trace
- "Excludable" flags: dashed border or different opacity
- Used in: recipe list cards, ingredient rows, line item rows, recipe editor
---
## Phase 5: Dashboard Integration & Internal API
### 5a: Dashboard Widgets
- **Main dashboard (Dashboard.tsx)**: Small card showing:
- Unmapped ingredients count (links to `/ingredients?unmapped=true`)
- Recipes without complete costing count
- Links to `/recipes` overview
### 5b: Recipe Overview Stats (on `/recipes` page)
- Total recipes / components / plated
- Unmapped ingredients count
- Recipes with incomplete flag coverage
- Recently updated recipes
### 5c: Internal API for In-House Apps
**Authentication**: API key in request header (not JWT). Kitchen identified from API key lookup.
```
X-API-Key: {kitchen_settings.api_key}
```
**Endpoints** (prefix `/api/external/`):
- `GET /api/external/recipes/plated` — List non-archived plated recipes
- Query params:
- `include_ingredients=flat` (consolidated ingredient list) | `nested` (shows sub-recipe ingredient breakdown) | `none`
- `include_costs=true|false` (whether to include cost data — default false)
- `exclude_flags=1,5,7` (filter out recipes containing specific flags by ID)
- Returns: id, name, description, menu_section, images, flags (with excludable markers), ingredients (if requested)
- `GET /api/external/recipes/{id}` — Single plated recipe with same query param options
- `GET /api/external/food-flags` — List all flag categories and flags (for external app to understand flag IDs)
**Use case**: Menu display app queries plated recipes → selects corresponding recipe for a menu item → reads flags to calculate and display allergen/dietary information.
**Settings page**: "API Access" section under Settings
- Generate / regenerate API key button
- Toggle API key enabled/disabled
- Copy key to clipboard
- Show when key was last used (optional future enhancement)
---
## Phase 6: Event/Function Ordering
### 6a: Backend
**Endpoints:**
- `GET /api/event-orders` — List all event orders (filterable by status, date range)
- `POST /api/event-orders` — Create event order (name, event_date, notes)
- `PATCH /api/event-orders/{id}` — Update metadata/status
- `DELETE /api/event-orders/{id}` — Delete (DRAFT only)
- `POST /api/event-orders/{id}/items` — Add recipe × quantity (both plated and component recipes)
- `PATCH /api/event-order-items/{id}` — Update quantity
- `DELETE /api/event-order-items/{id}` — Remove
- `GET /api/event-orders/{id}/shopping-list` — **Aggregated ingredient shopping list**:
- Walks all selected recipes (including sub-recipes) × quantities
- Aggregates total quantity needed per ingredient (in standard units, yield-adjusted)
- Groups by ingredient category
- For each ingredient: shows total needed, available sources with pack sizes, suggested packs to order (rounded up)
- Can group by supplier for generating per-supplier order lists
- `POST /api/event-orders/{id}/generate-po` — Optional: auto-generate purchase orders from shopping list (links to existing PO system)
### 6b: Frontend
**`/event-orders` page (EventOrders.tsx):**
- List of event orders with name, date, status, recipe count, estimated total cost
- Create new event order
**`/event-orders/:id` page (EventOrderEditor.tsx):**
- **Header**: Event name, date, status, notes
- **Recipe selection**:
- Searchable dropdown of all recipes (plated and component, with menu section grouping)
- For component recipes: shows batch_portions for context (e.g., "Burger Sauce — batch of 20 portions")
- Quantity input per recipe (servings for plated, batches for component)
- Shows: recipe name, type badge, cost/portion, quantity, subtotal
- Running total at bottom
- **Shopping list view** (toggle/tab):
- Aggregated ingredients grouped by category
- Each row: ingredient name, total quantity needed (standard unit), yield-adjusted quantity
- Expandable: which recipes need this ingredient and how much each
- Source info: supplier(s), pack size, suggested packs to order, cost per pack, subtotal
- Group-by-supplier view: generates per-supplier order lists
- "Generate Purchase Orders" button → creates POs in existing system per supplier
- **Cost summary**: Total ingredient cost, cost per head, GP comparison
---
## Phase 7: KDS Recipe Link
### 7a: Schema
Already included in main `recipes` table schema (Phase 1) as `kds_menu_item_name VARCHAR(255)`. No separate migration needed.
### 7b: Backend (added to existing `backend/api/kds.py`)
- `GET /api/kds/recipe-link/{menu_item_name}` — Look up linked recipe for a KDS order item
- Display recipe summary (plating photo, key steps, flag badges) in a KDS-friendly format
### 7c: Frontend (KDS page enhancement)
- When a KDS order item has a linked recipe: show small recipe icon
- Tap to view: plating photo, ingredient list, key steps, flag badges
- Useful for new staff or complex dishes
- Lightweight overlay that doesn't disrupt KDS workflow
---
## Implementation Order
| Phase | Scope | Key Deliverables |
|-------|-------|------------------|
| 2a | Migration + models | All DB tables, pg_trgm extension, line_item.ingredient_id, pre-seeded data, SQLAlchemy models |
| 2b | Ingredient backend | CRUD endpoints, source mapping (product_code + description_pattern), auto-price hook, flag latching, duplicate detection |
| 2c | Ingredient frontend | `/ingredients` page, category management, yield %, duplicate warnings, flag display |
| 2d | Ingredient mapping modal | Modal dialog in Review.tsx replacing inline expansion, ingredient_id linking |
| 3a | Recipe backend | CRUD, costing, cycle check, change logging, cost snapshots (upsert), menu sections, print HTML |
| 3b | Recipe frontend | `/recipes` list + `/recipes/:id` editor with scaling, cost trend chart, print button |
| 4a | Flag management backend | Flag categories/flags CRUD, ingredient flagging, line item flagging + latching, recipe propagation via ingredient_flags |
| 4b | Flag frontend | Line item flag button, settings management, recipe flag notification block + matrix + badges |
| 5a-c | Dashboard + internal API | Dashboard widgets, recipe stats, API key auth, external endpoints for in-house apps |
| 6a-b | Event ordering backend + frontend | Event orders (plated + component), aggregated shopping list, PO generation |
| 7a-c | KDS recipe link | Link plated recipes to KDS menu items, recipe overlay on KDS |
### Verification Plan
1. **Phase 2a**: Run migration → verify all tables + pre-seeded data via `psql`. Verify pg_trgm extension active. Verify line_items.ingredient_id column exists
2. **Phase 2b**: Create ingredients via API, map line items as sources → verify unit conversion + yield-adjusted price. Test duplicate detection via pg_trgm on similar names. Process a new invoice → verify auto-price update (supplier_id resolved via invoice join). Test description_pattern matching for no-SKU suppliers. Verify ingredient_id set on matched line items
3. **Phase 2c-d**: Create ingredient from `/ingredients` page with yield %. Open ingredient mapping modal on a line item → verify auto-populate from description. Map to ingredient → verify source created with correct conversion and line_item.ingredient_id set
4. **Phase 3**: Create "Burger Patty" component in "Preparations" section (batch: 4 portions). Create "Beef Burger" plated in "Mains" section using 1 portion of Burger Patty + bun → verify cost = (1/4 × patty total) + bun cost. Test scaling calculator at different portion counts. Verify cost trend chart after ingredient price changes (upsert for same-day updates). Print recipe card via HTML preview
5. **Phase 4**: Flag ingredient "Butter" with "Contains: Milk". Flag a line item → verify latching creates ingredient_flag. Open recipe flag matrix → verify "Contains: Milk" propagates from Butter via any-match. Verify "suitable_for" propagates via all-must-match. Verify amber ❓ shows for unassessed ingredients. Test override with mandatory note + audit log. Test excludable_on_request on plated. Verify notification block shows missing flag count
6. **Phase 5**: Dashboard widget shows unmapped count. Generate API key in Settings. Use API key to query `/api/external/recipes/plated` → verify returns recipes with flags. Test `include_ingredients` and `include_costs` query params
7. **Phase 6**: Create event order for "Wedding Reception", add 50× Beef Burger (plated) + 3× Burger Sauce (component, 20-portion batch) → verify aggregated shopping list totals ingredients correctly across recipes. Test suggested packs calculation. Generate POs per supplier
8. **Phase 7**: Link "Beef Burger" recipe to KDS menu item. Verify recipe overlay appears on KDS when that item is ordered
### Key Design Decisions
- **Standard units: g, kg, ml, ltr, each** — chefs choose the appropriate standard per ingredient (saffron in g, beef in kg)
- **Yield percentage** on ingredients adjusts effective cost for waste/trim (e.g., 85% yield carrots, 65% whole chicken)
- **Most recent purchase price** used as default recipe cost (not pinned suppliers) — auto-updates as new invoices are processed. Min/max show the range across all sources
- **ingredient_sources coexist with product_definitions** — existing portioning works unchanged. Ingredient mapping is additive
- **Dual matching: product_code first, then description_pattern** — supports both SKU-based and description-based suppliers. Same priority pattern as existing product_definitions
- **ingredient_id FK on line_items** — direct link from line item to ingredient, simplifies queries and enables flag latching
- **ingredient_flags as canonical flag source** — flags live on ingredients, not just line items. Line item flags are a data-entry mechanism that auto-latches to ingredients. Recipe propagation reads from ingredient_flags
- **Flag latching** — when a line item is flagged, the system auto-creates a permanent ingredient_flag. Flags only accumulate, never auto-remove. Manual removal by user only
- **Food flags computed on-read** — always fresh from ingredient_flags, no cache invalidation needed. `recipe_flags` table only stores manual additions + override state
- **Propagation type per category** — "contains" (allergens, any-match) vs "suitable_for" (dietary, all-must-match) enables correct semantics for both flag types
- **Unassessed ingredients shown as amber ❓** — clearly distinguishes "not yet assessed" from "assessed as clean", prevents false negatives in dietary flags
- **Flag matrix with grouped sub-recipe ingredients** — full ingredient × flag breakdown with component grouping headers, colour-coded by flag type
- **Notification block for incomplete flag coverage** — warns when ingredients are missing allergen details, notes when manual recipe flags are a stopgap
- **Duplicate ingredient detection** — pg_trgm trigram similarity (PostgreSQL extension) for fuzzy name matching, threshold > 0.3
- **Max 5 levels** sub-recipe nesting — enforced via recursive CTE depth check
- **Batch portions on components only** — plated recipes always represent 1 serving
- **Menu sections for both recipe types** — Starters/Mains/Desserts for plated, Sauces/Bases/Preparations for components. Shared table, filtered by recipe type in UI
- **Recipe scaling calculator** — frontend-only, multiplies quantities by target/batch ratio for display
- **Cost trend snapshots with upsert** — daily snapshots triggered by ingredient price changes. Multiple updates on same day upsert to latest values
- **Recipe cards via HTML + browser print** — follows existing PO preview pattern (`_build_po_html()`). Two formats: full detail and kitchen card. No new PDF library needed
- **Recipe image serving** — authenticated endpoint `GET /api/recipes/{id}/images/{image_id}`, same pattern as invoice image endpoints. Stored at `/app/data/{kitchen_id}/recipes/{uuid}.{ext}` on existing Docker volume
- **Event ordering supports both recipe types** — plated (servings) and component (batches) can be added to event orders
- **Internal API with API key auth** — `/api/external/` prefix, X-API-Key header, for in-house apps (e.g., menu display plugin querying recipes for allergen calculation). Not publicly unauthenticated
- **API key management in Settings** — generate/regenerate, enable/disable toggle, per-kitchen
- **KDS recipe link** — matches plated recipes to KDS menu items for quick recipe/plating reference
- **Change history as summary strings** — single log entry per save with old→new field values
- **Recipe images on local Docker volume** at `/app/data/{kitchen_id}/recipes/` — backed up via existing Nextcloud
- **Any user can create/edit** recipes — uses existing auth, no new roles needed
- **Inline ingredient creation** from both mapping modal and recipe editor — with duplicate detection and ability to search+map a line item source or set manual placeholder price
- **Recipe duplication** copies top-level content (ingredients, steps, images, flags) but links sub-recipes (not deep-copied)
- **Flag filters** offer both include AND exclude for every flag — three-state toggle (neutral/include/exclude)
- **Live filtering** on recipe list with debounce — no "Apply" button
- **New "Recipes" dropdown in top header nav** — matches existing Invoices/Bookings/Reports dropdown pattern, separate from invoice navigation
- **Ingredient mapping modal replaces inline expansion** — more space for ingredient search, pack fields, and conversion display
- **Scales icon colours preserved** — red/amber/green as before, tooltip shows ingredient name when mapped
- **Supplier_id resolved via invoice join** — line_items don't have direct supplier_id, the auto-price hook joins through invoice.supplier_id