Analyzes menu images using OCR, LLM classification, RAG, and MCP tools to identify vegetarian dishes and calculate totals. Built on LangGraph with FastAPI microservices.
Analyzes restaurant menu images to identify vegetarian dishes and calculate total prices using a microservices architecture with OCR, LLM classification, RAG validation, and MCP tools.
This skill helps you work with the GreenDish codebase, a multi-service application that:
When working with this codebase, follow these guidelines:
The system has three microservices:
**CRITICAL**: Within each service directory, use relative imports ONLY:
```python
from config import settings
from models import OCRResult
from services import OCRService
from api.config import settings
```
For cross-service imports (e.g., Streamlit importing API modules), use absolute paths after adding repo root to `sys.path`.
Always use **uv** (not pip) for dependencies:
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
cd api # or streamlit-ui or mcp-server
uv sync
uv add package-name
```
**Never create `requirements.txt` files** - use `pyproject.toml` only.
```bash
cd api
uv sync
python main.py # or: uv run uvicorn main:app --reload --port 8005
cd streamlit-ui
uv sync
streamlit run app.py
cd mcp-server
uv sync
python server.py
docker-compose up --build
```
All config is in `/api/config.py` using Pydantic. Copy `.env.example` to `.env` and set:
```bash
GROQ_API_KEY=your_groq_key
OPENROUTER_API_KEY=your_openrouter_key # fallback
LANGCHAIN_TRACING_V2=true
LANGCHAIN_API_KEY=your_langsmith_key
MCP_SERVER_URL=http://localhost:8001
OPENROUTER_PRIMARY_MODEL=deepseek/deepseek-chat-v3.1
GROQ_PRIMARY_MODEL=openai/gpt-oss-20b
```
| Path | Purpose |
|------|---------|
| `/api/main.py` | FastAPI app entrypoint |
| `/api/routers/menu.py` | `/api/v1/process-menu` endpoint |
| `/api/agents/menu_processor.py` | LangGraph state machine |
| `/api/llm/` | Shared LLM router (Groq + OpenRouter) |
| `/api/services/` | OCR, parser, classifier, RAG services |
| `/mcp-server/server.py` | MCP tool implementations |
| `/streamlit-ui/app.py` | Multi-page test UI |
| `/docs/phases/PHASE_WISE_PLAN.MD` | Phased development roadmap |
```bash
python scripts/test_ocr.py
pytest tests/ --cov=api --cov=mcp-server
pytest tests/test_ocr.py
```
Sample menus are in `tests/fixtures/images/` (Applebee's menu, cafe menu, etc.).
1. Keyword fallback checks dish name for vegetarian indicators
2. If uncertain, invoke LLM via `LLMRouter.complete_json()` for schema-validated response
3. If LLM confidence < threshold (default 0.7), trigger RAG node
4. RAG retrieves top-3 similar dishes from ChromaDB and re-classifies
5. Calculator node sums vegetarian dishes via MCP tool
When extending the system:
1. **New endpoint**: Add to `/api/routers/` with versioned path (`/api/v1/...`)
2. **New service**: Create in `/api/services/` following service layer pattern
3. **New MCP tool**: Add to `/mcp-server/server.py` following MCP protocol
4. **New agent node**: Add to `/api/agents/nodes/` and wire into `menu_processor.py` graph
5. **New test**: Add to `/tests/` with fixtures in `tests/fixtures/`
Commit format: `Phase X: [Description]`
Example:
```
Phase 5: LLM classification with Groq and OpenRouter fallback
```
All key operations are traced in LangSmith:
Check trace URLs in Streamlit UI or LangSmith dashboard.
```bash
git clone <repo-url> && cd greendish
cp .env.example .env
cd api && uv sync && cd ..
cd mcp-server && uv sync && cd ..
cd streamlit-ui && uv sync && cd ..
cd api && python main.py
cd mcp-server && python server.py
cd streamlit-ui && streamlit run app.py
curl -X POST http://localhost:8005/api/v1/process-menu \
-F "files=@tests/fixtures/images/menu1.jpeg"
```
Leave a review
No reviews yet. Be the first to review this skill!
# Download SKILL.md from killerskills.ai/api/skills/process-restaurant-menu-for-vegetarian-dishes/raw