LLM Gateway Development
Expert guidance for working with the LLM Gateway codebase - a full-stack TypeScript monorepo for routing, managing, and analyzing LLM requests across multiple providers with a unified API interface.
Overview
LLM Gateway is a monorepo containing multiple services:
**Core Services:**
**Gateway** (apps/gateway) - LLM request routing and provider management (Hono + Zod + OpenAPI)**API** (apps/api) - Backend API for user management, billing, analytics (Hono + Zod + OpenAPI)**UI** (apps/ui) - Frontend dashboard (Next.js App Router)**Playground** (apps/playground) - Interactive LLM testing environment (Next.js App Router)**Code** (apps/code) - Dev plans + coding tools landing & dashboard (Next.js App Router)**Docs** (apps/docs) - Documentation site (Next.js + Fumadocs)**Admin** (apps/admin) - Internal admin dashboard**Shared Packages:**
@llmgateway/db - Database schema, migrations, utilities (Drizzle ORM)@llmgateway/models - LLM provider definitions and model configurations@llmgateway/auth - Authentication utilities and session managementTechnology Stack
**Backend**: Hono, PostgreSQL (Drizzle ORM), Redis, Better Auth, Zod, OpenAPI**Frontend**: Next.js App Router, TanStack Query, Radix UI, Tailwind CSS**Development**: Turbo monorepo, pnpm workspaces, Vitest, TypeScript (strict mode)Step-by-Step Instructions
1. Initial Setup
When setting up the development environment:
1. Install dependencies: `pnpm install`
2. Start Docker services: `docker compose up -d` (PostgreSQL and Redis)
3. Run full setup: `pnpm setup` (resets DB, syncs schema, seeds data)
**Development URLs:**
UI: http://localhost:3002Playground: http://localhost:3003Code: http://localhost:3004API: http://localhost:4002Gateway: http://localhost:4001Docs: http://localhost:3005Admin: http://localhost:3006PostgreSQL: localhost:5432Redis: localhost:63792. Development Workflow
**Starting development servers:**
```bash
pnpm dev # Starts all services
```
**CRITICAL - All commands must be run from the root directory, not individual app directories.**
**Before committing code:**
1. Format code: `pnpm format` (ALWAYS run before committing)
2. Check linting: `pnpm lint`
3. Build for production: `pnpm build` (ALWAYS run after finishing features, especially if API routes were modified)
4. Run tests: `pnpm test:unit` and `pnpm test:e2e`
3. Code Standards
**TypeScript Requirements:**
This is a pure TypeScript projectNEVER use `any` or `as any` unless absolutely necessaryAlways use tabs for indentation (not spaces)Always use top-level `import`, never require or dynamic importsApply DRY principles for code reuseNo unnecessary code comments**Commit Standards:**
Use conventional commit message formatLimit commit message title to max 50 charactersDo NOT --amend commits after pushing to remoteNEVER force push on main/default branch (only acceptable on feature branches)When writing pull request titles, use conventional commit format and limit to max 50 characters4. Database Operations
**Using Drizzle ORM (latest object syntax):**
For reads: Use `db().query.<table>.findMany()` or `db().query.<table>.findFirst()`For schema changes: ALWAYS use `pnpm run setup` to regenerate migrationsNEVER write migrations manuallyOnly edit generated migration files if specifically asked**After table/column changes:**
```bash
pnpm run setup # Resets db, syncs schema, seeds data
```
**Available commands:**
```bash
pnpm --filter db push # Push database schema
pnpm --filter db seed # Seed database with initial data
```
**Resolving migration conflicts:**
```bash
git restore --source=origin/main packages/db/migrations/
pnpm migrations # Regenerate migrations automatically
```
**Resolving pnpm-lock.yaml conflicts:**
```bash
pnpm install # Automatically resolves conflicts
```
5. Testing
**Unit Tests:**
```bash
pnpm test:unit # Run *.spec.ts files
```
**End-to-End Tests:**
```bash
pnpm test:e2e # Run *.e2e.ts files (full suite)
Quick testing with specific models
TEST_MODELS="openai/gpt-4o-mini,anthropic/claude-3-5-sonnet-20241022" pnpm test:e2e
Include free models
FULL_MODE=true pnpm test:e2e
Enable detailed logging
LOG_MODE=true pnpm test:e2e
```
**E2E Test Structure:**
Tests run up to 16 in parallel (minimum 8 threads)`apps/gateway/src/api.e2e.ts` - Parallelized `.each()` tests`apps/gateway/src/api-individual.e2e.ts` - Individual test cases needing isolationMain suite uses `{ concurrent: true }` for parallel execution**Testing with local API:**
Use `test-token` as authentication when running curl commands against the local API.
6. Creating New Packages
When creating a new package in `packages/`, copy these config files from an existing package (e.g., `packages/models`):
`package.json` - Package configuration with build scripts`tsconfig.json` - TypeScript configuration extending root`.prettierignore` - Ignores `dist` build output`.lintstagedrc.json` - Lint-staged configuration`eslint.config.mjs` - ESLint configuration7. Frontend Development
**Next.js App Router conventions:**
Use `next/link` for linksUse `next/navigation`'s router for programmatic navigationUse React Server Components by defaultUse cookies for user-settings not saved in database (ensures SSR works)8. Utility Commands
```bash
pnpm clean # Clean build artifacts and cache directories
```
Key Features
**LLM Gateway:**
Multi-provider support (OpenAI, Anthropic, Google Vertex AI, etc.)OpenAI-compatible API interfaceRequest routing and load balancingResponse caching with RedisUsage tracking and cost analytics**Management Platform:**
User authentication with passkey supportAPI key managementProject and organization managementBilling integration with StripeReal-time usage monitoringProvider key managementImportant Notes
This project uses pnpm for package management (not npm or yarn)All monorepo commands must be run from the root directoryDatabase schema changes require running `pnpm run setup` to regenerate migrationsAlways run `pnpm format` before committing codeAlways run `pnpm build` after finishing features to ensure production builds workThe repository uses AGPLv3 license for core features; enterprise features in `ee/` directory require separate licenseConstraints
NEVER use `any` or `as any` types unless absolutely necessaryNEVER write database migrations manually - use `pnpm run setup` insteadNEVER force push to main/default branchALWAYS use tabs for indentation (not spaces)ALWAYS run commands from root directory, not individual app directoriesALWAYS format code with `pnpm format` before committing