A FastAPI-based OpenAI API proxy with multi-provider support, load balancing, key rotation, and authentication. Manages multiple API keys with automatic failover and health monitoring.
A Python FastAPI application that provides OpenAI API proxy and load balancing capabilities. Supports multi-provider configuration, API key management, automatic rotation, failure tracking, and health monitoring.
This skill provides guidance for working with the OpenAI Balance codebase, a sophisticated API proxy system that:
1. Client request arrives
2. `AuthMiddleware` validates cookie/token authentication
3. `SmartRoutingMiddleware` normalizes URLs
4. Router directs to `provider_routes` or `openai_routes`
5. `ProviderService` or `OpenAIChatService` handles business logic
6. `KeyManager` selects a healthy key using round-robin rotation
7. Request proxied to upstream API
**Application Entry** (`app/main.py`, `app/core/application.py`)
**Multi-Provider System** (`app/service/provider/`)
**Key Management** (`app/service/key/key_manager.py`)
**Router Priority** (`app/router/routes.py`)
```python
app.include_router(provider_routes.router) # Multi-provider routes (highest priority)
app.include_router(openai_routes.router) # Backward compatibility
app.include_router(openai_compatiable_routes.router)
```
**Configuration** (`app/config/config.py`)
| Route Pattern | Provider |
|--------------|----------|
| `/v1/*` | Default provider |
| `/{provider}/v1/*` | Named provider |
| `/hf/v1/*`, `/hf/{provider}/v1/*` | HuggingFace format |
| `/openai/v1/*`, `/openai/{provider}/v1/*` | OpenAI format |
**Setup:**
```bash
pip install -r requirements.txt
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
./start.sh # Linux/macOS
start.bat # Windows
```
**Testing:**
```bash
python -m pytest tests/ -v
python -m pytest tests/test_proxy.py -v
python -m pytest tests/test_key_redaction.py -v
```
**Docker:**
```bash
docker-compose up -d
```
| Variable | Default | Description |
|----------|---------|-------------|
| `API_KEYS` | `[]` | List of API keys for rotation |
| `ALLOWED_TOKENS` | `[]` | Allowed authentication tokens |
| `BASE_URL` | `https://api.openai.com/v1` | Upstream API base URL |
| `DEFAULT_PROVIDER` | `default` | Default provider name |
| `PROVIDERS_CONFIG` | `[]` | Multi-provider config (JSON) |
| `MAX_FAILURES` | `3` | Max failures before key disabled |
| `MAX_RETRIES` | `3` | Max retry attempts |
**File:** `.github/workflows/docker-publish.yml`
**Triggers:**
**Actions:**
**Required Secrets:**
**Release Process:**
```bash
echo "1.3.0" > VERSION
git add VERSION
git commit -m "Bump version to 1.3.0"
git push
git tag v1.3.0
git push origin v1.3.0
```
1. **Documentation:** All modules, classes, and public functions use Chinese docstrings
2. **Type Hints:** All function parameters and return values are type-annotated
3. **Async I/O:** All I/O operations use `async/await`
4. **Thread Safety:** Use `asyncio.Lock()` to protect shared state
5. **Commit Messages:**
- Use concise English descriptions
- Do NOT add Claude signatures or Co-Authored-By
- Do NOT use emojis or "Generated with Claude Code" markers
| Path | Purpose |
|------|---------|
| `app/main.py` | Application entry point |
| `app/core/application.py` | Factory function and lifecycle |
| `app/service/provider/provider_manager.py` | Multi-provider orchestration |
| `app/service/key/key_manager.py` | Key rotation and health tracking |
| `app/router/routes.py` | Router registration (order matters!) |
| `app/config/config.py` | Configuration loading and sync |
| `app/database/connection.py` | Async database connections |
Leave a review
No reviews yet. Be the first to review this skill!
# Download SKILL.md from killerskills.ai/api/skills/openai-balance-proxy/raw