DEX Aggregation Service Expert
Expert guidance for working with a real-time data aggregation service for cryptocurrency (specifically meme coins on Solana) that aggregates data from multiple DEX (Decentralized Exchange) APIs. The service provides both REST API endpoints and WebSocket connections for real-time updates.
Core Purpose
Aggregate, cache, and serve token data from DexScreener, Jupiter, and GeckoTerminal APIs with efficient caching and real-time updates.
Instructions
When working with this codebase, follow these guidelines:
1. Understanding the Architecture
**Core Data Flow:**
1. Client Request → REST API endpoint (`/api/tokens`)
2. Aggregation Service checks Redis cache
3. If cache miss: Parallel fetch from DexScreener + GeckoTerminal
4. Token Merging: Deduplicate and merge data from multiple sources
5. Cache: Store merged results in Redis (30s TTL)
6. Response: Return to client
7. WebSocket: Periodic updates broadcast to connected clients
**Service Layer Structure:**
`src/services/aggregation.service.ts` - Orchestrates multi-source fetching & merging`src/services/dexScreener.service.ts` - DexScreener API client`src/services/jupiter.service.ts` - Jupiter Price API client`src/services/geckoTerminal.service.ts` - GeckoTerminal API client`src/services/cache.service.ts` - Redis cache wrapper`src/services/websocket.service.ts` - Socket.io real-time updates`src/routes/` - API endpoints (tokens, health)`src/types/index.ts` - TypeScript interfaces`src/utils/` - Rate limiting, logging`src/config/index.ts` - Environment configuration2. Key Design Patterns to Follow
**Service Singleton Pattern:**
All services are instantiated once and exported:
```typescript
export const dexScreenerService = new DexScreenerService();
```
**Aggregation Pattern:**
The AggregationService is the central orchestrator that fetches from multiple sources in parallel, merges duplicate tokens by address, implements caching logic, and handles filtering, sorting, pagination.
**Token Merging Logic:**
When same token appears from multiple sources:
Price: Average across sourcesVolume/Liquidity: Use maximum value (avoid double-counting)Transaction Count: SumSources: Track all source names in array**Caching Strategy:**
Cache key: `dex:all_tokens`TTL: 30 seconds (configurable via `CACHE_TTL`)Cache-aside pattern: Check cache → if miss, fetch → populate cache**Rate Limiting:**
Token bucket algorithm per DEX serviceConfigurable window (default 60s) and max requests (default 300)Exponential backoff on failures (1s → 2s → 4s → 8s → 16s → 32s max)3. Important Implementation Rules
**Token Data Structure:**
All prices/volumes are in SOL (not USD)`source` is an array of strings tracking which APIs provided data`last_updated` is Unix timestampProtocol names come from the DEX (e.g., "Raydium CLMM", "Orca")**Configuration:**
Always use `config` object from `src/config/index.ts`NEVER use `process.env` directly in servicesAll environment variables are centralized in config**Error Handling:**
All services return empty arrays or null on errors (fail silently)Errors are logged via Winston loggerAPI routes return proper HTTP status codes (400, 404, 500)Graceful degradation: If one API fails, others still work**TypeScript ESM Modules:**
All imports must use `.js` extensions (even for `.ts` files)Use `import` instead of `require`Top-level `await` is supported**Code Style:**
Use TypeScript strict modePrefer `async/await` over promisesUse descriptive variable names (no single letters except loop counters)Logger levels: `debug` for verbose, `info` for important events, `error` for failuresExport service instances, not classes (singleton pattern)Unused variables: Use `_` prefix (e.g., `_req`, `_next`)4. Common Development Tasks
**Adding a New DEX API Source:**
1. Create new service in `src/services/newdex.service.ts`
2. Implement rate limiting and exponential backoff
3. Transform API response to `Token[]` format
4. Add to parallel fetch in `aggregation.service.ts:fetchAndAggregateTokens()`
5. Update README and CLAUDE.md
**Modifying Token Merging Logic:**
Edit `AggregationService.mergeTokens()` in `src/services/aggregation.service.ts:55`Be careful with averaging vs. max vs. sumConsider impact on data accuracy**Changing Cache Strategy:**
Modify `CacheService` in `src/services/cache.service.ts`TTL changes: Update `CACHE_TTL` env varCache keys: Use `keyPrefix` pattern (`dex:*`)Invalidation: Call `cacheService.del(key)` or `flush()`**Adjusting WebSocket Update Logic:**
Modify `WebSocketService.detectChanges()` in `src/services/websocket.service.ts:99`Current thresholds: 1% price change, 10% volume change5. Available Commands
**Development:**
`npm run dev` - Start development server with hot reload`npm run build` - Compile TypeScript to JavaScript`npm start` - Run production build`npm test` - Run all tests`npm run test:watch` - Run tests in watch mode**Docker:**
`docker-compose up -d` - Start full stack (app + Redis)`docker-compose down` - Stop all services`docker build -t dex-service .` - Build Docker image**Redis (local development):**
`docker run -d -p 6379:6379 redis:7-alpine`6. Technical Constraints
**API Rate Limits:**
DexScreener: 300 requests/min (most restrictive)Jupiter: Unknown (appears unlimited)GeckoTerminal: Unknown (appears generous)**Redis Dependency:**
The service REQUIRES Redis to function. Without Redis, cache service will throw errors and application won't start.
**Memory Considerations:**
Token data stored in-memory in `AggregationService.tokens` Map~1KB per token, 1000 tokens ≈ 1MB7. Troubleshooting
**Redis Connection Issues:**
1. Verify Redis is running: `redis-cli ping` should return `PONG`
2. Check REDIS_HOST and REDIS_PORT in `.env`
3. For Docker: Ensure Redis container is on same network
**API Rate Limiting:**
1. Increase `UPDATE_INTERVAL` to reduce fetch frequency
2. Increase `CACHE_TTL` to reduce cache misses
3. Check rate limiter configuration in services
**WebSocket Not Updating:**
1. Check UPDATE_INTERVAL in environment
2. Verify clients connected via `/api/health` endpoint
3. Check change detection thresholds
4. Ensure data is actually changing
8. API Endpoints Reference
`GET /api/tokens` - Query params: `timePeriod`, `sortBy`, `sortOrder`, `limit`, `cursor``GET /api/tokens/:address` - Fetch specific token (bypasses cache)`POST /api/tokens/refresh` - Clear cache and force new fetch`GET /api/health` - Health check (200 if healthy, 503 if degraded)9. WebSocket Events
**Server → Client:**
`initial_data` - Sent on connection (30 tokens max)`token_updates` - Periodic broadcasts (every 5s by default)`filtered_data` - Response to subscribe event`error` - Error messages**Client → Server:**
`subscribe` - Send TokenFilter object to get filtered updates`unsubscribe` - Stop receiving filtered updatesTesting Patterns
Use Jest with mocksMock external services at module levelUse `jest.clearAllMocks()` in `beforeEach`Mock Redis and logger to avoid dependenciesIntegration tests mock at service boundary, not internal methodsDeployment Requirements
Node.js v20+Redis instance (local or cloud)Minimum 512MB RAM (1GB+ recommended)Configured for Render (see `render.yaml`)