Compiler Explorer Router Development
You are working on the ce-router codebase, a TypeScript Express service that handles compilation and CMake requests for Compiler Explorer. This service replaces an AWS Lambda function and provides both queue-based routing (via SQS/WebSocket) and direct HTTP forwarding.
Available Commands
When working with this codebase, use these npm scripts:
`npm run dev` - Start development server with hot reload (uses tsx watch)`npm run build` - Compile TypeScript to JavaScript in dist/ directory`npm start` - Run the built application from dist/`npm test` - Run all tests using Vitest`npm run test:watch` - Run tests in watch mode for iterative development`npm run test:coverage` - Generate test coverage report`npm run lint` - Format and lint code using Biome (auto-fixes issues)`npm run lint:check` - Check linting without making changes`npm run typecheck` - Run TypeScript type checking without compilationProject Architecture Overview
Core Services
The application is built around these key services:
1. **Express Router Service**: Main HTTP server handling compilation and cmake requests
2. **HTTP Forwarder** (`src/services/http-forwarder.ts`): Handles direct HTTP forwarding to target URLs with proper header normalization
3. **WebSocket Manager** (`src/services/websocket-manager.ts`): Provides robust WebSocket client with automatic reconnection, subscription management, and ping/pong keepalive
4. **AWS Clients** (`src/services/aws-clients.ts`): Pre-configured AWS SDK v3 clients for DynamoDB, S3, SQS, SSM, and STS
API Endpoints
The service exposes these primary endpoints:
`POST /api/compiler/:compilerid/compile` - Compilation requests for specific compilers`POST /api/compiler/:compilerid/cmake` - CMake build requests`POST /:env/api/compiler/:compilerid/compile` - Environment-prefixed compilation (beta, staging)`POST /:env/api/compiler/:compilerid/cmake` - Environment-prefixed CMake requests`GET /healthcheck` - Health status including WebSocket connection stateRouting Methods
The service implements two routing strategies:
**Queue-based Routing (Default)**
Requests sent to SQS queues for asynchronous processingResults returned via WebSocket connectionsSupports automatic failover and load balancingUsed for most compilation requests**URL-based Routing**
Direct HTTP forwarding to specific target URLsConfigured via DynamoDB routing tableUsed for specialized compilers (e.g., GPU compilers)Includes HTTP header normalization to prevent protocol violationsConfiguration
The service is configured via environment variables and CLI flags:
**Port**: `PORT` env var or `--port` flag (default: 3000)**WebSocket URL**: `WEBSOCKET_URL` env var or `--websocket` flag (default: wss://events.compiler-explorer.com/beta)**AWS Region**: `AWS_REGION` env var (default: us-east-1)Technical Stack
**Language**: TypeScript (ES2022 target, NodeNext module resolution)**Module System**: ESM (use .js extensions in imports)**Linting/Formatting**: Biome (4-space indentation, 120 char line width, single quotes)**Testing**: Vitest with coverage support and mocking**Runtime**: Node.js with ExpressCode Style Requirements
When writing or modifying code, adhere to these style guidelines:
Use 4 spaces for indentationSingle quotes for stringsInclude trailing commas in multi-line structuresMaximum line width of 120 charactersStrict TypeScript configuration (detect unused parameters and locals)Use ES module syntax with .js file extensions in importsDevelopment Workflow
When making changes to this codebase:
1. **Read the relevant files first** - Always read existing code before proposing changes
2. **Make focused changes** - Keep modifications minimal and directly related to the task
3. **Run linting and type checks** - Always run `npm run lint` and `npm run typecheck` after code changes
4. **Write or update tests** - Prefer unit tests for pure functions
5. **Use appropriate logging**:
- Debug-level logging for detailed troubleshooting
- Info-level logging for essential operational messages
6. **Test your changes** - Run `npm test` or `npm run test:watch` before committing
Important Implementation Notes
**HTTP Response Handling**
Automatically remove conflicting headers (e.g., `transfer-encoding` when `content-length` is set)Large responses (>1MB) trigger warnings due to ALB limitsCORS headers are automatically added to all responses**WebSocket Manager Features**
Automatic reconnection with exponential backoffTopic-based pub/sub for subscription managementMessage serialization/deserializationConnection health monitoring via ping/pong keepalive**AWS Integration**
All AWS SDK v3 clients are pre-configured in `src/services/aws-clients.ts`Services include: DynamoDB, S3, SQS, SSM, and STSUse these pre-configured clients rather than creating new instancesCommon Tasks
**Adding a new endpoint**: Define route in Express router, implement handler, add tests, update health check if needed
**Modifying routing logic**: Check both queue-based and URL-based routing paths, update DynamoDB schema if needed, test failover behavior
**Updating WebSocket behavior**: Modify `src/services/websocket-manager.ts`, ensure reconnection logic is preserved, test subscription/unsubscription flows
**Adding AWS service integration**: Use pre-configured clients from `src/services/aws-clients.ts`, follow existing patterns for error handling
Testing Strategy
Write unit tests for pure functions and business logicUse integration tests sparingly for complex service interactionsMock external dependencies (AWS services, WebSocket connections)Aim for meaningful coverage, not just high percentagesUse `npm run test:watch` during development for fast feedback