TestBench CLI Development
Expert assistant for developing and maintaining TestBench, a TypeScript CLI application that provides automated testing capabilities for remote servers.
What This Skill Does
Helps you build, test, and maintain the TestBench CLI tool - a comprehensive testing solution that combines API calls, browser automation, and test execution against remote servers.
Architecture Overview
**Entry Point**: `src/index.ts` - Main CLI using Commander.js**Commands**: - `update-server` - Updates remote server environment via API calls (non-interactive)
- `smoketest` - Runs end-to-end tests against the configured server
- `get-token` - Generates bearer tokens for server authentication
**Configuration**: `config.ts` - Environment variable management**Actions**: `src/actions/` - Executable scripts for server setup and configuration**Playwright Actions**: `src/actions/playwright/` - Browser automation scripts for server setup**End-to-End Tests**: `src/tests/` - Test suites that validate server functionality**Unit Tests**: `src/__tests__/` - Tests for internal project componentsKey Technologies
**commander**: CLI framework**playwright**: Browser automation**axios**: HTTP client for API calls**dotenv**: Environment variable management**vitest**: Unit testing frameworkInstructions
When working with TestBench, follow these guidelines:
1. Environment Setup
Always validate required environment variables: `SERVER_URL` and `BEARER_TOKEN`Use `dotenv` for environment variable managementFail fast with clear error messages if environment is not properly configured2. Code Standards
Use TypeScript strict modeFollow async/await for all asynchronous operationsImplement proper error handling with user-friendly messagesSupport verbose logging for debuggingKeep actions self-contained and non-interactive3. Test Organization
**End-to-End Tests** (`src/tests/`):
Validate actual server functionalityRun against configured remote serverFocus on integration and behavior testing**Unit Tests** (`src/__tests__/`):
Test internal project componentsTest utilities, actions, and command logicFocus on isolated component behavior**Playwright Actions** (`src/actions/playwright/`):
Browser automation for server setup/configurationNot tests themselves, but setup scriptsMay include assertions for validation4. Update Server Command
When implementing or modifying the `update-server` command:
**Default behavior**: Run all scripts in `src/actions/` (excluding subfolders)**Specific script**: Support `--script=<name>` flag to run single script**Non-interactive**: No user prompts - fully automated**Sequential**: Execute scripts one after another**Error handling**: Stop on first error with clear messages5. Development Workflow
**Development**:
```bash
npm run dev # Hot reload development mode
```
**Build**:
```bash
npm run build # Compile TypeScript
npm start # Run compiled version
```
**Testing**:
```bash
npm test # Unit tests (vitest)
npm run smoketest # E2E tests (playwright)
npm run update-server # Run all server config scripts
npm run update-server -- --script=configure-email # Run specific script
npm run get-token # Generate bearer tokens
```
6. Adding New Commands
When adding new CLI commands:
Register in `src/index.ts` using Commander.jsCreate implementation in appropriate moduleValidate environment variables upfrontProvide clear help text and examplesHandle errors gracefully with actionable messages7. Adding New Actions
When creating server configuration actions:
Place in `src/actions/` for API-based actionsPlace in `src/actions/playwright/` for browser automationMake scripts idempotent when possibleUse axios with bearer token authenticationInclude descriptive loggingReturn clear success/failure status8. Authentication
Use bearer token authentication for API callsToken generated via `get-token` command using PlaywrightInclude token in `Authorization: Bearer <token>` headerValidate token before making API calls9. Error Handling
Validate inputs before executionProvide specific error messagesExit with appropriate status codesLog errors with sufficient context for debugging10. Documentation
When modifying code:
Update inline comments for complex logicKeep README in sync with commands and featuresDocument environment variables and their purposesProvide examples for command usageExample Usage
Creating a New Action Script
```typescript
// src/actions/configure-feature.ts
import axios from 'axios';
import config from '../config';
export async function configureFeature() {
if (!config.serverUrl || !config.bearerToken) {
throw new Error('SERVER_URL and BEARER_TOKEN are required');
}
try {
const response = await axios.post(
`${config.serverUrl}/api/configure`,
{ feature: 'example' },
{
headers: {
Authorization: `Bearer ${config.bearerToken}`,
'Content-Type': 'application/json'
}
}
);
console.log('Feature configured successfully');
return response.data;
} catch (error) {
console.error('Failed to configure feature:', error.message);
throw error;
}
}
```
Creating a New E2E Test
```typescript
// src/tests/feature.test.ts
import { test, expect } from '@playwright/test';
test('feature works correctly', async ({ page }) => {
await page.goto(process.env.SERVER_URL);
// Test actual functionality
await expect(page.locator('.feature')).toBeVisible();
});
```
Constraints
All actions must be non-interactive and fully automatedEnvironment variables must be validated before executionStop execution on first error when running multiple scriptsMaintain separation between setup actions and actual testsFollow TypeScript strict mode - no implicit any typesUse proper async/await patterns - no callback pyramids