SIDFlow Development Assistant

Expert AI agent for developing the SIDFlow platform - a CLI-first pipeline that streams C64 SID music with ML-powered recommendations based on your listening feedback and mood preferences.

Overview

SIDFlow is a TypeScript monorepo that fetches SID music collections, classifies audio characteristics, trains ML models on user feedback, and provides intelligent playback with mood-based stations. This skill guides you through the architecture, conventions, and workflows for extending and maintaining the platform.

Core Architecture

SIDFlow operates as a multi-stage pipeline with four primary packages:

1. **@sidflow/fetch** - Downloads SID collections (currently HVSC, extensible to any local collection)

2. **@sidflow/classify** - Renders WAV cache, extracts heuristic features, generates auto-tags, emits JSONL metadata

3. **@sidflow/train** - Consumes classified data + user feedback, trains ML models, persists LanceDB artifacts

4. **@sidflow/play** - Provides playback, recommendations, and mood-based station creation

All stages read/write structured data to `data/` in JSONL format and respect configuration from `.sidflow.json`.

Development Workflow

Required Reading

1. **Always start by reading `AGENTS.md`** at repo root for:

- Repository-wide agent behavior patterns

- ExecPlan usage for multi-step work

- Multi-hour workflow expectations

- Autonomous agent guidelines

2. **Follow `PLANS.md`** for any multi-step work:

- Load and update the central execution plan

- Treat it as single source of truth

- Log progress, surprises, decisions, and outcomes

- Maintain continuity across long sessions

3. **Honor `.cursorrules`** if user is on Cursor (mirrors these guardrails)

Agent Autonomy Principles

**Don't stop early**: Continue until the user's request is fully satisfied or you're genuinely blocked by missing credentials/external access

**Prefer research over clarification**: Make reasonable assumptions and document them in the plan's Decision Log

**Run validation after edits**: Build, lint/typecheck, tests - record PASS/FAIL in Progress section

**Iterate on failures**: Attempt up to three targeted fixes before surfacing summary to user

**Keep going across sessions**: Use plan sections to maintain state over hours of work

Making Code Changes

1. **Minimal, additive edits only**: Keep public APIs stable unless task requires changes

2. **Preserve conventions**: Follow repository coding patterns and shared utilities

3. **Read before modifying**: Never propose changes to code you haven't read

4. **Validate quickly**: Run fast checks (build, test) after substantive edits

Shared Libraries & Conventions

@sidflow/common

All shared utilities live here - always import rather than reimplementing:

**`loadConfig`** - Load `.sidflow.json` configuration, honor `--config` overrides

**`resetConfigCache`** - Reset config cache in long-running tools

**`stringifyDeterministic`** - Serialize JSON to avoid diff churn, normalize nested structures

**`ensureDir` / `pathExists`** - Async fs helpers (extend these, don't mix sync APIs)

**`createProgressLogger`** - Throttled progress reporting for CLI output

**`buildDatabase`** - LanceDB builder for similarity search

**`generateManifest`** - Deterministic checksum generation for derived artifacts

**`SidflowConfigError`** - Shared error type for configuration issues

Configuration

**`.sidflow.json`** defines:

- `sidPath` - Local SID collection path (works with any collection, not just HVSC)

- `audioCachePath` - WAV cache storage

- `tagsPath` - Auto-generated tags location

- `classifiedPath` - (optional) Classification output

Default to config values but accept explicit paths when provided by callers

Always load through `loadConfig` helper

Data Persistence

**Classification metadata**: Uses `resolve*` helpers, stores WAV hashes in sidecar `.hash` files - preserve this caching scheme

**Feedback logs**: Date-partitioned under `data/feedback/<year>/<month>/<day>/events.jsonl`

- Consumers must tolerate missing folders

- Skip corrupt lines with warnings, not hard failures

**Derived artifacts**: Store with deterministic checksums using `generateManifest`

**JSONL format**: All pipeline stages communicate via JSONL files in `data/`

CLI Patterns

Follow this structure for all CLI tools:

1. **Parse args** in `cli.ts`

2. **Call `plan*` function** to validate inputs

3. **Run pure helpers** that accept explicit dependencies (see `runClassifyCli`, `runFetchCli`)

4. **Provide progress reporting** via callbacks with throttling

5. **Keep CLI UX stable** - scripts in `scripts/` are thin wrappers defining the end-to-end contract

Archive Extraction

Relies on bundled `7zip-min` npm dependency

Inject via shared utilities, don't reimport directly

Testing & Quality Gates

Local Commands (CI-Aligned)

```bash

Build / typecheck

bun run build

Unit tests with coverage

bun run test

E2E tests (Playwright)

bun run test:e2e

Config validation

bun run validate:config

First-time test setup (installs Playwright browsers)

bun run setup:tests

```

Coverage

Coverage enabled automatically with `bun run test` (see root `package.json` for flags/excludes)

Reported to Codecov in CI

Don't hard-code coverage percentages in docs - describe mechanism instead

Stability Requirements

**Keep tests deterministic**: No `waitForTimeout` in Playwright

**Use explicit selectors**: Prefer `data-testid` attributes

**Three consecutive green runs**: Required for behavior/test changes (see `AGENTS.md`)

Coding Practices

TypeScript

**Strict mode**: Follow `tsconfig.base.json` settings

**No `any` types**: Keep types explicit

**Expose pure helpers**: For unit testing

**Compose small functions**: Build functionality from focused units

Error Handling

Use `SidflowConfigError` for config issues

Preserve informative messaging when wrapping errors

Log warnings for skippable errors (e.g., corrupt feedback lines)

Hard fail only when genuinely blocked

Documentation

Use concise comments only for non-obvious steps (e.g., cache heuristics)

Prefer self-documenting code and clear naming

Update `AGENTS.md` / `PLANS.md` when changing workflows

Code Reuse

Never duplicate logging/serialization patterns

Extend `@sidflow/common` if new cross-cutting helper is needed

Follow existing utility patterns (async fs, retry logic, etc.)

Tooling & Dependencies

Runtime

**Bun** is the runtime for all operations:

- Install deps: `bun install`

- Build: `bun run build`

- Tests/CLIs: `bun run ...`

Key Dependencies

**LanceDB** (`vectordb`) - Powers similarity search, call `buildDatabase` before generating manifests

**7zip-min** - Archive extraction (bundled)

**Playwright** - E2E testing (`packages/sidflow-web`)

Common Tasks

Adding a New Classification Feature

1. Extend `heuristicFeatureExtractor` in `@sidflow/classify`

2. Update JSONL schema if needed

3. Preserve WAV cache + hash scheme

4. Add tests for feature extraction

5. Validate with `bun run test && bun run build`

Modifying Feedback Weighting

1. Edit weight constants in `@sidflow/train`

2. Update feedback merge logic

3. Regenerate model artifacts with new weights

4. Document weight rationale in Decision Log

Supporting New SID Collection Sources

1. Extend `@sidflow/fetch` with new source

2. Keep classification/training/playback source-agnostic

3. Ensure `.sidflow.json` `sidPath` supports new layout

4. Test end-to-end pipeline with new collection

Extending Shared Utilities

1. Add to appropriate module in `@sidflow/common`

2. Keep pure and explicitly typed

3. Add unit tests

4. Update this documentation if cross-cutting pattern emerges

When to Ask for Clarification

You should work autonomously, but ask when:

**Missing credentials** for external services

**Unclear requirements** that could lead to wasted work

**API-breaking changes** needed that affect multiple consumers

**Architecture decisions** with significant long-term implications

Otherwise, make reasonable assumptions, document them in the Decision Log, and proceed.

Success Criteria

A task is complete when:

1. All requested functionality is implemented

2. Code follows repository conventions

3. Build passes: `bun run build`

4. Tests pass: `bun run test` (+ `test:e2e` if UI changed)

5. Config validation passes: `bun run validate:config`

6. Progress/Decisions/Outcomes logged in `PLANS.md`

7. No regressions introduced in existing functionality

Additional Resources

**Architecture deep-dive**: See package READMEs in `packages/`

**CI configuration**: `.github/workflows/` for build/test/deploy pipeline

**Execution plan template**: `PLANS.md` at repo root

**Agent behavior guide**: `AGENTS.md` at repo root

SIDFlow Development Assistant

SIDFlow Development Assistant

Overview

Core Architecture

Development Workflow

Required Reading

Agent Autonomy Principles

Making Code Changes

Shared Libraries & Conventions

@sidflow/common

Configuration

Data Persistence

CLI Patterns

Archive Extraction

Testing & Quality Gates

Local Commands (CI-Aligned)

Build / typecheck

Unit tests with coverage

E2E tests (Playwright)

Config validation

First-time test setup (installs Playwright browsers)

Coverage

Stability Requirements

Coding Practices

TypeScript

Error Handling

Documentation

Code Reuse

Tooling & Dependencies

Runtime

Key Dependencies

Common Tasks

Adding a New Classification Feature

Modifying Feedback Weighting

Supporting New SID Collection Sources

Extending Shared Utilities

When to Ask for Clarification

Success Criteria

Additional Resources

Reviews (0)