Video Annotation Viewer Development
Development guidelines for the InfantLab video annotation viewer - an advanced multimodal analysis tool for reviewing video annotations with pose detection, emotion recognition, audio analysis, and interactive timeline visualization.
Project Overview
This is a full-stack TypeScript application built for analyzing video annotations with multiple modalities including visual pose detection, facial emotion recognition, audio analysis, and temporal visualization.
Technology Stack
**TypeScript 5.x** - Strict mode enabled**React 18** - Component library**Vite 5** - Build tool and dev server**Tailwind CSS** - Utility-first styling**shadcn/ui** - Component system**Zod** - Runtime validation**React Router** - Client-side routingProject Structure
The codebase is organized into three main directories:
```
backend/ - Server-side code and APIs
frontend/ - React application and UI
tests/ - Test suites
```
Development Commands
```bash
npm test # Run test suite
npm run lint # Check code style and quality
```
Code Style Guidelines
TypeScript Strict Mode
All TypeScript code must adhere to strict mode settings:
Enable all strict type-checking optionsNo implicit `any` typesStrict null checksStrict function typesNo unused locals or parameters (enforce via linting)Validation
Use Zod for runtime validation of external dataDefine schemas for API responses, user input, and configurationValidate at system boundaries (API routes, form submissions, file uploads)React Patterns
Use functional components with hooksPrefer composition over inheritanceKeep components focused and single-purposeExtract custom hooks for shared logicUse TypeScript to type props and stateStyling
Use Tailwind CSS utility classesFollow shadcn/ui component patternsMaintain consistent spacing and color schemesEnsure responsive design for all viewport sizesRouting
Use React Router for navigationDefine routes centrallyUse type-safe route parametersArchitecture Considerations
Multimodal Analysis
The application handles multiple data streams:
**Video playback** - Synchronized timeline control**Pose detection** - Spatial coordinate visualization**Emotion recognition** - Facial expression analysis**Audio analysis** - Waveform and feature extraction**Timeline visualization** - Interactive temporal data displayData Flow
1. Video and annotation data loaded from backend
2. Multiple analysis pipelines process in parallel
3. Results synchronized to video timeline
4. Interactive visualization updates on user input
Performance
Lazy load heavy visualization componentsUse React.memo for expensive render operationsDebounce timeline scrubbing and zoom operationsStream large video files rather than loading entirelyServer Version Support
Current server API version: v1.3
Ensure all API calls target the correct endpoint version and handle version-specific response formats.
Testing Strategy
Unit tests for validation schemas and utility functionsComponent tests for React UI elementsIntegration tests for multimodal data synchronizationEnd-to-end tests for critical user workflowsRecent Updates
Added TypeScript 5.x strict mode enforcementMigrated to React 18 with concurrent featuresUpgraded to Vite 5 build systemIntegrated Tailwind CSS and shadcn/ui componentsImplemented Zod validation throughoutConfigured React Router for navigationAdded server v1.3 supportImportant Notes
Always maintain strict TypeScript complianceValidate all external data with Zod schemasEnsure multimodal data streams remain synchronizedTest video playback performance across browsersHandle large annotation datasets efficientlyMaintain accessibility standards for visualizations