Sophisticated multi-agent AI system for crawling, processing, and generating content using LangGraph orchestration, FastAPI backend, and Next.js frontend with dependency injection architecture.
You are working with a sophisticated multi-agent AI content aggregation system built with modern enterprise-grade architecture patterns. The system automatically crawls, processes, and generates AI-related articles using LangGraph orchestration, dependency injection, and microservices design principles.
The system follows a 6-layer enterprise architecture pattern:
1. **Presentation Layer**: Next.js 15 + React 19 + TypeScript + Tailwind CSS v4
2. **API Gateway Layer**: FastAPI + CORS + Request/Response Validation
3. **Business Logic Layer**: LangGraph Multi-Agent Orchestration
4. **Service Layer**: AI, Storage, Crawler, Content Processing Services
5. **Data Access Layer**: Repository Pattern + Enhanced Storage Service
6. **Persistence Layer**: File System + JSON + YAML Configuration Storage
When the user asks to start or run the application:
1. **Modern Unified System (Recommended)**:
```bash
cd backend
python start_modern.py # Start modern API server (port 8000)
```
2. **Traditional Startup**:
```bash
./start.sh -d # Start both backend and frontend with live logs
```
3. **Backend Only**:
```bash
cd backend
uv run python main.py # Start unified modern API (port 8000)
```
4. **Frontend Only**:
```bash
cd frontend
npm run dev # Start development server with Turbopack (port 3000)
```
5. **Health Check**:
```bash
curl http://localhost:8000/health # Check backend status with tool info
```
**Backend (Python with uv)**:
**Frontend (Next.js + Turbopack)**:
The system uses four specialized LangGraph agents in sequence:
1. **CrawlerAgent** (`backend/agents/crawler_agent.py`)
- Multi-source content crawling (RSS, HTML, API)
- Concurrent processing with rate limiting
- Authentication support and error recovery
2. **ProcessorAgent** (`backend/agents/processor_agent.py`)
- AI-powered content analysis using ARK API
- Content categorization and relevance scoring
- Quality filtering (≥0.6 relevance threshold)
3. **ResearchAgent** (`backend/agents/research_agent.py`)
- Deep research and fact verification
- Topic analysis and insight generation
- ReactAgent pattern with tool integration
4. **WriterAgent** (`backend/agents/writer_agent.py`)
- Comprehensive article generation (800-1200 words)
- Content grouping by category
- Professional article structure with quality control
```
Crawler → Processor → Research → Writer
↓ ↓ ↓ ↓
Storage Analysis Insights Articles
```
The DI container (`backend/core/container.py`) manages:
When working with services, understand these key components:
```
backend/
├── agents/ # LangGraph multi-agent system
├── api/ # FastAPI endpoints
├── core/ # DI container, interfaces, exceptions
├── services/ # Business logic services
├── repositories/ # Data access layer
├── models/ # Data models and schemas
├── config/ # Configuration management
├── utils/ # Utility functions
└── conf.yml # Main configuration file
frontend/
├── src/app/ # Next.js App Router
├── src/components/ # React components
├── src/lib/ # API client and utilities
└── src/types/ # TypeScript definitions
data/
├── articles/ # Generated articles
├── content/ # Processed content
├── sources/ # Source configurations
├── workflows/ # Workflow state persistence
└── logs/ # System logs
```
Edit `backend/conf.yml` for system configuration:
```yaml
app:
name: "AI Content Aggregator"
version: "1.0.0"
models:
ark:
api_key: "${ARK_API_KEY}" # Required environment variable
base_url: "https://ark.cn-beijing.volces.com/api/v3"
model: "ep-20250617155129-hfzl9"
agents:
crawler:
max_sources: 50
timeout: 30
processor:
relevance_threshold: 0.6
writer:
min_word_count: 800
max_word_count: 1200
services:
storage:
base_path: "./data"
```
1. **Define Interface First**: Add interface to `backend/core/interfaces.py`
2. **Implement Service**: Create service class implementing the interface
3. **Register in Container**: Add to DI container in `backend/core/container.py`
4. **Add API Endpoint**: Create endpoint in `backend/api/main.py`
5. **Update Frontend**: Add corresponding UI components and API calls
1. **Understand Agent Pipeline**: Know where agent fits in Crawler → Processor → Research → Writer flow
2. **Maintain State Schema**: Keep state definitions consistent across agents
3. **Handle Errors Gracefully**: Use try-except with detailed logging
4. **Test Agent Integration**: Run `python test_unified_modern.py`
Use `./start.sh -d` for color-coded live logs from both backend and frontend.
Check system health and tool status:
```bash
curl http://localhost:8000/health
```
Visit `http://localhost:8000/docs` for interactive API documentation.
1. **Scalability**: Microservices architecture with async processing
2. **Maintainability**: Clean architecture with clear separation of concerns
3. **Reliability**: Comprehensive error handling and data persistence
4. **Developer Experience**: Type safety, hot reload, rich documentation
```bash
./start.sh -d # Start with live logs
```
```bash
cd backend
python test_unified_modern.py
```
```bash
cd backend
uv add <package>
cd frontend
npm install <package>
```
```bash
curl -X POST http://localhost:8000/articles/cache/clear
```
```bash
curl -X POST http://localhost:8000/articles/deduplicate
```
---
Follow these instructions when working with this sophisticated multi-agent AI content aggregation system. Maintain the architectural patterns, respect the dependency injection system, and ensure all code meets the quality standards outlined above.
Leave a review
No reviews yet. Be the first to review this skill!
# Download SKILL.md from killerskills.ai/api/skills/enterprise-ai-content-aggregator/raw