N8N News Pipeline Expert
Expert guidance for working with the **News Pipeline N8N** repository - an enterprise-grade automated news intelligence workflow with AI-powered analysis and topic tracking.
What This Skill Does
This skill provides comprehensive knowledge of the News Pipeline N8N project architecture, helping you:
Understand the N8N workflow structure and 51-node execution flowWork with the AI-powered topic analysis and relevance scoring systemManage RSS feed sources and content filtering logicMaintain Airtable integration and data storageTroubleshoot workflow issues and optimize performanceFollow established development and release patternsProject Context
**News Pipeline N8N** transforms scattered tech news sources into organized, topic-focused intelligence through automated daily monitoring with AI-powered analysis.
Key Technical Specifications
**Workflow Files**: `Daily_News_Topic_Tracker.json` (original), `Daily_News_Topic_Tracker_v2.1.json` (enhanced)**Complexity**: 686 lines, 35KB JSON, 51 nodes (15 core processing)**AI Engine**: OpenRouter GPT-4o-mini with 10KB+ prompts**Performance**: 99.2% reliability, 2-3 minute execution, 70-85% relevance accuracy**Daily Volume**: 100-300 tech articles processed**Airtable Base**: `app42MWoBdW4bj8Ba` ("News Pipeline Base")**Topics Table**: `tbl0UGDeOm5zulwqA` ("Topics to Monitor")Premium RSS Sources (12 Tech/AI Feeds)
**Core Technology News**: TechCrunch, The Verge, Ars Technica, VentureBeat
**AI-Specific Sources**: OpenAI Blog, Google AI Blog, Anthropic Blog, Hugging Face Blog
**Developer & Business**: Hacker News, MIT Technology Review, Bloomberg Technology
Step-by-Step Instructions
1. Understanding the Workflow Architecture
When exploring the workflow structure:
Read `Daily_News_Topic_Tracker_v2.1.json` to understand the enhanced tech/AI focused versionReview the 7-stage execution flow: Schedule Trigger → Airtable Topics → RSS Collection → JavaScript Filtering → AI Analysis → Relevance Scoring → Airtable StorageExamine the 15 core processing nodes that handle RSS parsing, content filtering, and AI analysisCheck `/rss-feeds/tech-ai-feeds.json` for the curated list of 12 premium sources2. Working with RSS Feed Management
When adding or modifying RSS sources:
Test new feeds with `python3 test_rss_feeds.py` before adding to workflowAdd feeds to `/rss-feeds/tech-ai-feeds.json` following the existing JSON structureUpdate the workflow's RSS node configurations to include new sourcesEnsure content filtering logic excludes repair/maintenance/automotive topicsValidate feed structure matches expected format (title, link, pubDate, description)3. Managing AI Topic Analysis
When optimizing AI analysis:
Review the LangChain LLM integration prompts (10,160 characters)Understand the 35 tech-specific category taxonomyAdjust relevance scoring thresholds (emotional impact, global relevance, specificity)Modify entity extraction rules for companies, products, and key peopleTest prompt changes with sample articles before production deployment4. Maintaining Airtable Integration
When working with data storage:
Use Personal Access Token for Airtable API authenticationUpdate "Topics to Monitor" table (`tbl0UGDeOm5zulwqA`) for keyword changesEnsure all 25+ metadata fields are mapped correctly in storage nodesReview structured output format (source info, AI analysis, metadata, business intelligence)Monitor data consistency across article records5. Troubleshooting and Performance Optimization
When diagnosing issues:
Check execution logs for rate limiting (10-second wait nodes prevent API overload)Verify error handling is working for each RSS source (robust failure recovery)Validate JSON structure with `jq . "Daily_News_Topic_Tracker_v2.1.json" > /dev/null`Review success rate metrics (target: 99.2% reliability)Monitor processing time (target: 2-3 minutes per execution)6. Development Workflow Commands
When importing/exporting workflows:
```bash
Import workflow into N8N
n8n import:workflow --input="Daily_News_Topic_Tracker_v2.1.json"
Validate JSON structure
jq . "Daily_News_Topic_Tracker_v2.1.json" > /dev/null && echo "Valid"
Export updated workflow
n8n export:workflow --id=[workflow-id] --output="Daily_News_Topic_Tracker_v2.1.json"
```
7. Release Management and Documentation
When preparing releases:
Follow semantic versioning (current: v2.1.1)Update `CHANGELOG.md` with categorized improvementsCreate GitHub releases with comprehensive notesUpdate README.md badges and version informationVerify GitHub Pages deployment at https://jeremylongshore.github.io/news-pipeline-n8n/Important Constraints
**Never use `--no-edit` flag with git rebase** - Invalid option for git rebase**Avoid rate limiting** - Respect 10-second wait nodes for API calls**Content filtering is critical** - Must exclude repair/maintenance/automotive articles**Validate JSON before committing** - All workflow files must pass `jq` validation**API credentials security** - Store keys in N8N environment, never in code**Daily execution timing** - Scheduled for 8:01 AM to avoid peak API trafficKey Files Reference
`/Daily_News_Topic_Tracker_v2.1.json` - Enhanced workflow (primary)`/rss-feeds/tech-ai-feeds.json` - Curated premium RSS sources`/rss-feeds/comprehensive-news-feeds.json` - Extended feed catalog (45+ sources)`/CHANGELOG.md` - Detailed version history`/README.md` - Project overview and setup instructions`/.github/workflows/` - CI/CD pipeline configurationExpected Output Structure
Each processed article includes:
**Source Information**: Publication name, RSS feed, publication date**AI Analysis**: Topic tags (35 categories), relevance scores (70-85%), sentiment analysis**Metadata**: Word count, reading time, key entities**Business Intelligence**: Significance level, development status, competitive analysis**Storage**: Structured Airtable records with 25+ fields for team collaborationPerformance Targets
**Daily Volume**: 100-300 tech articles**Match Rate**: 70-85% relevance**Processing Time**: 2-3 minutes complete execution**Success Rate**: 99.2% workflow reliability**Content Quality**: Premium tech/AI sources onlyResources
**Interactive Documentation**: https://jeremylongshore.github.io/news-pipeline-n8n/**GitHub Repository**: https://github.com/jeremylongshore/news-pipeline-n8n**Current Version**: v2.1.1 (Production ready)**Last Updated**: 2025-09-28