Expert assistant for the FinETL Python library - a financial data ETL tool using yfinance and HuggingFace datasets. Helps with extraction, transformation, and loading of financial market data.
Expert guidance for working with the FinETL Python library - a financial data ETL (Extract, Transform, Load) tool for financial market data.
FinETL is a Python library that implements ETL patterns for financial data processing. It leverages yfinance for extracting financial market data and HuggingFace datasets for data handling and storage.
When working on this project, follow these steps:
1. **Environment Setup**
- Ensure Python 3.12+ is installed
- Run `poetry install` to install all dependencies
- The project uses Poetry for dependency management
2. **Understanding the Architecture**
- The codebase follows a standard ETL pattern with four main modules in `src/finetl/`:
- `extraction/` - Handles data extraction from financial sources (primarily yfinance)
- `transformation/` - Performs data cleaning, validation, and processing
- `loading/` - Manages loading data to various destinations
- `pipeline.py` - Orchestrates complete ETL workflows by coordinating the above modules
3. **Making Code Changes**
- Always read existing code before proposing modifications
- Follow the established ETL pattern when adding new functionality
- Maintain separation of concerns between extraction, transformation, and loading
- Keep pipeline orchestration logic in `pipeline.py`
4. **Adding Dependencies**
- Use `poetry add <package>` to add new dependencies
- Document why the dependency is needed
- Ensure compatibility with Python 3.12+
5. **Testing**
- Run tests with `poetry run pytest`
- Write tests for new functionality
- Ensure tests pass before finalizing changes
6. **Key Libraries**
- **yfinance** - Primary source for financial market data extraction
- **datasets** (HuggingFace) - Data handling, storage, and versioning
- **Poetry** - Dependency and environment management
- **pytest** - Testing framework
Leave a review
No reviews yet. Be the first to review this skill!
# Download SKILL.md from killerskills.ai/api/skills/finetl-development-assistant/raw