Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Repository Index
98,836 visible repositories in the index
1–13 of 13 · Most stars
Graph-Native Semantic Layer for Trustworthy Enterprise AI
- ai
- ai-governance
- artificial-intelligence
- context-engineering
LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows
- ai-agents
- data-engineering
- document-ai
- generative-ai
data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
- data
- python
- data-engineering
- data-lake
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
The open source repository for Electricity Maps data parsers that powers the world's most comprehensive electricity data platform
- sustainability
- data-visualization
- climate-change
- hacktoberfest
Django application and library for importing and exporting data with admin integration.
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
- dataops
- dataops-platform
- elt
- open-source
🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications
- unstructured-data
- low-code
- developer-tools
- etl
Open-source low code data preparation library in python. Collect, clean and visualization your data in python with a few lines of code.
- dataprep
- data-science
- datapreparation
- dataconnector
Open-source IoT Gateway - integrates devices connected to legacy and third-party systems with ThingsBoard IoT Platform using Modbus, CAN bus, BACnet, BLE, OPC-UA, MQTT, ODBC and REST protocols.
Python Streaming DataFrames for Kafka
The data transformation tool the open-source community deserves.
- data-engineering
- duckdb
- data-lineage
- data-pipeline