Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Repository Index
87,075 visible repositories in the index
1–19 of 19 · Most stars
Graph-Native Infrastructure for Context and Accountable AI Systems
- ai
- ai-governance
- artificial-intelligence
- context-engineering
LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows
- ai-agents
- data-engineering
- document-ai
- generative-ai
data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
- data
- python
- data-engineering
- data-lake
A system for agentic LLM-powered data processing and ETL
Cartography is a Python tool that pulls infrastructure assets and their relationships into a Neo4j graph database.
The open source repository for Electricity Maps data parsers that powers the world's most comprehensive electricity data platform
- sustainability
- data-visualization
- climate-change
- hacktoberfest
Towhee is a framework that is dedicated to making neural data processing pipelines simple and fast.
- machine-learning
- convolutional-networks
- embedding-vectors
- embeddings
The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure
- claude-code
- codex
- data-processing
- harness-engineering
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
- dataops
- dataops-platform
- elt
- open-source
🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications
- unstructured-data
- low-code
- developer-tools
- etl
Open-source low code data preparation library in python. Collect, clean and visualization your data in python with a few lines of code.
- dataprep
- data-science
- datapreparation
- dataconnector
Open-source IoT Gateway - integrates devices connected to legacy and third-party systems with ThingsBoard IoT Platform using Modbus, CAN bus, BACnet, BLE, OPC-UA, MQTT, ODBC and REST protocols.
Scalable data pre processing and curation toolkit for LLMs
- data-curation
- llm
- data
- data-prep
Python Streaming DataFrames for Kafka
The data transformation tool the open-source community deserves.
- data-engineering
- duckdb
- data-lineage
- data-pipeline
A decision-model-based ontology & knowledge-graph builder — builds governed knowledge graphs from your documents, every fact gated and traceable to its evidence.
- graphrag
- information-extraction
- knowledge-graph
- llm
ETL scripts for EOS.
- eos
- eosio
- etl
- apache-beam
Parse cowrie honeypot logs into a neo4j database
- cowrie
- honeypot
- cybersecurity
- graph-database