Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Repository Index
142,187 visible repositories in the index
1–22 of 22 · Most stars
Graph-Native Semantic Layer for Trustworthy Enterprise AI
- ai
- ai-governance
- artificial-intelligence
- context-engineering
LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows
- ai-agents
- data-engineering
- document-ai
- generative-ai
Cartography is a Python tool that pulls infrastructure assets and their relationships into a Neo4j graph database.
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
The open source repository for Electricity Maps data parsers that powers the world's most comprehensive electricity data platform
- sustainability
- data-visualization
- climate-change
- hacktoberfest
A Dockerized python Script to fetch Garmin health data and populate that in a InfluxDB Database, for visualization long term health trends with Grafana
- garmin
- garmin-api
- garmin-connect
- garmin-data
Django application and library for importing and exporting data with admin integration.
Self-hosted platform to unify wearable health data through one AI-ready API.
- wearables
- apple-health
- apple-watch
- fitness
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
- dataops
- dataops-platform
- elt
- open-source
🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications
- unstructured-data
- low-code
- developer-tools
- etl
Open-source low code data preparation library in python. Collect, clean and visualization your data in python with a few lines of code.
- dataprep
- data-science
- datapreparation
- dataconnector
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
- etl
- data-integration
- python
- postgresql
Scalable data pre processing and curation toolkit for LLMs
- data-curation
- llm
- data
- data-prep
Python Streaming DataFrames for Kafka
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
- etl-pipeline
- etl-framework
- spark
- apache-spark
Postgres, MySQL, or MariaDB to Elasticsearch/OpenSearch sync
- elasticsearch
- elasticsearch-sync
- postgresql
- change-data-capture
A decision-model-based ontology & knowledge-graph builder — builds governed knowledge graphs from your documents, every fact gated and traceable to its evidence.
- graphrag
- information-extraction
- knowledge-graph
- llm
Interfaces ham radio APRS-IS servers and saves packet data into an influxdb database
ETL scripts for EOS.
- eos
- eosio
- etl
- apache-beam
Parse cowrie honeypot logs into a neo4j database
- cowrie
- honeypot
- cybersecurity
- graph-database
Data ingestion engine — Singer taps to Parquet in cloud buckets