Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Repository Index
47,133 repositories indexed
1–24 of 46 · Most stars
Automation engine to build, test and ship any codebase. Runs locally, in CI, or directly in the cloud
Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
Where data access meets operational intelligence
- mysql
- postgresql
- ai-agent
- data-catalog
tbls is a CI-Friendly tool to document a database, written in Go.
- database-schema
- mysql
- database-document
- documentation-tool
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
- bigquery
- copy-database
- data-ingestion
- data-integration
Elasticsearch integrations for ActiveModel/Record and Ruby on Rails
Deepnote is a drop-in replacement for Jupyter with an AI-first design, sleek UI, new blocks, and native data integrations. Use Python, R, and SQL locally in your favorite IDE, then scale to Deepnote cloud for real-time collaboration, Deepnote agent, and deployable data apps. https://deepnote.com/
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
- etl
- data-integration
- python
- postgresql
Ruby integrations for Elasticsearch
- ruby
- elasticsearch
- elastic
- search
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
- data-engineering
- data-integration
- data-pipeline
- data-quality
Test your code without writing mocks with ephemeral Docker containers 📦 Setup popular services with just a couple lines of code ⏱️ No bash, no yaml, only code 💻
- go
- golang
- testing
- integration-testing
An integration library for Laravel and Doctrine ORM
MySQL Spatial Data Extension integration with Laravel.
Free, open-source AWS emulator. LocalStack alternative: 107 services, 7,657 operations, true 100% Smithy conformance (258,545/258,545 variants pass). No account, no auth token, no paid tier.
Immutable transaction isolated sql driver for golang
- sql-driver
- integration-testing
- go
- golang
PHP Censor is an open source self-hosted continuous integration server for PHP projects.
- php
- continuous-integration
- ci
- php-censor
AlgoliaSearch integration to your favorite ORM
- algolia
- search
- rails
- gem
Shift-left API Integration | Auto-fix integration drift before integration. Specmatic turns API specifications into executable contracts, continuously verifying that every change made by AI coding agents and engineers stays true to the desired behaviour. Ship AI-ready APIs 10x faster, from intent to implementation, with Spec-Driven API Development.
- contract-testing
- microservices
- contract-driven-development
- cdd
WebTau (web test automation) is a testing API, command line tool and a framework to write unit, integration and end-to-end tests. Test across REST-API, WebSocket, GraphQL, Browser, Database, CLI and Business Logic with a consistent set of matchers and concepts. REPL mode speeds-up tests development. Rich reporting cuts down investigation time.
Guardian DB integration for tracking tokens and ensuring logout cannot be replayed.
- guardian
- database
- guardian-db
Hetionet: an integrative network of disease