[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
- data
- data-cleaning
- data-pipelines
- data-processing
Repository Index
47,133 repositories indexed
49–72 of 225 · Most stars
Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
- data-drift
- jupyter-notebook
- pandas-dataframe
- machine-learning
The leader in Customer Data Infrastructure
- snowplow
- analytics
- data
- data-pipeline
Integrate React.js with Rails views and controllers, the asset pipeline, or webpacker.
🍴 Web applications made easy. Since 2011.
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
📚 Parameterize, execute, and analyze notebooks
- jupyter
- notebooks
- notebook-generator
- nteract
TranslatedMedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. Training medical large models, implementing incremental pre-training (PT), supervised fine-tuning (SFT), RLHF, DPO, ORPO, and GRPO.
ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.
- mlops
- machine-learning
- data-science
- production-ready
🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.
- cloud-native
- streaming
- rust
- real-time
Digger is an open source IaC orchestration tool. Digger allows you to run IaC in your existing CI pipeline ⚡️
- infrastructure-as-code
- terraform
- terraformcloud
- tacos
Where data access meets operational intelligence
- mysql
- postgresql
- ai-agent
- data-catalog
Self-hostable workflow orchestrator for teams whose main work isn't orchestration. Declarative YAML over your scripts, SSH commands, containers, etc; keep workflows separate from business logic. One binary, no database, runs on limited H/W resources. Alternative to Airflow / Cron / Job Scheduler.
- cron
- workflow-engine
- workflow-scheduler
- workflow-orchestration
Privacy and Security focused Segment-alternative, in Golang and React
- privacy
- warehouse-management
- data-warehouse
- customer-data
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
- bigquery
- copy-database
- data-ingestion
- data-integration
AI Infra / AI Orchestration / AI Control Plane
Open-source Infrastructure as Code (IaC) orchestration platform: GitOps workflows, orchestration, code generation, observability, drift detection, asset management, policies, Slack notifications, and more. Integrates with Terraform, OpenTofu, Terragrunt, Kubernetes, GitHub Actions, GitLab CI/CD, BitBucket Pipelines, and any other CI/CD platform.
pySLAM is a hybrid Python/C++ Visual SLAM pipeline supporting monocular, stereo, and RGB-D cameras. It provides a broad set of modern local and global feature extractors, multiple loop-closure strategies, a volumetric reconstruction module, integrated depth-prediction models, and semantic segmentation capabilities for enhanced scene understanding.
- 3d-reconstruction
- depth-prediction
- feature-matching
- global-features
Browser compilation library – an asset pipeline for applications that run in the browser
Surging is a micro-service engine that provides a lightweight, high-performance, modular RPC request pipeline. support Event-based Asynchronous Pattern and reactive programming.
- microservices
- apigateway
- surging
- rabbitmq
Kubernetes-native platform to run massively parallel data/streaming jobs
- kubernetes
- stream-processing
- data-processing
- k8s
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
- dataops
- dataops-platform
- elt
- open-source
High-performance time-series aggregation for PostgreSQL
- realtime
- stream-processing
- analytics
- sql
Little command line REST client that you can use in pipelines (bash or zsh).