Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
- kedro
- hacktoberfest
- mlops
- python
Repository Index
47,133 repositories indexed
25–48 of 90 · Most stars
Agent Reinforcement Trainer: train multi-step agents for real-world tasks using GRPO. Give your agents on-the-job training. Reinforcement learning for Qwen3.6, GPT-OSS, Llama, and more!
StreamDiffusion: A Pipeline-Level Solution for Real-Time Interactive Generation
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
The Python package installer
- pip
- python
- packaging
AlphaFold 3 inference pipeline.
CLI tool and python library that converts the output of popular command-line tools, file-types, and common strings to JSON, YAML, or Dictionaries. This allows piping of output to tools like jq and simplifying automation scripts.
- parsers
- json
- linux
- command-line
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
- data
- data-cleaning
- data-pipelines
- data-processing
📚 Parameterize, execute, and analyze notebooks
- jupyter
- notebooks
- notebook-generator
- nteract
TranslatedMedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. Training medical large models, implementing incremental pre-training (PT), supervised fine-tuning (SFT), RLHF, DPO, ORPO, and GRPO.
ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.
- mlops
- machine-learning
- data-science
- production-ready
pySLAM is a hybrid Python/C++ Visual SLAM pipeline supporting monocular, stereo, and RGB-D cameras. It provides a broad set of modern local and global feature extractors, multiple loop-closure strategies, a volumetric reconstruction module, integrated depth-prediction models, and semantic segmentation capabilities for enhanced scene understanding.
- 3d-reconstruction
- depth-prediction
- feature-matching
- global-features
adds flavor of interactive filtering to the traditional pipe concept of UNIX shell
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
- dataops
- dataops-platform
- elt
- open-source
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
- kv-cache
- llm-inference
- local-llm
- quantization
CLI task management & automation tool
- python
- build-tool
- build-automation
- task-runner
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
- etl
- data-integration
- python
- postgresql
Elyra extends JupyterLab with an AI centric approach.
- notebooks
- notebook-jupyter
- jupyterlab
- jupyterlab-extensions
A full spaCy pipeline and models for scientific/biomedical documents.
- scientific-documents
- spacy
- custom-pipes
- nlp
Nix User Repository: User contributed nix packages [maintainer=@Pandapip1]
- nur
- nixpkgs
- overlays
- nix-packages
MongoDB data stream pipeline tools by YouGov (adopted from MongoDB)
A deep learning toolkit for healthcare AI: EHR (MIMIC, eICU, OMOP), clinical text, signals and imaging. 30+ models, reproducible pipelines.
- healthcare
- data-mining
- deep-learning
- preprocessing
Python Streaming DataFrames for Kafka
Pipeline is an asset packaging library for Django.