Sign in

Repository Index

47,133 repositories indexed

Filters

Filters

Clear all filters

1–24 of 134 · Most stars

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

  • batch-processing
  • kafka
  • pathway
  • python

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Apache Spark - A unified analytics engine for large-scale data processing

  • python
  • scala
  • r
  • java

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. Stored as pure Python. All in a modern, AI-native editor.

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

  • python
  • luigi
  • orchestration-framework
  • scheduling

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learn...

The most widely used Python to C compiler

  • python
  • cython
  • cpython
  • cpython-extensions