Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
Repository Index
47,133 repositories indexed
1–12 of 12 · Most stars
☁️ Build multimodal AI applications with cloud-native stack
- neural-search
- cloud-native
- deep-learning
- machine-learning
Turns Data and AI algorithms into production-ready web applications in no time.
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
- python
- luigi
- orchestration-framework
- scheduling
GUI for ChatGPT API and many LLMs. Supports agents, file-based QA, GPT finetuning and query with web search. All with a neat UI.
- chatbot
- chatgpt-api
- chatglm
- claude
🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.
- python
- utilities
- cache
- data-structures
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
CLI task management & automation tool
- python
- build-tool
- build-automation
- task-runner
Python Streaming DataFrames for Kafka
pyspark methods to enhance developer productivity 📣 👯 🎉
Build and deploy a serverless data pipeline on AWS with no effort.
- aws-cdk
- data-pipeline
- stepfunctions
- glue
Pipeline de ingestão e busca semântica de documentos para RAG — com rate limiting, circuit breaker e fila resiliente.