Repository Index

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

• crawler
• crawling
• crawling-python
• playwright

LlamaIndex is the document processing platform for AI

• agents
• application
• data
• fine-tuning

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

• python
• workflow
• data-engineering
• data-science

Superduper: End-to-end framework for building custom AI applications and agents.

• ai
• mlops
• torch
• transformers

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

• etl
• data-integration
• python
• postgresql