🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Repository Index
47,133 repositories indexed
1–11 of 11 · Most stars
LlamaIndex is the document processing platform for AI
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
A curated list of awesome big data frameworks, ressources and other awesomeness.
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
Superduper: End-to-end framework for building custom AI applications and agents.
A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
Data processing & ETL framework for Ruby
An extensible framework for linking databases and interactive views.
Implementations of the repository pattern for .NET to support the domain-driven modeling