Repository Index

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

• crawler
• crawling
• crawling-python
• playwright

AKShare is an elegant and simple financial data interface library for Python, built for human beings! 开源财经数据接口库

• futures
• financial-data
• data-science
• quant

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

• data
• pipeline
• data-analysis
• data-engineering

CKAN is an open-source DMS (data management system) for powering data hubs and data portals. CKAN makes it easy to publish, share and use data. It powers catalog.data.gov, open.canada.ca/data, data.humdata.org among many other sites.

• python
• open-data
• data
• catalog

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

• dataops
• dataops-platform
• elt
• open-source

Clean APIs for data cleaning. Python implementation of R package Janitor

• pandas
• dataframe
• data
• cleaning-data

Pandas, Polars, Spark, and Snowpark DataFrame comparison for humans and more!

• python
• pandas
• spark
• data