Repository Index

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

• crawler
• crawling
• crawling-python
• playwright

The 30 Days of Python programming challenge is a step-by-step guide to learn the Python programming language in 30 days. This challenge may take more than 100 days. Follow your own pace. These videos may help too: https://www.youtube.com/channel/UC7PNRuno1rzYPb1xLa4yktw

• 30-days-of-python
• python
• flask
• github

LlamaIndex is the document processing platform for AI

• agents
• application
• data
• fine-tuning

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

• python
• workflow
• data-engineering
• data-science

Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.

• llm
• pandas
• ai
• data-analysis

AKShare is an elegant and simple financial data interface library for Python, built for human beings! 开源财经数据接口库

• futures
• financial-data
• data-science
• quant

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

• data
• pipeline
• data-analysis
• data-engineering

:orange_book: 中华新华字典数据库。包括歇后语，成语，词语，汉字。

• data
• scraper
• chinese-traditional
• python3

[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.

• data
• data-cleaning
• data-pipelines
• data-processing

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️

• data
• python
• data-engineering
• data-lake

Superduper: End-to-end framework for building custom AI applications and agents.

• ai
• mlops
• torch
• transformers

CKAN is an open-source DMS (data management system) for powering data hubs and data portals. CKAN makes it easy to publish, share and use data. It powers catalog.data.gov, open.canada.ca/data, data.humdata.org among many other sites.

• python
• open-data
• data
• catalog

Mimesis is a Python library for generating fake but realistic data in multiple languages and locales.

• mimesis
• fake
• data
• generator

Deepnote is a drop-in replacement for Jupyter with an AI-first design, sleek UI, new blocks, and native data integrations. Use Python, R, and SQL locally in your favorite IDE, then scale to Deepnote cloud for real-time collaboration, Deepnote agent, and deployable data apps. https://deepnote.com/

• artificial-intelligence
• data
• data-analysis
• data-science

Colour Science for Python

• color
• color-science
• color-space
• color-spaces

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

• dataops
• dataops-platform
• elt
• open-source

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

• etl
• data-integration
• python
• postgresql

Clean APIs for data cleaning. Python implementation of R package Janitor

• pandas
• dataframe
• data
• cleaning-data

visual data prep powered by python

• data
• data-pipelines
• etl
• structured-data

Based on my own experience, I believe this roadmap provides clear answers to all the key questions about becoming a data analyst from scratch, including which technologies and programming languages to learn, what soft skills are essential, and how to start a professional career in this field.

• blogs
• businessintelligence
• courses
• data

Pandas, Polars, Spark, and Snowpark DataFrame comparison for humans and more!

• python
• pandas
• spark
• data

Main yt repository

• data
• visualization
• analysis
• python

A Python toolkit for processing tabular data

• pandas
• csv
• xml
• xlsx

No description provided by the repository host.

• hacktoberfest
• data
• datalineage
• dbt