Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
- airflow
- apache
- apache-airflow
- python
Repository Index
67,262 visible repositories in the index
73–96 of 5,415 · Most stars
Learn how to design systems at scale and prepare for system design interviews
- architecture
- distributed-systems
- system-design
- system-design-interview
Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼
👩💻👨💻 Awesome cheatsheets for popular programming languages, frameworks and development tools. They include everything you should know in one single file.
- cheatsheets
- javascript
- bash
- nodejs
Milvus is a high-performance, cloud-native vector database built for scalable vector ANN search
- anns
- nearest-neighbor-search
- faiss
- vector-search
Streamlit — A faster way to build and share data apps.
免费大模型API，支持免费调用GPT、DeepSeek等主流模型，免费额度10000点，每日刷新！另付费价格最低官方1-2折！
This is a repo with links to everything you'd ever want to learn about data engineering
A unified trading API with more than 100 crypto exchanges and prediction markets in JavaScript / TypeScript / Python / C# / PHP / Go / Java / Rust
TranslatedA visual no-code/code-free web crawler/spider. 易采集 (EasySpider, "easy collection"): a visual browser automation testing/data collection/web crawler software that lets you design and execute crawler tasks graphically without code. Alias: ServiceWrapper, an intelligent service wrapping system for Web applications.
✨ Innovative and open-source visualization application that transforms various data formats, such as JSON, YAML, XML and CSV into interactive graphs.
- json
- tool
- react
- visualization
Apache Spark - A unified analytics engine for large-scale data processing
- python
- scala
- r
- java
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Build and share delightful machine learning apps, all in Python. 🌟 Star to support our work!
- machine-learning
- models
- ui
- ui-components
Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
A command line tool and library for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, MQTTS, POP3, POP3S, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET, TFTP, WS and WSS. libcurl offers a myriad of powerful features
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
DuckDB is a high-performance analytical database system
Translated微舆 (Weiyu, "micro public opinion"): a multi-Agent public opinion analysis assistant that anyone can use. It breaks information cocoons, restores the original picture of public opinion, predicts future trends, and assists decision-making! Implemented from scratch, without relying on any framework.
- agent-framework
- data-analysis
- multi-agent-system
- nlp
Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
- context-database
- agentic-rag
- agent-memory
- self-evolving
Making large AI models cheaper, faster and more accessible
- deep-learning
- hpc
- large-scale
- data-parallelism
Platform to build admin panels, internal tools, and dashboards. Integrates with 25+ databases and any API.
- low-code
- admin-dashboard
- app-builder
- internal-tools
Easily and securely send things from one computer to another :crocodile: :package:
- file-sharing
- tcp
- golang
- peer-to-peer
Convert PDF to markdown + JSON quickly with high accuracy