A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
- etl
- data-integration
- python
- postgresql
Repository Index
78,351 visible repositories in the index
145–168 of 295 · Most stars
Automatically generate a RESTful API service for your legacy database. No code required!
- python
- sqlalchemy
- database
- rest-api
Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.
- data
- data-engineering
- data-engineering-pipeline
- etl-pipeline
A repository that contains models, datasets, and fine-tuning techniques for DB-GPT, with the purpose of enhancing model performance in Text-to-SQL
Full-stack AI app generator — FastAPI + Next.js with AI Agents, RAG, streaming, auth, and 20+ integrations out of the box.
A fast, user friendly ORM and query builder which supports asyncio.
- piccolo
- asyncio
- python
- python3
A complete, structured Quran database — verses, translations and metadata, free for any project.
- mysql
- quran
- gait
NucleiFuzzer is a robust automation tool that efficiently detects web application vulnerabilities, including XSS, SQLi, SSRF, and Open Redirects, leveraging advanced scanning and URL enumeration techniques
- nuclei
- nuclei-templates
- fuzzing-templates
- nucleifuzzer
QuantMind（量化大脑）开源版是一款面向个人开发者与投研团队的 AI 原生多市场量化交易平台。深度集成微软 Qlib、RD-Agent 因子演化与 QuantBot全能工作台，提供从 300+ 维因子挖掘、13 种机器学习与深度学习模型工场、Qlib 高性能回测、截面批量推理、7x24 实时舆情情绪分析，到通达信深度联动（板块推送/预警雷达/闪电下单）与实盘模拟交易的完整闭环。支持 A股、港股、美股、期货及区块链。可免费商用，系统采用 Docker Compose 一键私有化部署，数据与模型完全本地化，保障策略隐私，功能零门槛无限制。
- docker
- lightgbm
- python3
- qlib
The Future of Data Engineering — A CLI SQL client for the modern data stack, enabling AI-native context engineering for data.
Django and MongoDB database connector
:rocket: CLI tool for FastAPI. Generating new FastAPI projects & boilerplates made easy.
Django tenants using PostgreSQL Schemas
Implementation of X/Twitter v1, v2, and GraphQL APIs
- api
- automation
- client
- scrape
aiomysql is a library for accessing a MySQL database from the asyncio
- asyncio
- mysql
- mariadb
- sqlalchemy
Docker 基本教學 - 從無到有 Docker-Beginners-Guide 教你用 Docker 建立 Django + PostgreSQL 📝
Highly available elephant herd: HA PostgreSQL cluster using Docker
- python
- docker
- docker-image
- postgresql
mobilenetv3 with pytorch，provide pre-train model
A Python based web application scanner to gather OSINT and fuzz for OWASP vulnerabilities on a target website.
- web
- application
- scanner
- osint
Learning Python: from Beginner to Master. https://lqlab.readthedocs.io
Search and download public exploits from the Vulners database — online, or fully offline from a local SQLite FTS5 index.
AI on the way. An auto deep learning pipe dream. An RDBMS approach to deep learning. Declarative, explainable, scalable, optimizable, easy to deploy, all that good stuff.
YouTube Full Text Search - Search all of YouTube from the command line
- chromadb
- full-text-search
- semantic-search
- youtube
More than 2000+ Data engineer interview questions.