Personal Data Engineering Projects
- data-lake
- ingest-data
- data-engineering
- data-warehouse
142,187 visible repositories in the index
49–61 of 61 · Most stars
Personal Data Engineering Projects
Collection of scripts corresponding to LucidProgramming YouTube tutorials
Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist
Simple but useful Python web scraping tutorial code.
A list of scrapers from around the web.
Universal Reddit Scraper - Works on any Subreddit or User
TranslatedAI coding perks navigation: a roundup of relay stations and public-service stations for Claude Code, Codex, and Cursor, organizing free quotas, sign-up bonuses, check-in activities, and evaluation records, and providing a visual dashboard and a Python data collection tool.
🍿 15K+ Company-wise LeetCode Interview Questions and System Design. Filter by company, topic, and difficulty.
A curated list of awesome packages, articles, and other cool resources from the Scrapy community.
Guide, reference and cheatsheet on web scraping using rvest, httr and Rselenium.
:mag: Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript
Open field guide to 39 CAPTCHA types: identification signals, free and open-source bypass methods, and paid alternatives. Includes a searchable web version.
An app to search startup jobs scraped from websites written in Elixir, Phoenix, React and styled-components.