TranslatedFocused on big data learning and interviews, the road to becoming a big data god begins. Flink/Spark/Hadoop/Hbase/Hive...
Repository Index
47,133 repositories indexed
25–48 of 207 · Most stars
A simple, beautiful, and embeddable JavaScript Markdown editor. Delightful editing for beginners and experts alike. Features built-in autosaving and spell checking.
Python SQL Parser and Transpiler
- transpiler
- sql
- python
- parser
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
List of 128 languages for Laravel Framework, Laravel Jetstream, Laravel Fortify, Laravel Breeze, Laravel Cashier, Laravel Nova and Laravel UI.
H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
the portable Python dataframe library
Upserts, Deletes And Incremental Processing on Big Data.
- hudi
- apachehudi
- datalake
- bigdata
macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.
▁▂▃▅▂▇ in your shell.
lakeFS - Data version control for your data lake | Git for data
- data-engineering
- data-versioning
- go
- object-storage
Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark
Share and collaborate by syncing with any Git repository instantly. Linux, macOS, and Windows
- sync
- git
- version-control
- self-hosted
Mechanize is a ruby library that makes automated web interaction easy.
- scraping
- web
- ruby
State of the Art Natural Language Processing
A better compressed bitset in Java: used by Apache Spark, Netflix Atlas, Apache Pinot, Tablesaw, and many others
- java
- bitset
- roaring-bitmaps
- roaringbitmap
The Hunting ELK
- hunting
- elasticsearch
- kibana
- logstash
TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
- dataquality
- spark
- unit-testing
- scala
Apache Linkis builds a computation middleware layer to facilitate connection, governance and orchestration between the upper applications and the underlying data engines.
Koalas: pandas API on Apache Spark
DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
- workflow
- governance
- azkaban
- davinci
Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.
REST job server for Apache Spark