SeaweedFS is a distributed storage system for object storage (S3), file systems, Iceberg table buckets, and Lance vector buckets, designed to handle billions of files with O(1) disk access and effortless horizontal scaling.
Repository Index
47,133 repositories indexed
1–10 of 10 · Most stars
Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
Ceph is a distributed object, block, and file storage platform
TranslatedBig Data Beginner's Guide :star:
- hadoop
- hdfs
- yarn
- mapreduce
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
TranslatedFocused on big data learning and interviews, the road to becoming a big data god begins. Flink/Spark/Hadoop/Hbase/Hive...
Utils for streaming large files (S3, HDFS, gzip, bz2...)
- python
- s3
- hdfs
- webhdfs
The Universal Storage Engine
- tiledb
- arrays
- storage-engine
- scientific-computing
A native go client for HDFS
- hdfs
- go
- commandline
Big Data essentials: Hadoop, MapReduce, Spark. Explore tutorials and demos in Jupyter notebooks—most are self-contained and live, ready to run with a click.