SeaweedFS is a distributed storage system for object storage (S3), file systems, Iceberg table buckets, and Lance vector buckets, designed to handle billions of files with O(1) disk access and effortless horizontal scaling.
Repository Index
95,842 visible repositories in the index
1–14 of 14 · Most stars
Data Agent Ready Warehouse : One for Analytics, Search, AI, Python Sandbox. — rebuilt from scratch. Unified architecture on your S3.
- rust
- database
- serverless
- bigdata
Apache Iceberg
- iceberg
- apache
- hacktoberfest
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
Open Lakehouse Format for Multimodal AI. Convert from Parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..
- machine-learning
- computer-vision
- data-format
- deep-learning
Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
A native Rust library for Delta Lake, with bindings into Python
- delta
- rust
- delta-lake
- databricks
pg_lake: Postgres with Iceberg and data lake access
Nessie: Transactional Catalog for Data Lakes with Git-like semantics
- data
- spark
- java
- iceberg
Apache Iceberg REST Catalog in Rust — access control, credential vending and audit for every engine and AI agent. Apache 2.0.
Apache Iceberg
- apache
- hacktoberfest
- iceberg
- rust
Apache XTable (incubating) is a cross-table converter for lakehouse table formats that facilitates interoperability across data processing systems and query engines.
- apache-hudi
- apache-iceberg
- delta-lake
Open Lakehouse Compute Fabric: an open-source lakehouse platform with Microsoft Fabric Compute Extension.
- apache-iceberg
- apache-spark
- data-engineering
- delta-lake