Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.
Repository Index
105,102 visible repositories in the index
1–9 of 9 · Most stars
Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.
💎多语言实现,高性能生成唯一数字ID。 💎优化的雪花算法(SnowFlake)——雪花漂移算法,在缩短ID长度的同时,具备极高瞬时并发处理能力(50W/0.1s)。 💎原生支持 C#/Java/Go/Rust/C/JavaScript/TypeScript/Python/Pascal 多语言,提供其它适用于其它语言的多线程安全调用动态库(FFI)。💎支持容器环境自动扩容(自动注册 WorkerId ),单机或分布式唯一IdGenerator。💎顶尖优化,超强效能。
- snowflake
- idgenerator
- dotnet
- rust
IPython Parallel: Interactive Parallel Computing in Python
AI based entity resolution, identity resolution, master data management and deduplication at scale for humans and agents
- fuzzymatch
- fuzzy-matching
- deduplication
- dedupe
Cloud-native genomic dataframes and batch computing
- genetics
- vcf
- genomics
- gwas
Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
- hpc
- workflow
- workflow-automation
- workflow-management
Python MapReduce library written in Cython. Visit us in #hadoopy on freenode. See the link below for documentation and tutorials.
Serverless ML Framework