Repository Index

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

• bloom
• deep-learning
• distributed-systems
• language-models

Modin: Scale your Pandas workflows by changing a single line of code

• dataframe
• pandas
• distributed
• datascience

Build, Manage and Deploy AI/ML Systems

• machine-learning
• model-management
• ai
• ml

《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验

• accelerator
• ai-infra
• ai-infrastructure
• book

Python Stream Processing

• kafka-streams
• kafka
• python
• asyncio

Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.

• asyncio
• kafka
• python
• rabbitmq

Accelerated deep learning R&D

• deep-learning
• reinforcement-learning
• machine-learning
• computer-vision

Python Stream Processing. A Faust fork

• asyncio
• python-streaming
• kafka
• redis

A library for event sourcing in Python.

• domain-driven-design
• ddd
• eventsourcing
• cqrs

MLBox is a powerful Automated Machine Learning python library.

• machine-learning
• auto-ml
• kaggle
• deep-learning

YAS3FS (Yet Another S3-backed File System) is a Filesystem in Userspace (FUSE) interface to Amazon S3. It was inspired by s3fs but rewritten from scratch to implement a distributed cache synchronized by Amazon SNS notifications. A web console is provided to easily monitor the nodes of a cluster.

InterPlanetary Wayback: A distributed and persistent archive replay system using IPFS

• ipfs
• warc
• wayback
• web-archiving

Reinforcement learning library(framework) designed for PyTorch, implements DQN, DDPG, A2C, PPO, SAC, MADDPG, A3C, APEX, IMPALA ...

• reinforcement-learning
• deep-learning
• pytorch
• pytorch-reinforcement-learning

A web front-end for password cracking and analytics

• python
• hashcat
• web
• flask

Maze Applied Reinforcement Learning Framework

• reinforcement-learning
• python
• machine-learning
• simulation

Distributed Honeypot

• distributed
• honeypot
• deception
• security

Stream Processing Made Easy

• machine-learning
• reactive-programming
• kafka
• reactive-systems

Conflict-free merge for DataFrames, JSON, ML models & distributed agents — powered by CRDTs. The first merge library where every operation is mathematically guaranteed to converge.

• collaborative-editing
• conflict-resolution
• crdt
• data-engineering