Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
Repository Index
47,133 repositories indexed
1–17 of 17 · Most stars
GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs
- apache-spark
- big-data
- connected-components
- dataframe
Data structure and algorithm library for go, designed to provide functions similar to C++ STL
.NET Stream Processing Library for Apache Kafka 🚀
- kafka-streams-dotnet
- kafka-streams
- kafka-streams-net
- kafka
Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.
- data-analytics
- data-cleaning
- data-cleansing
- data-engineering
RabbitMQ wire tap and swiss army knife
- rabbitmq
- tap
- wiretap
- golang
A plugin to push to Algolia based on graphQl queries
- gatsby
- algolia
- search
- static-site-generator
An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.
- archivespark
- spark-framework
- spark
- web-archiving
Hadoop Data Integration with various databases, ftp servers, salesforce. Incremental update, dedup, append, merge your data on Hadoop.
Distributed batch data processing framework
Support vector machines (SVMs) and related kernel-based learning algorithms are a well-known class of machine learning algorithms, for non-parametric classification and regression. liquidSVM is an implementation of SVMs whose key features are: fully integrated hyper-parameter selection, extreme speed on both small and large data sets, full flexibility for experts, and inclusion of a variety of different learning scenarios: multi-class classification, ROC, and Neyman-Pearson learning, and least-squares, quantile, and expectile regression.
- svm
- regression
- classification
- machine-learning
Algorand's data pipeline framework.
Wisp CLI — Interactive trading platform for backtesting and live trading across spot, perpetual futures, options, and prediction markets
A Splitable Hadoop InputFormat for Concatenated GZIP Files and *.(w)arc.gz
💬 Lightweight AlgoChat client for corvid-agent — GitHub Pages hosted
- algochat
- algorand
- blockchain
- encrypted-messaging
Partition (W)ARC Files by MIME Type and Year
- web-archiving
- webarchive
- hadoop
- warc
Implementation of Partitioning Around Medoids (PAM) in H2O-3
- clustering
- machine-learning
- h2o-3
- java