SubQuery is an Open, Flexible, Fast and Universal data indexing framework for web3. Our mission is to help developers create the decentralised products of the future.
Repository Index
47,133 repositories indexed
1–24 of 32 · Most stars
An orchestration platform for the development, production, and observation of data assets.
- data-pipelines
- dagster
- workflow
- data-science
Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.
- deep-learning
- document-parsing
- machine-learning
- nlp
Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.
- change-data-capture
- kafka-connect
- apache-kafka
- debezium
Open-Source Web UI for Apache Kafka Management
- kafka-ui
- kafka-brokers
- kafka
- kafka-streams
Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
- kedro
- hacktoberfest
- mlops
- python
The leader in Customer Data Infrastructure
- snowplow
- analytics
- data
- data-pipeline
Generic command line non-JVM Apache Kafka producer and consumer
ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.
- mlops
- machine-learning
- data-science
- production-ready
Franz is a free messaging app for services like WhatsApp, Slack, Messenger and many more.
Maxwell's daemon, a mysql-to-json kafka producer
Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.
- machine-learning
- ml
- model-validation
- data-validation
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
- dataops
- dataops-platform
- elt
- open-source
Project Flogo is an open source ecosystem of opinionated event-driven capabilities to simplify building efficient & modern serverless functions, microservices & edge apps.
- flogo
- iot
- integration
- golang
Ruby and Rails efficient Kafka processing framework
- karafka-framework
- kafka
- kafka-client
- kafka-clients
The Apache Kafka C/C++ library
- kafka
- kafka-consumer
- apache-kafka
- high-performance
Polo travels through your database and creates sample snapshots so you can work with real world data in development.
- database
- snapshot
- productivity
- activerecord
pyspark methods to enhance developer productivity 📣 👯 🎉
An open source, a high-performance hybrid integration platform that allows developers quick integration with any application, data, or system.
- integration
- middleware
- wso2
- esb
Modular, Production-Ready, Open-Source EHR
- ehr
- electronic-health-record
- fhir
- health
A CLI tool for Kafka
High performance Kafka consumer for InfluxDB. Supports collectd message formats.
- kafka-influxdb
- kafka-consumer
- logstash
- influxdb
An opinionated Elixir wrapper around brod, the Erlang Kafka client, that supports encrypted connections to Heroku Kafka out of the box.
- elixir
- kafka
Real-time decision features without streaming infra. Turn live events into product reflexes — no Kafka, no Flink, no feature store.
- analytics
- feature-store
- fraud-detection
- python