A LiDAR odometry pipeline that just works
- 3d-mapping
- icp
- lidar-slam
- slam
Repository Index
47,133 repositories indexed
97–120 of 299 · Most stars
HTML processing filters and utilities
Translated【A common used C++ & Python DAG framework】 A general-purpose, third-party-dependency-free, cross-platform parallel computing framework based on flow graphs, listed in awesome-cpp. Welcome to star & fork & discuss
The fastest unified streaming SQL Engine in a single C++ binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.
- clickhouse
- kakfa
- sql
- redpanda
A lightweight stream processing library for Go
CLI task management & automation tool
- python
- build-tool
- build-automation
- task-runner
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
- etl
- data-integration
- python
- postgresql
Elyra extends JupyterLab with an AI centric approach.
- notebooks
- notebook-jupyter
- jupyterlab
- jupyterlab-extensions
A full spaCy pipeline and models for scientific/biomedical documents.
- scientific-documents
- spacy
- custom-pipes
- nlp
Azure Pipelines Agent 🚀
Composable concurrency toolkit for Go
- concurrency
- channels
- go
- golang
Go HardWare discovery/inspection library
- hardware
- discovery
- golang-hardware-discovery
- inspection-library
MongoDB data stream pipeline tools by YouGov (adopted from MongoDB)
Twitter Image Pipeline is a robust and performant image loading and caching framework for iOS clients
- image-pipeline
- ios
- swift
- objective-c
Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.
- analytics
- bigquery
- data-modeling
- data-pipelines
A categorized collection of FFmpeg commands for video automation pipelines.
- automation
- cheatsheet
- ffmpeg
- ffprobe
Pipe your nix-build output through the nix-output-monitor a.k.a nom to get additional information while building.
- nix
- haskell
- hacktoberfest
- nom
A deep learning toolkit for healthcare AI: EHR (MIMIC, eICU, OMOP), clinical text, signals and imaging. 30+ models, reproducible pipelines.
- healthcare
- data-mining
- deep-learning
- preprocessing
🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.
- data-engineering
- reverse-etl
- data-pipeline
- data-activation
Cookbook for Node on FHIR and the clinical Meteor.js build pipeline. Create FDA, HIPPA, and HL7 compliant Javascript/Node applications!
Python Streaming DataFrames for Kafka
Fastest end-to-end CSV ingestion for Ruby (with C acceleration). SmarterCSV auto-detects formats, applies smart defaults, and returns Rails-ready hashes for seamless use with ActiveRecord, Sidekiq, parallel jobs, and S3 pipelines — even for messy user-uploaded real-world data.
- csv
- csv-converter
- csv-export
- csv-file
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
- data-engineering
- data-integration
- data-pipeline
- data-quality
MLeap: Deploy ML Pipelines to Production
- scikit-learn
- spark
- data-pipelines
- transformers