Sign in

Repository Index

142,273 visible repositories in the index

Tags: ETL
Tags

A repository shows when it has any selected Tag.

Clear Tags
Filters

Filters

Clear all filters

25–48 of 66 · Most stars

MongoShake is a universal data replication platform based on MongoDB's oplog. Redundant replication and active-active replication are two most important functions. 基于mongodb oplog的集群复制工具,可以满足迁移和同步的需求,进一步实现灾备和多活功能。

  • mongo-database
  • mongodb-oplog
  • replication
  • active-active

Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.

  • analytics
  • bigquery
  • data-modeling
  • data-pipelines

High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.

  • flow
  • rust
  • rust-lang
  • stream

Open Source research tool to search, browse, analyze and explore large document collections by Semantic Search Engine and Open Source Text Mining & Text Analytics platform (Integrates ETL for document processing, OCR for images & PDF, named entity recognition for persons, organizations & locations, metadata management by thesaurus & ontologies, search user interface & search apps for fulltext search, faceted search & knowledge graph)

go-stash is a high performance, free and open source server-side data processing pipeline that ingests data from Kafka, processes it, and then sends it to ElasticSearch.

Kylo is a data lake management software platform and framework for enabling scalable enterprise-class data lakes on big data technologies such as Teradata, Apache Spark and/or Hadoop. Kylo is licensed under Apache 2.0. Contributed by Teradata Inc.

omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.

  • transform
  • etl
  • xml
  • json

MetaForge is integrated software for data transformation, integration, and visualization. Support a variety of data sources, file types, big data related database, relational database, NoSQL database, etc. Through the software can realize the management of multiple data sources, the data under the source of various operations conversion, making dat

qData is a lightweight, feature-complete, AI-native open-source data platform. It covers data integration, development, modeling, governance, assets and API services, with self-hosted deployment, flexible scaling and commercial use.

  • data-middle-platform
  • data-development
  • data-integration
  • data-modeling

ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)