Sign in

Repository Index

132,241 visible repositories in the index

Tags: ETL
Tags

A repository shows when it has any selected Tag.

Clear Tags
Filters

Filters

Clear all filters

25–48 of 56 · Most stars

A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.

  • data
  • ingestion
  • replication
  • management

LarkMidTable 是一站式开源的数据中台,实现中台的 基础建设,数据治理,数据开发,监控告警,数据服务,数据的可视化,实现高效赋能数据前台并提供数据服务的产品。

  • flink
  • flinkx
  • seatunnel
  • datax

Dozer is a real-time data movement tool that leverages CDC from various sources and moves data into various sinks.

  • apis
  • data
  • rust
  • sql

Open Source research tool to search, browse, analyze and explore large document collections by Semantic Search Engine and Open Source Text Mining & Text Analytics platform (Integrates ETL for document processing, OCR for images & PDF, named entity recognition for persons, organizations & locations, metadata management by thesaurus & ontologies, search user interface & search apps for fulltext search, faceted search & knowledge graph)

Python Fast Dataflow programming framework for Data pipeline work( Web Crawler,Machine Learning,Quantitative Trading.etc)

Intake is a lightweight package for finding, investigating, loading and disseminating data.

  • python
  • data-access
  • data-catalog

omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.

  • transform
  • etl
  • xml
  • json

MetaForge is integrated software for data transformation, integration, and visualization. Support a variety of data sources, file types, big data related database, relational database, NoSQL database, etc. Through the software can realize the management of multiple data sources, the data under the source of various operations conversion, making dat

:lipstick: Durable and asynchronous data imports for consuming data at scale and publishing testable SDKs.

  • porter
  • data-import
  • framework
  • data-transformation

Download, unpack from a ZIP/TAR/GZ/BZ2 archive, parse, correct, convert units and import Google Spreadsheets, XLS, ODS, XML, CSV, HTML, etc. into your ActiveRecord models. Uses RemoteTable gem internally.

A decision-model-based ontology & knowledge-graph builder — builds governed knowledge graphs from your documents, every fact gated and traceable to its evidence.

  • graphrag
  • information-extraction
  • knowledge-graph
  • llm