Trending repositories

Ranked by Attention Score: stars gained over the last 180 complete days

Period

Sort order

Tag: Data pipelines and messaging

Tag

Search Tags

All Tags
Top-level Tags
AI agents
Audio and video
Automation, bots, and scraping
Backend and APIs
Blockchain
Cloud and DevOps
CMS and websites
Command line
Data pipelines and messaging
Data science
Databases
Design, fonts, and icons
Desktop apps
Developer tools
Finance and trading
Frontend and UI
Games
Graphics and 3D
Hardware and IoT
Lists and learning
LLM tools
Machine learning
Maps and geospatial
Mobile
Monitoring and observability
Neovim and editors
Networking
Operating systems and systems
Productivity and documents
Programming languages and compilers
Science and math
Security
Self-hosted
Testing
Finer Tags
2D graphics libraryGraphics and 3D
3D modeling toolGraphics and 3D
3D printingHardware and IoT
3D rendering libraryGraphics and 3D
AccountingFinance and trading
Ad blockerNetworking
Admin dashboard templateFrontend and UI
Agent frameworkAI agents
Agent memoryAI agents
Agent sandboxAI agents
Agent skillAI agents
AI code assistantLLM tools
AlertingMonitoring and observability
Algorithms collectionLists and learning
Analytical databaseDatabases
Android appMobile
Android libraryMobile
Android moddingOperating systems and systems
Android UI componentMobile
Angular libraryFrontend and UI
Animation libraryFrontend and UI
API client appDeveloper tools
API gatewayBackend and APIs
API testingTesting
App launcherDesktop apps
Application performance monitoringMonitoring and observability
Arduino libraryHardware and IoT
Audio libraryAudio and video
Authentication librarySecurity
Authorization librarySecurity
AutoMLMachine learning
Awesome listLists and learning
AWS toolCloud and DevOps
Backend as a serviceBackend and APIs
BacktestingFinance and trading
Backup toolSelf-hosted
BDD testingTesting
BenchmarkingTesting
BioinformaticsScience and math
BitcoinBlockchain
BitTorrentNetworking
Blockchain nodeBlockchain
Blogging platformCMS and websites
BookLists and learning
Bookmark managerProductivity and documents
Browser agentAI agents
Browser automationAutomation, bots, and scraping
Browser extensionProductivity and documents
Build systemDeveloper tools
Build your ownLists and learning
Bundler pluginFrontend and UI
Business intelligenceData science
CacheDatabases
CalendarProductivity and documents
CAPTCHA solverAutomation, bots, and scraping
Career guideLists and learning
Censorship circumventionNetworking
Change data captureData pipelines and messaging
Chart libraryData science
Chat botAutomation, bots, and scraping
Cheat sheetLists and learning
CI/CDCloud and DevOps
Classical machine learning libraryMachine learning
CLI frameworkCommand line
Code examplesLists and learning
Code formatterDeveloper tools
Code review toolDeveloper tools
Code searchDeveloper tools
Coding agentAI agents
Coding agent managerAI agents
Coding agent pluginAI agents
Coding challenge solutionsLists and learning
Color themeDesign, fonts, and icons
Color toolDesign, fonts, and icons
ComfyUI extensionMachine learning
CompilerProgramming languages and compilers
Computational chemistryScience and math
Computational geometryScience and math
Computer vision libraryMachine learning
Concurrency libraryProgramming languages and compilers
Configuration managementCloud and DevOps
Container image toolCloud and DevOps
Container runtimeCloud and DevOps
Content management systemCMS and websites
CourseLists and learning
Creative codingGraphics and 3D
Cron schedulerBackend and APIs
Cross-platform mobile frameworkMobile
Cryptography librarySecurity
CSS frameworkFrontend and UI
CTFSecurity
Data app frameworkData science
Data labelingMachine learning
Data orchestrationData pipelines and messaging
Data qualityData pipelines and messaging
Database backupDatabases
Database clientDatabases
Database driverDatabases
Database migrationDatabases
Dataframe libraryData science
DebuggerDeveloper tools
Deep learning frameworkMachine learning
Deep research agentAI agents
Dependency managementDeveloper tools
Design patternsLists and learning
Design resourceDesign, fonts, and icons
Desktop app frameworkDesktop apps
Device driverOperating systems and systems
Device firmwareHardware and IoT
Diagramming toolProductivity and documents
Digital forensicsSecurity
Distributed computingData pipelines and messaging
Distributed trainingMachine learning
DNSNetworking
Document databaseDatabases
Document parsing for LLMsLLM tools
Documentation generatorDeveloper tools
Documentation site generatorCMS and websites
DotfilesCommand line
Download managerNetworking
Drawing appGraphics and 3D
Durable executionBackend and APIs
E-book toolProductivity and documents
E-commerce platformCMS and websites
eBPF toolOperating systems and systems
Editor color schemeNeovim and editors
Electronics designHardware and IoT
Embedded databaseDatabases
EmbeddingsLLM tools
EmojiDesign, fonts, and icons
EmulatorGames
Encryption toolSecurity
End-to-end testingTesting
ERPFinance and trading
ESP32Hardware and IoT
Ethereum and EVMBlockchain
ETLData pipelines and messaging
Event streamingData pipelines and messaging
Experiment trackingMachine learning
Exploit proof of conceptSecurity
Face recognitionMachine learning
Fake data generatorTesting
Feature flagBackend and APIs
File managerCommand line
File syncSelf-hosted
File systemOperating systems and systems
File transferNetworking
FirewallNetworking
Flutter packageMobile
FontDesign, fonts, and icons
Form libraryFrontend and UI
Free resources listLists and learning
Frontend frameworkFrontend and UI
Frontend meta-frameworkFrontend and UI
Frontend routerFrontend and UI
Functional programming libraryProgramming languages and compilers
FuzzingTesting
Fuzzy finderCommand line
Game development toolGames
Game engineGames
Game frameworkGames
Game launcherGames
Game modGames
Game source portGames
GISMaps and geospatial
Git clientDeveloper tools
Git extensionDeveloper tools
GitHub ActionCloud and DevOps
Godot addonGames
GPU computeGraphics and 3D
Graph databaseDatabases
Graphics API wrapperGraphics and 3D
GraphQL serverBackend and APIs
GUI toolkitDesktop apps
Home Assistant integrationSelf-hosted
Home automationSelf-hosted
Homelab dashboardSelf-hosted
HoneypotSecurity
HTTP clientBackend and APIs
HTTP serverBackend and APIs
Icon setDesign, fonts, and icons
IDENeovim and editors
Identity providerSecurity
Image editorGraphics and 3D
Image generationMachine learning
Image generation UIGraphics and 3D
Image processing libraryGraphics and 3D
Image segmentationMachine learning
Infrastructure as codeCloud and DevOps
Infrastructure monitoringMonitoring and observability
Instrumentation libraryMonitoring and observability
Internationalization libraryFrontend and UI
InterpreterProgramming languages and compilers
Interview preparationLists and learning
Intrusion detectionSecurity
Invoicing and billingFinance and trading
iOS appMobile
iOS libraryMobile
JavaScript runtimeProgramming languages and compilers
Jetpack Compose libraryMobile
Job queueBackend and APIs
jQuery pluginFrontend and UI
JSON processorCommand line
Kernel moduleOperating systems and systems
Key-value storeDatabases
Kubernetes distributionCloud and DevOps
Kubernetes operatorCloud and DevOps
Kubernetes toolCloud and DevOps
Language bindingsProgramming languages and compilers
Language serverDeveloper tools
LaTeX toolProductivity and documents
Linear algebraScience and math
LinterDeveloper tools
Linux desktop appDesktop apps
Linux distributionOperating systems and systems
Live streaming serverAudio and video
LLM chat interfaceLLM tools
LLM evaluationLLM tools
LLM fine-tuningLLM tools
LLM gatewayLLM tools
LLM inference serverLLM tools
LLM observabilityLLM tools
LLM SDKLLM tools
Load balancerNetworking
Load testingTesting
Local LLM runnerLLM tools
Log viewerMonitoring and observability
Logging libraryMonitoring and observability
Machine learning datasetMachine learning
macOS appDesktop apps
Mail serverSelf-hosted
Malware analysisSecurity
Map appMaps and geospatial
Map libraryMaps and geospatial
Markdown editorProductivity and documents
Markdown parserProductivity and documents
Market dataFinance and trading
MCP frameworkAI agents
MCP serverAI agents
Media automationSelf-hosted
Media serverSelf-hosted
Menu bar appDesktop apps
Message brokerData pipelines and messaging
Microservice frameworkBackend and APIs
MinecraftGames
Mobile OS modificationMobile
MockingTesting
Model quantizationMachine learning
MQTTHardware and IoT
Multi-agent orchestrationAI agents
Multiplayer game networkingGames
Music playerAudio and video
Music productionAudio and video
Neovim configurationNeovim and editors
Neovim pluginNeovim and editors
Network scannerNetworking
NixOperating systems and systems
NLP libraryMachine learning
Note-taking appProductivity and documents
NotebookData science
Notification serviceBackend and APIs
Numerical computingScience and math
Object detectionMachine learning
Object storageCloud and DevOps
Observability platformMonitoring and observability
OCRMachine learning
Office document libraryProductivity and documents
Office suiteProductivity and documents
On-device inferenceMachine learning
OpenAPI toolBackend and APIs
OpenStreetMapMaps and geospatial
Operating systemOperating systems and systems
Optimization solverScience and math
ORMDatabases
OSINTSecurity
Package managerDeveloper tools
Packet analyzerNetworking
Paper implementationMachine learning
Parser generatorProgramming languages and compilers
Parser libraryProgramming languages and compilers
Password managerSecurity
Payment processingFinance and trading
PDF libraryProductivity and documents
PDF toolProductivity and documents
Peer-to-peerNetworking
Penetration testingSecurity
Personal AI assistantAI agents
Personal financeFinance and trading
Photo managerGraphics and 3D
Physics engineGames
Physics simulationScience and math
Platform as a serviceCloud and DevOps
Playable gameGames
PostgreSQL extensionDatabases
PowerShell moduleCommand line
Presentation toolProductivity and documents
Product analyticsData science
ProfilerDeveloper tools
Programming exercisesLists and learning
Programming languageProgramming languages and compilers
Project management toolProductivity and documents
Prompt collectionLLM tools
Prompt toolingLLM tools
Quantitative financeFinance and trading
Query builderDatabases
RAG frameworkLLM tools
Raspberry PiHardware and IoT
React componentFrontend and UI
React Native libraryMobile
Real-time communicationAudio and video
Recommender systemMachine learning
Reinforcement learningMachine learning
Relational databaseDatabases
Remote desktopNetworking
Reverse engineeringSecurity
Reverse proxyNetworking
Rich text editorFrontend and UI
RoadmapLists and learning
RoboticsHardware and IoT
ROS packageHardware and IoT
RPA and desktop automationAutomation, bots, and scraping
RPC frameworkBackend and APIs
RSS readerProductivity and documents
Screen recorderAudio and video
Screenshot toolDesktop apps
Search engineDatabases
Secrets managementSecurity
Secrets scannerSecurity
Self-hosting platformSelf-hosted
Serialization libraryBackend and APIs
ServerlessCloud and DevOps
ShaderGraphics and 3D
ShellCommand line
Shell pluginCommand line
Shell promptCommand line
Social media botAutomation, bots, and scraping
Speech recognitionMachine learning
SSH toolNetworking
State managementFrontend and UI
Static analysisDeveloper tools
Static site generatorCMS and websites
Static site themeCMS and websites
Statistics libraryData science
Stream processingData pipelines and messaging
Structured outputLLM tools
Study notesLists and learning
Style guideLists and learning
Svelte libraryFrontend and UI
SVG toolGraphics and 3D
SwiftUI componentMobile
Sync engineDatabases
Syntax highlighterNeovim and editors
System designLists and learning
System monitorOperating systems and systems
System tweakerOperating systems and systems
Telegram botAutomation, bots, and scraping
Telemetry collectorMonitoring and observability
Terminal emulatorCommand line
Terminal multiplexerCommand line
Terminal output libraryCommand line
Test frameworkTesting
Text editorNeovim and editors
Text to speechMachine learning
Threat intelligenceSecurity
Time series analysisData science
Time-series databaseDatabases
TLS certificate toolSecurity
To-do appProductivity and documents
Trading botFinance and trading
TranspilerProgramming languages and compilers
TUI frameworkCommand line
TunnelNetworking
TutorialLists and learning
Type checkerProgramming languages and compilers
UI component libraryFrontend and UI
UIKit componentMobile
Unity packageGames
Uptime monitorMonitoring and observability
UserscriptProductivity and documents
Validation libraryBackend and APIs
Vector databaseDatabases
Version managerDeveloper tools
Video downloaderAudio and video
Video editingAudio and video
Video generationMachine learning
Video libraryAudio and video
Video playerAudio and video
Vim pluginNeovim and editors
VirtualizationOperating systems and systems
VPNNetworking
VS Code extensionNeovim and editors
Vue libraryFrontend and UI
Vulnerability scannerSecurity
WallpaperDesign, fonts, and icons
Web browserDesktop apps
Web componentFrontend and UI
Web frameworkBackend and APIs
Web scraperAutomation, bots, and scraping
Web3 client libraryBlockchain
WebAssembly runtimeProgramming languages and compilers
WebAssembly toolchainProgramming languages and compilers
WebSocket libraryBackend and APIs
WeChat botAutomation, bots, and scraping
Wiki softwareCMS and websites
Window managerOperating systems and systems
Windows appDesktop apps
Workflow automationAutomation, bots, and scraping
Workflow engineBackend and APIs
XAML UI libraryDesktop apps

No Tag matches.

Filters (1)

Apply

Filters

Primary language

License

Total stars

Last push

Created

Archived

Clear all filters

22 repositories by Attention Score · May 8, 2026 – Nov 3, 2026 (UTC)

Language: Scala
Tag: Data pipelines and messaging
Clear all

1.apache / spark

Apache Spark - A unified analytics engine for large-scale data processing

python
scala
r
java
big-data
jdbc
+2 more

Scala APACHE-2.044K total30K forks

+1,398 stars

from 43,008 · score 7.24

2.delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

spark
acid
big-data
analytics
delta-lake

Scala APACHE-2.09.1K total2.2K forks

+339 stars

from 8,727 · score 5.83

3.softwaremill / elasticmq

In-memory message queue with an Amazon SQS-compatible interface. Runs stand-alone or embedded.

scala
sqs-interface
elasticmq
amazon-sqs
aws
aws-sqs
+1 more

Scala APACHE-2.03K total201 forks

+119 stars

from 2,834 · score 4.79

4.apache / kyuubi

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

spark
hive
sql
thrift
jdbc
spark-sql
+4 more

Scala APACHE-2.02.4K total1K forks

+66 stars

from 2,310 · score 4.20

5.awslabs / deequ

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.

dataquality
spark
unit-testing
scala

Scala APACHE-2.03.7K total585 forks

+61 stars

from 3,596 · score 4.13

6.snowplow / snowplow

The leader in Customer Data Infrastructure

snowplow
analytics
data
data-pipeline
data-collection
product-analytics
+3 more

Scala APACHE-2.07K total1.2K forks

+60 stars

from 6,988 · score 4.11

7.graphframes / graphframes

GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs

apache-spark
big-data
connected-components
dataframe
dataframes
graphs
+8 more

Scala APACHE-2.01.2K total275 forks

+43 stars

from 1,169 · score 3.78

8.yahoo / CMAK

CMAK is a tool for managing Apache Kafka clusters

kafka
scala
cluster-management
big-data

Scala APACHE-2.012K total2.5K forks

+39 stars

from 11,879 · score 3.69

9.TouK / nussknacker

Low-code tool for automating actions on real time data | Stream processing for the users.

flink
flink-kafka
apache-flink
gui
touk
real-time
+13 more

Scala APACHE-2.0745 total104 forks

+23 stars

from 722 · score 3.18

10.typelevel / fs2

Compositional, streaming I/O library for Scala

scala
stream
streaming-computation
fp
functional-programming

Scala Other license2.5K total633 forks

+20 stars

from 2,436 · score 3.04

11.apache / livy

Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.

livy
bigdata
spark
apachelivy

Scala APACHE-2.0961 total623 forks

+18 stars

from 943 · score 2.94

12.bigdatagenomics / adam

ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.

spark
big-data
bioinformatics
genomics
parquet
avro
+4 more

Scala APACHE-2.01.1K total311 forks

+17 stars

from 1,043 · score 2.89

13.apache / cassandra-spark-connector

Apache Spark to Apache Cassandra connector

cassandra
scala
spark

Scala APACHE-2.02K total931 forks

+12 stars

from 1,943 · score 2.56

14.linkedin / isolation-forest

A distributed Spark/Scala implementation of the isolation forest and extended isolation forest algorithms for unsupervised outlier detection, featuring support for scalable training and ONNX export for easy cross-platform inference.

isolation-forest
spark
linkedin
outlier-detection
unsupervised-learning
machine-learning
+3 more

Scala Other license265 total54 forks

+10 stars

from 255 · score 2.40

15.h2oai / sparkling-water

Sparkling Water provides H2O functionality inside Spark cluster

h2o
spark
machine-learning
integration
pysparkling
rsparkling
+3 more

Scala APACHE-2.0982 total357 forks

+6 stars

from 976 · score 1.95

16.neo4j / neo4j-spark-connector

Neo4j Connector for Apache Spark, which provides bi-directional read/write access to Neo4j from Spark, using the Spark DataSource APIs

spark
neo4j-connector
cypher
neo4j-driver
bolt
hacktoberfest

Scala APACHE-2.0324 total120 forks

+6 stars

from 318 · score 1.95

17.helgeho / ArchiveSpark

An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.

archivespark
spark-framework
spark
web-archiving
webarchive
internet-archive
+1 more

Scala MIT164 total19 forks

+5 stars

from 159 · score 1.79

18.archivesunleashed / aut

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

spark
hadoop
webarchives
analysis
apache-spark
scala
+8 more

Scala APACHE-2.0159 total34 forks

+4 stars

from 155 · score 1.61

19.Hydrospheredata / mist

Serverless proxy for Spark cluster

apache-spark
api
big-data
serverless

Scala APACHE-2.0325 total69 forks

+2 stars

from 323 · score 1.10

20.twitter / summingbirdArchived

Streaming MapReduce with Scalding and Storm

Scala APACHE-2.02.1K total255 forks

+1 stars

from 2,120 · score 0.69

21.cloudera / livy

Livy is an open source REST interface for interacting with Apache Spark from anywhere

Scala No license1K total311 forks

+1 stars

from 1,005 · score 0.69

22.pawelkaczor / akka-ddd

Akka CQRS/ES framework

ddd
event-sourcing
cqrs
microservices
akka

Scala MIT360 total58 forks

+1 stars

from 359 · score 0.69

Order: Attention Score

Most net stars gained in the window first. The score only puts this gain on a log scale, so the order is the order of the gain.

How the ranking works

Ranked by stars gained in the window. The score is the natural log of one plus that gain, so a repository's rank is exactly its star gain's rank.

score = log1p(net stars gained in the trailing T complete UTC days)

A repository is ranked only when its Star History covers every day of the window and its latest Observation is not missing.

The longest window is 180 days. Star History from the Repository Host goes back at most 180 days before the first daily record that AwesomeRepos made for the repository. A longer window needs more of these daily records first. A repository younger than the window does not rank in it.

The filters select from the same eligible repositories. A repository matches the topics when it has any of them. Total stars is the count now. Last push and creation on the Repository Host count back from the start of today (UTC). The counts beside the languages, licenses, and topics include all ranked repositories in the window. A Tag holds the repositories that have it. See Tags.

Repositories enter the index from the starting dataset of the index, or by gaining stars faster than the published admission threshold, whatever their size. A repository stays in the index after it enters. Thus an archived repository can still rank.

Browse the Repository Index