datajuicer / data-juicer
Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷
- Stars
- 7,241 stars
- Forks
- 451 forks
- Language
- Python
- License
- APACHE-2.0
- Created
- Aug 1, 2023
- Last push
- Nov 11, 2026
- Updated
- Nov 11, 2026
Why these Tags: an agent read the README and quoted it
- ETL from the readme:
**RAG & Analytics**: Extraction, normalization, semantic chunking, deduplication, and data profiling
An agent gave these Tags from the fixed Tag list in the AwesomeRepos source code. A code check kept each Tag only when its quote is in the text. A wrong Tag? Report an error with the form at the end of this page.
Admitted to the Repository Index on Nov 10, 2026: 7,232 stars on 2026-11-10.
Star History
188 days recorded, May 8, 2026 – Nov 11, 2026 (UTC). Net change +918 stars.
The star count of each day is the latest count minus the net stars of the later days.
Add this Star History to a README
The images follow the light or dark theme of the page that shows them, and update every hour.
Add ?theme=light or ?theme=dark
to the chart address to keep one theme.
Star History chart
[](https://awesomerepos.com/datajuicer/data-juicer) Badge
[](https://awesomerepos.com/datajuicer/data-juicer) Timeline
It gained stars on 184 of the 188 days recorded since May 8, 2026.
Its largest 7 days were Jul 4, 2026 to Jul 10, 2026: 77 stars.
In the 30 days to Nov 11, 2026 it got 112 stars, 4% more than in the 30 days before.
It had 6,323 stars before May 8, 2026 and has 7,241 stars now: 2,759 fewer than 10,000.
- Created on the Repository Host
- Largest star day recorded: +15 stars
- +14 stars in one day
- +15 stars in one day
- Release v1.5.4
- Release v1.5.5
- Release v1.6.0
- Release v1.6.1
- Release v1.6.2
- Entered the Repository Index: 7,232 stars on 2026-11-10
- Latest push
The timeline comes from the creation date and the releases on the Repository Host, the Admission, the Star History, and the Observations. Star milestones show only when the Star History records the day. The list shows up to 5 stable releases from the latest 10 releases recorded. When none of them is stable, one item counts the pre-releases. On the chart of net stars each day, a solid line marks each recorded release.
Forks and pushes
From 2 Observations, Nov 11, 2026 – Nov 12, 2026. Forks +2. AwesomeRepos saw a new push on 1 of 1 days.
- Nov 12, 2026: new push
A filled day: the last push time changed since the Observation before it.
Health facts
Measured facts in four separate dimensions, each with a date and a source. AwesomeRepos does not add them into one score. AwesomeRepos reads the weekly commits, the changelog, and the community files each week for the 5,000 visible repositories with the most stars and for each rising repository. How AwesomeRepos defines each check
Maintenance
2 of 4 checks (2 without data)-
Yes: Not archived
GitHub does not mark the repository as archived. Source
-
Yes: A push in the last 90 days
Last push on 2026-11-11, 1 day ago. Source
-
No data: Commits in the last 13 weeks
No data: AwesomeRepos has not read the weekly commits.
-
No data: Commits in 26 or more of the last 52 weeks
No data: AwesomeRepos has not read the weekly commits.
Releases
2 of 3 checks (1 without data)-
Yes: A release in the last 365 days
Newest release v1.6.2 on 2026-11-04, 8 days ago. 10 or more releases in the last 365 days. Source
-
Yes: The newest release tag is a semantic version
The newest release tag is v1.6.2. Source
-
No data: A changelog file at the root
No data: AwesomeRepos has not scanned this repository.
Security practice
0 of 1 check (1 without data)-
No data: A security policy
No data: AwesomeRepos has not scanned this repository.
AwesomeRepos has not asked OpenSSF Scorecard about this repository.
Community files
1 of 3 checks (2 without data)-
Yes: A licence
GitHub finds the licence APACHE-2.0. Source
-
No data: A contributing guide
No data: AwesomeRepos has not scanned this repository.
-
No data: A code of conduct
No data: AwesomeRepos has not scanned this repository.
Dependencies in the index
This section shows the packages that connect this repository to other repositories in the index.
The GitHub dependency graph of this repository names 165 direct packages (scanned on Nov 12, 2026). Packages from 65 other repositories in the index are among them.
No scanned repository names a package from this repository.
Uses (in the index) · 65
-
PyPI: transformers
- pytorch/pytorch 105K
PyPI: torch
-
PyPI: vllm
- astral-sh/uv 92K
PyPI: uv
-
PyPI: ultralytics
- BerriAI/litellm 63K
PyPI: litellm
- psf/requests 55K
PyPI: requests
-
PyPI: pandas
-
PyPI: streamlit
- grpc/grpc 45K
PyPI: grpcio
Show all 65
- apache/spark 44K
PyPI: pyspark
- ray-project/ray 44K
PyPI: ray
- psf/black 42K
PyPI: black
-
PyPI: timm
-
PyPI: diffusers
- explosion/spaCy 34K
PyPI: spacy
- numpy/numpy 33K
PyPI: numpy
-
PyPI: openai
- tqdm/tqdm 31K
PyPI: tqdm
- JaidedAI/EasyOCR 30K
PyPI: easyocr
-
PyPI: pydantic
-
PyPI: label-studio
-
PyPI: fire
-
PyPI: rembg
-
PyPI: mcp
- Delgan/loguru 24K
PyPI: loguru
-
PyPI: deepface
-
PyPI: matplotlib
-
PyPI: datasets
-
PyPI: funasr
- openai/tiktoken 20K
PyPI: tiktoken
- pallets/click 18K
PyPI: click
-
PyPI: networkx
- ipython/ipython 17K
PyPI: ipython
- encode/httpx 16K
PyPI: httpx
-
PyPI: pre-commit
- scipy/scipy 15K
PyPI: scipy
- nltk/nltk 15K
PyPI: nltk
-
PyPI: pytest
-
PyPI: pillow
-
PyPI: sentencepiece
-
PyPI: uvloop
- giampaolo/psutil 11K
PyPI: psutil
- wandb/wandb 11K
PyPI: wandb
-
PyPI: ffmpeg-python
-
PyPI: pdfplumber
-
PyPI: wordcloud
- BYVoid/OpenCC 10K
PyPI: opencc
-
PyPI: accelerate
- boto/boto3 9.9K
PyPI: boto3
- arogozhnikov/einops 9.6K
PyPI: einops
-
GitHub Actions: actions/checkout
-
PyPI: python-dotenv
- librosa/librosa 8.9K
PyPI: librosa
- sphinx-doc/sphinx 8.1K
PyPI: sphinx
- pypa/hatch 7.3K
GitHub Actions: pypa/hatch
-
PyPI: python-docx
- opencv/opencv-python 5.4K
PyPI: opencv-contrib-python
- rspeer/python-ftfy 4.1K
PyPI: ftfy
- PyAV-Org/PyAV 3.3K
PyPI: av
- yaml/pyyaml 3K
PyPI: pyyaml
-
PyPI: torchaudio
- dmlc/decord 2.5K
PyPI: decord
-
PyPI: audiomentations
- bmcfee/resampy 284
PyPI: resampy
Used by (in the index) · 0
None.
Sources: the GitHub dependency graph of each repository, and deps.dev for the source repository of each package (checked on Nov 12, 2026). A dependency graph also names development packages. deps.dev takes most source repositories from package metadata. How the dependency graph is made
Observations
Latest Observation, Nov 12, 2026: 7,241 stars, 451 forks, last push Nov 11, 2026, not archived. One of 2 Observations since Nov 11, 2026.