huggingface / evaluation-guidebook
Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!
- Stars
- 2,167 stars
- Forks
- 124 forks
- Language
- Jupyter Notebook
- License
- OTHER
- Created
- Oct 9, 2024
- Last push
- Dec 3, 2025
- Updated
- Nov 13, 2026
Why these Tags: an agent read the README and quoted it
- Tutorial from the readme:
It covers the different ways you can evaluate a model, guides on designing your own evaluations, and tips and tricks from practical experience.
An agent gave these Tags from the fixed Tag list in the AwesomeRepos source code. A code check kept each Tag only when its quote is in the text. A wrong Tag? Report an error with the form at the end of this page.
Admitted to the Repository Index on Nov 11, 2026: 2,167 stars on 2026-11-11.
Star History
190 days recorded, May 8, 2026 – Nov 13, 2026 (UTC). Net change +85 stars.
The star count of each day is the latest count minus the net stars of the later days.
Add this Star History to a README
The images follow the light or dark theme of the page that shows them, and update every hour.
Add ?theme=light or ?theme=dark
to the chart address to keep one theme.
Star History chart
[](https://awesomerepos.com/huggingface/evaluation-guidebook) Badge
[](https://awesomerepos.com/huggingface/evaluation-guidebook) Timeline
It gained stars on 76 of the 190 days recorded since May 8, 2026.
Its largest 7 days were Aug 9, 2026 to Aug 15, 2026: 7 stars.
In the 30 days to Nov 13, 2026 it got 17 stars, 70% more than in the 30 days before.
It had 2,082 stars before May 8, 2026 and has 2,167 stars now: 2,833 fewer than 5,000.
- Created on the Repository Host
- Latest push
- Largest star day recorded: +2 stars
- +2 stars in one day
- +2 stars in one day
- Entered the Repository Index: 2,167 stars on 2026-11-11
The timeline comes from the creation date and the releases on the Repository Host, the Admission, the Star History, and the Observations. Star milestones show only when the Star History records the day. The list shows up to 5 stable releases from the latest 10 releases recorded. When none of them is stable, one item counts the pre-releases. On the chart of net stars each day, a solid line marks each recorded release.
Forks and pushes
From 3 Observations, Nov 12, 2026 – Nov 14, 2026. Forks 0. AwesomeRepos saw a new push on 0 of 2 days.
- Nov 13, 2026: no new push
- Nov 14, 2026: no new push
A filled day: the last push time changed since the Observation before it.
Health facts
Measured facts in four separate dimensions, each with a date and a source. AwesomeRepos does not add them into one score. AwesomeRepos reads the weekly commits, the changelog, and the community files each week for the 5,000 visible repositories with the most stars and for each rising repository. How AwesomeRepos defines each check
Copies a prompt for a coding agent. It has 1 task for the maintainers of huggingface/evaluation-guidebook.
Improve the Health Facts of the GitHub repository huggingface/evaluation-guidebook. AwesomeRepos measures facts about how a repository is maintained, released, secured, and documented. Its page for this repository is https://awesomerepos.com/huggingface/evaluation-guidebook. A change to the repository can improve each of the item below. Each item gives the measured fact or score, and a task. 1. A release in the last 365 days Fact: No GitHub release (read on 2026-11-11). Rule: The newest published GitHub release (a pre-release counts; a draft does not) is 365 days old or less. A repository that publishes only tags, or only to a package registry, has no GitHub release. The release refresh reads the 10 newest releases after each push to the repository, and at least each 30 days. The index refresh reads the date of the last push each day. Task: Prepare a release, but do not publish it. Find the changes since the newest release, propose the next semantic version and the release notes, and ask me to publish the GitHub release. If nothing changed since the newest release, skip this item. If the project publishes only to a package registry, add a CI step that also makes a GitHub release for each version tag. Rules for this work: - Each file must be true for this project. Do not copy text that does not apply to it. - Do not choose a licence, a contact, or a release version for me. Ask me. - Do not publish a release, change a repository setting, or push to the default branch. Make the changes on a new branch. - Keep each change small, and run the checks of this repository before you finish. - At the end, list each item as DONE, SKIPPED (with the reason), or NEEDS A DECISION (with the question). AwesomeRepos reads these facts again in its next scan. The definitions of the checks: https://awesomerepos.com/health-facts
Maintenance
1 of 4 checks (2 without data)-
Yes: Not archived
GitHub does not mark the repository as archived. Source
-
No: A push in the last 90 days
Last push on 2025-12-03, 345 days ago. Source
-
No data: Commits in the last 13 weeks
No data: AwesomeRepos has not read the weekly commits.
-
No data: Commits in 26 or more of the last 52 weeks
No data: AwesomeRepos has not read the weekly commits.
Releases
0 of 3 checks (2 without data)-
No: A release in the last 365 days
No GitHub release (read on 2026-11-11). Source
-
No data: The newest release tag is a semantic version
No data: no release.
-
No data: A changelog file at the root
No data: AwesomeRepos has not scanned this repository.
Security practice
0 of 1 check (1 without data)-
No data: A security policy
No data: AwesomeRepos has not scanned this repository.
AwesomeRepos has not asked OpenSSF Scorecard about this repository.
Community files
1 of 3 checks (2 without data)-
Yes: A licence
GitHub finds a licence file, but cannot name the licence. Source
-
No data: A contributing guide
No data: AwesomeRepos has not scanned this repository.
-
No data: A code of conduct
No data: AwesomeRepos has not scanned this repository.
Observations
Latest Observation, Nov 14, 2026: 2,167 stars, 124 forks, last push Dec 3, 2025, not archived. One of 3 Observations since Nov 12, 2026.