Evaluate your LLM's response with Prometheus and GPT4 💯
- evaluation
- litellm
- llm
- llmops
142,273 visible repositories in the index
961–984 of 1,943 · Most stars
Evaluate your LLM's response with Prometheus and GPT4 💯
Caveman Compression is a semantic compression method for LLM contexts. It removes predictable grammar while preserving the unpredictable, factual content that defines meaning.
OpenJudge: A Unified Framework for Holistic Evaluation and Quality Rewards
A MemAgent framework that can be extrapolated to 3.5M, along with a training framework for RL training of any agent workflow.
Measuring and evolving with the frontier of agent work
Based on Nano-vLLM, a simple replication of vLLM with self-contained paged attention and flash attention implementation
Local models support for Microsoft's graphrag using ollama (llama3, mistral, gemma2 phi3)- LLM & Embedding extraction
[NeurIPS 2025 Main] SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications
The collaborative spreadsheet for AI. Chain cells into powerful pipelines, experiment with prompts and models, and evaluate LLM responses in real-time. Work together seamlessly to build and iterate on AI applications.
Evaluation tool for LLM QA chains
T3Bench: Benchmarking Current Progress in Text-to-3D Generation
WHartTest 是一款AI驱动的测试自动化平台,实现从需求到可执行测试用例的自动化生成与管理,帮助测试团队提升效率与覆盖率。 (WHartTest is an AI-driven test automation platform that automates the generation and management of executable test cases from requirements, helping testing teams improve efficiency and coverage.)
🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support
Tencent Pre-training framework in PyTorch & Pre-trained Model Zoo
An Open Source Toolkit For LLM Distillation
Tenacious tool calling built on LangGraph
[EMNLP 2024 Demo] MarkLLM: An Open-Source Toolkit for LLM Watermarking
Benchmarking Generalized Out-of-Distribution Detection
The repository host gives no description.
Arena-Hard-Auto: An automatic LLM benchmark.
sentence embedding by Smooth Inverse Frequency weighting scheme
AIConfig is a config-based framework to build generative AI applications.
AutoEvals is a tool for quickly and easily evaluating AI model outputs using best practices.
A series of math-specific large language models of our Qwen2 series.