Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
Repository Index
47,133 repositories indexed
25–48 of 57 · Most stars
TranslatedChinese LLaMA & Alpaca LLMs + local CPU/GPU training and deployment
- llm
- plm
- pre-trained-language-models
- alpaca
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).
Structured Outputs
- generative-ai
- llms
- prompt-engineering
- symbolic-ai
GUI for ChatGPT API and many LLMs. Supports agents, file-based QA, GPT finetuning and query with web search. All with a neat UI.
- chatbot
- chatgpt-api
- chatglm
- claude
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
structured outputs for llms
- openai
- python
- pydantic-v2
- openai-functions
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
TranslatedChatGLM3 series: Open Bilingual Chat LLMs
HexStrike AI MCP Agents is an advanced MCP server that lets AI agents (Claude, GPT, Copilot, etc.) autonomously run 150+ cybersecurity tools for automated pentesting, vulnerability discovery, bug bounty automation, and security research. Seamlessly bridge LLMs with real-world offensive security capabilities.
Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)
Pocket Flow: Codebase to Tutorial
Access large language models from the command-line
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
- llm
- llmops
- model-inference
- fine-tuning
Retrieval and Retrieval-augmented LLMs
- embeddings
- information-retrieval
- llm
- sentence-embeddings
The official GitHub page for the survey paper "A Survey of Large Language Models".
- chain-of-thought
- chatgpt
- in-context-learning
- instruction-tuning
AI Observability & Evaluation
- llmops
- ai-monitoring
- ai-observability
- llm-eval
Low-code framework for building custom LLMs, neural networks, and other AI models
An Open-Source Asynchronous Coding Agent
Agent Reinforcement Trainer: train multi-step agents for real-world tasks using GRPO. Give your agents on-the-job training. Reinforcement learning for Qwen3.6, GPT-OSS, Llama, and more!
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
- academia
- arxiv
- brave
- deep-research
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
- data
- data-cleaning
- data-pipelines
- data-processing
Semantic cache for LLMs. Fully integrated with LangChain and llama_index.