An open-source, code-first Python toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.
Repository Index
47,133 repositories indexed
25–48 of 218 · Most stars
Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
The LLM Evaluation Framework
- evaluation-metrics
- evaluation-framework
- llm-evaluation
- llm-evaluation-framework
State-of-the-Art Embeddings, Retrieval, and Reranking
SkillOpt is a text-space optimizer that trains reusable natural-language skills for frozen LLM agents through trajectory-driven edits, validation-gated updates, and deployable best_skill.md artifacts.
- agent-skills
- self-evolving-agents
Private AI platform for agents, assistants and enterprise search. Built-in Agent Builder, Deep research, Document analysis, Multi-model support, and API connectivity for agents.
Topic Modelling for Humans
- gensim
- topic-modeling
- information-retrieval
- machine-learning
Supercharge Your LLM Application Evaluations 🚀
- llm
- llmops
- evaluation
Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view
- agentneo
- agents
- ai-performance-optimization
- llm-testing
Research and development (R&D) is crucial for the enhancement of industrial productivity, especially in the AI era, where the core aspects of R&D are mainly focused on data and models. We are committed to automating these high-value generic R&D processes through R&D-Agent, which lets AI drive data-driven AI. 🔗https://aka.ms/RD-Agent-Tech-Report
- agent
- ai
- automation
- data-mining
Unified framework for building enterprise RAG pipelines with small, specialized models
- parsing
- retrieval-augmented-generation
- agents
- generative-ai-tools
A framework for few-shot evaluation of language models.
- evaluation-framework
- language-model
- transformer
structured outputs for llms
- openai
- python
- pydantic-v2
- openai-functions
[MLsys2026 Best Paper]: https://arxiv.org/abs/2506.08276. RAG on Everything with LEANN. Enjoy 97% storage savings while running a fast, accurate, and 100% private RAG application on your personal device.
💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows
- python
- search
- nlp
- semantic-search
Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)
Agent S: the state-of-the-art open source computer use agents
- agent-computer-interface
- ai-agents
- computer-automation
- gui-agents
🏄 Scalable embedding, reasoning, ranking for images and sentences with CLIP
- bert
- sentence-encoding
- deep-learning
- clip-model
Retrieval and Retrieval-augmented LLMs
- embeddings
- information-retrieval
- llm
- sentence-embeddings
AI Observability & Evaluation
- llmops
- ai-monitoring
- ai-observability
- llm-eval
BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include: GenAI workflow, RAG, Agent, Unified model management, Evaluation, SFT, Dataset Management, Enterprise-level System Management, Observability and more.
Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.
- weak-supervision
- data-cleaning
- data-quality
- noisy-labels
Pocket Flow: 100-line LLM framework. Let Agents build Agents!
- agents
- flow-based-programming
- flow-engineering
- llm-agent
ValueCell is a community-driven, multi-agent platform for financial applications.
- agentic-ai
- agents
- ai
- assitant