Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
Repository Index
47,133 repositories indexed
121–144 of 1,080 · Most stars
Machine Learning Engineering Open Book
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
Turns Data and AI algorithms into production-ready web applications in no time.
The LLM Evaluation Framework
- evaluation-metrics
- evaluation-framework
- llm-evaluation
- llm-evaluation-framework
Open source UI framework written in Python, running on Windows, Linux, macOS, Android and iOS
The interactive graphing library for Python :sparkles:
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
- python
- luigi
- orchestration-framework
- scheduling
TranslatedProduction-grade platform for building agentic IM bots - Agent, knowledge base orchestration, plugin system / Bots for Discord / Slack / LINE / Telegram / WeChat(企业微信 (Qiye Weixin, "Enterprise WeChat"), 企微智能机器人 (Qiwei Zhineng Jiqiren, "Enterprise WeChat smart bot"), 公众号 (Gongzhonghao, "official account")) / 飞书 (Feishu, "flying book") / 钉钉 (Dingding, "nail nail") / QQ / Matrix e.g. Integrated with ChatGPT(GPT), DeepSeek, Dify, n8n, Langflow, Coze, Claude, Gemini, GLM, Ollama, SiliconFlow, Kimi, openclaw / hermes agent, deerflow
A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
- machine-translation
- speaker-recognition
- asr
- tts
ARIS ⚔️ (Auto-Research-In-Sleep) — Lightweight Markdown-only skills for autonomous ML research: cross-model review loops, idea discovery, and experiment automation. No framework, no lock-in — works with Claude Code, Codex, OpenClaw, or any LLM agent.
- ai-research
- autonomous-agent
- claude-code
- codex
Accelerate your web app development | Build fast. Run fast.
- python
- framework
- asyncio
- api-server
Code samples for my book "Neural Networks and Deep Learning"
🐫 CAMEL: The first and the best multi-agent framework. Finding the Scaling Law of Agents. https://www.camel-ai.org
<⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.
- agi
- ai
- autonomous-agents
- python
Apprise - Push Notifications that work with just about every platform!
- notifications
- notification-hub
- notification-api
- python
PyTorch implementations of Generative Adversarial Networks.
Memori is agent-native memory infrastructure. A LLM-agnostic layer that turns agent execution and conversation into structured, persistent state for production systems. Built for enterprise, Memori works with the data infrastructure you already run, no rip-and-replace, and deploys across managed cloud, single-tenant cloud, VPC, and on-premises.
Network Analysis in Python
- python
- complex-networks
- graph-theory
- graph-algorithms
Agent framework and applications built upon Qwen>=3.0, featuring Function Calling, MCP, Code Interpreter, RAG, Chrome extension, etc.
A curated list of practical Codex skills for automating workflows across the Codex CLI and API.
Open Source framework for voice agents, multimodal apps, and realtime AI. Maintained by Daily and the community.
- ai
- real-time
- voice
- voice-assistant
Translated🏛️ 三省六部制 (Sansheng Liubu Zhi, "Three Departments and Six Ministries system") · OpenClaw Multi-Agent Orchestration System — 9 specialized AI agents with real-time dashboard, model config, and full audit trails
- ai-orchestration
- dashboard
- llm
- multi-agent
A curated list of Artificial Intelligence (AI) courses, books, video lectures and papers.
- machine-learning
- machine-intelligence
- artificial-intelligence
- reinforcement-learning