Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Repository Index
47,133 repositories indexed
25–48 of 100 · Most stars
Translated🎭 277 plug-and-play AI expert roles — supporting 20 tools such as Claude Code/Cursor/Copilot, covering 20 departments including engineering/design/marketing/finance. Includes 64 original agents for the Chinese market (Xiaohongshu/Douyin/WeChat/Feishu/DingTalk/Qt host computer software/mechanical design). Paired with the orchestrator agency-orchestrator, a single sentence lets multiple experts collaborate automatically as a DAG.
How Python does AI. Agents, realtime voice, image generation, embeddings. Every model, every interface, typed end to end.
- agent-framework
- pydantic
- python
- genai
Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
The LLM Evaluation Framework
- evaluation-metrics
- evaluation-framework
- llm-evaluation
- llm-evaluation-framework
The Generative UI framework
Topic Modelling for Humans
- gensim
- topic-modeling
- information-retrieval
- machine-learning
A 100% free modern JS SaaS boilerplate (React, NodeJS, Prisma). Full-featured: Auth (email, google, github, slack, MS), Email sending, Background jobs, Landing page, Payments (Stripe, Polar.sh), Shadcn UI, S3 file upload. AI-ready with tailored AGENTS.md, skills, and Claude Code plugin. One cmd deploy. Powered by Wasp full-stack framework.
- full-stack
- open-source
- saas
- nodejs
Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view
- agentneo
- agents
- ai-performance-optimization
- llm-testing
Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.
- deep-learning
- document-parsing
- machine-learning
- nlp
Unified framework for building enterprise RAG pipelines with small, specialized models
- parsing
- retrieval-augmented-generation
- agents
- generative-ai-tools
A framework for few-shot evaluation of language models.
- evaluation-framework
- language-model
- transformer
A very simple framework for state-of-the-art Natural Language Processing (NLP)
LangChain4j is an idiomatic, open-source Java library for building LLM-powered applications on the JVM. It offers a unified API over popular LLM providers and vector stores, and makes implementing tool calling (including MCP support), agents and RAG easy. It integrates seamlessly with enterprise Java frameworks like Quarkus and Spring Boot.
💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows
- python
- search
- nlp
- semantic-search
TranslatedLangGPT: Empowering everyone to become a prompt expert! 🚀 📌 Proposer of Structured Prompt 📌 Initiator of Meta-Prompt 📌 The most popular paradigm for putting prompts into practice | Language of GPT The pioneering framework for structured & meta-prompt design 10,000+ ⭐ | Battle-tested by thousands of users worldwide Created by 云中江树 (Yunzhong Jiangshu, "river and trees amid the clouds")
BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include: GenAI workflow, RAG, Agent, Unified model management, Evaluation, SFT, Dataset Management, Enterprise-level System Management, Observability and more.
Low-code framework for building custom LLMs, neural networks, and other AI models
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
- avx2
- c99
- cpu-inference
- deep-learning
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
🌌 A complete search engine and RAG pipeline in your browser, server or edge network with support for full-text, vector, and hybrid search in less than 2kb.
- data-structures
- full-text
- search
- typo-tolerance
Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
- data-drift
- jupyter-notebook
- pandas-dataframe
- machine-learning
TranslatedOpen-source manuscript of 《深入理解 AI Infra：量化分析与系统设计》 (In-Depth Understanding of AI Infra: Quantitative Analysis and System Design), by 李博杰 (Li Bojie): starting from hardware constraints and model architecture, it quantitatively derives the system design of LLM inference and training. Includes the full text of the book, a PDF, and companion calculation tools and experiments
- accelerator
- ai-infra
- ai-infrastructure
- book