Reference BLEU implementation that auto-downloads test sets and reports a version string to facilitate cross-lab comparisons
Repository Index
142,273 visible repositories in the index
673–696 of 1,816 · Most stars
A framework for comprehensive diagnosis and optimization of agents using simulated, realistic synthetic interactions
- agent
- evaluation
- llmops
- simulator
A single interface to use and evaluate different agent frameworks
Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform
The official implementation of Self-Play Fine-Tuning (SPIN)
- deep-learning
- fine-tuning
- large-language-models
- self-play
Symfony AI is a set of components that integrate AI capabilities into PHP applications
Distributed training & inference for multimodal perception: shared embeddings across text, images, video, audio, code & graphs from 64 to 1024 dimensions 🖼️
- huggingface-transformers
- language-vision
- multimodal
- pytorch
Toy Python implementation of http://www-nlp.stanford.edu/projects/glove/
The open-source alternative to Carbon.ai. Build powerful RAG applications with any data source, at any scale.
- nextjs
- rag
- typescript
- tailwindcss
TranslatedAI Cross-Border E-commerce Operations Agent Toolkit · Breaks operations SOPs down into 7 reusable AI Agent workflows, including prompts, templates, and real-world cases
Simple shell script to use OpenAI's ChatGPT and DALL-E from the terminal. No Python or JS required.
- chatbot
- chatgpt
- chatgpt-api
- chatgpt-api-wrapper
GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation
- ai4science
- data-generation
- llm-training
- sft-data
This repository contains a collection of the best system prompts for ChatGPT, a conversational AI model developed by OpenAI. Star this repository to help us reach 5,000 stars!
Langtrace 🔍 is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorDBs and more.. Integrate using Typescript, Python. 🚀💻📊
Independently authored prompt templates for AI coding agents — system prompts, tool prompts, agent delegation, memory management, and multi-agent coordination. Informed by studying Claude Code.
- ai-coding-agent
- ai-tools
- anthropic-claude
- claude-code
The open-source AI engine for AMD Strix Halo. Text, embeddings, speech, image and video models in one binary, with OpenAI-compatible APIs, caching for coding agents and model extensions. Linux and Windows.
- amd
- llm
- strix-halo
- gfx1151
PDEBench: An Extensive Benchmark for Scientific Machine Learning
🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.
Chinese safety prompts for evaluating and improving the safety of LLMs. 中文安全prompts,用于评估和提升大模型的安全性。
A Self-adaptation Framework🐙 that adapts LLMs for unseen tasks in real-time!
Paper accepted by CIKM 2024. Codes of GongBU, a LLM fine-tuning platform for domain-specific adaptation.
A non-saturating, open-ended environment for evaluating LLMs in Factorio
Open-source tool to visualise your RAG 🔮
Elixir implementation of a LangChain style framework that lets Elixir projects integrate with and leverage LLMs.