Post-training with Tinker
Repository Index
132,241 visible repositories in the index
1–24 of 33 · Most stars
A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.
Evaluation and Tracking for LLM Experiments and AI Agents
- machine-learning
- neural-networks
- explainable-ml
- llmops
[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
- agent
- artificial-intelligence
- benchmark
- multimodal
Formal verification for AI: a Lean 4 + Dafny proof checker that catches fake AI proofs (reward hacking), and verified recursive self-improvement (RSI) for coding agents. AI for math, physics and software engineering.
- proof-assistant
- lean4
- ai-agents
- ai-for-math
Benchmarking large language models' complex reasoning ability with chain-of-thought prompting
Repository for benchmarking graph neural networks (JMLR 2023)
- graph-representation-learning
- graph-neural-networks
- benchmark-framework
- graph-deep-learning
ADR secures enterprise AI agents through observability, security benchmarking, and threat detection. Deployed at Uber.
An on-premises, OCR-free unstructured data extraction, markdown conversion and benchmarking toolkit. (https://idp-leaderboard.org/)
- document
- document-analysis
- extraction
- llms
Benchmark for tracking model capability after release.
A powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers identify and mitigate potential jailbreaks in their LLM APIs.
[ACL 2024 🔥] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted for spatiotemporal video representation. We also introduce a rigorous 'Quantitative Evaluation Benchmarking' for video-based conversational models.
Llama-3 agents that can browse the web by following instructions and talking to you
PinchBench is a benchmarking system for evaluating LLM models as OpenClaw coding agents. Made with 🦀 by the humans at https://kilo.ai
A Toolbox for Adversarial Robustness Research
- pytorch
- adversarial-examples
- adversarial-example
- adversarial-perturbations
Stop your AI from making things up — it proposes, deterministic tools decide, every claim checked against ground truth with evidence. Grounded facts and context survive resets. Reverse engineering is the proving ground. MCP server + CLI.
- ai-agents
- developer-tools
- binary-analysis
- ctf
character-policy: Official Benchmark & Codebase for Multimodal 3D Character & Holistic Co-Speech Gesture Policy
- co-speech-gesture
- talking-face-generation
- 3d-human-motion
- audio-driven-animation
CommerceAgentBench: Benchmarking Long-Horizon Agents in High-Fidelity, Stateful, and Reproducible Replicas of Real Online Services
This repository allows reproduction of Poetiq's record-breaking submission to the ARC-AGI-1 and ARC-AGI-2 benchmarks.
HOTA (and other) evaluation metrics for Multi-Object Tracking (MOT).
Benchmarking Chat Assistants on Long-Term Interactive Memory (ICLR 2025)
A generalized information-seeking agent system with Large Language Models (LLMs).
- agi
- autogpt
- autonomous-agents
- chatgpt
T3Bench: Benchmarking Current Progress in Text-to-3D Generation
- 3d
- text-to-3d
- diffusion
- nerf
[EMNLP 2024 Demo] MarkLLM: An Open-Source Toolkit for LLM Watermarking
- llm
- toolkit
- watermark
- large-language-models