TranslatedA Codex jailbreak prompt and test pack for gpt.
Repository Index
132,241 visible repositories in the index
1–24 of 49 · Most stars
Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
- data-drift
- jupyter-notebook
- pandas-dataframe
- machine-learning
A full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.
This repo powers my experiment where ChatGPT manages a real-money micro-cap stock portfolio.
🧩 FrontierAgent, our agent framework, open-sourced alongside it — native command-line TUI, ReAct and Agent Team modes, one command on macOS and Linux, no preinstall, no hard Docker dependency.
- agent-orchestration
- ai-agents
- harness
- agentic-ai
Open-source AI red teaming toolkit from Microsoft. Test generative AI systems with automated scans, adaptive multi-turn attacks, and a web UI.
- ai-red-team
- generative-ai
- red-team-tools
- responsible-ai
Align Anything: Training All-modality Model with Feedback
- large-language-models
- multimodal
- rlhf
- chameleon
Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
- agentic-ai
- evaluation
- agent-evaluation
- agent-security
Evaluation and Tracking for LLM Experiments and AI Agents
- machine-learning
- neural-networks
- explainable-ml
- llmops
LLM-as-a-Verifier is a general-purpose framework that provides fine-grained feedback for any agent without requiring additional training. It achieves SOTA performance across coding, robotics, and medical agentic benchmarks.
Same agent, Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.
- observability
- llmops
- ai-observability
- openai
Formal verification for AI: a Lean 4 + Dafny proof checker that catches fake AI proofs (reward hacking), and verified recursive self-improvement (RSI) for coding agents. AI for math, physics and software engineering.
- proof-assistant
- lean4
- ai-agents
- ai-for-math
Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
- evaluation
- evaluation-framework
- evaluation-metrics
- huggingface
Build tested agent skills and govern their lifecycle through a user-defined marketplace: evidence, discovery, updates, rollback, quarantine, and 17-platform distribution.
Doing simple retrieval from LLM models at various context lengths to measure accuracy
UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.
- machine-learning
- experimentation
- llm-prompting
- llm-test
Beyond Accuracy: Behavioral Testing of NLP models with CheckList
:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI
👾 nao is the #1 open source agentic BI. (1) Create context with nao-core cli, (2) deploy nao chat interface for everyone
- agentic-analytics
- analytics
- analytics-engineering
- bigquery
Benchmark for tracking model capability after release.
Measuring Massive Multitask Language Understanding | ICLR 2021
- muti-task
- transfer-learning
- gpt-3
- few-shot-learning
An alignment auditing agent capable of quickly exploring alignment hypothesis
Advbox is a toolbox to generate adversarial examples that fool neural networks in PaddlePaddle、PyTorch、Caffe2、MxNet、Keras、TensorFlow and Advbox can benchmark the robustness of machine learning models. Advbox give a command line tool to generate adversarial examples with Zero-Coding.
- adversarial-examples
- paddlepaddle
- machine-learning
- security
The MATH Dataset (NeurIPS 2021)
- math
- reasoning
- logic