AI Observability & Evaluation
- llmops
- ai-monitoring
- ai-observability
- llm-eval
Repository Index
47,133 repositories indexed
1–21 of 21 · Most stars
BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include: GenAI workflow, RAG, Agent, Unified model management, Evaluation, SFT, Dataset Management, Enterprise-level System Management, Observability and more.
Tool for real-time visualization, monitoring and collaborative analysis of AI/ML experiments and live data. Supports Python, PyTorch/Torch, NumPy, TensorFlow/Keras https://visdom.dev
StreamAlert is a serverless, realtime data analysis framework which empowers you to ingest, analyze, and alert on data from any environment, using datasources and alerting logic you define.
- security
- kinesis
- serverless
- terraform
A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.
- ai
- fairness-ai
- fairness
- fairness-testing
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
- ai-gateway
- observability
- simulation
- ai-agents
Krawl is a customizable, lightweight, cloud-native web deception server and anti-crawler that creates fake web applications with low-hanging vulnerabilities using realistic, randomly generated decoy data and AI-generated HTML templates.
- anti-crawling
- cloud-native
- crawler
- deception
PostgreSQL Remote Control
- postgresql
- database
- rdbms
- management
Tool that extracts and provides metrics on your PostgreSQL database
Get a detailed, real-time view of your PostgreSQL database and system metrics
Data validation toolkit for assessing and monitoring data quality.
- data-validation
- tabular-data
- data-testing
- data-quality
Maze Applied Reinforcement Learning Framework
CLI utility and Python module for analyzing log files and other data.
- log-analysis
- log-parser
- cli
- security
Dreamento (DReam ENgenieering TOolbox): a Python-based software for dream engineering while monitoring/analyzing real-time EEG data.
- automatic-sleep-analysis
- dream
- eeg-analysis
- lucid-dreaming
Python script to monitor Oracle Databases
- python
- monitoring
- oracle-database
- zabbix
Graphite InfluxDB backend. InfluxDB storage finder / plugin for Graphite API.
Realistic synthetic events for testing, demos, and pipelines — streamed live or generated in bulk
- clickhouse
- data-engineering
- data-generation
- devops
Network probing tool crafted for datacenters (but not only)
- datacenter
- network
- probing
- scapy
Nova DSO Tracker is an open-source Flask application for astrophotographers, providing real-time tracking of deep-sky objects' altitude, observability prediction, and integration with Stellarium and SIMBAD databases.
A detective for your data. Zero-config data quality monitoring — works with dbt, Postgres, BigQuery, Snowflake. No YAML.
- anomaly-detection
- bigquery
- cli
- data-engineering
Monitoring OpenStreetMap with Datadog