A high-throughput and memory-efficient inference and serving engine for LLMs
Repository Index
47,133 repositories indexed
1–24 of 43 · Most stars
Turn any PDF or image document into structured data for your AI. A powerful, lightweight OCR toolkit that bridges the gap between images/PDFs and LLMs. Supports 100+ languages.
- ocr
- chineseocr
- pdf2markdown
- pp-ocr
Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.
- fine-tuning
- llama
- llms
- gemma
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private.
A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.
A modular graph-based Retrieval-Augmented Generation (RAG) system
The #1 AI Harness for Building Resumes, PDFs, Cover Letters & more, locally with 100+ LLMs support.
- text-similarity
- natural-language-processing
- word-embeddings
- python
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
- machine-learning
- ai
- ml
- mlflow
An open-source RAG-based tool for chatting with your documents.
Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.
Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)
- llm
- plm
- pre-trained-language-models
- alpaca
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).
Structured Outputs
- generative-ai
- llms
- prompt-engineering
- symbolic-ai
GUI for ChatGPT API and many LLMs. Supports agents, file-based QA, GPT finetuning and query with web search. All with a neat UI.
- chatbot
- chatgpt-api
- chatglm
- claude
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
structured outputs for llms
- openai
- python
- pydantic-v2
- openai-functions
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
ChatGLM3 series: Open Bilingual Chat LLMs | 开源双语对话语言模型
HexStrike AI MCP Agents is an advanced MCP server that lets AI agents (Claude, GPT, Copilot, etc.) autonomously run 150+ cybersecurity tools for automated pentesting, vulnerability discovery, bug bounty automation, and security research. Seamlessly bridge LLMs with real-world offensive security capabilities.
Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)
Pocket Flow: Codebase to Tutorial
Access large language models from the command-line