A high-throughput and memory-efficient inference and serving engine for LLMs
Repository Index
57,138 visible repositories in the index
1–24 of 94 · Most stars
Turn any PDF or image document into structured data for your AI. A powerful, lightweight OCR toolkit that bridges the gap between images/PDFs and LLMs. Supports 100+ languages.
- ocr
- chineseocr
- pdf2markdown
- pp-ocr
Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
Wrap Antigravity, ChatGPT Codex, Claude Code, Grok Build, Muse Code, Devin as an OpenAI/Gemini/Claude/Codex compatible API service, allowing you to enjoy the free Gemini Series, GPT Series, Grok Series, Claude model through API
Convert PDF to markdown + JSON quickly with high accuracy
[EMNLP2025] LightRAG: Simple and Fast Retrieval-Augmented Generation
- knowledge-graph
- large-language-models
- retrieval-augmented-generation
- genai
SGLang is a high-performance serving framework for large language models and multimodal models.
DeepInfer: High-performance, large-scale inference framework for DeepSeek models
Open source observability platform for logs, metrics, traces, RUM (web, android, ios), Session replay, pipelines, SLO and LLM observability. A sophisticated, simple and highly performant alternative to Datadog, Splunk, and Elasticsearch with 140x lower storage costs and single binary deployment.
gpt-oss-120b and gpt-oss-20b are two open-weight language models by OpenAI
SkillOpt is a text-space optimizer that trains reusable natural-language skills for frozen LLM agents through trajectory-driven edits, validation-gated updates, and deployable best_skill.md artifacts.
- agent-skills
- self-evolving-agents
The absolute trainer to light up AI agents.
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.
PyMuPDF is a high performance Python library for data extraction, analysis, conversion & manipulation of PDF (and other) documents.
- mupdf
- xps
- pdf-documents
- epub
An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)
High-speed Large Language Model Serving for Local Deployment
Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.
High accuracy RAG for answering questions from scientific documents with citations
Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets and MCP Code Mode. Works with any SDK or agent and adds under 200µs of overhead at 5k RPS.
- gateway
- llm-gateway
- guardrails
- mcp-gateway
Lightweight (7MB) terminal-first AI-native development environment
❯ Lightweight, beautiful and user-friendly interactive prompts
- prompt
- prompts
- cli
- command-line
An app that brings language models directly to your phone.