A high-throughput and memory-efficient inference and serving engine for LLMs
Repository Index
55,140 visible repositories in the index
1–24 of 27 · Most stars
Convert PDF to markdown + JSON quickly with high accuracy
SGLang is a high-performance serving framework for large language models and multimodal models.
DeepInfer: High-performance, large-scale inference framework for DeepSeek models
Open source observability platform for logs, metrics, traces, RUM (web, android, ios), Session replay, pipelines, SLO and LLM observability. A sophisticated, simple and highly performant alternative to Datadog, Splunk, and Elasticsearch with 140x lower storage costs and single binary deployment.
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.
PyMuPDF is a high performance Python library for data extraction, analysis, conversion & manipulation of PDF (and other) documents.
- mupdf
- xps
- pdf-documents
- epub
An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)
High-speed Large Language Model Serving for Local Deployment
High accuracy RAG for answering questions from scientific documents with citations
Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets and MCP Code Mode. Works with any SDK or agent and adds under 200µs of overhead at 5k RPS.
- gateway
- llm-gateway
- guardrails
- mcp-gateway
💼 An enterprise-grade Next.js boilerplate for high-performance, maintainable apps. Packed with features like Tailwind CSS, TypeScript, ESLint, Prettier, testing tools, and more to accelerate your development.
AI suite powered by state-of-the-art models and providing advanced AI/AGI functions. Includes AI personas, AGI functions, world-class Beam multi-model chats, text-to-image, voice, response streaming, code highlighting and execution, PDF import, presets for developers, much more. Deploy on-prem or in the cloud.
Venom is a high-performance system developed with JavaScript to create a bot for WhatsApp, support for creating any interaction, such as customer service, media sending, sentence recognition based on artificial intelligence and all types of design architecture for WhatsApp.
Apache Maka (Incubating) is a high-performance agent workspace that keeps a complete record of everything it did.
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
Enterprise-grade AI agent framework with multi-provider LLM management, secure knowledge bases and high-precision RAG, visual workflow orchestration, and multi-agent coordination.
TranslatedA high-performance, 100% client-side tool for removing Gemini AI image & video watermarks. Built with pure JavaScript using mathematically precise Reverse Alpha Blending. / A JavaScript-based, purely browser-side lossless watermark removal tool for Gemini AI images and videos, using a mathematically precise reverse Alpha blending algorithm
- gemini
- gemini-watermark-remover
- skills
- userscript
[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.
A high-performance inference engine for AI models
🦎 A lightweight, streaming-native, multi-protocol gateway for AI, APIs, and real-time data. Route LLM requests across providers, govern MCP tools, and connect Kafka, MQTT, and HTTP through one high-performance engine with shared security and observability.
A production-grade, high-efficiency Agentic RL framework for frontier LLMs with validated recipes.
Youtu-Parsing: Perception, Structuring and Recognition via High-Parallelism Decoding && Youtu-Parsing-Omni: One Encoder, One Schema, All Modalities