LLaMA-Omni is a low-latency and high-quality end-to-end speech interaction model built upon Llama-3.1-8B-Instruct, aiming to achieve speech capabilities at the GPT-4o level.
[EMNLP 2023 Demo] Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
Implementation for MatMul-free LM.
Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).
AI Search & RAG Without Moving Your Data. Get instant answers from your company's knowledge across 100+ apps while keeping data secure. Deploy in minutes, not months.
InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions
- chatgpt
- visual-language-learning
- multi-modality
- foundation
Large-scale, Informative, and Diverse Multi-round Chat Data (and Models)
A 13B large language model developed by Baichuan Intelligent Technology
Official Implementation of "Graph of Thoughts: Solving Elaborate Problems with Large Language Models"
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime
- low-precision
- pruning
- sparsity
- auto-tuning
[ICLR 2024] Official implementation of " 🦙 Time-LLM: Time Series Forecasting by Reprogramming Large Language Models"
Project Page for "LISA: Reasoning Segmentation via Large Language Model"
Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)
[EMNLP-2024] Build multimodal language agents for fast prototype and production
The Cradle framework is a first attempt at General Computer Control (GCC). Cradle supports agents to ace any computer task by enabling strong reasoning abilities, self-improvment, and skill curation, in a standardized general environment with minimal requirements.
- ai-agent
- ai-agents-framework
- computer-control
- cradle
Awesome Reasoning LLM Tutorial/Survey/Guide
Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3 (NeurIPS'25).
✨✨[NeurIPS 2025] VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction
- large-multimodal-models
- multimodal-large-language-models
- omni-language-model
- omni-modal-video-understanding
mPLUG-Owl: The Powerful Multi-modal Large Language Model Family
Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.
A snappy, keyboard-centric terminal user interface for interacting with large language models. Chat with ChatGPT, Claude, Llama 3, Phi 3, Mistral, Gemma and more.
A simple, performant, and scalable Jax LLM!
verl-agent is an extension of veRL, designed for training LLM/VLM agents via RL. verl-agent is also the official code for paper "Group-in-Group Policy Optimization for LLM Agent Training"
mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding
- chart-understanding
- document-understanding
- mllm
- multimodal