GitHub-проекты

A high-throughput and low-latency inference and serving framework for LLMs and VLs

Inference for text-embeddings in Python

Inference for text-embeddings in Rust, HFOIL Licence.

MII makes low-latency and high-throughput inference, similar to vLLM powered by DeepSpeed.

Fine-tune, serve, deploy, and monitor any open-source LLMs in production. Used in production at BentoML for LLMs-based applications.

Run LLMs and batch jobs on any cloud. Get maximum cost savings, highest GPU availability, and managed execution -- all with a simple interface.

Blazingly fast LLM inference.

A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

NVIDIA Framework for LLM Inference(Transitioned to TensorRT-LLM)

Nvidia Framework for LLM Inference

️ GigaChat 3 Lightning в GGUF: максимум скорости для локального запуска Мы подготовили три официальные сборки модели.

Open-source framework for fine-tuning and evaluating LLMs. It simplifies the process of experimenting with different training configurations and makes it easy to reproduce and share results, supporting features like LoRA, QLoRA, DeepSpeed, PEFT, and multi-GPU setups.

An Easy-to-use, Scalable and High-performance RLHF Framework (70B+ PPO Full Tuning & Iterative DPO & LoRA & RingAttention & RFT).

A library for accelerating Transformer model training on NVIDIA GPUs.

An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.

A simple, performant and scalable Jax LLM!

Mesh TensorFlow: Model Parallelism Made Easier.

Efficient Training for Big Models.

veRL is a flexible and efficient RL framework for LLMs.

An Efficient and User-Friendly Scaling Library for Reinforcement Learning with Large Language Models.

A Native-PyTorch Library for LLM Fine-tuning.

DeepSpeed version of NVIDIA's Megatron-LM that adds additional support for several features such as MoE model training, Curriculum Learning, 3D Parallelism, and others.

A native PyTorch Library for large model training.

Ongoing research training transformer models at scale.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Minimalistic large language model 3D-parallelism training.

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

a framework that helps you evaluate your Retrieval Augmented Generation (RAG) pipelines.

Testing & evaluation library for LLM applications, in particular RAGs

This repository contains code to quantitatively evaluate instruction-tuned models such as Alpaca and Flan-T5 on held-out tasks.

Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models.

A reliable click-and-go evaluation suite compatible with both open-source and proprietary models, supporting MixEval and other benchmarks.

a repository for evaluating open language models.

Eval tools by OpenAI.

a lightweight LLM evaluation suite that Hugging Face has been using internally.

A framework for few-shot evaluation of language models.

A powerful tool for creating high-quality training datasets for Large Language Models

Dingo: A Comprehensive Data Quality Evaluation Tool

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

Open-Source Toolkit for Efficient Unstructured Data Processing with Pre-built Modules and Local to Cluster Scalability.

</details>

T5

an expert-driven benchmark for Chineses LLMs.

an evaluation benchmark focused on ancient Chinese language comprehension.

This paper list focuses on the theoretical or empirical analysis of language models, e.g., the learning dynamics, expressive capacity, interpretability, generalization, and other interesting topics.

Applying Large language models (LLMs) for diverse optimization tasks (Opt) is an emerging research area. This is a collection of references and papers of LLM4Opt.

整理开源的中文大语言模型，以规模较小、可私有化部署、训练成本较低的模型为主，包括底座模型，垂直领域微调及应用，数据集与教程等。