This repository contains code to quantitatively evaluate instruction-tuned models such as Alpaca and Flan-T5 on held-out tasks.
Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of language models.
A reliable click-and-go evaluation suite compatible with both open-source and proprietary models, supporting MixEval and other benchmarks.
a repository for evaluating open language models.
Eval tools by OpenAI.
a lightweight LLM evaluation suite that Hugging Face has been using internally.
A framework for few-shot evaluation of language models.
A powerful tool for creating high-quality training datasets for Large Language Models
Dingo: A Comprehensive Data Quality Evaluation Tool
Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.
Open-Source Toolkit for Efficient Unstructured Data Processing with Pre-built Modules and Local to Cluster Scalability.
</details>
T5
an expert-driven benchmark for Chineses LLMs.
an evaluation benchmark focused on ancient Chinese language comprehension.
This paper list focuses on the theoretical or empirical analysis of language models, e.g., the learning dynamics, expressive capacity, interpretability, generalization, and other interesting topics.
Applying Large language models (LLMs) for diverse optimization tasks (Opt) is an emerging research area. This is a collection of references and papers of LLM4Opt.
整理开源的中文大语言模型，以规模较小、可私有化部署、训练成本较低的模型为主，包括底座模型，垂直领域微调及应用，数据集与教程等。
a curated collection of datasets specifically designed for chatbot training, including links, size, language, usage, and a brief description of each dataset
A curated list of Multi-modal Large Language Model in 3D world, including 3D understanding, reasoning, generation, and embodied agents.
A curated list of Awesome LLM Inference Paper with codes.
The paper list of the review on LLMs in medicine.
日本語LLMまとめ - Overview of Japanese LLMs.
A collection of open source, actively maintained web apps for LLM applications.
Awesome LLM systems research papers.
Awesome LLM compression research papers and tools.
An awesome and curated list of best code-LLM for research.
A collection of papers and resources about aligning large language models (LLMs) with human.
A curation of awesome tools, documents and projects about LLM Security.
A paper & resource list on model editing for large language models.
possibly useful materials and tutorial for learning RWKV.
a collection of human preference datasets for LLM instruction tuning, RLHF and evaluation.
a collection of demos and articles about the OpenAI GPT-3 API.
A curated list of awesome projects and resources related to GPT, ChatGPT, OpenAI, LLM, and more.
Measuring LLMs' Reasoning Performance
Collection of papers and resources on Reasoning using Language Models.
A paper & resource list of large language models.
A trend starts from Natrural-Instruction (ACL 2022), FLAN (ICLR 2022) and T0 (ICLR 2022).
How to ask LLMs to produce reliable reasoning and make reason-responsive decisions.
A trend starts from "Chain of Thought Prompting Elicits Reasoning in Large Language Models.
Curated list of resources for ChatGPT and GPT-3 from OpenAI.
A Chinese collection of prompt examples to be used with the ChatGPT model.
A curated list of practical guide resources of LLMs
List of papers on hallucination detection in LLMs.
LLM hallucination paper list.
MoE language model with 32B active and 1T total parameters.
First-generation reasoning models from DeepSeek.
Explore examples and solutions for using Claude on AWS infrastructure.