AgentVFS
People
Toolathlon

llm-training-dataset

Fill the ptdata sheet of the LLM Pre-training Data spreadsheet with the pre-training datasets of LLaMA and GPT-Neo: name, which model uses it, size in GB and a link, preferably to Hugging Face, sorted by size in descending order and listing a dataset that both models use once for each.

Task Prompt

I've been learning about large-scale language models recently, and I've decided to train a decoder-only language model on my own. The first step is to prepare pre-training data, so I need your help organizing the pre-training data for llama and gpt-neo into the `ptdata` sheet in `LLM Pre-training Data` spreadsheet. The pre-training data should be sorted in descending order by data size, with columns named `name`, `use in llm` (the value in each row must be either llama or gpt-neo), `size` (the numeric value in GB, but do not include "GB" in cell), and `link` in order. If a dataset is used by both models, list it separately for llama and gpt-neo because the model-specific data sizes may be different. The link to huggingface is preferred; if huggingface doesn't have the dataset, we'll try to provide other links.