The most powerful open source instructions dataset: Flan. 378 Million samples. (~300GB) [1] - Link: huggingface.co/datasets/Open… Why should you care? 🤔 - Flan is an incredibly powerful dataset [2] and some famous models trained on it (FlanT5, UL2..) hold the top positions on various leaderboards to this day. - The main reason for it is the quality and diversity of the data. - It is huge: Ever wondered "What would happen if we just merged all instructions datasets together into a single huge one?", this is basically the motivation behind the Flan dataset. - It is balanced (!!) which promotes the models trained on it to generalize better to arbitrary tasks down the line. Flexibility: - Zero-Shot vs Few-Shot: For many of the tasks you can fetch the same task either for Zero-Shot: No solved for demonstration or Few-Shot. - Chain of thought built in on some of the tasks. The "next step".. A small part of Flan had been augmented with additional explanations in the past. The result of this was the first model ever to rival ChatGPT on vicuna's benchmark. And again.. This was just a small part of Flan.. ---- [1] ai.googleblog.com/2023/02/th… [2] arxiv.org/pdf/2301.13688.pdf (* This paper is a must if you are building text datasets)