LightningDataModule

正文

A datamodule encapsulates the five steps involved in data processing in PyTorch:

1. Download / tokenize / process.

2. Clean and (maybe) save to disk.

3. Load inside Dataset.

4. Apply transforms (rotate, tokenize, etc…).

5. Wrap inside a DataLoader.

prepare_data() 的作用

prepare_data() 主要放一次性数据准备逻辑，例如：

下载数据解压数据tokenize 文本清洗后保存到磁盘

核心原则是：只做“准备数据文件”的事情，不创建训练时要用的 Dataset 状态。

setup

There are also data operations you might want to perform on every GPU. Use setup() to do things like:

• count number of classes

• build vocabulary

• perform train/val/test splits

• create datasets

• apply transforms (defined explicitly in your datamodule)