[AutoParallel] Fix dataloader in to_static mode. #64334
Conversation
|
你的PR提交成功，感谢你对开源项目的贡献! |
| @@ -274,7 +274,8 @@ def __init__( | |||
| def _prepare_data_spec_from_dataloader(self, dataloader): | |||
| inputs_spec = [] | |||
| labels_spec = [] | |||
| data = next(iter(dataloader)) | |||
|
|
|||
| data = dataloader._get_input_spec() | |||
There was a problem hiding this comment.
if hasattr(dataloader, _get_input_spec):
data = dataloader._get_input_spec()
else:
data = next(iter(dataloader))
There was a problem hiding this comment.
只有ShardDataloader会调用_prepare_data_spec_from_dataloader
There was a problem hiding this comment.
_prepare_data_spec直接取dataset，对于IterableDataset，不支持Shuffle，不存在这个问题。对于普通Dataset，直接取第 0 个数据，不会影响随机性。
| # of BatchSampler when `Shuffle=True`. It may cause difference of data feeding | ||
| # between dynamic and to_static mode. | ||
| def _get_input_spec(self): | ||
| batch_data = self._dataloader.batch_sampler.dataset.__getitem__(0) |
There was a problem hiding this comment.
what about IterableDataset?
There was a problem hiding this comment.
IterableDataset 也有__getitem__(self, index)方法
There was a problem hiding this comment.
这里会再修改一下，对IterableDataset还是需要特殊处理一下。
| collate_fn = self._dataloader.collate_fn | ||
| batch_data = collate_fn(batch_data) | ||
| if isinstance(batch_data, dict): | ||
| batch_data = [batch_data] |
There was a problem hiding this comment.
batch data is already a tensor?
There was a problem hiding this comment.
getitem(0)取出来的是一个np.ndarray
There was a problem hiding this comment.
这里的顺序写错了。collate_fn 会把 batch 遍历一遍，需要输入 batch_data 是一个 list。完成collate_fn 后，_get_batch 需要输入为 Tensor 。
… fix_dataloader
PR Category
Auto Parallel
PR Types
Bug fixes
Description
Pcard-73145
Fix dataloader in to_static mode. Get data from DataLoader iterator directly may affect data generation randomness of BatchSampler when
Shuffle=True. It may cause difference of data feeding between dynamic and to_static mode.