Skip to content
This repository has been archived by the owner on Nov 16, 2023. It is now read-only.

[BUG] No such file or directory #621

Open
Farhad-Heybati opened this issue May 20, 2021 · 1 comment
Open

[BUG] No such file or directory #621

Farhad-Heybati opened this issue May 20, 2021 · 1 comment
Labels
bug Something isn't working

Comments

@Farhad-Heybati
Copy link

Description

tc_bert_azureml.ipynb
Error message:
wc: /mnt/batch/tasks/shared/LS_root/jobs/gaia-ml-wks/azureml/3576038c-f399-4522-8674-383ad7cd316b/mounts/workspaceblobstore/azureml/88f85fcf-eaf4-4d58-bff2-712b35cb50aa/train0,: No such file or directory

How do we replicate the bug?

Just run the notebook tc_bert_azureml

Expected behavior (i.e. solution)

Other Comments

@Farhad-Heybati Farhad-Heybati added the bug Something isn't working label May 20, 2021
@Farhad-Heybati
Copy link
Author

Farhad-Heybati commented May 20, 2021

I think the issue is coming from the fact that the output of preprocessing is defined by PipelineData:
output_data = PipelineData(name="train{}".format(str(i)), datastore=ds,
output_path_on_compute='mnli_data/processed_train/batch{}.csv'.format(str(i)))
By default, the path to the data file is based on an environment variable: $AZUREML_DATAREFERENCE
for example:
processed_train_files[0] => $AZUREML_DATAREFERENCE_train0
When I print the value of the file path in preprocessing step and in the training step, these are differents:
preprocess step:
/mnt/batch/tasks/shared/LS_root/jobs/gaia-ml-wks/azureml/273296fe-21e3-44fa-84d1-6a6b33c04031/mounts/workspaceblobstore/azureml/273296fe-21e3-44fa-84d1-6a6b33c04031/train0'
Bert training step:
/mnt/batch/tasks/shared/LS_root/jobs/gaia-ml-wks/azureml/51dd8b67-13a7-40a1-a9ff-eef158a300c5/mounts/workspaceblobstore/azureml/273296fe-21e3-44fa-84d1-6a6b33c04031/train0

The $AZUREML_DATAREFERENCE value change based on the runID

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant