You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Here is a suggestion to improve the hyperparameter storage of pipelines. Currently we have a lot of files of the style pipeline_name/pipeline_name_dataset.json to denote hyperparameter changes of the pipeline for the purpose of benchmarking. To reduce the number of files in a particular folder, we suggest the following:
Under a particular pipeline folder (pipeline_name/), we can have benchmark-meta.json with all the datasets defined and their corresponding hyperparameter. The corresponding hyperparameter can be defined as:
a dictionary denoting the primitive and hyperparameter change
a path to a file that contains the hyperparameter changes, this is useful when we have a complex set of hyperparameters to be changed.
Pipelines and templates
We technically don't differentiate between pipelines and templates using Orion API. It might even get confusing if we display pipelines and we store them as only templates. For lstm_dt and lstm pipelines (dynamic threshold, and fixed threshold respectively), I suggest having two pipeline folders to eliminate confusion. This will result in:
lstm_dt folder
lstm folder
The text was updated successfully, but these errors were encountered:
Here is a suggestion to improve the hyperparameter storage of pipelines. Currently we have a lot of files of the style
pipeline_name/pipeline_name_dataset.json
to denote hyperparameter changes of the pipeline for the purpose of benchmarking. To reduce the number of files in a particular folder, we suggest the following:Under a particular pipeline folder (
pipeline_name/
), we can havebenchmark-meta.json
with all the datasets defined and their corresponding hyperparameter. The corresponding hyperparameter can be defined as:See example here
Pipelines and templates
We technically don't differentiate between pipelines and templates using Orion API. It might even get confusing if we display pipelines and we store them as only templates. For
lstm_dt
andlstm
pipelines (dynamic threshold, and fixed threshold respectively), I suggest having two pipeline folders to eliminate confusion. This will result in:lstm_dt
folderlstm
folderThe text was updated successfully, but these errors were encountered: