Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support snapshots #28

Open
seba-1511 opened this issue Jun 15, 2018 · 1 comment
Open

Support snapshots #28

seba-1511 opened this issue Jun 15, 2018 · 1 comment

Comments

@seba-1511
Copy link
Owner

Snapshopts are unfinished experiments.

The idea is to save partial results (in a dedicated folder) that could be used to implement fancier random search algorithms. (e.g. Population Based Training.) They are also useful for very long / tedious experiments where saving intermediary results is useful.

One way to do it is to create a dedicated subfolder for these in same vein as attachments. However careful (and efficient) annotations need to be added in order to support "versioning" of snapshots. (both dumping and loading) Those should support the usual result + attachments.

Another way to do it is taking advantage of the current infrastructure, and use experiment names for versioning. This is simpler (and maybe cleaner) but it requires adding functions to modify/extend JSON summaries and saving these modified JSON summaries in a separate experiment.

@seba-1511
Copy link
Owner Author

Ideally, if a snapshot is extended you would be able to retrieve the history of hyperparameters that were used at each snapshot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant