-
Notifications
You must be signed in to change notification settings - Fork 91
Description
I've found that in the case of train_model, none of logging handlers ever close (i.e. selene_sdk.train_model.log, selene_sdk.train_model.train.txt, and selene_sdk.train_model.validation.txt). I assume this happens in evaluate_model too but I haven't looked too closely.
This creates problems because I frequently write code that involves creating multiple trainers. Since this happens in the same script, each trainer ends up sharing loggers. If trainer 1 is made before trainer 2, then trainer 2 logs to the trainer 1 log as well as the trainer 2 log.
It would be useful if somewhere within the TrainModel object, the Handlers could be removed from the loggers at some point. To me it seems most useful after calling train_and_validate but it could also be done in an implementation of __del__.
In a related issue, it would be helpful if I could set a flag telling the logger not to write to stderr. Since the logs are getting written elsewhere anyways, it feels redundant to write the logs to stderr, especially since it gets mixed in with my own custom logging.