Incorporate an evaluate function into model controller & a number of other refactoring steps towards a training and evaluation pipeline. #25
Conversation
…ove the odd seq length/bin length requirement in online sampler
…n strand specific module wrapper to use when sequence-level models can group predictions from the forward & reverse seqs
config_examples/parameters.yml
Outdated
| @@ -1,25 +1,37 @@ | |||
| --- | |||
| model: { | |||
| non_strand_specific_module: True, | |||
There was a problem hiding this comment.
need to include a mode = "mean" or "max" parameter
| test_holdout: [8, 9], | ||
| validation_holdout: [6, 7], | ||
| random_seed: 127, | ||
| sequence_length: 1001, | ||
| center_bin_to_predict: 201, | ||
| default_threshold: 0.5, | ||
| feature_thresholds: 0.5, |
There was a problem hiding this comment.
check that the proper handling for this input is in IntervalsSampler (or OnlineSampler).
models/non_strand_specific_module.py
Outdated
| def __init__(self, model, mode="mean"): | ||
| super(NonStrandSpecific, self).__init__() | ||
|
|
||
| print(mode) |
There was a problem hiding this comment.
remove print statements
|
|
||
| def forward(self, input): | ||
|
|
||
| reverse_input = flip( |
There was a problem hiding this comment.
This is based on the assumption that the sequence is encoded in such a way that we can just "flip" the indices in the matrix and get the reverse sequence. I'll need to document that encoding
| [default: False] | ||
| --verbosity=<level> Logging verbosity level (0=WARN, 1=INFO, 2=DEBUG) | ||
| [default: 1] | ||
| <config-yml> Model-specific parameters |
There was a problem hiding this comment.
Improve the documentation here.
| @@ -1,36 +1,109 @@ | |||
| """TODO: nothing in this file works right now. Please do not use/review | |||
There was a problem hiding this comment.
This file will be updated soon. It was originally implemented to account for the forward & reverse predictions but we are now just incorporating forward/reverse strand predictions directly into the model architecture.
This pull request is still in progress. It will address multiple issues, which I will continue to update in this post.
After this pull request, I will avoid addressing multiple changes in a pull request as much as possible. I'll also go back to the pull request model that I started initially (submitting pull requests from a fork of the repository, rather than a branch in the FunctionLab repo).
Changes made:
parameters.ymlandpaths.ymlinto a single file. I also removed the processing code for IntervalsSampler inselene.pyto ensure that we are truly making the Sampler modules dynamically configurable just from the YAML file.forward.model_train.pyperformance monitoring should be more flexible/modular #3