Fix: tuneThreshold - Minimization for measures needing maximization #2857
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When using
tuneThreshold
to get the best threshold for a measure that needs maximization it does not work, but provides the minima (worst threshold) and returns it with the wrong sign.Reproduce:
Cause
The defined callback function in tune threshold always makes it a minimization problem for all measures (e.g. for measures that need maximization as well):
ifelse(measure$minimize, 1, -1) * performance(setThreshold(pred, x), measure, task, model, simpleaggr = TRUE)
When
optimizeSubInts
is called themaximum
flag is set depending on the measure'sminimize
flag (even though it was already handled and it will be always a minimization problem at this point). This leads to finding the overall minima not maxima of a measure that needs to be maximized (e.g. bac, acc etc.).Fix
After changing the call of optimizeSubInts to search for the minima always it works correctly: