DOC add an example on how to optimize a metric with a constraint in TunedThresholdClassifierCV

[glemaitre]

We merged TunedThresholdClassifierCV in #26120.
However, we don't expose any way to optimize a metric that is constrained by another as one would do when choosing a point on the ROC or PR curves.

We should have an example that shows how to do such optimization as discussed here:
#26120 (review)

This would be a temporary trick until we settle on the best possible API regarding this constrained scorer.

[adrinjalali]

Probably also figure out a nice api that we can use for SearchCV as well

[jeremiedbb]

I think that having a single scoring param able to handle constrained metrics as well would be ideal. In addition to make an example for that, we can maybe implement a helper for the main uses cases, precision at recall and tpr at tnr, that returns a ready-to-use scorer.

Probably also figure out a nice api that we can use for SearchCV as well

Do you think it would make sense to search over constraint values ? My understanding was that such constraint is determined by an external factor that you have no flexibility over.

[adrinjalali]

At least in fairness related context, you might have a constraint on a fairness metric, and among those you want the best accuracy.

Or if you take the classical example, you want the model which has the least AIC/BIC within the 1 std of the test error of the best model.