Exploration/Exploitation Ratio explicitly made available to users

[juliecious]

The current default acquisition function is to optimise the exploration and exploration ratio of the recommending batches, would it be possible to make the ratio available or visible to users?

[Scienfitz]

There seems to be the expectation that an acquisition function is explicitly built by weighing between exploration and exploitation. This is not the case, so I think in general its difficult to provide such an assessment.

For UCB and qUCB I think we have an interesting interpretation about the beta parameter that relates to your question, but I'll let @AdrianSosic explain this as he formed a good interpretation of it.

Regarding other acqfs where you can clearly "set" exploration/exploitation tradeoff I know things like this work but its using a modeling approach not that commonly used in practice and thus not in baybe

[AdrianSosic]

Agree! Without making an excursion into acquisition functions, there are two issues that make this a nontrivial task:

• There is generally no explicit measure of exploration vs exploitation that can be easily accessed
• What you are asking for is a "local" quantity, i.e. a measure per recommended point. This generally clashes with the concept that points of a batch are jointly optimized, i.e. they are batch-diversified. This makes it hard to attribute a individual number per point.

However, that said, I still think it's fair to ask for a simple proxy, and I can imagine several ways how to do it – as long as we keep the above caveats in mind. It's probably worth to do a little literature research, but from the top of my head, a simple approach would be:

• Take the predicted standard deviation as the measure for exploration
• Take the expected improvement as a measure for exploitation

This would work regardless of the acquisition function type used. And perhaps one could refine for specific types such as UCB. But to get a first idea / show some tendency in dashboard, I think it's good enough.

What do you think, @Scienfitz?

[AdrianSosic]

Ah, and you'd probably need to be a bit careful on which domain these quantities are computed. I guess it's most natural to compute them on the transformed domain, because that is where the acquisition function is evaluated.

[Scienfitz]

@AdrianSosic

can you elaborate why you see the EI of a point as good proxy for exploitation? Its kind of higher the higher the std of that point is too, so highly correlated to the measure for exploration.

Here also an alternative suggestion: I think absolute measures for exploration/exploitation are not that useful. I am always under the impression people want two numbers that add up to 1, so kind of a normalized bisection o the "character" of each point. For that we could calculate the std_point of a point, divide it by the global maximum std of all points std_global_max which gives the exploration factor f_exploration = std_point / std_global_max which is inside [0,1]. The exploitation is then f_exploitation = 1 - f_exploration. This would also give high exploitation values to points that are measured and are bad, but in a sense thats correct because theres noting to be explored about them?

Judging from the question I am also not 100% sure just the characterization of recommended points is evertghing, @juliecious do you also want to configure the algorithm by explicitly setting exploration / exploitation? This is soemthing not adressed by our proposals so far. The only way to do this currently is using UCB/qUCB

[AdrianSosic]

can you elaborate why you see the EI of a point as good proxy for exploitation?

Sorry, my bad. I actually meant just the posterior mean.

Here also an alternative suggestion: [...]
Without having thought too much about it yet, it feels to me that this is not what we'd want and is also impracticable because

1. You pretty much don't have std_global_max in continuous case or maybe even in large discrete spaces
2. I don't think we can easily bring these two numbers onto the same scale. Like, for example, what would happen in your approach if I give you two predictions: A) mu=0, std=1 and B) mu=1, std=1 --> your exploration/exploitation number will be the same, but this clearly doesn't align with what you would intuitively say about these two points, no?