Bayesian Committee Machine (BCM)

[byooooo]

Feature description

Add option to parallelize training of GPs using BCM method (factorized GPs). Need to consider potential conflict between existing parallelization for multiple objectives.

Implementation idea

Create _train_BCM()/_predict_BCM() class function for sklearn instance (and GPy instance if applicable)

Alternatives

[kjappelbaum]

I think the interesting thing about BCM would be that we could use the online learning implementation based on the Kalman filter

[kjappelbaum]

https://github.com/jopago/GPyBCM/blob/master/GPyBCM.py uses GPy, so we could give it a shot, but I personally would give the jax issue (#115 ) a higher priority as it will give us speedups with any model

[byooooo]

yea we can make this a low priority

[kjappelbaum]

wondering what the API would look like, i.e., how we initialize the PAL class. The minimum that the user would need to be able to change would be the number of experts and the model BCM/rBCM and somehow we need to know that the user wants to use this.

so an initial thought would be to add

bcm: bool = False
m: int = 6
bcm_model: str = 'rbcm'

the downside of this is that the list of arguments would be quite long

[kjappelbaum]

@byooooo do you know a benchmark comparing this to low-rank approximation as the SparseGP one can already use with PyePAL via GPy?

[byooooo]

yea i think that would make the instantiation cumbersome. maybe instead we can have another function after initialization with PAL that the user can call (in case they want to use BCM) and have the BCM attributes set to None initially in the PAL class.

So it would be something like:
PALinstance(*args, **kwargs)
followed by
PALinstance.BCM(*args, **kwargs)
where the args could be the ones you listed above

[byooooo]

@byooooo do you know a benchmark comparing this to low-rank approximation as the SparseGP one can already use with PyePAL via GPy?

Nope. 😅 maybe should try this ourselves first before implementing this into the package

[kjappelbaum]

Yea and GPflow also has a bunch of different approximations, e.g., https://www.jmlr.org/papers/v18/16-579.html

[github-actions]

Stale issue message