Multi-Output Regression Models

[narasimhard]

It would be fantastic to have models that can handle multiple target regression problems, for instance, in a scenario where the same set of independent variables can be used to learn and predict multiple dependent outcomes. This would simplify the training process by using just one model and also make deployment easier.

[wendycwong]

@narasimhard

Needing some clarification here. If I have one set of predictors, I want to be able to build a model to predict say dependent outcome 1, outcome 2 and outcome 3. If we are thinking about using GLM, are you saying that we want to have three sets of parameters for the 3 outcomes right?

Thanks, Wendy

[tomasfryda]

@narasimhard please correct me if I'm wrong but I think the point could be joint estimation of the parameters (e.g. betas). @wendycwong GLM is a little bit special in this area since there is a generalization of GLM that is trying to do that (VGLM). It still uses N sets of "betas" but they are different than independently estimated betas using N independent GLMs, i.e., the N sets of betas can take an advantage of correlation structure in the data and build a better estimator.

For DeepLearning, I think the solution would be just more neurons in the output layer.

Am I correct @narasimhard ?

[narasimhard]

Yes, that's what I am thinking as well. When using DL models, there would be more neurons in the output that can do the predictions for N target variables. @tomasfryda @wendycwong

[wendycwong]

Thank you @tomasfryda and @narasimhard : From reading your notes, this means that we are doing the following:

1. for one output Y1 only, we will be doing this:
   adjust system parameters W to optimize objective f(Y1, W);

2. If we have multiple regression outputs Y1, Y2, Y3, we will be doing this:
   adjust system parameters W to optimized a linear combination of the objective functions: a1f(Y1, W)+a2f(Y2, W)+a3*f(Y3, W)

Please let me know.