Improved feature importance support

[ablaom]

The MLJ model API only says that model reporting feature importances
should report them in the report output by fit. But it says
nothing about the actual format of this output, and I can see
inconsistencies in the implementations. Feature importances are used
by some meta-alogorithms, such as RecursiveFeatureElimination (#426) so this
might be worth sorting out.

I propose adding a new method feature_importance(model::Model, report) to the model API to report the scores, according to some
fixed convention. Some models (e.g., LightGBM models) report multiple
types of importance scores. So I propose this method return a named
tuple keyed on the type, whose values are Float64 vectors.

edit See suggestion for format below.

edit The proposal follows that same interface patter that we have already for training_losses.

Thoughts anyone?

TODO:

• Add reports_feature_importances trait to StatisticalTraits, defaulting to false
• Add feature_importances(model, report) stub to MLJModelInterface (in model_api.jl); fallback to return nothing.
• In MLJBase: Overload MMI.feature_importance(mach::Machine) following this pattern
• Update MLJ model API docs
• In MLJ: Bump MLJBase compat and re-export feature_importances #954
• Roll out implementations for packages that already report
  importances in their report (including linear models for which absolute value of
  coefficients serve). These include:
  • MLJDecisionTreeInterface
  • EvoTrees models
  • MLJLinearModels
  • MLJGLMInterface
  • MLJXGBoostInterface
  • LightGBM
    It may make sense to roll out data front-ends for some of these models at the same time, mimicking the EvoTrees case where this already done.
• Get a list of scikit learn models that expose importances or
  coefficients and get these models to report the scores in their
  report, and to implement the above method and trait. See Have linear models report names of coefficients along with values. MLJScikitLearnInterface.jl#30 and implement feature importance rankings MLJScikitLearnInterface.jl#26

[ablaom]

cc @boliu-christine

[ablaom]

Here's an update on my suggestion for the format of feature importances, as returned by the proposed method feature_importances(model, report).

I think allowing models to expose multiple types of feature importance is overkill / excessively complicated. Of course multiple scores can still be declared in the report itself.

So I suggest a vector of name => float pairs, where name is a symbol:

v= [:gender =>0.23, :height =>, :weight => 0.1] 

[zsz00]

What is the current state of this ??
I need feature importance support !

[OkonSamuel]

What is the current state of this ?? I need feature importance support !

Am still working on this. Will be done soon.

[zsz00]

What is the current state of this ?
@OkonSamuel