Nano: pytorch val and test metric logs has wrong format in `LightningModuleFromTorch`

[shane-huang]

Problem:

The current log will output something like val_function_0 while it should be val_mean_squared_error_0.

Solution:

"val/{}_{}".format(type(metric).__name__, i) use the name of the type of metric (metric is an instance of torchmetrics.metric.Metric so the type of it is function), that's why the output looks like val_function_0. It should use the name of the metric itself, i.e. metric.__name__.

• https://github.com/intel-analytics/BigDL/blob/main/python/nano/src/bigdl/nano/pytorch/lightning.py#L87)
• https://github.com/intel-analytics/BigDL/blob/main/python/nano/src/bigdl/nano/pytorch/lightning.py#L95

[pranjalks]

Hi @shane-huang, can you please have a look at this PR that I have raised to resolve this issue: #10145. Thanks!

[shane-huang]

Sure. In general the fix looks good to me. I've started the PR validation tasks and let's wait for its results.

[pranjalks]

Sure. In general the fix looks good to me. I've started the PR validation tasks and let's wait for its results.

Ok, so all the tasks did not pass. On further looking into the issue, the error is being thrown because of changing the code to metric.__name__. This would work fine with functions from sklearn.metrics but would throw error with any metric class imported from torchmetrics.metric.

I further looked into the file lightning.py to check which all kind of metric functions are being passed to the parameter self.metrics. It is clear from the constructor parameters that the parameter metrics is actually a list of Metric (List[Metric]) imported from torchmetrics.metric. Hence, we need to change the code such that we can extract the name from such metrics.

Therefore, to view the name of such a metric function, we can change the code to:

acc = {"val/{}_{}".format(metric.__class__.__name__, i): metric(y_hat, batch[-1])
                   for i, metric in enumerate(self.metrics)}

acc = {"test/{}_{}".format(metric.__class__.__name__, i): metric(y_hat, batch[-1])
                   for i, metric in enumerate(self.metrics)}

Please have a look at this. Will raise a PR if that's ok.