Expected parameter covariance_matrix (Tensor of shape (95, 95)) of distribution MultivariateNormal(loc: torch.Size([95]), covariance_matrix: torch.Size([95, 95])) to satisfy the constraint PositiveDefinite(), but found invalid values

[vengdeng]

Dear Tang,
I have a question is this repo the newest code ? I directly fork the git and run the given fedcorr command for both Fminist and Cifar-10, however, both of them meets PositiveDefinite issue when calculate the gpr.Predict_Loss(test_data,test_idx,pred_idx).

Thanks!

[Yoruko-Tang]

Hi,

Thank you for your issue. We noticed that there was a wrong default hyperparameter in our code. The default gpr_begin should be 0 instead of 10. We have fixed this in the new version. Sorry for your inconvenience.

[vengdeng]

Hi,

Thank you for your issue. We noticed that there was a wrong default hyperparameter in our code. The default gpr_begin should be 0 instead of 10. We have fixed this in the new version. Sorry for your inconvenience.

Thank you very much for your quick reply!!! I will try the new hyp. In the meantime, may I ask what's the except condition ValueError. Actually I solved that by changing the ecexpt condition from runtime-error to value-error, since I thought you may wish to skip and increase the noise when not PositiveDefinite.

[Yoruko-Tang]

We hope to increase the noise scale (namely, adding a larger diagonal matrix to the covariance matrix) when the covariance matrix is singular. If the covariance matrix is singular, the MultivariateNormal function in Pytorch will raise a RuntimeError.

[vengdeng]

We hope to increase the noise scale (namely, adding a larger diagonal matrix to the covariance matrix) when the covariance matrix is singular. If the covariance matrix is singular, the MultivariateNormal function in Pytorch will raise a RuntimeError.

Oh I see, thank you very much for your explain!!

[vengdeng]

Hi tang, sottry to bother you again, a quick update that I reforked the rep and use the newest paramter but the error still occurs after the warmup at epoch 21 in cifar case.

[Yoruko-Tang]

I did not get the same error on my server. Please check if you are using the same Pytorch version as mine which is 1.7.1.

[vengdeng]

I followed your environment and now everything is good, thank you very much!