You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The GMLogLikelihood metric was added to cover the metrics that existed in the original SDGym iteration, which used GM Log Likelhood metric over datasets that were simulated using GaussianMixtures.
However, even though the implementation was optimized and improved to make the output as stable and meaningful as possible, the scores produced when this metric is run on datasets which not simulated from GMs tends to be very noisy and may produce inconsistent results between runs. As a consequence of this, the ranking-based integration test fails randomly.
We may want to remove this metric from the ranking test and have a separated one which is tested using GM simulated data, and also add a disclaimer in the documentation indicating what this metric is best suited for.
The text was updated successfully, but these errors were encountered:
The GMLogLikelihood metric was added to cover the metrics that existed in the original SDGym iteration, which used GM Log Likelhood metric over datasets that were simulated using GaussianMixtures.
However, even though the implementation was optimized and improved to make the output as stable and meaningful as possible, the scores produced when this metric is run on datasets which not simulated from GMs tends to be very noisy and may produce inconsistent results between runs. As a consequence of this, the ranking-based integration test fails randomly.
We may want to remove this metric from the ranking test and have a separated one which is tested using GM simulated data, and also add a disclaimer in the documentation indicating what this metric is best suited for.
The text was updated successfully, but these errors were encountered: