Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding mixed type in privacy metrics silently fails with no feedback #647

Closed
npatki opened this issue Nov 23, 2021 · 0 comments · Fixed by sdv-dev/SDMetrics#108
Closed
Labels
feature:evaluation Related to running metrics or visualizations feature request Request for a new feature

Comments

@npatki
Copy link
Contributor

npatki commented Nov 23, 2021

Problem Description

If I accidentally include mixed types in a Privacy Metric calculation, the metric returns a nan with no indication as to whether there was an error or what made the computation fail.

Using the same demo data from the User Guide, I tried to run the following code:

>>> NumericalLR.compute(real_data, synthetic_data
        key_fields=['second_perc', 'mba_perc', 'gender'],
        sensitive_fields=['degree_perc'])
nan

(This is happening because gender is a categorical column being used in a Numerical metric. It correctly works if I remove gender.)

Expected Outcome

In sdv-dev/SDMetrics#134, I filed an issue about supporting mixed types.

However even if we don't support mixed types, we should validate the inputs and throw a nice, descriptive Error to the user.

@npatki npatki added feature request Request for a new feature feature:evaluation Related to running metrics or visualizations labels Nov 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature:evaluation Related to running metrics or visualizations feature request Request for a new feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant