Bias in models?

[braaannigan]

There is a question about data privacy. Should there also be a question about potential sources of bias that could arise from using the model and how this is monitored?

[eugeneyan]

Hey @braaannigan, yes, data privacy is a key concern in systems involving customer data and is covered in Section 6.5.

Your point on bias is spot on. Yes, monitoring for bias, data drift, model validation metrics can also be part of the design doc and should be added if it's critical to the system. I've not included it here so as not to overwhelm.

[erikcvisser]

I would also argue that it would be useful to add this to 6.6 (Monitoring). You want to monitor not only for technical issues, but also for data drift. I want the ML dev team to think about when a model needs to be retrained, or potentially deprecated altogether (lifecycle management).