Skip to content

Scan your AI/ML models for problems before you put them into production.

License

Notifications You must be signed in to change notification settings

IQTLabs/daisybell

daisybell

A scanner that will scan your AI models for problems. Currently it focuses on bias testing. It is currently alpha.

How to Use

First install it:

pip install daisybell

Run it in this manner (currently supports models from HuggingFace's repository):

daisybell roberta-base

The scan can output files for further analysis:

daisybell roberta-base --output results/roberta-base

We will infer the task(s) of model by default but to provide specific tasks to test explicitly use the --task switch:

daisybell cross-encoder/nli-distilroberta-base --task zero-shot-classification

Daisybell can even scan chatbot models like StableLM for AI safety and alignment issues:

daisybell StabilityAI/stablelm-tuned-alpha-3b -d cuda:0

That's it for now. More will come.

Future Work

  • More bias tests. More metrics for bias testing based on the research in the field.
  • Integration with other types of testing (eg. adversarial robustness)
  • More kinds of models besides HuggingFace models. We are especially interested in MLFlow integration.
  • Documentation.

Please contribute if you can. Help is always helpful.

License

Apache

Credit

A project of IQT Labs.