Diffusion-based Synthetic Anomalies for Industrial Inspection (DSAII)

  synthetic_output
  ├─ audio cable
  │  ├─ img_url.json
  │  │
  │  ├─ normal
  │  │  ├─ 000.png
  │  │  ├─ 001.png
  │  │    ...
  │  │  └─ 019.png
  │  │
  │  ├─ anomaly
  │  │  ├─ 000.png
  │  │  ├─ 001.png
  │  │    ...
  │  │  └─ 017.png
  │  │
  │  └─ mask
  │     ├─ 000.png
  │     ├─ 001.png
  │       ...
  │     └─ 017.png
  ├─ audiopipe
  ...

Dataset Structure

DSAII dataset serves as a performance evaluation dataset for industrial anomaly detection and segmentation task. These images are obtained by crawling and arranging similar images from the web. Each class (0~77) contains 10~20 images without defect, which is saved under path synthetic_output/{CLASS}/normal. The image reference link is listed in img_url.json. The whole dataset contains 2906 images (including normal and defect images).

For each normal images, we apply Stable Diffusion for defect inpainting respectively. The inpainting mask is acquired by random sampling ground_truth mask from MVTec AD dataset, then applying augmentations as our input inpainting masks. The inpainting masks are saved under path synthetic_output/{CLASS}/mask. The output defect images are saved under path synthetic_output/{CLASS}/defect. The failure cases are eliminated.

Why use DSAII?

DSAII dataset contains normal images with large intra-class variation, while defective images are generated by diffusion model. This setting aims to evaluate the model’s generalizability and few-shot capabilities. The model needs to learn the abstract pattern of the object/texture and the its normal appearance to achieve better performance. Besides, we provide a flexible way for creating synthetic anomalies. With any image, you can customize your own dataset, and simply edit prompt for controlling defect types. By Following our instructions, you can customize your own image anomaly detection dataset.

Limitation

Due to Copyright issue, we should be careful when crawling images from the internet. Please make sure the images are available for free or contact author for further authorization.

Instructions

Phase 1: Crawler

• Step 1: Prepare representative images for each class you want to create.
  For crawling bunches of similar images, first prepare one image for each class, for example:

  /path/to/dir
  ├─ audiopipe
  │  └─ 000.png
  ├─ bucket
  │  └─ 000.png
  ...

• Step 2: Prepare a Google Chrome browser.
• Step 3: Download chromedriver.exe from official website.
• Step 4: Edit SRC, DST path to directory in crawler_picture.py then execute python crawler_picture.py. After executing crawler_picture.py, you will get output directory, for example:

  /path/to/dir
  ├─ audiopipe
  │  ├─ 000.png
  │  ├─ 001.png
  │  │    ...
  │  └─ 019.png
  ├─ bucket
  │  ├─ 000.png
  ...

Phase 2: Ground Truth Mask Generation

• Step 1: Follow Segment Anything to acquiring object mask, for furthur generating ground-truth mask. The output mask candidates should looks like:

  /object/mask/candidates/
  ├─ audiopipe
  │  ├─ 000
  │  │  ├─ 0.png
  │  │  ├─ 1.png
  │  │    ...
  │  │  └─ 16.png
  │  ├─ 001
  │  │  ├─ 0.png
  │  │    ...
  │  │  └─ 8.png
  │  │    ...
  │  └─ 019
  │     ├─ 0.png
  │      ...
  ├─ bucket
  │  ├─ 0.png
  ...

• Step 2: Edit SRC, DST path to directory in mask_extractor.py then execute python mask_extractor.py. It will output the final object mask.
• Step 3: Prepare a bunch of mask you want to use, in this case we use MVTec AD testing set mask.
• Step 4: Update SKIP list in gen_mask.py for skipping object mask when augmenting ground truth mask. SRC, MASK, DST for input/mask/output directories.
• Step 5: Run python gen_mask.py

Phase 3: Diffusion Inpainting

• Step 1: Generate masks of anomaly region from exist masks or prepare your own masks
• Step 2: Download and install stable diffusion web-ui follow the instructions in here.
• Step 3: Apply inpaint function to the image with corresponding mask given the prompt (in “img2img” block and choose the “batch” section).

  prompt for objects: broken, cracked, severely shattered
  prompt for textures: broken, cracked, severely shattered(“stains, holes” are applied on classes that don’t go well with default prompts)

Experiment

We test synthetic dataset on Patchcore (CVPR, 2022), SimpleNet (CVPR, 2023). The performance is record as follow.

Method	Mean Image AUC (%)	Mean Pixel AUC (%)
Patchcore	57.2	87.5
SimpleNet	69.1	65.3

NTHU CVLAB

For more details, please visit CVLAB.

CHENG-YU HO: [email protected]
YU-HSUAN HSIEH: [email protected]

License

MIT Licence