Retreaval-based-Adversarial-Defense

Adversarial defense by retreaval-based methods

---

⚠ status: it is seemingly a degenerated version of the MAE defense which also failed against PGD attack, leaving for future research :(

Ideas of the retreaval-based methods:

• input pixel patch
• input pixel textual patch (f')
• cnn fmap patch
• cnn fmap textual patch

quick start

⚪ Preparation

• download the datasets here, unzip to data/ folder
  • NIPS17 & ssa-cwa-200: clean and pre-generated adversarial images from Attack-Bard
  • imagenet-1k: 1000 cherry-picked images from the imagenet validation set

⚪ Warmup

• run vis_NIPS17.py, try understand what happens
• run run_NIPS17_clf.py, try understand what happens
• run run.py, try understand what happens
• run run.py --atk, try understand what happens

⚪ Your Tasks

Use imagenet-1k as the ref-data to remove adv noise on ssa-cwa-200 (pregen adv of NIPS17) Our final goal: let run.py --atk --dfn work! :)

• implement defenses.vector_db
• implement defenses.img_hifreq
• implement defenses.patch_replace

references

• rvc-project: https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI
• the-ever-lasting-adversarial-war: https://github.com/Kahsolt/the-ever-lasting-adversarial-war

---

by Armit 2023/10/26