This repository is based off of Microsoft's USB: A Unified Semi-supervised learning Benchmark for CV, NLP, and Audio Classification repository. We specifically only include network traffic data related content in this repository. Refer to the original USB repository for the full original content.
Table of Contents
USB is a Pytorch-based Python package for Semi-Supervised Learning (SSL). It is easy-to-use/extend, affordable to small groups, and comprehensive for developing and evaluating SSL algorithms. USB provides the implementation of 14 SSL algorithms based on Consistency Regularization, and 15 tasks for evaluation from CV, NLP, and Audio domain.
This is an example of how to set up USB locally. To get a local copy up, running follow these simple example steps.
USB is built on pytorch, with torchvision, torchaudio, and transformers.
To install the required packages, you can create a conda environment:
conda create --name usb python=3.8then use pip to install required packages:
pip install -r requirements.txtFrom now on, you can start use USB by typing
python train.py --c config/usb_cv/fixmatch/fixmatch_cifar100_200_0.yamlWe provide a Python package semilearn of USB for users who want to start training/testing the supported SSL algorithms on their data quickly:
pip install semilearnYou can also develop your own SSL algorithm and evaluate it by cloning USB:
git clone https://github.com/microsoft/Semi-supervised-learning.gitThe detailed instructions for downloading and processing are shown in Dataset Download. Please follow it to download datasets before running or developing algorithms.
USB is easy to use and extend. Going through the bellowing examples will help you familiar with USB for quick use, evaluate an existing SSL algorithm on your own dataset, or developing new SSL algorithms.
Please see Installation to install USB first. We provide colab tutorials for:
Step1: Check your environment
You need to properly install Docker and nvidia driver first. To use GPU in a docker container
You also need to install nvidia-docker2 (Installation Guide).
Then, Please check your CUDA version via nvidia-smi
Step2: Clone the project
git clone https://github.com/microsoft/Semi-supervised-learning.gitStep3: Build the Docker image
Before building the image, you may modify the Dockerfile according to your CUDA version.
The CUDA version we use is 11.6. You can change the base image tag according to this site.
You also need to change the --extra-index-url according to your CUDA version in order to install the correct version of Pytorch.
You can check the url through Pytorch website.
Use this command to build the image
cd Semi-supervised-learning && docker build -t semilearn .Job done. You can use the image you just built for your own project. Don't forget to use the argument --gpu when you want
to use GPU in a container.
Here is an example to train FixMatch on the balanced UNSW-NB15 dataset with 100 labels per class. Training other supported algorithms (on other datasets with different label settings) can be specified by a config file:
python train.py --c config/usb_network/fixmatch/fixmatch_unsw_img_balanced_1k_0.yamlAfter training, you can check the evaluation performance on training logs, or running evaluation script:
python eval.py --dataset cifar100 --num_classes 100 --load_path /PATH/TO/CHECKPOINT
Distributed under the MIT License. See LICENSE.txt for more information.
The USB comunity is maintained by:
- Robin Bhoo (rbhoo@andrew.cmu.edu), Carnegie Mellon University
We thank the following projects for reference: