Release v1.3.1 of NNCF on Github

Author: vshampor

.gitattributes

@@ -0,0 +1 @@
+*.png filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,111 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+env/
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+results.html
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# dotenv
+.env
+
+# virtualenv
+.venv
+venv/
+ENV/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+
+# PyCharm
+.idea
+
+# snapshots
+*.tar
+
+# object detection eval results
+examples/object_detection/eval/

.pylintrc

@@ -0,0 +1,37 @@
+[MASTER]
+disable = arguments-differ,
+          cell-var-from-loop,
+          fixme,
+          global-statement,
+          invalid-name,
+          logging-format-interpolation,
+          missing-docstring,
+          no-self-use,
+          not-callable,
+          too-few-public-methods,
+          too-many-arguments,
+          too-many-instance-attributes,
+          too-many-locals,
+          unbalanced-tuple-unpacking,
+          ungrouped-imports,
+          unpacking-non-sequence,
+          unused-argument,
+          wrong-import-order,
+          attribute-defined-outside-init,
+          import-outside-toplevel
+
+max-line-length = 120
+ignore-docstrings = yes
+ignored-modules = numpy,torch,cv2,openvino
+extension-pkg-whitelist = torch,cv2
+
+[SIMILARITIES]
+ignore-imports = yes
+
+[BASIC]
+bad-functions = print
+good-names = logger,fn
+
+[DESIGN]
+max-statements=60
+max-branches=13

README.md

@@ -0,0 +1,141 @@
+# Neural Network Compression Framework (NNCF)
+
+This module contains a PyTorch\*-based framework and samples for neural networks compression. The framework is organized as a Python\* package that can be built and used in a standalone mode. The framework architecture is unified to make it easy to add different compression methods. The samples demonstrate the usage of compression algorithms for three different use cases on public models and datasets: Image Classification, Object Detection and Semantic Segmentation.
+
+## Key Features
+
+- Support of various compression algorithms, applied during a model fine-tuning process to achieve best compression parameters and accuracy:
+    - [Quantization](./docs/compression_algorithms/Quantization.md)
+    - [Binarization](./docs/compression_algorithms/Binarization.md)
+    - [Sparsity](./docs/compression_algorithms/Sparsity.md)
+    - [Filter pruning](./docs/compression_algorithms/Pruning.md)
+- Automatic, configurable model graph transformation to obtain the compressed model. The source model is wrapped by the custom class and additional compression-specific layers are inserted in the graph.
+- Common interface for compression methods
+- GPU-accelerated layers for faster compressed model fine-tuning
+- Distributed training support
+- Configuration file examples for each supported compression algorithm.
+- Git patches for prominent third-party repositories ([mmdetection](https://github.com/open-mmlab/mmdetection), [huggingface-transformers](https://github.com/huggingface/transformers)) demonstrating the process of integrating NNCF into custom training pipelines
+- Exporting compressed models to ONNX\* checkpoints ready for usage with [OpenVINO™ toolkit](https://github.com/opencv/dldt).
+
+## Usage
+The NNCF is organized as a regular Python package that can be imported in your target training pipeline script.
+The basic workflow is loading a JSON configuration script containing NNCF-specific parameters determining the compression to be applied to your model, and then passing your model along with the configuration script to the `nncf.create_compressed_model` function.
+This function returns a wrapped model ready for compression fine-tuning, and handle to the object allowing you to control the compression during the training process:
+
+```python
+import nncf
+from nncf import create_compressed_model, Config as NNCFConfig
+
+# Instantiate your uncompressed model
+from torchvision.models.resnet import resnet50
+model = resnet50()
+
+# Load a configuration file to specify compression
+nncf_config = NNCFConfig.from_json("resnet50_int8.json")
+
+# Provide data loaders for compression algorithm initialization, if necessary
+nncf_config = register_default_init_args(nncf_config, loss_criterion, train_loader)
+
+# Apply the specified compression algorithms to the model
+comp_ctrl, compressed_model = create_compressed_model(model, nncf_config)
+
+# Now use compressed_model as a usual torch.nn.Module to fine-tune compression parameters along with the model weights
+
+# ... the rest of the usual PyTorch-powered training pipeline
+
+# Export to ONNX or .pth when done fine-tuning
+comp_ctrl.export_model("compressed_model.onnx")
+torch.save(compressed_model.state_dict(), "compressed_model.pth")
+```
+
+For a more detailed description of NNCF usage in your training code, see [Usage.md](./docs/Usage.md). For in-depth examples of NNCF integration, browse the [sample scripts](#Model Compression Samples) code, or the [example patches](#Third-party repository integration) to third-party repositories.
+
+For more details about the framework architecture, refer to the [NNCFArchitecture.md](./docs/NNCFArchitecture.md).
+
+
+### Model Compression Samples
+
+For a quicker start with NNCF-powered compression, you can also try the sample scripts, each of which provides a basic training pipeline for classification, semantic segmentation and object detection neural network training correspondingly.
+
+To run the samples please refer to the corresponding tutorials:
+- [Image Classification sample](examples/classification/README.md)
+- [Object Detection sample](examples/object_detection/README.md)
+- [Semantic Segmentation sample](examples/semantic_segmentation/README.md)
+
+### Third-party repository integration
+NNCF may be straightforwardly integrated into training/evaluation pipelines of third-party repositories.
+See [third_party_integration](./third_party_integration) for examples of code modifications (Git patches and base commit IDs are provided) that are necessary to integrate NNCF into select repositories.
+
+
+### System requirements
+- Ubuntu\* 16.04 or later (64-bit)
+- Python\* 3.6 or later
+- NVidia CUDA\* Toolkit 10.2 or later
+- PyTorch\* 1.5 or higher.
+
+### Installation
+We suggest to install or use the package in the [Python virtual environment](https://docs.python.org/3/tutorial/venv.html).
+
+#### As a package built from checked-out repository:
+1) Install the following system dependencies:
+
+`sudo apt-get install python3-dev`
+
+2) Install the package and its dependencies by running the following in the repository root directory:
+
+- For CPU & GPU-powered execution:
+`python setup.py install`
+- For CPU-only installation
+`python setup.py install --cpu-only`
+
+#### As a Docker image
+Use one of the Dockerfiles in the [docker](./docker) directory to build an image with an environment already set up and ready for running NNCF [sample scripts](#Model Compression Samples).
+
+
+## NNCF compression results
+
+Achieved using sample scripts and NNCF configuration files provided with this repository. See README.md files for [sample scripts](#Model Compression Samples) for links to exact configuration files and final PyTorch checkpoints.
+
+
+|Model|Compression algorithm|Dataset|PyTorch FP32 baseline|PyTorch compressed accuracy|
+| :---: | :---: | :---: | :---: | :---: |
+|ResNet-50|None|ImageNet|-|76.13|
+|ResNet-50|INT8|ImageNet|76.13|76.05|
+|ResNet-50|Mixed, 44.8% INT8 / 55.2% INT4|ImageNet|76.13|76.3|
+|ResNet-50|INT8 + Sparsity 61% (RB)|ImageNet|76.13|75.28|
+|ResNet-50|Filter pruning, 30%, magnitude criterion|ImageNet|76.13|75.7|
+|ResNet-50|Filter pruning, 30%, geometric median criterion|ImageNet|76.13|75.7|
+|Inception V3|None|ImageNet|-|77.32|
+|Inception V3|INT8|ImageNet|77.32|76.92|
+|Inception V3|INT8 + Sparsity 61% (RB)|ImageNet|77.32|76.98|
+|MobileNet V2|None|ImageNet|-|71.81|
+|MobileNet V2|INT8|ImageNet|71.81|71.34|
+|MobileNet V2|Mixed, 46.6% INT8 / 53.4% INT4|ImageNet|71.81|70.89|
+|MobileNet V2|INT8 + Sparsity 52% (RB)|ImageNet|71.81|70.99|
+|SqueezeNet V1.1|None|ImageNet|-|58.18|
+|SqueezeNet V1.1|INT8|ImageNet|58.18|58.02|
+|SqueezeNet V1.1|Mixed, 54.7% INT8 / 45.3% INT4|ImageNet|58.18|58.84|
+|ResNet-18|None|ImageNet|-|69.76|
+|ResNet-18|XNOR (weights), scale/threshold (activations)|ImageNet|69.76|61.61|
+|ResNet-18|DoReFa (weights), scale/threshold (activations)|ImageNet|69.76|61.59|
+|ResNet-18|Filter pruning, 30%, magnitude criterion|ImageNet|69.76|68.69|
+|ResNet-18|Filter pruning, 30%, geometric median criterion|ImageNet|69.76|68.97|
+|ResNet-34|None|ImageNet|-|73.31|
+|ResNet-34|Filter pruning, 30%, magnitude criterion|ImageNet|73.31|72.54|
+|ResNet-34|Filter pruning, 30%, geometric median criterion|ImageNet|73.31|72.60|
+|SSD300-BN|None|VOC12+07|-|78.28|
+|SSD300-BN|INT8|VOC12+07|78.28|78.07|
+|SSD300-BN|INT8 + Sparsity 70% (Magnitude)|VOC12+07|78.28|78.01|
+|SSD512-BN|None|VOC12+07|-|80.26|
+|SSD512-BN|INT8|VOC12+07|80.26|80.02|
+|SSD512-BN|INT8 + Sparsity 70% (Magnitude)|VOC12+07|80.26|79.98|
+|UNet|None|CamVid|-|71.95|
+|UNet|INT8|CamVid|71.95|71.66|
+|UNet|INT8 + Sparsity 60% (Magnitude)|CamVid|71.95|71.72|
+|ICNet|None|CamVid|-|67.89|
+|ICNet|INT8|CamVid|67.89|67.87|
+|ICNet|INT8 + Sparsity 60% (Magnitude)|CamVid|67.89|67.24|
+|UNet|None|Mapillary|-|56.23|
+|UNet|INT8|Mapillary|56.23|56.12|
+|UNet|INT8 + Sparsity 60% (Magnitude)|Mapillary|56.23|56.0|
+

ReleaseNotes.md

@@ -0,0 +1,55 @@
+# Release Notes
+
+## Introduction
+*Neural Network Compression Framework (NNCF)* is a toolset for Neural Networks model compression.
+The framework organized as a Python module that can be built and used as standalone or within
+samples distributed with the code.  The samples demonstrate the usage of compression methods on
+public models and datasets for three different use cases: Image Classification, Object Detection,
+and Semantic Segmentation.
+
+## New in Release 1.3.1
+- Now using PyTorch 1.5 and CUDA 10.2 by default
+- Support for exporting quantized models to ONNX checkpoints with standard ONNX v10 QuantizeLinear/DequantizeLinear pairs (8-bit quantization only)
+- Compression algorithm initialization moved to the compressed model creation stage
+
+## New in Release 1.3:
+- Filter pruning algorithm added
+- Mixed precision quantization with manual and automatic (HAWQ-powered) precision setup
+- Support for DistilBERT
+- Selecting quantization parameters based on hardware configuration preset (CPU, GPU, VPU)
+- Propagation-based quantizer position setup mode (quantizers are position as early in the network control flow graph as possible while keeping inputs of target operation quantized)
+- Improved model graph tracing with introduction of input nodes and intermediate tensor shape tracking
+- Updated third-party integration patches for consistency with NNCF release v1.3
+- CPU-only installation mode for execution on machines without CUDA GPU hardware installed
+- Docker images supplied for easier setup in container-based environments
+- Usability improvements (NNCF config .JSON file validation by schema, less boilerplate code, separate logging and others)
+
+## New in Release 1.2:
+- Support for transformer-based networks quantization (tested on BERT and RoBERTa)
+- Added instructions and Git patches for integrating NNCF into third-party repositories ([mmdetection](https://github.com/open-mmlab/mmdetection), [transformers](https://github.com/huggingface/transformers))
+- Support for GNMT quantization
+- Regular expression format support for specifying ignored/target scopes in config files - prefix the regex-enabled scope with {re}
+
+## New in Release 1.1
+
+- Binary networks using XNOR and DoReFa methods
+- Asymmetric quantization scheme and per-channel quantization of Convolution
+- 3D models support
+- Support of integration into the [mmdetection](https://github.com/open-mmlab/mmdetection) repository
+- Custom search patterns for FakeQuantize operation insertion
+- Quantization of the model input by default
+- Support of quantization of non-ReLU models (ELU, sigmoid, swish, hswish, and others)
+
+## New in Release 1.0
+
+- Support of symmetric quantization and two sparsity algorithms with fine-tuning
+- Automatic model graph transformation. The model is wrapped by the custom class and additional layers are inserted in the graph. The transformations are configurable.
+- Three training samples which demonstrate usage of compression methods from the NNCF:
+    - Image Classification:  torchvision models for classification and custom models on ImageNet and CIFAR10/100 datasets.
+    - Object Detection: SSD300, SSD512, MobileNet SSD on Pascal VOC2007, Pascal VOC2012, and COCO datasets.
+    - Semantic Segmentation: UNet, ICNet on CamVid and Mapillary Vistas datasets.
+- Unified interface for compression methods.
+- GPU-accelerated *Quantization* layer for fast model fine-tuning.
+- Distributed training support in all samples.
+- Configuration file examples for sparsity, quantization and sparsity with quantization for all three samples. Each type of compression requires only one additional stage of fine-tuning.
+- Export models to the ONNX format that is supported by the [OpenVINO](https://github.com/opencv/dldt) toolkit.

docker/README.md

@@ -0,0 +1,30 @@
+## Step 1. Install docker
+Review the instructions for installation docker [here](https://docs.docker.com/engine/install/ubuntu/) and configure HTTP or HTTPS proxy behavior as [here](https://docs.docker.com/config/daemon/systemd/).
+
+## Step 2. Install nvidia-docker
+
+*Skip this step if you don't have GPU.*
+
+Review the instructions for installation docker [here](https://github.com/NVIDIA/nvidia-docker)
+
+## Step 3. Build image
+In the project folder run in terminal:
+ ```
+ sudo docker image build --network=host --build-arg http_proxy=http://example.com:80 --build-arg https_proxy=http://example.com:81 --build-arg 
+ftp_proxy=http://example.com:80 <PATH_TO_DIR_WITH_DOCKERFILE>
+ ```
+
+*Use `--http_proxy` , `--https_proxy`, `--ftp_proxy`, `--network` to duplicate the network settings of your localhost into context build*
+  
+## Step 4. Run container
+Run in terminal:
+```
+sudo docker run --name <NAME_CONTAINER> --runtime=nvidia -it --network=host --mount type=bind,source=<PATH_TO_DATASETS_ON_HOST>,target=<PATH_TO_DATSETS_IN_CONTAINER>  --mount type=bind,source=<PATH_TO_NNCF_HOME_ON_HOST>,target=/home/nncf/ <ID_IMAGE>
+ ```
+
+*You should not use `--runtime=nvidia` if you want to use `--cpu-only` mode.* 
+
+*Use `--shm-size` to increase the size of the shared memory directory.*
+
+Now you have a working container and you can run examples.
+