NVIDIA Neural Modules 1.0.0

Release 1.0.0

NeMo 1.0.0 release is a stable version of "1.0.0 release candidate". It substantially improves overall quality and documentation. This update adds support for new tasks such as neural machine translation and many new models pretrained in different languages. As a mature tool for ASR and TTS it also adds new features for text normalization and denormalization, dataset creation based on CTC-segmentation and speech data explorer. These updates will benefit researchers in academia and industry by making it easier for them to develop and train new conversational AI models.

To install this specific version from pip do:

apt-get update && apt-get install -y libsndfile1 ffmpeg
pip install Cython
pip install nemo-toolkit['all']==1.0.0

NVIDIA Neural Modules 1.0.0rc1

Release 1.0.0rc1

This release contains major new models, features and docs improvements.
It is a "candidate" release for 1.0.0.

To install from Pip do:

apt-get update && apt-get install -y libsndfile1 ffmpeg
pip install Cython
pip install nemo_toolkit['all']==1.0.0rc1

It adds the following model architectures:

• CitriNet and Conformer-CTC for ASR
• HiFiGan, MelGan, GlowTTS, UniGlow SqueezeWave for TTS

In NLP collections, a neural machine translation task (NMT) has been added with Transformer-based models.
This release includes pre-trained NMT models for these language pairs (in both directions):

• En<->Es
• En<->Ru
• En<->Zh
• En<->De
• En<->Fr

For ASR task, we also added QuartzNet models, trained on the following languages from Mozilla's Common Voice dataset: Zh, Ru, Es, Pl, Ca, It, Fr and De.
In total, this release adds 60 new pre-trained models.

This release also adds new NeMo tools for:

• Text normalization
• Dataset Creation Tool Based on CTC-Segmentation
• Speech Data Explorer

Known Issues

This version is not compatible with PyTorch 1.8.* Please use 1.7.* with it or use our container.

NVIDIA Neural Modules 1.0.0b4

Release 1.0.0b4

This release is compatible with Jarvis and TLT public beta.
It also updates versions of many dependencies and contains minor bug fixes over 1.0.0b3.

NVIDIA Neural Modules 1.0.0b3

Release 1.0.0b3

This release contains minor bug fixes over 1.0.0b2.
It sets compatible version ranges for Hugging Face Transformers and Pytorch Lightning packages.

NVIDIA Neural Modules 1.0.0b2

Release 1.0.0b2

This release contains stability improvements and bug fixes. It also adds beam search support for CTC based ASR models.

Highlights

• Added beam search and external LM rescoring support for character-based CTC ASR models.
• Switch to Pytorch Lightning version 1.0.5 or above.
• Switch to Hydra version 1.0.3 or above.
• Increase NVIDIA Pytorch container version to 20.09

Known Issues

This version will not work with Hugging Face transformers library version >=4.0.0. Please make sure your transformers library version is transformers>=3.1.0 and <4.0.0.

Toolkit in an early version software.

NVIDIA Neural Modules 1.0.0b1

Release 1.0.0b1

This release is a major re-design compared to previous version.
All NeMo models and modules are now compatible out-of-the box with Pytorch and Pytorch Lightning.
Every NeMo model is a LightningModule that comes equipped with all supporting infrastructure for training and reproducibility. Every NeMo model has an example configuration file and a corresponding script that contains all configurations needed for training. NeMo, Pytorch Lightning, and Hydra makes all NeMo models have the same look and feel so that it is easy to do Conversational AI research across multiple domains. New models such as Speaker verification and Megatron are added.

Highlights

• Pytorch Lightning based Core
• Hydra and Omegaconf configuration management
• All model's files tarred together as .nemo files make it easy for users to download models automatically from NGC
• NGC collections now includes a collection of all NeMo assets in one
• New Models & tutorials
  • ASR: SpeakerNet speaker verification model
  • NLP: Bio Megatron state of the art model trained on bio medical tasks
• ASR, NLP and TTS tutorials as interactive notebooks

Known Issues

Toolkit in an early version software. Breaking changes compared to previous version.

Resolved Issues

All models and modules can be used anywhere torch.nn.Module is expected.

NVIDIA Neural Modules v0.11.0

Release 0.11.0

This release improves ease of use and adds new features

Highlights

• Neural Graphs and NeMo models for ASR
• New models:
  • Voice activity detection
  • Speaker identification
  • Matchboxnet Speech commands
  • Megatron BERT trained on bio medical data
• Various improvements and bugfixes

NVIDIA Neural Modules v0.10.1

Release 0.10.1

This is a bug fix release

Highlights:

• Fixes an issue when distributed training would halt if Loss is NaN instead of skipping the batch
• Other minor fixes

NVIDIA Neural Modules v0.10.0

Release 0.10.0

This release improves overall stability of NeMo, revamps type system and adds new models

Highlights:

• Switch to a single package install (“nemo_toolkit”) with on-demand collections
• New, easier-to-use and more flexible neural type system
• Unified neural module import/export to configuration files
• Various bugfixes fixes and improvements
  • Codebase switched to black formatter
• New models:
  • Speech command recognition
  • TRADE dialog state tracking
  • Question answering
  • Roberta and Albert
  • FastSpeech

NVIDIA Neural Modules v0.9.0

Release 0.9.0

This release contains new features, new models and quality improvements for NeMo.

Highlights:

• Added "nemo_tts" - a Speech Synthesis collection with necessary modules for Tacotron2 and WaveGlow
• Added Mandarin support into nemo_asr and nemo_nlp
• Updated ASR and TTS checkpoints including Mandarin ASR
• Documentation now translated to Mandarin https://nvidia.github.io/NeMo/chinese/intro.html
• Export functionality for deployment
• General improvements and bugfixes