How to deploy a custom trained ASR model on Android #5425
Replies: 4 comments 21 replies
-
|
Thanks for your question. |
Beta Was this translation helpful? Give feedback.
-
|
You have two ways:
|
Beta Was this translation helpful? Give feedback.
-
|
The best compromises i could make to recreate your working environment where to have everything exactly like yours but set the following packages with such versions... espnet-onnx==0.1.10 Unfortunately the conversion still gave the following error Traceback (most recent call last): |
Beta Was this translation helpful? Give feedback.
-
|
I cannot be sure, but maybe the issue is the ONNX version. Regarding the conflict between packages, I did not care when I installed it, and did not show any issue with generating the ONNX model. So I would suggest to delete the pip installation from the yaml I shared and just create the environment with the conda package, then install one by one the pip packages as listed before: |
Beta Was this translation helpful? Give feedback.
-
|
I tried removing all packages installed using pip and then tried installing "espnet_onnx, espnet, espnet_zoo_download, and onnx" and eventually, the following eeros resurfaced again. Traceback (most recent call last): I then tried it with Ubuntu 22, and again i end with the following error when i try to duplicate your environment. The conflict is caused by: @Fhrozen, Thanks for all your help but is there anything else you would advise me to try. Because something just doesn't add up. How where you able to create that specific environment? and i just can't seem to replicate it. |
Beta Was this translation helpful? Give feedback.
-
|
Dear @Fhrozen, It seems that no matter what i do, I can't have all the specified packages installed together without package version conflicts. So now, I just installed the packages in the following order even when at the end of each package installation errors of package version conflicts where arising. I just neglected that. And now even though the packages are incorrectly installed, the conversion code works just fine. Ofcourse the conversion fires warnings as usual if i have optimize and quantize options set to True. I am doing the conversion on Windows 11, Anaconda 23.09-0, a seperate conda environment with python 3.9.18 and pytorch. Later the order of the packages installed using pip are espnet, espnet_onnx, espnet_model_zoo, and onnx. if i have optimize and quantize options set to True, C:\Users\Admin\AppData\Roaming\Python\Python39\site-packages\torch\onnx\symbolic_opset9.py:4476: UserWarning: Exporting a model to ONNX with a batch_size other than 1, with a variable length with LSTM can cause an error when running the ONNX model with a different batch size. Make sure to save the model with a batch size of 1, or define the initial states (h0/c0) as inputs of the model. C:\Users\Admin.cache\espnet_onnx\asr_train_asr_transformer_raw_ti_char_sp_specaug_lr_001_steps_27000_33hr_30min_valid.acc.ave\full\xformer_encoder.opt.onnx C:\Users\Admin\anaconda3\envs\onnx\lib\site-packages\espnet_onnx\export\asr\export_asr.py:336: UserWarning: You cannot optimize TransformerDecoder or TransformerLM without custom version of onnxruntime.Please follow the instruction on README.md to install onnxruntime for espnet_onnx WARNING:root:Please consider to run pre-processing before quantization. Refer to example: https://github.com/microsoft/onnxruntime-inference-examples/blob/main/quantization/image_classification/cpu/ReadMe.md Please let me know if this is something i have to worry about. Again, Thanks for your relentless help @Fhrozen! 🙏🙏🙏 |
Beta Was this translation helpful? Give feedback.
-
|
If you have docker, then you can test the following code; for building:
FROM ubuntu:22.04
# RUN conda create --name venv
# conda activate venv && \
RUN apt-get update && \
apt-get -y install --no-install-recommends \
build-essential \
wget \
&& \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
ENV PATH=/opt/miniconda/bin:${PATH}
RUN wget --no-check-certificate --tries=3 -nv "https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh" -O miniconda.sh && \
bash miniconda.sh -b -p /opt/miniconda && \
rm miniconda.sh
RUN conda install -y python=3.9 && \
conda install -y pytorch torchvision torchaudio cpuonly -c pytorch && \
conda install -y matplotlib
RUN pip install espnet_onnx && \
pip install espnet && \
pip install espnet_model_zoo && \
pip install onnx
ENV THIS_USER=ubuntu
RUN useradd -rm -d /home/${THIS_USER} -u 1001 -g root ${THIS_USER}
USER ${THIS_USER}
RUN mkdir -p /home/${THIS_USER}/.cache/espnet_onnx
WORKDIR /home/${THIS_USER}build with:
Then, in a folder I named
move to ~/models, and execute there you python code from espnet2.bin.asr_inference import Speech2Text
from espnet_onnx.export import ASRModelExport
m = ASRModelExport(cache_dir='/home/ubuntu/models')
m.set_export_config(max_seq_len=3000)
m.export_from_zip("/home/ubuntu/models/pretrained_model_asr.zip", "asr_model", quantize=False)You will get the onnx files in the Best |
Beta Was this translation helpful? Give feedback.
-
I think that it should be fine. Try testing the model with a different file: import librosa
from espnet_onnx import Speech2Text
speech2text = Speech2Text(tag_name='asr_model')
y, sr = librosa.load('/export/corpus01/LibriSpeech/test-clean/1089/134686/1089-134686-0000.flac', sr=16000)
nbest = speech2text(y)
print(nbest[0][0]) |
Beta Was this translation helpful? Give feedback.
-
|
I tried testing the model with an audio file and it generated the following error... 2023-10-14 16:31:04.5390720 [E:onnxruntime:, sequential_executor.cc:514 onnxruntime::ExecuteKernel] Non-zero status code returned while running Gather node. Name:'/make_pad_mask/Gather' Status Message: indices element out of data bounds, idx=1043 must be within the inclusive range [-512,511] I tried using the audio file i have attached in a zip file. |
Beta Was this translation helpful? Give feedback.
-
|
Dear @Fhrozen , I am happy to tell you that, the docker instructions you gave worked without any problems. I have even made an inference of an audio with that model. The instructions you gave me should be included in the espnet docs. It would sure help others alot. Now that i have the onnx models, how do i use them for the Android App? I visted the link you suggested but i it only has demo apps for image recognition. Thanks for all your help and guidance once again. 🙏🙏🙏 |
Beta Was this translation helpful? Give feedback.
-
|
Did you check: |
Beta Was this translation helpful? Give feedback.
-
|
Maybe, you can have a look at this example using Whisper: https://github.com/microsoft/onnxruntime-inference-examples/tree/main/mobile/examples/whisper/local/android. |
Beta Was this translation helpful? Give feedback.
-
|
@Fhrozen, Many thanks for your help. If anyone has tried it before, please let me know. Thanks |
Beta Was this translation helpful? Give feedback.
-
Dear Team,
Is it possible to deploy a custom trained ESPNET ASR model on Android?
If yes, could you please advise on how to do it?
PS. I am using ESPnet 2.
Thank you in advance!
Beta Was this translation helpful? Give feedback.
All reactions