Coqui Docker image on windows host using WSL2

[rurounijones]

With the Release of Windows 10H2 we now have the ability to access the GPU from WSL2.

This means that, as far as I can tell, we should be able to use the Coqui docker image with the Docker WSL2 backend to train our voice models using our GPUs on the windows host. However I cannot detect the gpu inside the docker image.

WARNING: The NVIDIA Driver was not detected.  GPU functionality will not be available.
Use 'nvidia-docker run' to start this container; see
https://github.com/NVIDIA/nvidia-docker/wiki/nvidia-docker .

root@047045fd31fc:/usr/local# nvidia-smi
bash: nvidia-smi: command not found

root@047045fd31fc:/usr/local# python3
Python 3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
2021-12-25 01:32:03.631404: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.11.0
WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
>>> device_name = tf.test.gpu_device_name()
2021-12-25 01:32:08.882683: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 4008000000 Hz
2021-12-25 01:32:08.883411: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x53fe1b0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-12-25 01:32:08.883449: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2021-12-25 01:32:08.885125: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda/compat/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64
2021-12-25 01:32:08.885164: E tensorflow/stream_executor/cuda/cuda_driver.cc:322] failed call to cuInit: UNKNOWN ERROR (303)
2021-12-25 01:32:08.885200: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (047045fd31fc): /proc/driver/nvidia/version does not exist

Has anyone got this working? Do we need to install extra stuff into the image like nvidia drivers? I am hesitant to do anything since, as far as I can tell, everything should be setup but I cannot find any solid documentation on coqui + WSL2 + gpu.

Running docker run --gpus all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark following instructions at https://docs.nvidia.com/cuda/wsl-user-guide/index.html#installing-wip seemed to work

Run "nbody -benchmark [-numbodies=<numBodies>]" to measure performance.
        -fullscreen       (run n-body simulation in fullscreen mode)
        -fp64             (use double precision floating point values for simulation)
        -hostmem          (stores simulation data in host memory)
        -benchmark        (run benchmark to measure performance)
        -numbodies=<N>    (number of bodies (>= 1) to run in simulation)
        -device=<d>       (where d=0,1,2.... for the CUDA device to use)
        -numdevices=<i>   (where i=(number of CUDA devices > 0) to use for simulation)
        -compare          (compares simulation results running once on the default GPU and once on the CPU)
        -cpu              (run n-body simulation on the CPU)
        -tipsy=<file.bin> (load a tipsy model file for simulation)

NOTE: The CUDA Samples are not meant for performance measurements. Results may vary when GPU Boost is enabled.

> Windowed mode
> Simulation data stored in video memory
> Single precision floating point simulation
> 1 Devices used for simulation
GPU Device 0: "Pascal" with compute capability 6.1

> Compute 6.1 CUDA device: [NVIDIA GeForce GTX 1080]
20480 bodies, total time for 10 iterations: 14.844 ms
= 282.561 billion interactions per second
= 5651.214 single-precision GFLOP/s at 20 flops per interaction

Any pointers by people with more knowledge would be much appreciated.

[botaurus]

were you ever able to get this working?

[rurounijones]

Negative

[wasertech]

This is tricky but we managed to make it work on WSL with @HarikalarKutusu so I know for a fact it can work.
First thing you need is to be able to access GPU from Docker inside WSL. You need Nvidia's container toolkit for that.

[HarikalarKutusu]

I think in a couple of days I'll have time to write a how-to for a working setup as of 2023/01...