CUDNN_STATUS_INTERNAL_ERROR error

[comgs2]

Posting rules

1. No duplicated posts, only 1 new post opened a day, and up to 2 opened a week. Otherwise, extrict user bans will occur.
   • Check the FAQ section, other GitHub issues, and general documentation before posting. E.g., low-speed, out-of-memory, output format, 0-people detected, installation issues, ...).
   • Keep posting all your issues in the same post.
   • No bans if you are unsure whether the post is duplicated!
2. Fill all the Your System Configuration section if you are facing an error or unexpected behavior. Some posts (e.g., feature requests) might not require it.
3. No questions about training or 3rd party libraries:
   • OpenPose only implements testing. For training, check OpenPose train.
   • Caffe errors/issues: Check Caffe documentation.
   • CUDA/cuDNN check failed errors: They are usually fixed by re-installing CUDA, then re-installing the proper cuDNN version, then rebooting, and then re-installing OpenPose. Otherwise, check Nvidia/CUDA/cuDNN forums.
   • OpenCV errors: Install the default/pre-compiled OpenCV or check for OpenCV online help.
4. Set a proper issue title: Add the OS (Ubuntu, Windows) and be specific (e.g., do not call it: Error).
5. Only English comments.
6. Remove these posting rules from your post but follow them!
   Posts which do not follow these rules will be ignored/deleted and those users banned with no further clarification.

Issue Summary

CUDNN_STATUS_INTERNAL_ERROR has occurred.

Executed Command (if any)

Note: add --logging_level 0 --disable_multi_thread to get higher debug information.

./build/examples/openpose/openpose.bin --video examples/media/video.avi --face --hand

OpenPose Output (if any)

$ ./build/examples/openpose/openpose.bin --video examples/media/video.avi --face --hand
Starting OpenPose demo...
Configuring OpenPose...
Starting thread(s)...
Auto-detecting all available GPUs... Detected 1 GPU(s), using 1 of them starting at GPU 0.
F1218 14:40:00.012526 763422 cudnn_relu_layer.cpp:13] Check failed: status == CUDNN_STATUS_SUCCESS (4 vs. 0) CUDNN_STATUS_INTERNAL_ERROR
*** Check failure stack trace: ***
@ 0x7f54e6fadb03 google::LogMessage::Fail()
@ 0x7f54e6fb59d1 google::LogMessage::SendToLog()
@ 0x7f54e6fad7c2 google::LogMessage::Flush()
@ 0x7f54e6faf78f google::LogMessageFatal::~LogMessageFatal()
@ 0x7f54e5da8850 caffe::CuDNNReLULayer<>::LayerSetUp()
@ 0x7f54e5e7f65c caffe::Net<>::Init()
@ 0x7f54e5e81550 caffe::Net<>::Net()
@ 0x7f54e6c1f49c op::NetCaffe::initializationOnThread()
@ 0x7f54e6c03bdd op::HandExtractorCaffe::netInitializationOnThread()
@ 0x7f54e6c051f6 op::HandExtractorNet::initializationOnThread()
@ 0x7f54e6c66cb5 op::Worker<>::initializationOnThreadNoException()
@ 0x7f54e6c66e28 op::SubThread<>::initializationOnThread()
@ 0x7f54e6c68838 op::Thread<>::initializationOnThread()
@ 0x7f54e6c6adbc op::Thread<>::threadFunction()
@ 0x7f54e66dc253 (unknown)
@ 0x7f54e6294ac3 (unknown)
@ 0x7f54e6326660 (unknown)
@ (nil) (unknown)
中止 (コアダンプ)

Errors (if any)

CUDNN_STATUS_INTERNAL_ERROR

Type of Issue

Select the topic(s) on your post, delete the rest:

• Execution error

Your System Configuration

1. Whole console output (if errors appeared), paste the error to PasteBin and then paste the link here: LINK

$ ./build/examples/openpose/openpose.bin --video examples/media/video.avi --face --hand
Starting OpenPose demo...
Configuring OpenPose...
Starting thread(s)...
Auto-detecting all available GPUs... Detected 1 GPU(s), using 1 of them starting at GPU 0.
F1218 14:40:00.012526 763422 cudnn_relu_layer.cpp:13] Check failed: status == CUDNN_STATUS_SUCCESS (4 vs. 0) CUDNN_STATUS_INTERNAL_ERROR
*** Check failure stack trace: ***
@ 0x7f54e6fadb03 google::LogMessage::Fail()
@ 0x7f54e6fb59d1 google::LogMessage::SendToLog()
@ 0x7f54e6fad7c2 google::LogMessage::Flush()
@ 0x7f54e6faf78f google::LogMessageFatal::~LogMessageFatal()
@ 0x7f54e5da8850 caffe::CuDNNReLULayer<>::LayerSetUp()
@ 0x7f54e5e7f65c caffe::Net<>::Init()
@ 0x7f54e5e81550 caffe::Net<>::Net()
@ 0x7f54e6c1f49c op::NetCaffe::initializationOnThread()
@ 0x7f54e6c03bdd op::HandExtractorCaffe::netInitializationOnThread()
@ 0x7f54e6c051f6 op::HandExtractorNet::initializationOnThread()
@ 0x7f54e6c66cb5 op::Worker<>::initializationOnThreadNoException()
@ 0x7f54e6c66e28 op::SubThread<>::initializationOnThread()
@ 0x7f54e6c68838 op::Thread<>::initializationOnThread()
@ 0x7f54e6c6adbc op::Thread<>::threadFunction()
@ 0x7f54e66dc253 (unknown)
@ 0x7f54e6294ac3 (unknown)
@ 0x7f54e6326660 (unknown)
@ (nil) (unknown)
中止 (コアダンプ)

2. OpenPose version: Latest GitHub code? Or specific commit (e.g., d52878f)? Or specific version from Release section (e.g., 1.2.0)?

master

4. General configuration:

   • Installation mode: default

   • Operating system (lsb_release -a in Ubuntu):
     No LSB modules are available.
     Distributor ID: Ubuntu
     Description: Ubuntu 22.04.3 LTS
     Release: 22.04
     Codename: jammy

   • Operating system version (e.g., Ubuntu 16, Windows 10, ...):
     ubnutu 22.04

   • Release or Debug mode? (by default: release): release

   • Compiler : gcc version 11.4.0

5. Non-default settings:

   • 3-D Reconstruction module added? (by default: no):
   • Any other custom CMake configuration with respect to the default version? (by default: no):

6. 3rd-party software:

   • Caffe version: Default from OpenPose, custom version, ...?
   • CMake version (cmake --version in Ubuntu):
   • OpenCV version: pre-compiled apt-get install libopencv-dev (only Ubuntu); OpenPose default (only Windows); compiled from source? If so, 2.4.9, 2.4.12, 3.1, 3.2?; ...?

7. If GPU mode issue:

   • CUDA version (cat /usr/local/cuda/version.txt in most cases): cuda 12.1
   • cuDNN version: libcudnn8-dev 8.9.3.28-1+cuda12.1
   • GPU model (nvidia-smi in Ubuntu):RTX 3090

8. If CPU-only mode issue:

   • CPU brand & model:
   • Total RAM memory available:

9. If Python API:

   • Python version: 2.7, 3.7, ...?:3.10
   • Numpy version (python -c "import numpy; print numpy.version.version" in Ubuntu):1.26.2

10. If Windows system:

    • Portable demo or compiled library?

11. If speed performance issue:

    • Report OpenPose timing speed based on the profiling documentation.