-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GPU Processing Issue after latest driver updates #4
Comments
Dear Sveeggen, were you able to resolve this issue? I am in the same rabbit hole and I have tried every driver there is. Any of them works. I know my Fiji is also communicating with my GPU. For example, CLIJ works but NanoJ does not. |
Sadly I am still having trouble. We have ended up using an older computer to run it - though it takes around 3 minutes per image, that's at least manageable. I worry there needs to be an update or something to work with newer computers, but I have no idea how to go about that. |
Congrats Dr. Henriques in starting your lab at IGC!! I have the same error and message about openCL: I tested the openCL with the NanoJ-core test for openCL and the animation and test runs properly. I have tried in my Lenovo Legion Y540 with a Geforce RTX 2060 and in my lab's microscope computer HP 4 with a Geforce Quadro . I have tried in windows 10 and in Ubuntu 20.04. From my Ubuntu install NanoJ-Core tests. I know that in linux the CUDA components changed recently. Would this be connected to it? |
This config works for me:
Please let me know if it works for you. |
Did not help me (windows). Still drops to CPU mode. |
Okay. I think it's something to do with the 440+ drivers. It did work. Red hat enterprise Linux 8 we are actually getting an EMCCD with SRRF, but I'd like to also do SRRF with my CMOS as well. I'm installing the most recent 440 right now to see if I can reproduce the error. EDIT1: Outputs from the app nvidia-smi tool in linux (to show versions) Fail: EDIT2: |
I am also experiencing this problem on a 2070S under Windows 10 +-----------------------------------------------------------------------------+ |
I was having the same error (...CL exec seems to have failed...) and I was able to resolve it on a Windows 10 machine with an NVIDIA Quadro P4000. To resolve the error, I reverted back to the original drivers from March, 16 2018. The driver version according to the Windows Device Manager is: 23.21.13.9125. To revert back to these drivers, I went in through: The earliest driver was from March 16, 2018 (presumably from when the system was built). So, I picked that driver and restarted the system. SRRF is now working at GPU speeds after doing the above. I am not the most savvy GPU person and so I don't know the difference between the Windows Device Manager version numbers and the 450, 440, 430 numbers listed on the Nvidia downloads page. However, I was not able to get SRRF to work with either the 440 or 430 driver installers that I downloaded from the Nvidia webpage. It was only when I went in through Device Manager that SRRF started working again. I hope this helps someone. |
Thank you for replying, unfortunately the newer graphic card models (like my 2070S and the upcoming 3XXX models) do not have the option of rolling back to older drivers, the earliest driver supporting my graphic card happens to have a higher version number than the one you are using. Repacking the project with a newer Aparapi version could fix the issue, but I lack the skills to do so. The whole project feels abandoned (Especially the Bitbucket dead links). |
I tried to open the Nano-Updater.jar, and just replace the naparapi .so files inside to see if that would work. Doing so breaks NanoJ. The package that needs the update is in NanoJ-core (updater). The .dll and .so files are in that github, and I believe they come from aparapi-jni |
I'm trying to get this recompiled with more recent aparapi. I am trying to recompile everything and I spent a fair amount of time finding maven libraries. |
I managed to make a jar with the latest aparapi. I have this problem in Fiji. If it fixed the nvidia driver issue,I will share the jars and try to fork the updates I did to nanoj-core into github.
|
@joaomamede have you managed to get a working version for the plugin? I belong to this unlucky group whose drivers are too new for the official version |
I did not. I compiled it but i have never done Java or fiji plug-ins. Some of the libs should come from fiji, some from maven and things got messed up and it does not run. |
Old bitbucket stuff can be found here: https://web.archive.org/web/20200622021429/https://bitbucket.org/rhenriqueslab/nanoj-core/wiki/Home Something interesting is, that nano-core always picks the best card. |
Thank you so much for the link! After so much trial and error, I finally understand much better what each setting is doing and that several settings are absolutely not useful for my setup (High density non blinking emitters). Regarding your linux issue, if you are running Ubuntu, you should be able to switch off the Nvidia card by using nvidia-prime. Do you think that editing the jar file and the relative classes inside would be easier than compiling the source? |
I tried, but the classes are compiled in binary code in the plugin in fiji, so it has to be recompiled. I managed to compile this github version without any modifications, only by adding the jars from fiji in the libraries declarations. Somehow the resources with the libaparapi.so are not being inserted into the jars. I need to find out how to do it, if anyone has tips on how to do it, please let me know. |
It's not an easy swap. (Fiji Is Just) ImageJ 2.1.0/1.53c; Java 1.8.0_172 [64-bit]; Linux 4.18.0-240.8.1.el8_3.x86_64; 329MB of 37163MB (<1%)
|
Thanks for having tried! Your test was definitely worth a shot to see if a simple swap could help us figure this out. I want to also report that few day ago, the GPU suddenly decided to work, there was no warning and I could definitely see a speed up in the execution speed. Sadly I did not manage to ever reproduce what triggered that state, the only thing I had open at the time was youtube in firefox. I thought that maybe the hardware video decoding somehow kept the GPU in an active state so I also tried to start SRRF while an opencl benchmark (luxmark) was running in the background without success. I am also thinking to buy a recent-ish amd gpu to use exclusively with SRRF. |
Since you are running linux, you might want to give POCL a go. If I understood how this work, after the installation, it should come up as a choice in the CL-Device menu in nanoj or could be selected by an ICD-loader (https://manpages.ubuntu.com/manpages/xenial/man7/libOpenCL.7.html). It supports nvidia opencl acceleration through cuda so it might be a roundabout way to restore GPU acceleration. |
my opencl works fine with both intel-neo and nvidia-cl. It's all of nanoJ-core that doesn't support recent intel GPU (neo), it doesn't even detect it, and nano-srrf specifically, >435 nvidia drivers. |
Apologies in advance, my coding experience is virtually nonexistent, but I can no longer get SRRF to run.
The error:
The last time I used SRRF, I was able to blaze through a .nji file in under 30 seconds, but now it's claiming to require an hour at 0 FPS. Based on my reading, it sounds like the program is no longer able to utilize my GPU? This is frustrating because this plug-in was giving us great data with zero issues prior to now!
Since I last used it about a month ago, I believe there have been updates to FIJI, Java, and my NVIDIA drivers. As a habit I updated these all automatically, but now I can't find old files to roll back to and prove it's the new drivers causing issues. Regarding the NVIDIA drivers, I have tried both the gaming and studio drivers to no avail.
My computer info:
Any ideas? This really is an amazing plug-in that our lab would hate to not be able to use.
Edit: Update
I have also checked OpenCL in ImageJ and tested that OpenCL is able to detect my graphics card as successfully test it with Mandelbrot. So ImageJ and OpenCL can recognize my graphics card but for some reason SRRF can no longer use OpenCL?
The text was updated successfully, but these errors were encountered: