CUDA error: device-side assert triggered

[Bhavay-2001]

Hi authors. I'm trying to merge your model n setup with a different repo here. I'm trying to solve the errors at test time when I run the classifier_sample_known.py file.
It runs for some time and then throws this error-
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions

I have also tried with PyTorch version - 2.1.0.dev20230523+cu117 as I saw some solution online but that didn't work for me.
Can you please tell what can be the possible solution?
Thanks