You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
/home/user/.conda/envs/bm/lib/python3.7/site-packages/torch/jit/_trace.py:152: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the gradient for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations.
if a.grad is not None:
checkpoint_path: checkpoint/resnet50/128x128_16_0.1_Hair_Color_Wednesday_07_April_2021_23h_21m_26s
Training Epoch: 1 [16/13670] Loss: 1.5864 Acc: 1.0000 LR: 0.000000
epoch 1 training time consumed: 241.82s, Elapsed time: 0:04:01.824466
/opt/conda/conda-bld/pytorch_1607370128159/work/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [0,0,0] Assertion `t >= 0 && t < n_classes` failed.
/opt/conda/conda-bld/pytorch_1607370128159/work/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [13,0,0] Assertion `t >= 0 && t < n_classes` failed.
Traceback (most recent call last):
File "train.py", line 289, in <module>
acc, total_time = eval_training(epoch, total_time)
File "/home/user/.conda/envs/bm/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context
return func(*args, **kwargs)
File "train.py", line 124, in eval_training
test_loss += loss.item()
RuntimeError: CUDA error: device-side assert triggered
The text was updated successfully, but these errors were encountered:
The text was updated successfully, but these errors were encountered: