My model inference is taking almost as long as training. Is this normal?

[codenoobforreal]

I followed the Basic Workflow: From Training to Inference guide from the burn book and finished training my model. But when I tried to run inference, I found the inference time is almost the same as the training time. Since it's my first time using burn and model training, I'm not sure if this is expected.

My code matches the example on GitHub. Also, I didn't see the Training Dashboard during training.

experiment.log
config.json

[nathanielsimard]

How do you measure inference time? Are you making sure the execution queue is fully executed ? Also, are you using the autodiff backend even during inference?

[codenoobforreal]

I apologize for initiating the discussion without rigorous verification—I mistakenly included the compilation time in the inference time.
After reviewing the entire project again, I noticed that the profile.release settings in the cargo.toml file of my Cargo workspace included some optimizations that inadvertently increased the compilation time.
After switching the autodiff backend, I reran the inference process. This time, excluding the compilation time, the actual inference took very little time.

[laggui]

Thanks for validating! Will close the discussion in this case 🙂