Question about "packed mode"

[99DHL]

Does 'packed mode' yield different training results compared to using 'packed=False'? I thought 'packed mode' does not apply any approximations, but it seems to produce slightly different results. (PSNR is almost the same, but the number of gaussians is slightly different)

[liruilong940607]

It could yield different results. In packed mode the Gaussians with zero gradients will be skipped from updates. While in normal mode these zero-gradient Gaussians will still be updated because of the momentum in optimizer.

[99DHL]

Thank you for your kind response.
I looked at the code, and it seems that when using packed mode, Adam is used instead of SparseAdam. So I thought that even if the gradients are sparse, the optimizer and weight updates are still performed for all parameters. Did I misunderstand this? Also, if packed mode behaves differently from normal mode, could you explain how it differs from the sparse_grad mode that uses SparseAdam?

[liruilong940607]

Ah I thought you were asking packed along with sparse_grad.

Setting packed itself does not affect gradient update, only if you also set sparse_grad.

If you only set packed, then it should not affect the optimization process in anyway. Maybe the difference you see between runs are from the nondeterministic randomness in pytorch.

[99DHL]

Got it, thanks for the clarification!