Solved

[muses0229]

No description provided.

[apchenstu]

Great question. I used 4 A100-40G GPUs with a batch size of 3. I haven't tried training on a single GPU, but I believe the current implementation supports small batch sizes such as 1 or 2.

[yosun]

yejr - did you get it to work on RTX3090

[ChenYutongTHU]

yejr - did you get it to work on RTX3090

Hi. I also experienced the problem.
When running on RTX3090,
CUDA out of memory. Tried to allocate 131062.89 GiB. GPU 0 has a total capacty of 23.69 GiB of which 22.45 GiB is free. Including non-PyTorch memory, this process has 1.24 GiB memory in use. Of the allocated memory 826.90 MiB is allocated by PyTorch,
When running on RTX4090, it has no OOM issues

[apchenstu]

Hmm, 131062.89 GiB seems to be a bug in the code, generally caused by matrix shape misalignment, where did occur this error?

[ChenYutongTHU]

Hmm, 131062.89 GiB seems to be a bug in the code, generally caused by matrix shape misalignment, where did occur this error?

It happens in 2DGS rasterization. I used the same code for two types of GPU, using the same conda environment, in slurm. [However, it may be because that the 2DGS-diff package is installed in a 4090-gpu environment. When running in a 3090-gpu env, it has some problems.

[apchenstu]

Do you have the issue when using 4090?

[ChenYutongTHU]

No. It works well for 4099

…

On Mon, 26 Aug 2024 at 16:12, Anpei Chen ***@***.***> wrote: Do you have the issue when using 4090? — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJWEWBF474EBJ7E7KX2BFS3ZTMZVVAVCNFSM6AAAAABKVD4MSSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJQGMYTSNJQGE> . You are receiving this because you commented.Message ID: ***@***.***>

[apchenstu]

I see, could you please try to reinstall using 3090