StandardDescriptorSetAllocator handles layouts with high VariableDescriptorCount badly

[Firestar99]

Template

• Version of vulkano: current master (9b6e307)
• OS: Ubuntu 23.10, KDE on Wayland
• GPU:
  • RADV
  • AMDVLK

Issue

I was messing about with VariableDescriptorCount, and noticed that the maximum amount of descriptors one can have in a set has to be declared when the PipelineLayout is created. As I want as many as possible, I've just passed in device.physical_device().properties().max_descriptor_set_sampled_images, which on RADV is 8388606 = 0x7ffffe and on AMDVLK is 4294967295 = 0xffffffff. If you then allocate a dynamically-sized descriptor set of any amount of elements, the StandardDescriptorSetAllocator will always pre-allocate layout.variable_descriptor_count() (just count below) * StandardDescriptorSetAllocatorCreateInfo::set_count (default 32) descriptors, even if you only asked for like 2 sampled_image descriptors.

vulkano/vulkano/src/descriptor_set/allocator.rs

Line 429 in 9b6e307

(ty, count * create_info.set_count as u32)

• On AMDVLK this multiplication overflows the u32.
• On RADV, it will only allocate 0x7ffffe * 32 = 268435392 descriptors, to which the driver returns an out of device memory error.

You can test it by modifying this line in the runtime-array example from wanting at most 2 descriptors to device.physical_device().properties().max_descriptor_set_sampled_images:

vulkano/examples/runtime-array/main.rs

Line 420 in 9b6e307

binding.descriptor_count = 2;

[marc0246]

Apart from the overflow, this is intended behavior. The number of preallocated sets is configurable for this reason: you're supposed to set it to 1 for bindless. But just asking for my understanding, what were you expecting setting the max count so high?

[marc0246]

I guess we could add another option that limits the number of descriptors allocated, which would allow you to avoid recreating the layout?

[marc0246]

For the record, StandardDescriptorSetAllocator is ill-suited for bindless, although you can sort of make it work. It's mostly intended for reusing a pool for multiple sets as you can probably tell. On the other hand, for bindless what you probably want is to allocate one pool per set, and have only one set at any point in time. I'm planning to implement that in vulkano in the near future, together with automatic descriptor management, if the user wants it.

[Firestar99]

The number of preallocated sets is configurable for this reason: you're supposed to set it to 1 for bindless

Even if you were to do that, it'll still allocate 4294967295 descriptor sets on AMDVLK, likely still resulting in an out-of-device-memory.

I guess we could add another option that limits the number of descriptors allocated, which would allow you to avoid recreating the layout?

I think that's a quite reasonable solution to only allow it to pool a configurable amount of descriptors (like <=1024) and anything above that receiving a dedicated pool, akin to memory allocation.

For the record, StandardDescriptorSetAllocator is ill-suited for bindless

I pretty much expected having to write my own allocator down the line anyways, so I could see just leaving it as is as a viable solution for the moment, and I write my own allocator rather sooner than later :D