Failed commits on Windows segfault instead of returning null

[elasota]

This is pretty easy to reproduce, just do something like this on x64:

rpmalloc_initialize(NULL);
rpmalloc( ((size_t)1) << 45 );

os_mcommit doesn't return success/failure, it just asserts on failure, so if asserts are disabled, rpmalloc won't return null, it will just segfault as soon as the memory that it expects to be committed is used (which is usually immediately).

[VnUgE]

I'm thinking though this issue myself. It appears the only difference when ENABLE_DECOMMIT=1 is that instead of setting the MEM_COMMIT flag for the entire page mapping, it defers it, and attempts to only commit the first page in the span, then demand commit the rest. As far as I can tell if a call to VirtualAlloc with a valid address and the MEM_COMMIT which should happen when os_commit is called. I don't think os_mcommit will ever get called without a valid address. It also means that demand commit would fail when attempting to access the uncommited pages of the block. But if the allocation succeeded, but commit fails, then it would mean the system is out of usable memory right? Unless you commit at the same time you alloc you can't really know if the entire block is available right? Although from what I understand about the docs, is that MEM_RESERVE should prevent that? This is a strange situation to me.

[mjansson]

Since Windows don't do overcommit, on that platform (with decommit enabled) we map in without committing and then commit pages as needed. You are correct that it will not correctly handle the OOM scenario.

I changed this to propagate the failures and return null under commit failures.