Hi, On 28.06.25 10:54, Segher Boessenkool wrote:
How do you disable memory overcommit? "Just" don't run problems that are bigger than you have memmory for?
That is dependent on the kernel you use. On Linux, you would use the "vm.overcommit_memory" sysctl:
- a value of 0 (default) allows overcommit, but rejects some clearly silly requests.
- a value of 1 allows overcommit, and does not reject silly requests- a value of 2 limits allocations to the setting of "vm.overcommit_ratio" percent of physical memory.
By default, that ratio is 50%, so the other 50% are used for buffers and caches only, never for program allocations -- so you may want to tune that as well.
E.g. on my 8 GB machine
void *hint = NULL;
int prot = PROT_READ|PROT_WRITE;
int flags = MAP_PRIVATE|MAP_ANONYMOUS;
int fd = -1;
off_t offset = 0;
// overcommit setting 0
mmap(hint, 68719476736, prot, flags, fd, offset); // -1 ENOMEM
mmap(hint, 8589934592, prot, flags, fd, offset); // works
// overcommit setting 1
mmap(hint, 68719476736, prot, flags, fd, offset); // works
mmap(hint, 8589934592, prot, flags, fd, offset); // works
// overcommit setting 2
mmap(hint, 68719476736, prot, flags, fd, offset); // -1 ENOMEM
mmap(hint, 8589934592, prot, flags, fd, offset); // -1 ENOMEM
If you add MAP_POPULATE to the flags, it will also immediately assign
physical pages, and begin swapping out all other processes. The 8 GB
allocation in mode 0 still succeeds, even though only 7.5 GB will be
resident at the end of the call, because the beginning of the allocation
already got evicted to make space for later pages.
So, something like
# sysctl vm.overcommit_ratio=80
# sysctl vm.overcommit_memory=2
should guarantee you that your compiles will either run at full speed,
or fail quickly, but as mentioned, that also applies to the rest of your
system -- if you are using systemd or a desktop environment, all of that
will *also* need to fit into physical RAM and cannot be swapped out even
if unused, but these tools are written with the assumption that having a
program loaded but not doing anything costs pretty much nothing because
unaccessed pages will be evicted as soon as something else needs them,
which is normally a reasonable assumption.
So this solves the original problem, but is not optimal, because the optimal behaviour would be to swap out other programs to give GCC more space.
Simon
Attachment:
OpenPGP_signature.asc
Description: OpenPGP digital signature
