Recode malloc and page size

By default, libc's malloc implementation already allocates large single memory allocations using mmap

and MAP_ANONYMOUS

.

To demonstrate, compile this:

#include <stdio.h>
#include <stdlib.h>

int
main (int argc, char **argv)
{
  void *i = malloc (100 * 1024 * 1024);
  exit (0);
}

      

        
        
        
      

    

Then run it under strace

:

$ strace ./x 2>&1

execve("./x", ["./x"], [/* 21 vars */]) = 0
brk(0)                                  = 0x135d000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7ad0fbb000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=105169, ...}) = 0
mmap(NULL, 105169, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f7ad0fa1000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\30\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1811128, ...}) = 0
mmap(NULL, 3925176, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f7ad09dc000
mprotect(0x7f7ad0b91000, 2093056, PROT_NONE) = 0
mmap(0x7f7ad0d90000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b4000) = 0x7f7ad0d90000
mmap(0x7f7ad0d96000, 17592, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f7ad0d96000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7ad0fa0000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7ad0f9f000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7ad0f9e000
arch_prctl(ARCH_SET_FS, 0x7f7ad0f9f700) = 0
mprotect(0x7f7ad0d90000, 16384, PROT_READ) = 0
mprotect(0x600000, 4096, PROT_READ)     = 0
mprotect(0x7f7ad0fbd000, 4096, PROT_READ) = 0
munmap(0x7f7ad0fa1000, 105169)          = 0
mmap(NULL, 104861696, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7aca5db000
exit_group(0)                           = ?

      

        
        
        
      

    

This line ...

mmap(NULL, 104861696, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7aca5db000

      

        
        
        
      

    

... is a selection of a large block.

And there is an answer to your question. You make it big and MAP_PRIVATE|MAP_ANONYMOUS

.

The man page for malloc

Usually malloc()

allocates memory from the heap and adjusts the heap size as needed using sbrk

(2). When allocating blocks of memory more than MMAP_THRESHOLD

bytes, the glibc malloc()

implementation allocates memory as a private anonymous mapping using mmap(2)

. MMAP_THRESHOLD

by default 128 kB, but configurable with mallopt(3)

. Allocations made using mmap

(2) are independent of the resource limit RLIMIT_DATA

(see getrlimit

(2)).

And from the man page for mallopt

M_MMAP_THRESHOLD

If an allocation request that exceeds the specified value cannot be satisfied with the existing free chunk, it is guaranteed that the memory will be retrieved using mmap (). Smaller queries can be extracted from mmap()

either or sbrk()

. mmap()

-allocated memory can be immediately returned to the OS when it is freed, but this is not true for all memory allocated with sbrk()

; however, memory allocated mmap()

and later freed is not pooled or reused, so there is more overhead. Default: 128 * 1024.

M_MMAP_MAX

This value sets the maximum number of mmap()

-allocated chunks allowed for use at a given time (even if the size of the allocation request exceeds the parameter value M_MMAP_THRESHOLD

). This is useful on systems where the implementation mmap()

does not scale well. A value of 0 disables use mmap()

. Default: 65536.

If you set M_MMAP_TRESHOLD

to zero (I don't know if it will work that far), I would assume that each selection will be done through mmap

, without changing a few lines of code, But anyway, the answer to your question mmap

with similar parameters is given above.

Note that it may not be desirable to have too many separate mmap

'd regions , in which case you will want to select within one region or multiple regions. Growing regions mmap

without changing their address can be difficult, and this can make it impossible to return memory to the OS when allocating fragments.