Determine if allocation via malloc () is supported by a huge page

You can determine the exact state of any page, including if it is supported by a transparent (or opaque) huge page, by looking at the "pfn" (page number of the page) in the file /proc/kpageflags

. You get a pfn for a page by reading from a file /proc/$PID/pagemap

for your process, which is indexed by a virtual address.

Unfortunately, both the value pfn

of pagemap

¹ and the entire file /proc/kpageflags

are only accessible to root users. That said, if you can run your process as root, at least in the testing or testing you're interested in, this works well.

I wrote a little library called page-info that does the appropriate parsing for you. Give it a memory range and it will return information to you on every page, including whether it is in memory, backed up by a huge page, etc.

For example, starting an included test process as sudo ./page-info-test THP

gives the following output:

PAGE_SIZE = 4096, PID = 18868
       size memset       FLAG     SET   UNSET UNAVAIL
   0.25 MiB BEFORE        THP       0       1      64
   0.25 MiB AFTER         THP       0      65       0
   0.50 MiB BEFORE        THP       0       1     128
   0.50 MiB AFTER         THP       0     129       0
   1.00 MiB BEFORE        THP       0       1     256
   1.00 MiB AFTER         THP       0     257       0
   2.00 MiB BEFORE        THP       0       1     512
   2.00 MiB AFTER         THP       0     513       0
   4.00 MiB BEFORE        THP       0       1    1024
   4.00 MiB AFTER         THP     512     513       0
   8.00 MiB BEFORE        THP       0       1    2048
   8.00 MiB AFTER         THP    1536     513       0
  16.00 MiB BEFORE        THP       0       1    4096
  16.00 MiB AFTER         THP    3584     513       0
  32.00 MiB BEFORE        THP       0       1    8192
  32.00 MiB AFTER         THP    7680     513       0
  64.00 MiB BEFORE        THP       0       1   16384
  64.00 MiB AFTER         THP   15872     513       0
 128.00 MiB BEFORE        THP       0       1   32768
 128.00 MiB AFTER         THP   32256     513       0
 256.00 MiB BEFORE        THP       0       1   65536
 256.00 MiB AFTER         THP   65024     513       0
 512.00 MiB BEFORE        THP       0       1  131072
 512.00 MiB AFTER         THP  124416    6657       0
1024.00 MiB BEFORE        THP       0       1  262144
1024.00 MiB AFTER         THP       0  262145       0
DONE

      

        
        
        
      

    

The column UNAVAIL

means that the mapping information is not available - usually because the page has never accessed it and therefore is not yet supported by any page. You can see that for these "large" allocations, the following allocation only shows one page since we haven't touched memory.

The lines AFTER

represent the same information after being called memset()

for the entire allocation, resulting in the physical allocation of all pages. Here we see that no allocations are supported by transparent huge pages until we hit the 4 MiB selection, at which point most of each allocation is supported by THP, except for 513 pages (which end up on the edges of the selection). At 512 MiB, the system starts to exhaust the huge pages available, but still satisfies most of the allocation, but at 1024 MiB, all allocation is done in small pages.

This library is not production ready, so don't use it for anything critical (for example, some crashes just cause exit()

). Contributions are welcome.

¹ Since kernel 4.0 is roughly around, pfn was previously available to user processes other than root. From 4.0 to 4.1 or so, everything pagemap

was disabled for non-root processes, but since then the file is available again, but with pfn closed (it will always appear to be null).