R
RobHicSunt
Guest
I need to allocate a large memory chunk, say 2GB, in RAM that is both read-only and locked. What is the best way to get it so that (sequential/scan) memory access is fast?
This is useful, for instance, to perform high-performance streaming analytics on data without incurring the issues of page faults. Apologies if this has been already asked, as my search could not find any answer in the forums.
I am facing two alternatives with VirtualAlloc but none is fully satisfying.
(1) Use VirtualProtect with PAGE_READONLY on the chunk allocated by VirtualAlloc. As page size is typically 4KB, this significantly increases the working set size and potentially degrades the access performance of the associated metadata.
(2) Use large-page VirtualAlloc with MEM_LARGE_PAGES and page size of 2MB. This is good as the number of pages is almost three orders of magnitude smaller than solution (1) and they are not part of the working set. However only PAGE_READWRITE is allowed whereas I want read-only: the official documentation (Large-Page Support - Win32 apps) says that "The memory is always read/write and nonpageable (always resident in physical memory)".
I am not mentioning CreateFileMapping + MapViewOfFile as it is giving worse performance than VirtualAlloc, but maybe I employed it in an improper way (followed https://docs.microsoft.com/en-us/windows/win32/memory/creating-a-file-mapping-using-large-pages). It seems to me that the pages are not locked in RAM in this way.
Thank you for any help
-Rob
Continue reading...
This is useful, for instance, to perform high-performance streaming analytics on data without incurring the issues of page faults. Apologies if this has been already asked, as my search could not find any answer in the forums.
I am facing two alternatives with VirtualAlloc but none is fully satisfying.
(1) Use VirtualProtect with PAGE_READONLY on the chunk allocated by VirtualAlloc. As page size is typically 4KB, this significantly increases the working set size and potentially degrades the access performance of the associated metadata.
(2) Use large-page VirtualAlloc with MEM_LARGE_PAGES and page size of 2MB. This is good as the number of pages is almost three orders of magnitude smaller than solution (1) and they are not part of the working set. However only PAGE_READWRITE is allowed whereas I want read-only: the official documentation (Large-Page Support - Win32 apps) says that "The memory is always read/write and nonpageable (always resident in physical memory)".
I am not mentioning CreateFileMapping + MapViewOfFile as it is giving worse performance than VirtualAlloc, but maybe I employed it in an improper way (followed https://docs.microsoft.com/en-us/windows/win32/memory/creating-a-file-mapping-using-large-pages). It seems to me that the pages are not locked in RAM in this way.
Thank you for any help
-Rob
Continue reading...