This is the final article in my two part series on my motivation and process for making MEM_TOP_DOWN a per-process flag. Previously, in Part 1, I explained user-mode solutions that I thought fell short. In Part 2, I will cover reverse engineering the OS mechanism behind MEM_TOP_DOWN and how to use it.
Knowing that NtAllocateVirtualMemory is responsible for allocating memory it is reasonable to assume that the code path that handles the MEM_TOP_DOWN flag passed in as a parameter is the same code path that handles the MEM_TOP_DOWN set in the AllocationPreferences in the registry.
For this exercise I’ll be using examples from the 32-bit kernel as the analysis is easier to follow, for our purposes the 64-bit kernel behaves the same way.
Loading ntoskrnl.exe into the disassembler and going to NtAllocateVirtualMemory function we can start searching for the MEM_TOP_DOWN flag. Searching for 100000h shows up a few locations but the immediately interesting one is right before the call to _MiFindEmptyAddressRangeDownTree. We couldn’t have gotten any luckier, this code path is clearly the one that chooses between a normal virtual address and one that starts near the top of the address space.
PAGE:005CA930 test [ebp+AllocationType], 100000h
PAGE:005CA937 jz short loc_5CA952
PAGE:005CA939 mov eax, [ebp+var_1C]
PAGE:005CA93C add eax, 238h
PAGE:005CA941 push eax
PAGE:005CA942 push [ebp+var_44]
PAGE:005CA945 mov eax, [ebp+var_4C]
PAGE:005CA948 mov ecx, [ebp+var_28]
PAGE:005CA94B call _MiFindEmptyAddressRangeDownTree@20
PAGE:005CA950 jmp short loc_5CA960
PAGE:005CA952 ; ---------------------------------------------------------------------------
PAGE:005CA952
PAGE:005CA952 loc_5CA952: ;
PAGE:005CA952 push [ebp+var_4C]
PAGE:005CA955 push [ebp+var_44]
PAGE:005CA958 mov eax, [ebp+var_28]
PAGE:005CA95B call _MiFindEmptyAddressRange@16
Now that we know _MiFindEmptyAddressRangeDownTree is what changes the allocation we can search for it elsewhere in NtAllocateVirtualMemory. Since it doesn’t show up anywhere else this must be the same code path executed when the registry value is set. Searching backwards from this location for 100000h we find the next interesting piece of the puzzle.
PAGE:005CA5C2 mov eax, [ebp+var_1C]
PAGE:005CA5C5 test dword ptr [eax+228h], 200000h
PAGE:005CA5CF jz short loc_5CA5D8
PAGE:005CA5D1 or [ebp+AllocationType], 100000h
PAGE:005CA5D8
PAGE:005CA5D8 loc_5CA5D8: ; CODE XREF:
PAGE:005CA5D8 mov esi, edx
Here we are checking bit position 21 (200000h) at offset 0x228 in the structure pointed to by EBP+0x1C and if that is set then the MEM_TOP_DOWN flag is set in AllocationType. To figure out what structure was at EBP+0x1C I took a look at the ReactOS version of NtAllocateVirtualMemory. Of the listed local variables the EPROCESS structure peaked my interest. Looking up the definition for _EPROCESS in the kernel debugger you will see.
kd> dt -b -v _EPROCESS
struct _EPROCESS, 125 elements, 0x270 bytes
+0x000 Pcb : struct _KPROCESS, 35 elements, 0x80 bytes
….
+0x228 Flags : Uint4B
+0x228 CreateReported : Bitfield Pos 0, 1 Bit
+0x228 NoDebugInherit : Bitfield Pos 1, 1 Bit
...
+0x228 VmTopDown : Bitfield Pos 21, 1 Bit
+0x228 ImageNotifyDone : Bitfield Pos 22, 1 Bit
+0x228 PdeUpdateNeeded : Bitfield Pos 23, 1 Bit
…
A flag listed as VmTopDown in bit position 21 at offset 0x228. Just to be sure I could have continued digging through the kernel to figure out where this bit is set but I felt like it was time to change gears from research to experimentation.
The next step turned out to be easy. Writing a simple device driver that installed a callback using PsSetCreateProcessNotify allowed me to monitor process creation. Then using PsLookupProcessByProcessId retrieved a pointer to the EPROCESS structure. Microsoft does not provide header details for this structure so my first test hardcoded the offset of the VmTopDown flag and set it. Doing this I was able to see the MEM_TOP_DOWN behavior on new processes without having to turn the behavior on globally.
To package all of this up, I created a device driver and a control program that allows you to control which processes this driver affects. To try and make it a little OS version independent there is user-mode code that loads the current kernel symbols to get the correct offset of VmTopDown and passes that into the driver. You can find the code at https://github.com/jcopenha/topdown. Using the device driver and control program you’ll be able to turn MEM_TOP_DOWN on for a single process and ensure that it happens from the first allocation to the last.
Share your thoughts on this custom device driver used to make MEM_TOP_DOWN per-process in the comments below.
