Paging

paging!

hey!
although i finally figured out how memory descriptors lists work in theory, 
thanks to this article: http://www.osronline.com/article.cfm?id=423
i dis assembled the MmBuildForNonPagedPool function anyways for the sake of 
interest :) , i found the 
logic used to get the physical address a bit confusing and out of sync from 
my  knowledge of paging. 

mov ecx,edx
shr ecx,12
and ecx,3FF8
mov edi,[ecx+C0600000]
nov ecx,[ecx+C0600004]

the above is an extract from the MmBuildForNonPagedPool function. the ecx 
register initially stores Mdl.startVa
which is the starting virtual address of the buffer described by the 
MDL(aligned to 4096 byte boundry or page frame size boundry). 
in the second instruction the ecx register is shifted right 18 times, which 
gives us the rightmost 14 bits of virtual address 
i.e bits (31:18) the third instruction removes 3 leftmost bits. This implies 
that the function multiplies rightmost 11 bits of the virutal address
with 8 (8 is the size of a page directory entry with PAE bit (5) of the CR4 
register enabled) for example if we started with a
virtual address of say 81274567 then.. :

1000 0001 0010 0111 0100 0101 0110 0111 = 81274567 , starting virual address
0000 0000 0000 0000 0010 0000 0100 1001 = 00002049 , result after shifting 
18 times
0000 0000 0000 0000 0010 0000 0100 1000 = 00002048 , after stripping the 
last 3 bits which is the same as
1000 0001 001 (this first 11 bits of the VA) * 8 = 10 0000 0100 1000

this number (00002048 in our example) is used to index the page directory 
(C0600000, page directory base virtual address when
PAE is set) notice that we fetch 8 bits from the page directory, 4 of them 
are stored in edi and the other 4 are stored in ecx
Question1: which paging mechanism uses 11 bits to index a page directory ? 
according to the intel manuals when PAE is on
2 bits are used to index the PDPT and 9 bits are used to index the page 
directory.

here is the next set of asm instructions from the MmBuildForNonPagedPool 
function :

mov [ebp-4],ecx ;  store the higher 4 bytes of the PDE on the stack as ecx 
is going to used in the following instructions
mov ecx,00000081;  this clears all bits in ecx except bits 0 and 7
mov [ebp-8],edi ;  store the lower 4 bytes of the PDE on the stack as edi is 
going to used in the following instructions
and edi,ecx;  
cmp edi,ecx;  compare the bits 0 and bits 7 of the PDE 
jnz 8050A00; 
the jump is taken if bits 0 and 7 of the PDE are set, now it makes sense why 
its checking for bit 0 as it indicates
wether the PDE is paged in or paged out, but the bit 7 according to the 
intel manuals is set to 0 always then why bother
checking it? that is Question 2.
the MmBuildForNonPagedPool function is used when the physical pages 
described by the virtual address buffer described by the MDL 
is in non paged pool i.e the caller is sure that all the phyical pages 
associated with the virutall address buffer are in nonpaged pool, so i am
guessing the jnz instruction is a check to insure that the PDE is paged in 
(as page tables can be paged out) but i am not sure why its checking
for bit 7 :S 

to confuse matters further :
once the jump is taken i.e the PDE is paged in :

SHR EDX,09 ; edx initially contains the mdl.startVa, 
AND EDX,007FFFF8; the above 2 instructions multiply the rightmost 20 bits of 
the Virtual address with 8
SUB EDX,40000000; is this compiler optimization? i dont understand how this 
is better than,  add EDX,C0000000??
		anyways the purpose of this instruction is to index the Page table (sub 
Question: according to the
		intel manuals the page table base is obtained from the PDE? why is it 
assumed that the Page table
		base is always C0000000??)
mov ecx,[edx]; store the lower 4 bytes of the PTE in ecx
mov edi,[edx+04]; store the higher 4 bytes of the  PTE in edi
SHRD ecx,edi,0C; shifts the 8 byte PTE 12 times to the right, its looking 
for the page address base
AND ecx,03FFFFFF; its keeping 26 bits and stripping of the rest hmm 
according to the manuals the page address base is only 24 bits
then why 26 bits? (this value is stored in the array below the MDL header so 
i guess its the page address base) that is Question 4

thanks for your help
0
Utf
6/13/2010 5:18:23 PM
development.device.drivers 941 articles. 0 followers. Follow

6 Replies
650 Views

Similar Articles

[PageSpeed] 10

codeFather <codeFather@discussions.microsoft.com> wrote:
>
>paging!
>...
>Question1: which paging mechanism uses 11 bits to index a page directory ? 
>according to the intel manuals when PAE is on
>2 bits are used to index the PDPT and 9 bits are used to index the page 
>directory.

Windows stores all of the page table directories sequentially.  So, even
though the 11 bits happens to span several directories, they know the fetch
will work.

>the jump is taken if bits 0 and 7 of the PDE are set, now it makes sense why 
>its checking for bit 0 as it indicates
>wether the PDE is paged in or paged out, but the bit 7 according to the 
>intel manuals is set to 0 always then why bother
>checking it? that is Question 2.

What manual are you looking at?  Bit 7 of the PDE is the large/small page
bit.  0 means it's a 4k page.  1 means it's a 4M page.

>Question: according to the
>intel manuals the page table base is obtained from the PDE? why is it 
>assumed that the Page table base is always C0000000??)

Because the Windows kernel created the page tables do begin with.  Like the
PDEs, Windows creates the page tables sequentially.  That makes it possible
to access them as if they were all one big table.

>mov ecx,[edx]; store the lower 4 bytes of the PTE in ecx
>mov edi,[edx+04]; store the higher 4 bytes of the  PTE in edi
>SHRD ecx,edi,0C; shifts the 8 byte PTE 12 times to the right, its looking 
>for the page address base
>AND ecx,03FFFFFF; its keeping 26 bits and stripping of the rest hmm 
>according to the manuals the page address base is only 24 bits
>then why 26 bits? (this value is stored in the array below the MDL header so 
>i guess its the page address base) that is Question 4

It should be clear that 24 bits is not nearly enough for a physical page
base.
-- 
Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.
0
Tim
6/14/2010 3:27:02 AM
thanks for the post

>Windows stores all of the page table directories sequentially. So, even
>though the 11 bits happens to span several directories, they know the fetch
>will work. 

"sequentially" hmmm. do you mean to say that by using 11 bits what we are 
actually doing is taking the rightmost 2 bits and multiplying  them by the 
size of a page directory (which is Math.pow(2, 11-2) =  512)  and to index 
the 8 byte PDE all we need to do is change the 9 bits (following the 2 bits 
which index the PDPT) for example if the first 2 bits were 10 then
10 : first 2 bits used to index the PDPT
10 000000000 : append 9 bits i.e multiplying the first 2 bits with 512 (size 
of a page directory)
10 000000000 000 : append 3 bits i.e multiplying the whole by 8 (size of the 
PDE)

10 000000000 000 : the first page directory entry.
10 000000001 000 : the second page directory entry.
10 000000010 000 : the third page directory entry and so on.....

and i guess similar reasoning applies for page table entries, there we use 
the first 20 bits and multiply them by 8 =>
10 000000001 : 11 bits for the Page directory index
10 000000001 000000000 : multiplyed by 512 (size of the Page table)
10 000000001 000000000 000 : multiplyed by 8 (size of the page table entry)

>What manual are you looking at? Bit 7 of the PDE is the large/small page
>bit. 0 means it's a 4k page. 1 means it's a 4M page. 

my bad. i did not look carefully, actually i assumed that every important 
bit would be labelled anyways when i rechecked the manual i found bit 7 to be 
clear for 4 byte pages when PAE is enabled and bit 7 set for 4 MB pages when 
PAE is enabled (it is the Page Size bit, its labelled in  figure 3-15, 
3.6.4), i am referring to the (Intel Architecture Software Developer’s Manual 
Volume 3: System Programming (Order Number 243190) heres the download link 
http://www.intel.com/design/pentiumii/manuals/243190.htm) chapter 3.6 
(paging) section 8 i.e 3.6.8 (physical address extension)

>It should be clear that 24 bits is not nearly enough for a physical page
>base. 

the only thing i do not still understand is the need for a 26 bit address 
base?since the page size 4KB (clear from the fact the JNZ instruction took 
the jump and bit 0 was set) we need 12 bits to index every byte. But if we 
use 26 bits as page address base then we only get (36-26) = 10 bits to index 
the 4KB page, but with 10 bits we can only index 2^10 = 1024 bytes = 1KB. On 
the other hand if we use 24 bits then we get (36-24) = 12 bits to index the 
4KB page, which is what we need. And the same is mentioned in (Figure 3-20. 
Format of Page-Directory-Pointer-Table, Page-Directory, and Page-Table 
Entries for 4-KByte Pages and 36-Bit Extended Physical Addresses)  of the 
intel manual. 96th page of the .pdf,

thanks again!
0
Utf
6/14/2010 8:34:26 AM
codeFather <codeFather@discussions.microsoft.com> wrote:
>
>the only thing i do not still understand is the need for a 26 bit address 
>base?since the page size 4KB (clear from the fact the JNZ instruction took 
>the jump and bit 0 was set) we need 12 bits to index every byte. But if we 
>use 26 bits as page address base then we only get (36-26) = 10 bits to index 
>the 4KB page, but with 10 bits we can only index 2^10 = 1024 bytes = 1KB. On 
>the other hand if we use 24 bits then we get (36-24) = 12 bits to index the 
>4KB page, which is what we need.

Which operating system are you using?  Even with PAE, XP and Vista will not
allow access to any memory beyond the 4GB mark, because of a licensing
restriction.  It's possible that's related here, but I'm not sure.

You'd have to check Server 2003 or Server 2008 to see how they handle it.
-- 
Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.
0
Tim
6/16/2010 5:05:40 AM
i am using windows XP service pack 2. But if access to only 4GB mark is 
allowed, then why waste clock cycles fetching 2 4byte entries, the paging 
mechanism without Physical Address Extension would be more efficient then, 
right? 
0
Utf
6/16/2010 8:52:23 AM
IIRC PAE paging has to be enabled to get NX (execute disable) 
functionality working (at the hardware level) and it's one of the main 
reasons why PAE is enabled on 32bit systems.

Kris

-----Original Message-----
From: codeFather [mailto:codeFather@discussions.microsoft.com]
Posted At: Wednesday, June 16, 2010 9:52 AM
Posted To: microsoft.public.development.device.drivers
Conversation: Paging
Subject: Re: Paging

i am using windows XP service pack 2. But if access to only 4GB mark is
allowed, then why waste clock cycles fetching 2 4byte entries, the paging
mechanism without Physical Address Extension would be more efficient then,
right?

0
Krzysztof
6/16/2010 11:00:36 AM
codeFather <codeFather@discussions.microsoft.com> wrote:
>
>i am using windows XP service pack 2. But if access to only 4GB mark is 
>allowed, then why waste clock cycles fetching 2 4byte entries, the paging 
>mechanism without Physical Address Extension would be more efficient then, 
>right? 

The 4GB limit is strictly a software thing.

And remember that page table entries are aggressively cached in the TLB. It
doesn't take any extra clock cycles to fetch two 4-byte entries.
-- 
Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.
0
Tim
6/18/2010 5:01:53 AM
Reply:

Similar Artilces: