x86: pud_clear: only reload cr3 if necessary
authorJeremy Fitzhardinge <jeremy@goop.org>
Mon, 4 Feb 2008 15:48:02 +0000 (16:48 +0100)
committerIngo Molnar <mingo@elte.hu>
Mon, 4 Feb 2008 15:48:02 +0000 (16:48 +0100)
Rather than unconditionally reloading cr3, only do so if the pud we're
updating is within the active pgd.

This eliminates TLB flushes most of the time.  The
performance-critical uses of pud_clear are during execve and exit, but
in those cases cr3 is referring to some other pagetable.  The only
other use of pud_clear is during a large (1Gbyte+) munmap, and those
are sufficiently rare that a couple of cr3 reloads won't hurt.

Signed-off-by: Jeremy Fitzhardinge <jeremy@xensource.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
include/asm-x86/pgtable-3level.h

index ed4c6f0e57ec28464b018f6a61e4fed80a6257dc..ad71960bca3acc9ac0fb803db76c3a412aa605a4 100644 (file)
@@ -93,17 +93,20 @@ static inline void native_pmd_clear(pmd_t *pmd)
 
 static inline void pud_clear(pud_t *pudp)
 {
+       unsigned long pgd;
+
        set_pud(pudp, __pud(0));
 
        /*
         * Pentium-II erratum A13: in PAE mode we explicitly have to flush
         * the TLB via cr3 if the top-level pgd is changed...
         *
-        * XXX I don't think we need to worry about this here, since
-        * when clearing the pud, the calling code needs to flush the
-        * tlb anyway.  But do it now for safety's sake. - jsgf
+        * Make sure the pud entry we're updating is within the
+        * current pgd to avoid unnecessary TLB flushes.
         */
-       write_cr3(read_cr3());
+       pgd = read_cr3();
+       if (__pa(pudp) >= pgd && __pa(pudp) < (pgd + sizeof(pgd_t)*PTRS_PER_PGD))
+               write_cr3(pgd);
 }
 
 #define pud_page(pud) \