ARM: mm: fix racy ASID rollover broadcast on SMP platforms

If ASID rollover is detected on a CPU in an SMP system, a synchronous
IPI call is made to force the secondaries to reallocate their current
ASIDs.

There is a problem where a CPU may be interrupted in the cpu_switch_mm
code with the context ID held in r1. After servicing the IPI, the
context ID register will be updated with an ASID from the previous
generation, polluting the TLB for when that ASID becomes valid in the
new generation.

This patch disables interrupts during cpu_switch_mm for SMP systems,
preventing incoming rollover broadcasts from being serviced while the
register state is inconsistent. Additionally, the context resetting code
is modified to call cpu_switch_mm, rather than setting the context ID
register directly, so that the TTBR always agrees with the ASID.

Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>

diff --git a/arch/arm/include/asm/proc-fns.h b/arch/arm/include/asm/proc-fns.h
index 8ec535e11fd73c81ebdc12500c1ad24d7a9add35..35c3fc93b88f44866efa829a2077e1a613069516 100644 (file)
--- a/arch/arm/include/asm/proc-fns.h
+++ b/arch/arm/include/asm/proc-fns.h
@@ -97,8 +97,22 @@ extern void cpu_resume(void);
 
 #ifdef CONFIG_MMU
 
+#ifdef CONFIG_SMP
+
+#define cpu_switch_mm(pgd, mm) \
+       ({                                                      \
+               unsigned long flags;                            \
+               local_irq_save(flags);                          \
+               cpu_do_switch_mm(virt_to_phys(pgd), mm);        \
+               local_irq_restore(flags);                       \
+       })
+
+#else /* SMP */
+
 #define cpu_switch_mm(pgd,mm) cpu_do_switch_mm(virt_to_phys(pgd),mm)
 
+#endif
+
 #define cpu_get_pgd()  \
        ({                                              \
                unsigned long pg;                       \

diff --git a/arch/arm/mm/context.c b/arch/arm/mm/context.c
index b0ee9ba3cfab41a52853eca727466abff88ddf27..131f381712c91fe488490c5bf34a3d7259418728 100644 (file)
--- a/arch/arm/mm/context.c
+++ b/arch/arm/mm/context.c
@@ -99,8 +99,7 @@ static void reset_context(void *info)
        set_mm_context(mm, asid);
 
        /* set the new ASID */
-       asm("mcr        p15, 0, %0, c13, c0, 1\n" : : "r" (mm->context.id));
-       isb();
+       cpu_switch_mm(mm->pgd, mm);
 }
 
 #else