Arun Chandran [Thu, 26 Jun 2014 09:46:03 +0000 (15:16 +0530)]
arm64: vdso: fix build error when switching from LE to BE
Building a kernel with CPU_BIG_ENDIAN fails if there are stale objects
from a !CPU_BIG_ENDIAN build. Due to a missing FORCE prerequisite on an
if_changed rule in the VDSO Makefile, we attempt to link a stale LE
object into the new BE kernel.
According to Documentation/kbuild/makefiles.txt, FORCE is required for
if_changed rules and forgetting it is a common mistake, so fix it by
'Forcing' the build of vdso. This patch fixes build errors like these:
arch/arm64/kernel/vdso/note.o: compiled for a little endian system and target is big endian
failed to merge target specific data of file arch/arm64/kernel/vdso/note.o
arch/arm64/kernel/vdso/sigreturn.o: compiled for a little endian system and target is big endian
failed to merge target specific data of file arch/arm64/kernel/vdso/sigreturn.o
Tested-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Arun Chandran <achandran@mvista.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
(cherry picked from commit
1915e2ad1cf548217c963121e4076b3d44dd0169)
Signed-off-by: Mark Brown <broonie@kernel.org>
Mark Salter [Thu, 24 Jul 2014 14:56:15 +0000 (15:56 +0100)]
arm64: fix soft lockup due to large tlb flush range
Under certain loads, this soft lockup has been observed:
BUG: soft lockup - CPU#2 stuck for 22s! [ip6tables:1016]
Modules linked in: ip6t_rpfilter ip6t_REJECT cfg80211 rfkill xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw vfat fat efivarfs xfs libcrc32c
CPU: 2 PID: 1016 Comm: ip6tables Not tainted 3.13.0-0.rc7.30.sa2.aarch64 #1
task:
fffffe03e81d1400 ti:
fffffe03f01f8000 task.ti:
fffffe03f01f8000
PC is at __cpu_flush_kern_tlb_range+0xc/0x40
LR is at __purge_vmap_area_lazy+0x28c/0x3ac
pc : [<
fffffe000009c5cc>] lr : [<
fffffe0000182710>] pstate:
80000145
sp :
fffffe03f01fbb70
x29:
fffffe03f01fbb70 x28:
fffffe03f01f8000
x27:
fffffe0000b19000 x26:
00000000000000d0
x25:
000000000000001c x24:
fffffe03f01fbc50
x23:
fffffe03f01fbc58 x22:
fffffe03f01fbc10
x21:
fffffe0000b2a3f8 x20:
0000000000000802
x19:
fffffe0000b2a3c8 x18:
000003fffdf52710
x17:
000003ff9d8bb910 x16:
fffffe000050fbfc
x15:
0000000000005735 x14:
000003ff9d7e1a5c
x13:
0000000000000000 x12:
000003ff9d7e1a5c
x11:
0000000000000007 x10:
fffffe0000c09af0
x9 :
fffffe0000ad1000 x8 :
000000000000005c
x7 :
fffffe03e8624000 x6 :
0000000000000000
x5 :
0000000000000000 x4 :
0000000000000000
x3 :
fffffe0000c09cc8 x2 :
0000000000000000
x1 :
000fffffdfffca80 x0 :
000fffffcd742150
The __cpu_flush_kern_tlb_range() function looks like:
ENTRY(__cpu_flush_kern_tlb_range)
dsb sy
lsr x0, x0, #12
lsr x1, x1, #12
1: tlbi vaae1is, x0
add x0, x0, #1
cmp x0, x1
b.lo 1b
dsb sy
isb
ret
ENDPROC(__cpu_flush_kern_tlb_range)
The above soft lockup shows the PC at tlbi insn with:
x0 = 0x000fffffcd742150
x1 = 0x000fffffdfffca80
So __cpu_flush_kern_tlb_range has 0x128ba930 tlbi flushes left
after it has already been looping for 23 seconds!.
Looking up one frame at __purge_vmap_area_lazy(), there is:
...
list_for_each_entry_rcu(va, &vmap_area_list, list) {
if (va->flags & VM_LAZY_FREE) {
if (va->va_start < *start)
*start = va->va_start;
if (va->va_end > *end)
*end = va->va_end;
nr += (va->va_end - va->va_start) >> PAGE_SHIFT;
list_add_tail(&va->purge_list, &valist);
va->flags |= VM_LAZY_FREEING;
va->flags &= ~VM_LAZY_FREE;
}
}
...
if (nr || force_flush)
flush_tlb_kernel_range(*start, *end);
So if two areas are being freed, the range passed to
flush_tlb_kernel_range() may be as large as the vmalloc
space. For arm64, this is ~240GB for 4k pagesize and ~2TB
for 64kpage size.
This patch works around this problem by adding a loop limit.
If the range is larger than the limit, use flush_tlb_all()
rather than flushing based on individual pages. The limit
chosen is arbitrary as the TLB size is implementation
specific and not accessible in an architected way. The aim
of the arbitrary limit is to avoid soft lockup.
Signed-off-by: Mark Salter <msalter@redhat.com>
[catalin.marinas@arm.com: commit log update]
[catalin.marinas@arm.com: marginal optimisation]
[catalin.marinas@arm.com: changed to MAX_TLB_RANGE and added comment]
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
05ac65305437e8ef63d2d19cac704138a2a05aa5)
Signed-off-by: Mark Brown <broonie@kernel.org>
Mark Brown [Wed, 21 Jan 2015 23:22:06 +0000 (23:22 +0000)]
Merge remote-tracking branch 'lsk/v3.10/topic/arm64-ptrace' into lsk-v3.10-arm64-misc
Catalin Marinas [Wed, 16 Jul 2014 11:07:17 +0000 (12:07 +0100)]
arm64: Do not initialise the fixmap page tables in head.S
The early_ioremap_init() function already handles fixmap pte
initialisation, so upgrade this to cover all of pud/pmd/pte and remove
one page from swapper_pg_dir.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Tested-by: Jungseok Lee <jungseoklee85@gmail.com>
(cherry picked from commit
7edd88ad7e59c2b7b49da0e00f251884fb785d4f)
Signed-off-by: Mark Brown <broonie@kernel.org>
Catalin Marinas [Fri, 18 Jul 2014 10:54:37 +0000 (11:54 +0100)]
arm64: Create non-empty ZONE_DMA when DRAM starts above 4GB
ZONE_DMA is created to allow 32-bit only devices to access memory in the
absence of an IOMMU. On systems where the memory starts above 4GB, it is
expected that some devices have a DMA offset hardwired to be able to
access the bottom of the memory. Linux currently supports DT bindings
for the DMA offsets but they are not (easily) available early during
boot.
This patch tries to guess a DMA offset and assumes that ZONE_DMA
corresponds to the 32-bit mask above the start of DRAM.
Fixes: 2d5a5612bc (arm64: Limit the CMA buffer to 32-bit if ZONE_DMA)
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Reported-by: Mark Salter <msalter@redhat.com>
Tested-by: Mark Salter <msalter@redhat.com>
Tested-by: Anup Patel <anup.patel@linaro.org>
(cherry picked from commit
d50314a6b0702c630c35b88148c1acb76d2e4ede)
Signed-off-by: Mark Brown <broonie@kernel.org>
Conflicts:
arch/arm64/mm/init.c
Mark Rutland [Wed, 16 Jul 2014 15:32:43 +0000 (16:32 +0100)]
arm64: add MIDR_EL1 field accessors
The MIDR_EL1 register is composed of a number of bitfields, and uses of
the fields has so far involved open-coding of the shifts and masks
required.
This patch adds shifts and masks for each of the MIDR_EL1 subfields, and
also provides accessors built atop of these. Existing uses within
cputype.h are updated to use these accessors.
The read_cpuid_part_number macro is modified to return the extracted
bitfield rather than returning the value in-place with all other fields
(including revision) masked out, to better match the other accessors.
As the value is only used in comparison with the *_CPU_PART_* macros
which are similarly updated, and these values are never exposed to
userspace, this change should not affect any functionality.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
89c4a306e7631bcb71cc537c8a029172af6047fe)
Signed-off-by: Mark Brown <broonie@kernel.org>
Lorenzo Pieralisi [Thu, 17 Jul 2014 17:19:19 +0000 (18:19 +0100)]
arm64: kernel: add __init marker to PSCI init functions
PSCI init functions must be marked as __init so that they are freed
by the kernel upon boot.
This patch marks the PSCI init functions as such since they need not
be persistent in the kernel address space after the kernel has booted.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
b9e97ef93c630404f305350d88d09391d1a55648)
Signed-off-by: Mark Brown <broonie@kernel.org>
Lorenzo Pieralisi [Thu, 17 Jul 2014 17:19:18 +0000 (18:19 +0100)]
arm64: kernel: enable PSCI cpu operations on UP systems
PSCI CPU operations have to be enabled on UP kernels so that calls
like eg cpu_suspend can be made functional on UP too.
This patch reworks the PSCI CPU operations so that they can be
enabled on UP systems.
Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
756854d9b99a735f86bc3b86df5c19be12e8746e)
Signed-off-by: Mark Brown <broonie@kernel.org>
Will Deacon [Wed, 9 Jul 2014 18:22:13 +0000 (19:22 +0100)]
arm64: vdso: move data page before code pages
Andy pointed out that binutils generates additional sections in the vdso
image (e.g. section string table) which, if our .text section gets big
enough, could cross a page boundary and end up screwing up the location
where the kernel expects to put the data page.
This patch solves the issue in the same manner as x86_32, by moving the
data page before the code pages.
Cc: Andy Lutomirski <luto@amacapital.net>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
601255ae3c98fdeeee3a8bb4696425e4f868b4f1)
Signed-off-by: Mark Brown <broonie@kernel.org>
Will Deacon [Wed, 9 Jul 2014 18:22:12 +0000 (19:22 +0100)]
arm64: vdso: move to _install_special_mapping and remove arch_vma_name
_install_special_mapping replaces install_special_mapping and removes
the need to detect special VMA in arch_vma_name.
This patch moves the vdso and compat vectors page code over to the new
API.
Cc: Andy Lutomirski <luto@amacapital.net>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
2fea7f6c98f5957e539eb8aa0ce849729b900342)
Signed-off-by: Mark Brown <broonie@kernel.org>
Jungseok Lee [Mon, 12 May 2014 09:40:30 +0000 (10:40 +0100)]
arm64: Use pr_* instead of printk
This patch fixed the following checkpatch complaint as using pr_*
instead of printk.
WARNING: printk() should include KERN_ facility level
Signed-off-by: Jungseok Lee <jays.lee@samsung.com>
Reviewed-by: Sungjinn Chung <sungjinn.chung@samsung.com>
Acked-by: Kukjin Kim <kgene.kim@samsung.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
ac7b406c1a9d50ddbf5e5cbce8ca4d68d36ac2db)
Signed-off-by: Mark Brown <broonie@kernel.org>
Robin Holt [Mon, 8 Jul 2013 23:01:35 +0000 (16:01 -0700)]
reboot: x86: prepare reboot_mode for moving to generic kernel code
Prepare for the moving the parsing of reboot= to the generic kernel code
by making reboot_mode into a more generic form.
Signed-off-by: Robin Holt <holt@sgi.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Miguel Boton <mboton.lkml@gmail.com>
Cc: Russ Anderson <rja@sgi.com>
Cc: Robin Holt <holt@sgi.com>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Cc: Guan Xuetao <gxt@mprc.pku.edu.cn>
Acked-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
(cherry picked from commit
edf2b1394611fef7806d4af72179dc3ac101f275)
Signed-off-by: Mark Brown <broonie@kernel.org>
Catalin Marinas [Thu, 10 Jul 2014 10:37:40 +0000 (11:37 +0100)]
arm64: Cast KSTK_(EIP|ESP) to unsigned long
This is for similarity with thread_saved_(pc|sp) and to avoid some
compiler warnings in the audit code.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
ebe6152e722b9df6b487a0d9464aeff216b6d364)
Signed-off-by: Mark Brown <broonie@kernel.org>
Laura Abbott [Wed, 25 Jun 2014 22:55:03 +0000 (23:55 +0100)]
arm64: Add CONFIG_CC_STACKPROTECTOR
arm64 currently lacks support for -fstack-protector. Add
similar functionality to arm to detect stack corruption.
Acked-by: Will Deacon <will.deacon@arm.com>
Acked-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
c0c264ae5112d1cdb7d37d4e208b7a7e766a7418)
Signed-off-by: Mark Brown <broonie@kernel.org>
Conflicts:
arch/arm64/Kconfig
Colin Cross [Wed, 18 Jun 2014 20:10:09 +0000 (21:10 +0100)]
arm64: implement TASK_SIZE_OF
include/linux/sched.h implements TASK_SIZE_OF as TASK_SIZE if it
is not set by the architecture headers. TASK_SIZE uses the
current task to determine the size of the virtual address space.
On a 64-bit kernel this will cause reading /proc/pid/pagemap of a
64-bit process from a 32-bit process to return EOF when it reads
past 0xffffffff.
Implement TASK_SIZE_OF exactly the same as TASK_SIZE with
test_tsk_thread_flag instead of test_thread_flag.
Cc: stable@vger.kernel.org
Signed-off-by: Colin Cross <ccross@android.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
fa2ec3ea10bd377f9d55772b1dab65178425a1a2)
Signed-off-by: Mark Brown <broonie@kernel.org>
Will Deacon [Wed, 18 Jun 2014 13:06:27 +0000 (14:06 +0100)]
arm64: mm: remove broken &= operator from pmd_mknotpresent
This should be a plain old '&' and could easily lead to undefined
behaviour if the target of a pmd_mknotpresent invocation was the same
as the parameter.
Fixes: 9c7e535fcc17 (arm64: mm: Route pmd thp functions through pte equivalents)
Signed-off-by: Will Deacon <will.deacon@arm.com>
Cc: <stable@vger.kernel.org> # v3.15
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
e3a920afc3482e954834a4ed95908c4bc5e4c000)
Signed-off-by: Mark Brown <broonie@kernel.org>
Suravee Suthikulpanit [Fri, 6 Jun 2014 22:07:16 +0000 (23:07 +0100)]
arm64/dma: Removing ARCH_HAS_DMA_GET_REQUIRED_MASK macro
Arm64 does not define dma_get_required_mask() function.
Therefore, it should not define the ARCH_HAS_DMA_GET_REQUIRED_MASK.
This causes build errors in some device drivers (e.g. mpt2sas)
Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: <stable@vger.kernel.org>
(cherry picked from commit
f3a183cb422574014538017b5b291a416396f97e)
Signed-off-by: Mark Brown <broonie@kernel.org>
Conflicts:
arch/arm64/include/asm/dma-mapping.h
Victor Kamensky [Tue, 3 Jun 2014 18:21:30 +0000 (19:21 +0100)]
arm64: ptrace: fix empty registers set in prstatus of aarch32 process core
Currently core file of aarch32 process prstatus note has empty
registers set. As result aarch32 core files create by V8 kernel are
not very useful.
It happens because compat_gpr_get and compat_gpr_set functions can
copy registers values to/from either kbuf or ubuf. ELF core file
collection function fill_thread_core_info calls compat_gpr_get
with kbuf set and ubuf set to 0. But current compat_gpr_get and
compat_gpr_set function handle copy to/from only ubuf case.
Fix is to handle kbuf and ubuf as two separate cases in similar
way as other functions like user_regset_copyout, user_regset_copyin do.
Signed-off-by: Victor Kamensky <victor.kamensky@linaro.org>
Acked-by: Will Deacon <will.deacon@arm.com>
Cc: stable@vger.kernel.org
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
2227901a0230d8fde81ba9c602d649839390f56b)
Signed-off-by: Mark Brown <broonie@kernel.org>
Conflicts:
arch/arm64/kernel/ptrace.c
Will Deacon [Mon, 2 Jun 2014 10:47:23 +0000 (11:47 +0100)]
arm64: ptrace: change fs when passing kernel pointer to regset code
Our compat PTRACE_POKEUSR implementation simply passes the user data to
regset_copy_from_user after some simple range checking. Unfortunately,
the data in question has already been copied to the kernel stack by this
point, so the subsequent access_ok check fails and the ptrace request
returns -EFAULT. This causes problems tracing fork() with older versions
of strace.
This patch briefly changes the fs to KERNEL_DS, so that the access_ok
check passes even with a kernel address.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
c168870704bcde6bb63d05f7882b620dd3985a46)
Signed-off-by: Mark Brown <broonie@kernel.org>
Sudeep Holla [Fri, 9 May 2014 09:33:02 +0000 (10:33 +0100)]
arm64: add ARCH_HAS_OPP to allow enabling OPP library
The Operating Performance Point (OPP) Layer library is a generic
library used by CPUFREQ and DEVFREQ. It can be enabled only on the
platforms that specify ARCH_HAS_OPP option.
This patch selects that option in order to allow ARM64 based platforms
to use OPP library.
Signed-off-by: Sudeep Holla <sudeep.holla@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
333d17e566192efda500769fb55b11da71c6d960)
Signed-off-by: Mark Brown <broonie@kernel.org>
Conflicts:
arch/arm64/Kconfig
Lorenzo Pieralisi [Thu, 29 May 2014 17:16:54 +0000 (18:16 +0100)]
arm64: kernel: initialize broadcast hrtimer based clock event device
On platforms implementing CPU power management, the CPUidle subsystem
can allow CPUs to enter idle states where local timers logic is lost on power
down. To keep the software timers functional the kernel relies on an
always-on broadcast timer to be present in the platform to relay the
interrupt signalling the timer expiries.
For platforms implementing CPU core gating that do not implement an always-on
HW timer or implement it in a broken way, this patch adds code to initialize
the kernel hrtimer based clock event device upon boot (which can be chosen as
tick broadcast device by the kernel).
It relies on a dynamically chosen CPU to be always powered-up. This CPU then
relays the timer interrupt to CPUs in deep-idle states through its HW local
timer device.
Having a CPU always-on has implications on power management platform
capabilities and makes CPUidle suboptimal, since at least a CPU is kept
always in a shallow idle state by the kernel to relay timer interrupts,
but at least leaves the kernel with a functional system with some working
power management capabilities.
The hrtimer based clock event device is unconditionally registered, but
has the lowest possible rating such that any broadcast-capable HW clock
event device present will be chosen in preference as the tick broadcast
device.
Reviewed-by: Preeti U Murthy <preeti@linux.vnet.ibm.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
(cherry picked from commit
9358d755bd5cba8965ea79f2a446e689323409f9)
Signed-off-by: Mark Brown <broonie@kernel.org>
Will Deacon [Tue, 27 May 2014 18:11:58 +0000 (19:11 +0100)]
arm64: mm: fix pmd_write CoW brokenness
Commit
9c7e535fcc17 ("arm64: mm: Route pmd thp functions through pte
equivalents") changed the pmd manipulator and accessor functions to
convert the target pmd to a pte, process it with the pte functions, then
convert it back. Along the way, we gained support for PTE_WRITE, however
this is completely ignored by set_pmd_at, and so we fail to set the
PMD_SECT_RDONLY for PMDs, resulting in all sorts of lovely failures (like
CoW not working).
Partially reverting the offending commit (by making use of
PMD_SECT_RDONLY explicitly for pmd_{write,wrprotect,mkwrite} functions)
leads to further issues because pmd_write can then return potentially
incorrect values for page table entries marked as RDONLY, leading to
BUG_ON(pmd_write(entry)) tripping under some THP workloads.
This patch fixes the issue by routing set_pmd_at through set_pte_at,
which correctly takes the PTE_WRITE flag into account. Given that
THP mappings are always anonymous, the additional cache-flushing code
in __sync_icache_dcache won't impose any significant overhead as the
flush will be skipped.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Acked-by: Steve Capper <steve.capper@arm.com>
Tested-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
(cherry picked from commit
ceb218359de22e70980801d4fa04fffbfc44adb8)
Signed-off-by: Mark Brown <broonie@kernel.org>
Steve Capper [Tue, 25 Feb 2014 10:02:13 +0000 (10:02 +0000)]
arm64: mm: Route pmd thp functions through pte equivalents
Rather than have separate hugetlb and transparent huge page pmd
manipulation functions, re-wire our thp functions to simply call the
pte equivalents.
This allows THP to take advantage of the new PTE_WRITE logic introduced
in:
c2c93e5 arm64: mm: Introduce PTE_WRITE
To represent splitting THPs we use the PTE_SPECIAL bit as this is not
used for pmds.
Signed-off-by: Steve Capper <steve.capper@linaro.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
9c7e535fcc1725fc2e2d4f0d9dd14137f0243e23)
Signed-off-by: Mark Brown <broonie@kernel.org>
Geoff Levand [Fri, 16 May 2014 17:26:01 +0000 (18:26 +0100)]
arm64: Fix linker script entry point
Change the arm64 linker script ENTRY() command to define _text as the
kernel entry point.
The arm64 boot protocol specifies that the kernel must be entered at the
beginning of the kernel image. The existing ENTRY() command defined the
symbol stext as the entry point, which emitted an incorrect entry point,
but would not cause a runtime error because the existing entry code
immediately jumps to stext.
Signed-off-by: Geoff Levand <geoff@infradead.org>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
af885f4022622331f17227dfcfbb2dafdee25a43)
Signed-off-by: Mark Brown <broonie@kernel.org>
zhichang.yuan [Mon, 28 Apr 2014 05:11:34 +0000 (13:11 +0800)]
arm64: lib: Implement optimized string length routines
This patch, based on Linaro's Cortex Strings library, adds
an assembly optimized strlen() and strnlen() functions.
Signed-off-by: Zhichang Yuan <zhichang.yuan@linaro.org>
Signed-off-by: Deepak Saxena <dsaxena@linaro.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
0a42cb0a6fa64cb17db11164a1ad3511b43acefe)
Signed-off-by: Mark Brown <broonie@kernel.org>
zhichang.yuan [Mon, 28 Apr 2014 05:11:33 +0000 (13:11 +0800)]
arm64: lib: Implement optimized string compare routines
This patch, based on Linaro's Cortex Strings library, adds
an assembly optimized strcmp() and strncmp() functions.
Signed-off-by: Zhichang Yuan <zhichang.yuan@linaro.org>
Signed-off-by: Deepak Saxena <dsaxena@linaro.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
192c4d902f19b66902d7aacc19e9b169bebfb2e5)
Signed-off-by: Mark Brown <broonie@kernel.org>
zhichang.yuan [Mon, 28 Apr 2014 05:11:32 +0000 (06:11 +0100)]
arm64: lib: Implement optimized memcmp routine
This patch, based on Linaro's Cortex Strings library, adds
an assembly optimized memcmp() function.
Signed-off-by: Zhichang Yuan <zhichang.yuan@linaro.org>
Signed-off-by: Deepak Saxena <dsaxena@linaro.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
d875c9b3724083cd2629cd8507e424cd3716cd28)
Signed-off-by: Mark Brown <broonie@kernel.org>
zhichang.yuan [Mon, 28 Apr 2014 05:11:31 +0000 (06:11 +0100)]
arm64: lib: Implement optimized memset routine
This patch, based on Linaro's Cortex Strings library, improves
the performance of the assembly optimized memset() function.
Signed-off-by: Zhichang Yuan <zhichang.yuan@linaro.org>
Signed-off-by: Deepak Saxena <dsaxena@linaro.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
b29a51fe0e0be63157d8661666be8bbfd8f0c5d7)
Signed-off-by: Mark Brown <broonie@kernel.org>
zhichang.yuan [Mon, 28 Apr 2014 05:11:30 +0000 (06:11 +0100)]
arm64: lib: Implement optimized memmove routine
This patch, based on Linaro's Cortex Strings library, improves
the performance of the assembly optimized memmove() function.
Signed-off-by: Zhichang Yuan <zhichang.yuan@linaro.org>
Signed-off-by: Deepak Saxena <dsaxena@linaro.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
280adc1951c0c9fc8f2d85571ff563a1c412b1cd)
Signed-off-by: Mark Brown <broonie@kernel.org>
zhichang.yuan [Mon, 28 Apr 2014 05:11:29 +0000 (06:11 +0100)]
arm64: lib: Implement optimized memcpy routine
This patch, based on Linaro's Cortex Strings library, improves
the performance of the assembly optimized memcpy() function.
Signed-off-by: Zhichang Yuan <zhichang.yuan@linaro.org>
Signed-off-by: Deepak Saxena <dsaxena@linaro.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
808dbac6b51f3441eb5a07724c0b0d1257046d51)
Signed-off-by: Mark Brown <broonie@kernel.org>
Mark Salter [Thu, 15 May 2014 14:19:22 +0000 (15:19 +0100)]
arm64: fix pud_huge() for 2-level pagetables
The following happens when trying to run a kvm guest on a kernel
configured for 64k pages. This doesn't happen with 4k pages:
BUG: failure at include/linux/mm.h:297/put_page_testzero()!
Kernel panic - not syncing: BUG!
CPU: 2 PID: 4228 Comm: qemu-system-aar Tainted: GF 3.13.0-0.rc7.31.sa2.k32v1.aarch64.debug #1
Call trace:
[<
fffffe0000096034>] dump_backtrace+0x0/0x16c
[<
fffffe00000961b4>] show_stack+0x14/0x1c
[<
fffffe000066e648>] dump_stack+0x84/0xb0
[<
fffffe0000668678>] panic+0xf4/0x220
[<
fffffe000018ec78>] free_reserved_area+0x0/0x110
[<
fffffe000018edd8>] free_pages+0x50/0x88
[<
fffffe00000a759c>] kvm_free_stage2_pgd+0x30/0x40
[<
fffffe00000a5354>] kvm_arch_destroy_vm+0x18/0x44
[<
fffffe00000a1854>] kvm_put_kvm+0xf0/0x184
[<
fffffe00000a1938>] kvm_vm_release+0x10/0x1c
[<
fffffe00001edc1c>] __fput+0xb0/0x288
[<
fffffe00001ede4c>] ____fput+0xc/0x14
[<
fffffe00000d5a2c>] task_work_run+0xa8/0x11c
[<
fffffe0000095c14>] do_notify_resume+0x54/0x58
In arch/arm/kvm/mmu.c:unmap_range(), we end up doing an extra put_page()
on the stage2 pgd which leads to the BUG in put_page_testzero(). This
happens because a pud_huge() test in unmap_range() returns true when it
should always be false with 2-level pages tables used by 64k pages.
This patch removes support for huge puds if 2-level pagetables are
being used.
Signed-off-by: Mark Salter <msalter@redhat.com>
[catalin.marinas@arm.com: removed #ifndef around PUD_SIZE check]
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: <stable@vger.kernel.org> # v3.11+
(cherry picked from commit
4797ec2dc83a43be35bad56037d1b53db9e2b5d5)
Signed-off-by: Mark Brown <broonie@kernel.org>
Ashwin Chaugule [Wed, 7 May 2014 14:18:36 +0000 (10:18 -0400)]
ARM: Check if a CPU has gone offline
PSCIv0.2 adds a new function called AFFINITY_INFO, which
can be used to query if a specified CPU has actually gone
offline. Calling this function via cpu_kill ensures that
a CPU has quiesced after a call to cpu_die. This helps
prevent the CPU from doing arbitrary bad things when data
or instructions are clobbered (as happens with kexec)
in the window between a CPU announcing that it is dead
and said CPU leaving the kernel.
Signed-off-by: Ashwin Chaugule <ashwin.chaugule@linaro.org>
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
c814ca029e1015bb0ecec312f4bb9751ba1a711a)
Signed-off-by: Mark Brown <broonie@kernel.org>
Conflicts:
arch/arm64/include/asm/cpu_ops.h
Rob Herring [Wed, 15 May 2013 01:51:38 +0000 (20:51 -0500)]
ARM: PSCI: remove unnecessary include of arm-gic.h
Now that gic_secondary_init is no longer needed to be called by SMP init
functions, the header is not needed.
Signed-off-by: Rob Herring <rob.herring@calxeda.com>
Cc: Russell King <linux@arm.linux.org.uk>
(cherry picked from commit
97fc4de3d93fea934b0ec8332465b9f1899a9adb)
Signed-off-by: Mark Brown <broonie@kernel.org>
Arnd Bergmann [Mon, 3 Jun 2013 22:09:14 +0000 (23:09 +0100)]
ARM: 7745/1: psci: fix building without HOTPLUG_CPU
The cpu_die field in smp_operations is not valid with CONFIG_HOTPLUG_CPU,
so we must enclose it in #ifdef, but at least that lets us remove
two other lines.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
(cherry picked from commit
fdeb94b5dc5bf9db7b3e36f3f38089a554f6a108)
Signed-off-by: Mark Brown <broonie@kernel.org>
Ashwin Chaugule [Thu, 17 Apr 2014 18:38:41 +0000 (14:38 -0400)]
PSCI: Add initial support for PSCIv0.2 functions
The PSCIv0.2 spec defines standard values of function IDs
and introduces a few new functions. Detect version of PSCI
and appropriately select the right PSCI functions.
Signed-off-by: Ashwin Chaugule <ashwin.chaugule@linaro.org>
Reviewed-by: Rob Herring <robh@kernel.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
e71246a23acbc89e9cb4ebf1558d60e65733479f)
Signed-off-by: Mark Brown <broonie@kernel.org>
Stefano Stabellini [Tue, 21 May 2013 14:24:11 +0000 (14:24 +0000)]
arm: introduce psci_smp_ops
Rename virt_smp_ops to psci_smp_ops and move them to arch/arm/kernel/psci_smp.c.
Remove mach-virt/platsmp.c, now unused.
Compile psci_smp if CONFIG_ARM_PSCI and CONFIG_SMP.
Add a cpu_die smp_op based on psci_ops.cpu_off.
Initialize PSCI before setting smp_ops in setup_arch.
If PSCI is available on the platform, prefer psci_smp_ops over the
platform smp_ops.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Will Deacon <will.deacon@arm.com>
CC: arnd@arndb.de
CC: marc.zyngier@arm.com
CC: linux@arm.linux.org.uk
CC: nico@linaro.org
CC: rob.herring@calxeda.com
(cherry picked from commit
05774088391c7430f6a4c1d5d18196ef17bb3ba9)
Signed-off-by: Mark Brown <broonie@kernel.org>
Will Deacon [Fri, 2 May 2014 15:24:15 +0000 (16:24 +0100)]
arm64: mm: use inner-shareable barriers for inner-shareable maintenance
In order to ensure ordering and completion of inner-shareable maintenance
instructions (cache and TLB) on AArch64, we can use the -ish suffix to
the dmb and dsb instructions respectively.
This patch updates our low-level cache and tlb maintenance routines to
use the inner-shareable barrier variants where appropriate.
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
dc60b777fcdddbadab111028e266fd69d4702b34)
Signed-off-by: Mark Brown <broonie@kernel.org>
Will Deacon [Fri, 2 May 2014 15:24:13 +0000 (16:24 +0100)]
arm64: head: fix cache flushing and barriers in set_cpu_boot_mode_flag
set_cpu_boot_mode_flag is used to identify which exception levels are
encountered across the system by CPUs trying to enter the kernel. The
basic algorithm is: if a CPU is booting at EL2, it will set a flag at
an offset of #4 from __boot_cpu_mode, a cacheline-aligned variable.
Otherwise, a flag is set at an offset of zero into the same cacheline.
This enables us to check that all CPUs booted at the same exception
level.
This cacheline is written with the stage-1 MMU off (that is, via a
strongly-ordered mapping) and will bypass any clean lines in the cache,
leading to potential coherence problems when the variable is later
checked via the normal, cacheable mapping of the kernel image.
This patch reworks the broken flushing code so that we:
(1) Use a DMB to order the strongly-ordered write of the cacheline
against the subsequent cache-maintenance operation (by-VA
operations only hazard against normal, cacheable accesses).
(2) Use a single dc ivac instruction to invalidate any clean lines
containing a stale copy of the line after it has been updated.
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
d0488597a1b7105957b6d7d1bb0b6ee88aa51b37)
Signed-off-by: Mark Brown <broonie@kernel.org>
Will Deacon [Fri, 2 May 2014 15:24:12 +0000 (16:24 +0100)]
arm64: barriers: use barrier() instead of smp_mb() when !SMP
The recently introduced acquire/release accessors refer to smp_mb()
in the !CONFIG_SMP case. This is confusing when reading the code, so use
barrier() directly when we know we're UP.
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
be6209a6107e0f63544e3e7d00fd5c95434ec80a)
Signed-off-by: Mark Brown <broonie@kernel.org>
Conflicts:
arch/arm64/include/asm/barrier.h
Will Deacon [Fri, 2 May 2014 15:24:11 +0000 (16:24 +0100)]
arm64: barriers: wire up new barrier options
Now that all callers of the barrier macros are updated to pass the
mandatory options, update the macros so the option is actually used.
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
(cherry picked from commit
493e68747e07b69da3d746352525a1ebd6b61d82)
Signed-off-by: Mark Brown <broonie@kernel.org>
Conflicts:
arch/arm64/include/asm/barrier.h
Mark Brown [Thu, 9 Oct 2014 17:02:47 +0000 (18:02 +0100)]
Merge tag 'kvm-for-lsk-v3.10-v1' of git://git.linaro.org/people/christoffer.dall/linux-kvm-arm into lsk-v3.10-kvm
KVM/ARM/arm64 Support for LSK v3.10
Sam Bobroff [Thu, 18 Sep 2014 23:40:41 +0000 (09:40 +1000)]
KVM: correct null pid check in kvm_vcpu_yield_to()
Correct a simple mistake of checking the wrong variable
before a dereference, resulting in the dereference not being
properly protected by rcu_dereference().
Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
27fbe64bfa63cfb9da025975b59d96568caa2d53)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Ard Biesheuvel [Fri, 12 Sep 2014 13:16:00 +0000 (15:16 +0200)]
KVM: check for !is_zero_pfn() in kvm_is_mmio_pfn()
Read-only memory ranges may be backed by the zero page, so avoid
misidentifying it a a MMIO pfn.
This fixes another issue I identified when testing QEMU+KVM_UEFI, where
a read to an uninitialized emulated NOR flash brought in the zero page,
but mapped as a read-write device region, because kvm_is_mmio_pfn()
misidentifies it as a MMIO pfn due to its PG_reserved bit being set.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Fixes: b88657674d39 ("ARM: KVM: user_mem_abort: support stage 2 MMIO page mapping")
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
85c8555ff07ef09261bd50d603cd4290cff5a8cc)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Ard Biesheuvel [Fri, 12 Sep 2014 20:17:23 +0000 (22:17 +0200)]
mm: export symbol dependencies of is_zero_pfn()
In order to make the static inline function is_zero_pfn() callable by
modules, export its symbol dependencies 'zero_pfn' and (for s390 and
mips) 'zero_page_mask'.
We need this for KVM, as CONFIG_KVM is a tristate for all supported
architectures except ARM and arm64, and testing a pfn whether it refers
to the zero page is required to correctly distinguish the zero page
from other special RAM ranges that may also have the PG_reserved bit
set, but need to be treated as MMIO memory.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
0b70068e47e8f0c813a902dc3d6def601fd15acb)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Santosh Shilimkar [Wed, 31 Jul 2013 16:44:42 +0000 (12:44 -0400)]
ARM: mm: Introduce virt_to_idmap() with an arch hook
On some PAE systems (e.g. TI Keystone), memory is above the
32-bit addressable limit, and the interconnect provides an
aliased view of parts of physical memory in the 32-bit addressable
space. This alias is strictly for boot time usage, and is not
otherwise usable because of coherency limitations. On such systems,
the idmap mechanism needs to take this aliased mapping into account.
This patch introduces virt_to_idmap() and a arch function pointer which
can be populated by platform which needs it. Also populate necessary
idmap spots with now available virt_to_idmap(). Avoided #ifdef approach
to be compatible with multi-platform builds.
Most architecture won't touch it and in that case virt_to_idmap()
fall-back to existing virt_to_phys() macro.
Cc: Russell King <linux@arm.linux.org.uk>
Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
(cherry picked from commit
4dc9a81715973cb137a14399420bb35b0ed7d6ef)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Will Deacon [Fri, 10 May 2013 17:07:19 +0000 (18:07 +0100)]
ARM: barrier: allow options to be passed to memory barrier instructions
On ARMv7, the memory barrier instructions take an optional `option'
field which can be used to constrain the effects of a memory barrier
based on shareability and access type.
This patch allows the caller to pass these options if required, and
updates the smp_*() barriers to request inner-shareable barriers,
affecting only stores for the _wmb variant. wmb() is also changed to
use the -st version of dsb.
Reported-by: Albin Tonnerre <albin.tonnerre@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
(cherry picked from commit
3ea128065ed20d33bd02ff6dab689f88e38000be)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Victor Kamensky [Tue, 25 Feb 2014 07:41:09 +0000 (08:41 +0100)]
ARM: 7990/1: asm: rename logical shift macros push pull into lspush lspull
Renames logical shift macros, 'push' and 'pull', defined in
arch/arm/include/asm/assembler.h, into 'lspush' and 'lspull'.
That eliminates name conflict between 'push' logical shift macro
and 'push' instruction mnemonic. That allows assembler.h to be
included in .S files that use 'push' instruction.
Suggested-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Victor Kamensky <victor.kamensky@linaro.org>
Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
(cherry picked from commit
d98b90ea22b0a28d9d787769704a9cf1ea5a513a)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Andrew Morton [Wed, 3 Jul 2013 22:02:11 +0000 (15:02 -0700)]
include/linux/mm.h: add PAGE_ALIGNED() helper
To test whether an address is aligned to PAGE_SIZE.
Cc: HATAYAMA Daisuke <d.hatayama@jp.fujitsu.com>
Cc: "Eric W. Biederman" <ebiederm@xmission.com>,
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
(cherry picked from commit
0fa73b86ef0797ca4fde5334117ca0b330f08030)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christoffer Dall [Thu, 2 Oct 2014 09:53:28 +0000 (11:53 +0200)]
Revert "arm, kvm: fix double lock on cpu_add_remove_lock"
This reverts commit
d77503eadd2f16f2900b9be79a1dc6f37e8cd579. The whole
register cpu hotplug fix series has not been applied, so LSK is released
without this fix. If we ever include that series in LSK later, then
this can be fixed later too.
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
Reviewed-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
(cherry picked from commit
553f809e23f00976caea7a1ebdabaa58a6383e7d)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Vladimir Murzin [Mon, 22 Sep 2014 14:52:48 +0000 (15:52 +0100)]
arm: kvm: fix CPU hotplug
On some platforms with no power management capabilities, the hotplug
implementation is allowed to return from a smp_ops.cpu_die() call as a
function return. Upon a CPU onlining event, the KVM CPU notifier tries
to reinstall the hyp stub, which fails on platform where no reset took
place following a hotplug event, with the message:
CPU1: smp_ops.cpu_die() returned, trying to resuscitate
CPU1: Booted secondary processor
Kernel panic - not syncing: unexpected prefetch abort in Hyp mode at: 0x80409540
unexpected data abort in Hyp mode at: 0x80401fe8
unexpected HVC/SVC trap in Hyp mode at: 0x805c6170
since KVM code is trying to reinstall the stub on a system where it is
already configured.
To prevent this issue, this patch adds a check in the KVM hotplug
notifier that detects if the HYP stub really needs re-installing when a
CPU is onlined and skips the installation call if the stub is already in
place, which means that the CPU has not been reset.
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Acked-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
37a34ac1d4775aafbc73b9db53c7daebbbc67e6a)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christoffer Dall [Fri, 26 Sep 2014 10:29:34 +0000 (12:29 +0200)]
arm/arm64: KVM: Report correct FSC for unsupported fault types
When we catch something that's not a permission fault or a translation
fault, we log the unsupported FSC in the kernel log, but we were masking
off the bottom bits of the FSC which was not very helpful.
Also correctly report the FSC for data and instruction faults rather
than telling people it was a DFCS, which doesn't exist in the ARM ARM.
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
0496daa5cf99741ce8db82686b4c7446a37feabb)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Joel Schopp [Wed, 9 Jul 2014 16:17:04 +0000 (11:17 -0500)]
arm/arm64: KVM: Fix VTTBR_BADDR_MASK and pgd alloc
The current aarch64 calculation for VTTBR_BADDR_MASK masks only 39 bits
and not all the bits in the PA range. This is clearly a bug that
manifests itself on systems that allocate memory in the higher address
space range.
[ Modified from Joel's original patch to be based on PHYS_MASK_SHIFT
instead of a hard-coded value and to move the alignment check of the
allocation to mmu.c. Also added a comment explaining why we hardcode
the IPA range and changed the stage-2 pgd allocation to be based on
the 40 bit IPA range instead of the maximum possible 48 bit PA range.
- Christoffer ]
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Joel Schopp <joel.schopp@amd.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
dbff124e29fa24aff9705b354b5f4648cd96e0bb)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christoffer Dall [Thu, 25 Sep 2014 16:41:07 +0000 (18:41 +0200)]
arm/arm64: KVM: Fix set_clear_sgi_pend_reg offset
The sgi values calculated in read_set_clear_sgi_pend_reg() and
write_set_clear_sgi_pend_reg() were horribly incorrectly multiplied by 4
with catastrophic results in that subfunctions ended up overwriting
memory not allocated for the expected purpose.
This showed up as bugs in kfree() and the kernel complaining a lot of
you turn on memory debugging.
This addresses: http://marc.info/?l=kvm&m=
141164910007868&w=2
Reported-by: Shannon Zhao <zhaoshenglong@huawei.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
0fea6d7628ed6e25a9ee1b67edf7c859718d39e8)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Marc Zyngier [Tue, 8 Jul 2014 11:09:07 +0000 (12:09 +0100)]
arm/arm64: KVM: vgic: make number of irqs a configurable attribute
In order to make the number of interrupts configurable, use the new
fancy device management API to add KVM_DEV_ARM_VGIC_GRP_NR_IRQS as
a VGIC configurable attribute.
Userspace can now specify the exact size of the GIC (by increments
of 32 interrupts).
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
(cherry picked from commit
a98f26f183801685ef57333de4bafd4bbc692c7c)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Marc Zyngier [Tue, 8 Jul 2014 11:09:06 +0000 (12:09 +0100)]
arm/arm64: KVM: vgic: delay vgic allocation until init time
It is now quite easy to delay the allocation of the vgic tables
until we actually require it to be up and running (when the first
vcpu is kicking around, or someones tries to access the GIC registers).
This allow us to allocate memory for the exact number of CPUs we
have. As nobody configures the number of interrupts just yet,
use a fallback to VGIC_NR_IRQS_LEGACY.
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
(cherry picked from commit
4956f2bc1fdee4bc336532f3f34635a8534cedfd)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Marc Zyngier [Tue, 8 Jul 2014 11:09:05 +0000 (12:09 +0100)]
arm/arm64: KVM: vgic: kill VGIC_NR_IRQS
Nuke VGIC_NR_IRQS entierly, now that the distributor instance
contains the number of IRQ allocated to this GIC.
Also add VGIC_NR_IRQS_LEGACY to preserve the current API.
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
(cherry picked from commit
5fb66da64064d0cb8dcce4cc8bf4cb1b921b13a0)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Marc Zyngier [Tue, 8 Jul 2014 11:09:04 +0000 (12:09 +0100)]
arm/arm64: KVM: vgic: handle out-of-range MMIO accesses
Now that we can (almost) dynamically size the number of interrupts,
we're facing an interesting issue:
We have to evaluate at runtime whether or not an access hits a valid
register, based on the sizing of this particular instance of the
distributor. Furthermore, the GIC spec says that accessing a reserved
register is RAZ/WI.
For this, add a new field to our range structure, indicating the number
of bits a single interrupts uses. That allows us to find out whether or
not the access is in range.
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
(cherry picked from commit
c3c918361adcceb816c92b21dd95d2b46fb96a8f)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Marc Zyngier [Tue, 8 Jul 2014 11:09:03 +0000 (12:09 +0100)]
arm/arm64: KVM: vgic: kill VGIC_MAX_CPUS
We now have the information about the number of CPU interfaces in
the distributor itself. Let's get rid of VGIC_MAX_CPUS, and just
rely on KVM_MAX_VCPUS where we don't have the choice. Yet.
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
(cherry picked from commit
fc675e355e705a046df7b635d3f3330c0ad94569)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Marc Zyngier [Tue, 8 Jul 2014 11:09:02 +0000 (12:09 +0100)]
arm/arm64: KVM: vgic: Parametrize VGIC_NR_SHARED_IRQS
Having a dynamic number of supported interrupts means that we
cannot relly on VGIC_NR_SHARED_IRQS being fixed anymore.
Instead, make it take the distributor structure as a parameter,
so it can return the right value.
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
(cherry picked from commit
fb65ab63b8cae510ea1e43e68b5da2f9980aa6d5)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Marc Zyngier [Tue, 8 Jul 2014 11:09:01 +0000 (12:09 +0100)]
arm/arm64: KVM: vgic: switch to dynamic allocation
So far, all the VGIC data structures are statically defined by the
*maximum* number of vcpus and interrupts it supports. It means that
we always have to oversize it to cater for the worse case.
Start by changing the data structures to be dynamically sizeable,
and allocate them at runtime.
The sizes are still very static though.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
(cherry picked from commit
c1bfb577addd4867a82c4f235824a315d5afb94a)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Marc Zyngier [Tue, 8 Jul 2014 11:09:00 +0000 (12:09 +0100)]
KVM: ARM: vgic: plug irq injection race
As it stands, nothing prevents userspace from injecting an interrupt
before the guest's GIC is actually initialized.
This goes unnoticed so far (as everything is pretty much statically
allocated), but ends up exploding in a spectacular way once we switch
to a more dynamic allocation (the GIC data structure isn't there yet).
The fix is to test for the "ready" flag in the VGIC distributor before
trying to inject the interrupt. Note that in order to avoid breaking
userspace, we have to ignore what is essentially an error.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
71afaba4a2e98bb7bdeba5078370ab43d46e67a1)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christoffer Dall [Sat, 14 Jun 2014 20:34:04 +0000 (22:34 +0200)]
arm/arm64: KVM: vgic: Clarify and correct vgic documentation
The VGIC virtual distributor implementation documentation was written a
very long time ago, before the true nature of the beast had been
partially absorbed into my bloodstream. Clarify the docs.
Plus, it fixes an actual bug. ICFRn, pfff.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
7e362919a59e6fc60e08ad1cf0b047291d1ca2e9)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christoffer Dall [Sat, 14 Jun 2014 20:30:45 +0000 (22:30 +0200)]
arm/arm64: KVM: vgic: Fix SGI writes to GICD_I{CS}PENDR0
Writes to GICD_ISPENDR0 and GICD_ICPENDR0 ignore all settings of the
pending state for SGIs. Make sure the implementation handles this
correctly.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
9da48b5502622f9f0e49df957521ec43a0c9f4c1)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christoffer Dall [Sat, 14 Jun 2014 19:54:51 +0000 (21:54 +0200)]
arm/arm64: KVM: vgic: Improve handling of GICD_I{CS}PENDRn
Writes to GICD_ISPENDRn and GICD_ICPENDRn are currently not handled
correctly for level-triggered interrupts. The spec states that for
level-triggered interrupts, writes to the GICD_ISPENDRn activate the
output of a flip-flop which is in turn or'ed with the actual input
interrupt signal. Correspondingly, writes to GICD_ICPENDRn simply
deactivates the output of that flip-flop, but does not (of course) affect
the external input signal. Reads from GICC_IAR will also deactivate the
flip-flop output.
This requires us to track the state of the level-input separately from
the state in the flip-flop. We therefore introduce two new variables on
the distributor struct to track these two states. Astute readers may
notice that this is introducing more state than required (because an OR
of the two states gives you the pending state), but the remaining vgic
code uses the pending bitmap for optimized operations to figure out, at
the end of the day, if an interrupt is pending or not on the distributor
side. Refactoring the code to consider the two state variables all the
places where we currently access the precomputed pending value, did not
look pretty.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
faa1b46c3e9f4d40359aee04ff275eea5f4cae3a)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christoffer Dall [Sat, 14 Jun 2014 20:37:33 +0000 (22:37 +0200)]
arm/arm64: KVM: vgic: Clear queued flags on unqueue
If we unqueue a level-triggered interrupt completely, and the LR does
not stick around in the active state (and will therefore no longer
generate a maintenance interrupt), then we should clear the queued flag
so that the vgic can actually queue this level-triggered interrupt at a
later time and deal with its pending state then.
Note: This should actually be properly fixed to handle the active state
on the distributor.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
cced50c9280ef7ca1af48080707a170efa1adfa0)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christoffer Dall [Mon, 9 Jun 2014 10:55:13 +0000 (12:55 +0200)]
arm/arm64: KVM: Rename irq_active to irq_queued
We have a special bitmap on the distributor struct to keep track of when
level-triggered interrupts are queued on the list registers. This was
named irq_active, which is confusing, because the active state of an
interrupt as per the GIC spec is a different thing, not specifically
related to edge-triggered/level-triggered configurations but rather
indicates an interrupt which has been ack'ed but not yet eoi'ed.
Rename the bitmap and the corresponding accessor functions to irq_queued
to clarify what this is actually used for.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
dbf20f9d8105cca531614c8bff9a74351e8e67e7)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christoffer Dall [Mon, 9 Jun 2014 10:27:18 +0000 (12:27 +0200)]
arm/arm64: KVM: Rename irq_state to irq_pending
The irq_state field on the distributor struct is ambiguous in its
meaning; the comment says it's the level of the input put, but that
doesn't make much sense for edge-triggered interrupts. The code
actually uses this state variable to check if the interrupt is in the
pending state on the distributor so clarify the comment and rename the
actual variable and accessor methods.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
227844f53864077ccaefe01d0960fcccc03445ce)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Will Deacon [Tue, 2 Sep 2014 09:27:36 +0000 (10:27 +0100)]
KVM: VFIO: register kvm_device_ops dynamically
Now that we have a dynamic means to register kvm_device_ops, use that
for the VFIO kvm device, instead of relying on the static table.
This is achieved by a module_init call to register the ops with KVM.
Cc: Gleb Natapov <gleb@kernel.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Alex Williamson <Alex.Williamson@redhat.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
80ce1639727e9d38729c34f162378508c307ca25)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Will Deacon [Tue, 2 Sep 2014 09:27:34 +0000 (10:27 +0100)]
KVM: ARM: vgic: register kvm_device_ops dynamically
Now that we have a dynamic means to register kvm_device_ops, use that
for the ARM VGIC, instead of relying on the static table.
Cc: Gleb Natapov <gleb@kernel.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
c06a841bf36340e9e917ce60d11a6425ac85d0bd)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Will Deacon [Tue, 2 Sep 2014 09:27:33 +0000 (10:27 +0100)]
KVM: device: add simple registration mechanism for kvm_device_ops
kvm_ioctl_create_device currently has knowledge of all the device types
and their associated ops. This is fairly inflexible when adding support
for new in-kernel device emulations, so move what we currently have out
into a table, which can support dynamic registration of ops by new
drivers for virtual hardware.
Cc: Alex Williamson <Alex.Williamson@redhat.com>
Cc: Alex Graf <agraf@suse.de>
Cc: Gleb Natapov <gleb@kernel.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
d60eacb07053142bfb9b41582074a89a790a9d46)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Eric Auger [Mon, 1 Sep 2014 08:36:08 +0000 (09:36 +0100)]
KVM: EVENTFD: remove inclusion of irq.h
No more needed. irq.h would be void on ARM.
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Eric Auger <eric.auger@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
(cherry picked from commit
0ba09511ddc3ff0b462f37b4fe4b9c4dccc054ec)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Ard Biesheuvel [Tue, 9 Sep 2014 10:27:09 +0000 (11:27 +0100)]
ARM/arm64: KVM: fix use of WnR bit in kvm_is_write_fault()
The ISS encoding for an exception from a Data Abort has a WnR
bit[6] that indicates whether the Data Abort was caused by a
read or a write instruction. While there are several fields
in the encoding that are only valid if the ISV bit[24] is set,
WnR is not one of them, so we can read it unconditionally.
Instead of fixing both implementations of kvm_is_write_fault()
in place, reimplement it just once using kvm_vcpu_dabt_iswrite(),
which already does the right thing with respect to the WnR bit.
Also fix up the callers to pass 'vcpu'
Acked-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
(cherry picked from commit
a7d079cea2dffb112e26da2566dd84c0ef1fce97)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christian Borntraeger [Thu, 4 Sep 2014 19:13:33 +0000 (21:13 +0200)]
KVM: remove redundant assignments in __kvm_set_memory_region
__kvm_set_memory_region sets r to EINVAL very early.
Doing it again is not necessary. The same is true later on, where
r is assigned -ENOMEM twice.
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
f2a25160887e00434ce1361007009120e1fecbda)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christian Borntraeger [Thu, 4 Sep 2014 19:13:32 +0000 (21:13 +0200)]
KVM: remove redundant assigment of return value in kvm_dev_ioctl
The first statement of kvm_dev_ioctl is
long r = -EINVAL;
No need to reassign the same value.
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
a13f533b2f1d53a7c0baa7490498caeab7bc8ba5)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christian Borntraeger [Thu, 4 Sep 2014 19:13:31 +0000 (21:13 +0200)]
KVM: remove redundant check of in_spin_loop
The expression `vcpu->spin_loop.in_spin_loop' is always true,
because it is evaluated only when the condition
`!vcpu->spin_loop.in_spin_loop' is false.
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
34656113182b704682e23d1363417536addfec97)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Radim Krčmář [Thu, 28 Aug 2014 13:13:03 +0000 (15:13 +0200)]
KVM: remove garbage arg to *hardware_{en,dis}able
In the beggining was on_each_cpu(), which required an unused argument to
kvm_arch_ops.hardware_{en,dis}able, but this was soon forgotten.
Remove unnecessary arguments that stem from this.
Signed-off-by: Radim KrÄ\8dmář <rkrcmar@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
13a34e067eab24fec882e1834fbf2cc31911d474)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Radim Krčmář [Thu, 28 Aug 2014 13:13:02 +0000 (15:13 +0200)]
KVM: static inline empty kvm_arch functions
Using static inline is going to save few bytes and cycles.
For example on powerpc, the difference is 700 B after stripping.
(5 kB before)
This patch also deals with two overlooked empty functions:
kvm_arch_flush_shadow was not removed from arch/mips/kvm/mips.c
2df72e9bc KVM: split kvm_arch_flush_shadow
and kvm_arch_sched_in never made it into arch/ia64/kvm/kvm-ia64.c.
e790d9ef6 KVM: add kvm_arch_sched_in
Signed-off-by: Radim KrÄ\8dmář <rkrcmar@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
0865e636aef751966e6e0f8950a26bc7391e923c)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Paolo Bonzini [Fri, 29 Aug 2014 12:01:17 +0000 (14:01 +0200)]
KVM: forward declare structs in kvm_types.h
Opaque KVM structs are useful for prototypes in asm/kvm_host.h, to avoid
"'struct foo' declared inside parameter list" warnings (and consequent
breakage due to conflicting types).
Move them from individual files to a generic place in linux/kvm_types.h.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
656473003bc7e056c3bbd4a4d9832dad01e86f76)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christoffer Dall [Tue, 26 Aug 2014 12:00:38 +0000 (14:00 +0200)]
KVM: Unconditionally export KVM_CAP_USER_NMI
The idea between capabilities and the KVM_CHECK_EXTENSION ioctl is that
userspace can, at run-time, determine if a feature is supported or not.
This allows KVM to being supporting a new feature with a new kernel
version without any need to update user space. Unfortunately, since the
definition of KVM_CAP_USER_NMI was guarded by #ifdef
__KVM_HAVE_USER_NMI, such discovery still required a user space update.
Therefore, unconditionally export KVM_CAP_USER_NMI and change the
the typo in the comment for the IOCTL number definition as well.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
44b5ce73c99c389817be71b9161bceb197d40ecb)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christoffer Dall [Tue, 26 Aug 2014 12:00:37 +0000 (14:00 +0200)]
KVM: Unconditionally export KVM_CAP_READONLY_MEM
The idea between capabilities and the KVM_CHECK_EXTENSION ioctl is that
userspace can, at run-time, determine if a feature is supported or not.
This allows KVM to being supporting a new feature with a new kernel
version without any need to update user space. Unfortunately, since the
definition of KVM_CAP_READONLY_MEM was guarded by #ifdef
__KVM_HAVE_READONLY_MEM, such discovery still required a user space
update.
Therefore, unconditionally export KVM_CAP_READONLY_MEM and change the
in-kernel conditional to rely on __KVM_HAVE_READONLY_MEM.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
0f8a4de3e088797576ac76200b634b802e5c7781)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Will Deacon [Tue, 26 Aug 2014 14:13:25 +0000 (15:13 +0100)]
KVM: vgic: declare probe function pointer as const
We extract the vgic probe function from the of_device_id data pointer,
which is const. Kill the sparse warning by ensuring that the local
function pointer is also marked as const.
Cc: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
de56fb1923ca11f428bf557870e0faa99f38762e)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Will Deacon [Tue, 26 Aug 2014 14:13:24 +0000 (15:13 +0100)]
KVM: vgic: return int instead of bool when checking I/O ranges
vgic_ioaddr_overlap claims to return a bool, but in reality it returns
an int. Shut sparse up by fixing the type signature.
Cc: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
1fa451bcc67fa921a04c5fac8dbcde7844d54512)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Will Deacon [Tue, 26 Aug 2014 14:13:23 +0000 (15:13 +0100)]
KVM: ARM/arm64: return -EFAULT if copy_from_user fails in set_timer_reg
We currently return the number of bytes not copied if set_timer_reg
fails, which is almost certainly not what userspace would like.
This patch returns -EFAULT instead.
Cc: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
bd218bce92d3868ba4fe5e9e3eb8199d2aa614af)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Will Deacon [Tue, 26 Aug 2014 14:13:22 +0000 (15:13 +0100)]
KVM: ARM/arm64: avoid returning negative error code as bool
is_valid_cache returns true if the specified cache is valid.
Unfortunately, if the parameter passed it out of range, we return
-ENOENT, which ends up as true leading to potential hilarity.
This patch returns false on the failure path instead.
Cc: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
18d457661fb9fa69352822ab98d39331c3d0e571)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Will Deacon [Tue, 26 Aug 2014 14:13:21 +0000 (15:13 +0100)]
KVM: ARM/arm64: fix broken __percpu annotation
Running sparse results in a bunch of noisy address space mismatches
thanks to the broken __percpu annotation on kvm_get_running_vcpus.
This function returns a pcpu pointer to a pointer, not a pointer to a
pcpu pointer. This patch fixes the annotation, which kills the warnings
from sparse.
Cc: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
4000be423cb01a8d09de878bb8184511c49d4238)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Will Deacon [Tue, 26 Aug 2014 14:13:20 +0000 (15:13 +0100)]
KVM: ARM/arm64: fix non-const declaration of function returning const
Sparse kicks up about a type mismatch for kvm_target_cpu:
arch/arm64/kvm/guest.c:271:25: error: symbol 'kvm_target_cpu' redeclared with different type (originally declared at ./arch/arm64/include/asm/kvm_host.h:45) - different modifiers
so fix this by adding the missing const attribute to the function
declaration.
Cc: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
6951e48bff0b55d2a8e825a953fc1f8e3a34bf1c)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christoffer Dall [Tue, 19 Aug 2014 10:18:04 +0000 (12:18 +0200)]
arm/arm64: KVM: Support KVM_CAP_READONLY_MEM
When userspace loads code and data in a read-only memory regions, KVM
needs to be able to handle this on arm and arm64. Specifically this is
used when running code directly from a read-only flash device; the
common scenario is a UEFI blob loaded with the -bios option in QEMU.
Note that the MMIO exit on writes to a read-only memory is ABI and can
be used to emulate block-erase style flash devices.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
98047888bb9fd57734028c44ec17413ddd623958)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christoffer Dall [Tue, 19 Aug 2014 10:15:00 +0000 (12:15 +0200)]
KVM: Introduce gfn_to_hva_memslot_prot
To support read-only memory regions on arm and arm64, we have a need to
resolve a gfn to an hva given a pointer to a memslot to avoid looping
through the memslots twice and to reuse the hva error checking of
gfn_to_hva_prot(), add a new gfn_to_hva_memslot_prot() function and
refactor gfn_to_hva_prot() to use this function.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
64d831269ccbca1fc6d739a0f3c8aa24afb43a5e)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Radim Krčmář [Thu, 21 Aug 2014 16:08:05 +0000 (18:08 +0200)]
KVM: add kvm_arch_sched_in
Introduce preempt notifiers for architecture specific code.
Advantage over creating a new notifier in every arch is slightly simpler
code and guaranteed call order with respect to kvm_sched_in.
Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
e790d9ef6405633b007339d746b709aed43a928d)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Christian Borntraeger [Tue, 19 Aug 2014 14:45:56 +0000 (16:45 +0200)]
KVM: avoid unnecessary synchronize_rcu
We dont have to wait for a grace period if there is no oldpid that
we are going to free. putpid also checks for NULL, so this patch
only fences synchronize_rcu.
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
7103f60de8bed21a0ad5d15d2ad5b7a333dda201)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Paolo Bonzini [Wed, 6 Aug 2014 12:24:45 +0000 (14:24 +0200)]
KVM: Move more code under CONFIG_HAVE_KVM_IRQFD
Commits
e4d57e1ee1ab (KVM: Move irq notifier implementation into
eventfd.c, 2014-06-30) included the irq notifier code unconditionally
in eventfd.c, while it was under CONFIG_HAVE_KVM_IRQCHIP before.
Similarly, commit
297e21053a52 (KVM: Give IRQFD its own separate enabling
Kconfig option, 2014-06-30) moved code from CONFIG_HAVE_IRQ_ROUTING
to CONFIG_HAVE_KVM_IRQFD but forgot to move the pieces that used to be
under CONFIG_HAVE_KVM_IRQCHIP.
Together, this broke compilation without CONFIG_KVM_XICS. Fix by adding
or changing the #ifdefs so that they point at CONFIG_HAVE_KVM_IRQFD.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
c77dcacb397519b6ade8f08201a4a90a7f4f751e)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Paul Mackerras [Mon, 30 Jun 2014 10:51:13 +0000 (20:51 +1000)]
KVM: Give IRQFD its own separate enabling Kconfig option
Currently, the IRQFD code is conditional on CONFIG_HAVE_KVM_IRQ_ROUTING.
So that we can have the IRQFD code compiled in without having the
IRQ routing code, this creates a new CONFIG_HAVE_KVM_IRQFD, makes
the IRQFD code conditional on it instead of CONFIG_HAVE_KVM_IRQ_ROUTING,
and makes all the platforms that currently select HAVE_KVM_IRQ_ROUTING
also select HAVE_KVM_IRQFD.
Signed-off-by: Paul Mackerras <paulus@samba.org>
Tested-by: Eric Auger <eric.auger@linaro.org>
Tested-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
297e21053a52f060944e9f0de4c64fad9bcd72fc)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Paul Mackerras [Mon, 30 Jun 2014 10:51:12 +0000 (20:51 +1000)]
KVM: Move irq notifier implementation into eventfd.c
This moves the functions kvm_irq_has_notifier(), kvm_notify_acked_irq(),
kvm_register_irq_ack_notifier() and kvm_unregister_irq_ack_notifier()
from irqchip.c to eventfd.c. The reason for doing this is that those
functions are used in connection with IRQFDs, which are implemented in
eventfd.c. In future we will want to use IRQFDs on platforms that
don't implement the GSI routing implemented in irqchip.c, so we won't
be compiling in irqchip.c, but we still need the irq notifiers. The
implementation is unchanged.
Signed-off-by: Paul Mackerras <paulus@samba.org>
Tested-by: Eric Auger <eric.auger@linaro.org>
Tested-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
e4d57e1ee1ab59f0cef0272800ac6c52e0ec814a)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Paul Mackerras [Mon, 30 Jun 2014 10:51:11 +0000 (20:51 +1000)]
KVM: Move all accesses to kvm::irq_routing into irqchip.c
Now that struct _irqfd does not keep a reference to storage pointed
to by the irq_routing field of struct kvm, we can move the statement
that updates it out from under the irqfds.lock and put it in
kvm_set_irq_routing() instead. That means we then have to take a
srcu_read_lock on kvm->irq_srcu around the irqfd_update call in
kvm_irqfd_assign(), since holding the kvm->irqfds.lock no longer
ensures that that the routing can't change.
Combined with changing kvm_irq_map_gsi() and kvm_irq_map_chip_pin()
to take a struct kvm * argument instead of the pointer to the routing
table, this allows us to to move all references to kvm->irq_routing
into irqchip.c. That in turn allows us to move the definition of the
kvm_irq_routing_table struct into irqchip.c as well.
Signed-off-by: Paul Mackerras <paulus@samba.org>
Tested-by: Eric Auger <eric.auger@linaro.org>
Tested-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
9957c86d659a4d5a2bed25ccbd3bfc9c3f25e658)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Paul Mackerras [Mon, 30 Jun 2014 10:51:10 +0000 (20:51 +1000)]
KVM: irqchip: Provide and use accessors for irq routing table
This provides accessor functions for the KVM interrupt mappings, in
order to reduce the amount of code that accesses the fields of the
kvm_irq_routing_table struct, and restrict that code to one file,
virt/kvm/irqchip.c. The new functions are kvm_irq_map_gsi(), which
maps from a global interrupt number to a set of IRQ routing entries,
and kvm_irq_map_chip_pin, which maps from IRQ chip and pin numbers to
a global interrupt number.
This also moves the update of kvm_irq_routing_table::chip[][]
into irqchip.c, out of the various kvm_set_routing_entry
implementations. That means that none of the kvm_set_routing_entry
implementations need the kvm_irq_routing_table argument anymore,
so this removes it.
This does not change any locking or data lifetime rules.
Signed-off-by: Paul Mackerras <paulus@samba.org>
Tested-by: Eric Auger <eric.auger@linaro.org>
Tested-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
8ba918d488caded2c4368b0b922eb905fe3bb101)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Paul Mackerras [Mon, 30 Jun 2014 10:51:09 +0000 (20:51 +1000)]
KVM: Don't keep reference to irq routing table in irqfd struct
This makes the irqfd code keep a copy of the irq routing table entry
for each irqfd, rather than a reference to the copy in the actual
irq routing table maintained in kvm/virt/irqchip.c. This will enable
us to change the routing table structure in future, or even not have a
routing table at all on some platforms.
The synchronization that was previously achieved using srcu_dereference
on the read side is now achieved using a seqcount_t structure. That
ensures that we don't get a halfway-updated copy of the structure if
we read it while another thread is updating it.
We still use srcu_read_lock/unlock around the read side so that when
changing the routing table we can be sure that after calling
synchronize_srcu, nothing will be using the old routing.
Signed-off-by: Paul Mackerras <paulus@samba.org>
Tested-by: Eric Auger <eric.auger@linaro.org>
Tested-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(cherry picked from commit
56f89f3629ffd1a21d38c3d0bea23deac0e284ce)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Marc Zyngier [Fri, 1 Aug 2014 11:00:36 +0000 (12:00 +0100)]
arm64: KVM: fix 64bit CP15 VM access for 32bit guests
Commit
f0a3eaff71b8 (ARM64: KVM: fix big endian issue in
access_vm_reg for 32bit guest) changed the way we handle CP15
VM accesses, so that all 64bit accesses are done via vcpu_sys_reg.
This looks like a good idea as it solves indianness issues in an
elegant way, except for one small detail: the register index is
doesn't refer to the same array! We end up corrupting some random
data structure instead.
Fix this by reverting to the original code, except for the introduction
of a vcpu_cp15_64_high macro that deals with the endianness thing.
Tested on Juno with 32bit SMP guests.
Cc: Victor Kamensky <victor.kamensky@linaro.org>
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
dedf97e8ff2c7513b1370e36b56e08b6bd0f0290)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Marc Zyngier [Thu, 31 Jul 2014 10:42:18 +0000 (11:42 +0100)]
KVM: arm64: GICv3: mandate page-aligned GICV region
Just like GICv2 was fixed in
63afbe7a0ac1
(kvm: arm64: vgic: fix hyp panic with 64k pages on juno platform),
mandate the GICV region to be both aligned on a page boundary and
its size to be a multiple of page size.
This prevents a guest from being able to poke at regions where we
have no idea what is sitting there.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
fb3ec67942e92e5713e05b7691b277d0a0c0575d)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Marc Zyngier [Thu, 31 Jul 2014 13:16:39 +0000 (14:16 +0100)]
arm64: KVM: GICv3: move system register access to msr_s/mrs_s
Commit
72c583951526 (arm64: gicv3: Allow GICv3 compilation with
older binutils) changed the way we express the GICv3 system registers,
but couldn't change the occurences used by KVM as the code wasn't
merged yet.
Just fix the accessors.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Christoffer Dall <christoffer.dall@linaro.org>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
f4c321eb268e932786c112e0b902ee942d91a336)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Will Deacon [Fri, 25 Jul 2014 15:29:12 +0000 (16:29 +0100)]
kvm: arm64: vgic: fix hyp panic with 64k pages on juno platform
If the physical address of GICV isn't page-aligned, then we end up
creating a stage-2 mapping of the page containing it, which causes us to
map neighbouring memory locations directly into the guest.
As an example, consider a platform with GICV at physical 0x2c02f000
running a 64k-page host kernel. If qemu maps this into the guest at
0x80010000, then guest physical addresses 0x80010000 - 0x8001efff will
map host physical region 0x2c020000 - 0x2c02efff. Accesses to these
physical regions may cause UNPREDICTABLE behaviour, for example, on the
Juno platform this will cause an SError exception to EL3, which brings
down the entire physical CPU resulting in RCU stalls / HYP panics / host
crashing / wasted weeks of debugging.
SBSA recommends that systems alias the 4k GICV across the bounding 64k
region, in which case GICV physical could be described as 0x2c020000 in
the above scenario.
This patch fixes the problem by failing the vgic probe if the physical
base address or the size of GICV aren't page-aligned. Note that this
generated a warning in dmesg about freeing enabled IRQs, so I had to
move the IRQ enabling later in the probe.
Cc: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Gleb Natapov <gleb@kernel.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Joel Schopp <joel.schopp@amd.com>
Cc: Don Dutile <ddutile@redhat.com>
Acked-by: Peter Maydell <peter.maydell@linaro.org>
Acked-by: Joel Schopp <joel.schopp@amd.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
(cherry picked from commit
63afbe7a0ac184ef8485dac4914e87b211b5bfaa)
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>