arm64: Relax the kernel cache requirements for boot