net/mlx4_core: Enforce irq affinity changes immediatly