When checking if a load can be folded, we check if there is any non-direct