Remove the block from the LoopInfo, rather than just the Loop.