Have DenseMap auto-shrink itself on clear(). This improves the time to optimize