ASoC: Use late enable handling for direct voice, speaker and headphone