spi: txx9: fix reference leak to master in txx9spi_remove()