perf symbols: Retain bfd reference to lookup source line numbers