I’m afraid yes – that or remove the custom code regarding the attachments, which shows the related posts.
I have looked through the source code to see if there is any way to make this happen, but there is simply no way to account for the changes in the results list during the post process. To be able to do that, I would have to change the plugin code to process all of the results all at once, then count how many there are actually left when everything is finished – but as simple as it sounds, that is the worst possible thing to do. The search performance would drop drastically and the server load will increase by a lot, because everything is processed all at once.