Hi,
If the configuration was not changed whatsoever, then the plugin is definitely not at fault here.
I logged in to check, I found some areas of improvement in the configuration. First, I turned on the index table engine – it made a major improvement on the peformance to around 2 seconds from 8 seconds. I have also enabled the search cache, turned off some unused filters. I also turned on temporarily the performance tracker, to see the new the average search time dropped to 0.03 seconds. With this it seems like this is as fast as it is possible with then number of plugins active.