Hello!
May I ask you which version of the search were you running?
The versions before 2.0 were a lot slower then the current one. The plugin had been seriously reworked since then. Generally there are only a very few php files loaded, doing very few operations.
I’m attaching the newest version of the plugin to this post (this contains a few bugfixes which are not uploaded to codecanyon yet), hopefully you will get better results with the speedtest.