No problem 🙂 That makes sense now. I recommend checking the Fine tuning the search configuration guide. Especially the suggestions in chapter 1, as well as the index table engine. I believe Tthe index table engine in your case will make a huge difference – at it uses much faster, more optimized queries.
There is also a possibility to use a file/database cache – this can be useful for common repetitive queries.