Hi!
Thank you very much for the details, it helps me a lot!
Your configuration looks perfect, there was one missing thing though. The index table was generated, however the index table engine was not in use, so I enabled it here: https://i.imgur.com/1w1vM12.png
I have also enabled the custom ajax handler, that will give you an additional performance boost. There are other tips and tricks in that documentation chapter related to performance, but I believe in your case no further change is needed.
After testing on the performance tracker, the average execution time dropped from 1.7s to 0.15s, which is much better.
You should notice a much better response time now.