Continuation of Previous Thread Regarding New Features, Improvements, and Bugs

[Michael SamsonParticipant]

Hi Ernest,

I hope you’re doing well!

Yesterday I updated to the latest version of Ajax Search Pro and was very happy to see that you had created some of the features we discussed. Thank you so much for taking my suggestions seriously and making them happen! =)

I spent some time setting up the new features and testing them, and did find a few issues. Since our previous thread had become so long I thought it best to start a new one here. For reference here is our original conversation:

https://wp-dreams.com/forums/topic/4-unrelated-issues-on-a-new-ajax-search-pro-integration/

To help organize this new thread I have broken it up into a few sections starting with the new features. I know there is a lot here to review, and so I thank you ahead of time for looking through all of this. It’s very important to me that I get this all done, as Ajax Search Pro is very important within our platform.

…..

PROBLEMS WITH NEWLY ADDED FEATURES

Addition of Post-Format Filter for Default Format

A. First, a simple feature request to add to this. The default format is being called “Standard.” On our site we call this format “Article.” What I think would be best here is to be able to customize the names of all the post-formats in the filters. You already have this ability for the other filter types, so why not this one too. If it isn’t possible to let the user rename all the formats, then at the least they should be able to rename the default format. Right now this is the one I need to rename.

B. I found that after adding the default post-format to the filters and pressing save in the admin, that it would then not display in the list of post-format filters in the admin. On the front-end of the search the default filter would display, but in the admin it is simply missing from the list after saving.

C. This is the biggest issue I found. When selecting specific post-formats (other than the default), the default post-format posts are not being excluded in search results. For example, if I select to display the “Image” post-format only, then that is all I should see in the search results. But instead I am seeing the Image posts mixed together with the default format posts. That’s the problem. By comparison, if I selected only the default post-format, that was working and all I saw was the default format posts.

Content Type Filter

This one feature is going to make such a big difference for our site. Thank you so much for adding this!

A. I found that in order to include standard “Posts” as a filter you have to use the “Custom post types” option. This was a little counter-intuitive to me, as I wanted to just include normal posts (which is what this option did). Is this what you intended?

B. For the “Taxonomy Terms” filter, is it possible to divide this into two separate filters, one for “Categories” and one for “Tags.” This would work a little better for us (as having it combined means I need to write a much longer title for this filter). It would also be better for search to split this up.

Feature to Disable the Isotopic Default Image

I almost didn’t see you had added this ability. 😉
This is working very nicely, and I only have one additional request. You’ve added the following options:

Show the default image
Show the description
Don’t show that result

Can you please add an option to show only the title. The reason I’m asking for this is that the search results usually only display the title. I’d prefer to keep it that way even on results without an image.

…..

NEW BUGS / ISSUES

Date Filter Bug

I just found this issue today. I have the date filters enabled and set to always start with a date range of exactly one year ago through the current date. I’m using the Relative Date feature and setting the To Date to 0 years, 0 months, 0 days before current date. That should result in any search results on today’s date being included in the search results. But this is where I found the problem. Posts that I made today were not displaying in the results (even after updating the index table manually).

I did some testing and found this was a date related problem. If I set the search to display results through tomorrow’s date then suddenly the posts from today were displaying. So for some reason when setting the date range to display results through the current date, it is not including the current date in the results.

Image Caching Not Functioning in Conjunction with WP Offload S3 Plugin

I’ve recently setup the plugin WP Offload S3 (https://deliciousbrains.com/wp-offload-s3/) which allows us to automatically upload all media library items to an AWS S3 Bucket. Whenever any media is uploaded to the media library it is automatically copied over to S3, and then the original media is deleted from the uploads directory. This is a very important ability for us as it will substantially lower our media storage costs.

What I discovered here today is that unfortunately the image caching in Ajax Search Pro is no longer functioning because of this. I noticed that the search results were all using the full size images from S3, and not from the cache. So I checked the cache folder and found it was empty. I did some testing and can see that the search cache in the asp_upload directory is working fine. It’s only the bfi_thumb folder that is affected by this. It looks like the offloading process to S3 is somehow interfering with the caching function.

I know this is a specialized plugin (WP Offload S3), but it is a very popular plugin and we certainly need to use it. It’s very important that the image caching can function in combination with this plugin. I’m hoping this is something you can help us to solve.

…..

NEW FEATURE SUGGESTION

Select All / Select None Options for Categories Filter

This is actually a new idea. We have a lot of categories on our site, 50 in fact. So in the categories filter it lists all 50 with a scrollbar. This all works nicely. But if you want to select say just one category, you then need to unselect 49 of them which takes forever. I had an idea that we could just add two options at the top of the list as follows:

Select All
Select None

These would display at the top of the categories list with check-boxes so the user can more easily select only the categories they want. You could also include an option perhaps in the admin to turn this feature on or off.

…..

REMAINING ITEMS FROM ORIGINAL THREAD

Linking Attachment Search Results to Parent Posts

You and I discussed this a little in the other thread and I think I have a good solution now:

We’re going to be doing a little project here in the coming weeks to make sure that all of our media files in the media library are properly linked (attached) to their parent posts. The attach feature is a standard part of the WP media library. What I’d like to be able to do in Ajax Search Pro is specify the target link for media search results. I envision this as an option where you can select the behavior of the media results links like this:

Link to media attached content
Link to media directly in light-box
Link to media on default page

Of these options above it is that first one that I need. This would link the media directly to the attached content specified for that media in the library. All this option would need to do is reference the attached content in the media library and use it for the link.

You could structure these options above to also work together with fall-backs. For example, you specify that you want the link to first use the attached content. If there is no attached content then it would fall back to the next option, displaying in the light-box. The user would be able to control which options are enabled and their priority order. That last option I put above is something we’re not using: It’s what you see when clicking “View” from the media library (which displays the media on a default page).

Customizable Headers in Settings Screen for “Generic Selectors” and Date Filters

I know you took care of the title for the Generic Selectors. Thank you for that! It would also be helpful to have a customizable title for the Date Filters, so I could write something like “Filter by Date”…

Attachments Not Displaying When Using the Index Table

You had written:

Make sure that the attachments are also indexed under the index table submenu: https://i.imgur.com/ZyyE3lp.png
This option is still under development/beta testing, just been added in the latest release, so there is a chance that it may not work 100% correctly at all times.

I checked and we do indeed have “Attachments” added to the post-types. This appears to be a bug in this new feature. I’m sure you’ll get this fixed with more testing. In the meantime I’ll leave the attachments set to use the standard search method.

Question 1 / Parts 1-3

After reading your comments on the advantages to having just one folder for the search cache and css, I agree, this makes more sense. I was thinking more about the organization, and not how efficient it was. I agree with you now on this.

Since there would just be the two folders, you could name them this way:

asp_cache_images (instead of bfi_thumb)
asp_cache_results (instead of asp_upload)

The css would still be stored in the same folder as search results, but I still think it makes sense to name the folder as a cache.

• This topic was modified 6 years, 3 months ago by  Michael Samson.
• This topic was modified 6 years, 3 months ago by  Michael Samson.

[Michael SamsonParticipant]

Hi Ernest,

It’s been nearly a week since I posted this thread and you have not responded at all. I’d greatly appreciate you letting me know that you saw this.

I know there are many issues in my post and it will take you time to reply to it all. I’m not rushing you. I only want to ensure that you saw this so that everything will ultimately be addressed.

I look forward to hearing from you…

~ Michael

[Ernest MarcinkoKeymaster]

Hi Michael,

I have read your post, please excuse me, I haven’t had time to complete my reply yet. Usually tickets consist of 1 or 2 minor issues listed, which I can guarantee to reply on within 48 hours of time. In this case unfortunately will take much longer, but I am working on it as soon as possible.

Best,
Ernest Marcinko
If you like my products, don't forget to rate them on codecanyon :)

[Michael SamsonParticipant]

Hi Ernest,

Thank you for letting me know that you saw everything. There’s no rush here at all, and I’d much rather you take your time carefully examining everything that I reported. I simply wanted to make sure you had seen my post.

While it’s true there are many items above, I did my very best to organize it clearly for you. With that said, I know it’s going to take some time to fully review everything.

I want you to know how much I appreciate your responsiveness and willingness to improve your plugin from this kind of user feedback. I’m very impressed both by you and Ajax Search Pro. My compliments!

Our platform is making great use of your plugin, and I’m really looking forward to showing you our implementation when we launch in a few more months!

Talk soon,

~ Michael

[Ernest MarcinkoKeymaster]

Hi!

Sorry for the late response, I wanted to test all the reported issues before responding to you. Thank you again for taking the time to letting me know, it does help a lot.

Addition of Post-Format Filter for Default Format
A. This was originally my idea to resolve this, however there is are two problems with customized term titles.
1. The translation plugins. Many customers do use WPML, qTranslatex or Polylang to translate the term titles, and use them in multiple languages. This would make it much more difficult to translate them, actually they would have to do it twice, which is not acceptable.
2. WordPress arleady has a UI to edit terms, so it would be redundant to add another one there.

Since this issue only affects this one single term, I have came up with better, but less convenient solution, which is already built-in, but not documented yet:

The filter ‘asp_post_format_standard’ gives you access to this one single term before printing, allowing you to change the name and the label, as you like it. See this custom code below.

B. You are correct, I don’t know how that got trhough the testing. I have corrected this for the ucpming release.

C. I can’t seem to recreate this issue. This was the most problematic feature to get through, but no matter what configuration I try, if the ‘Standard’ post is unchecked, I don’t see any posts from that format on my test servers.
I feel like this is going to be related to a very specific configuration.

Can you please send a screshot of your configuration on the following panels:
– Frontend search settings -> Categories & Taxonomy terms
– Frontend search settings -> Advanced
– The search box on the front-end, with the options visible

If you have any exclusions on the Advanced -> Exclude results, let me know that as well. I might be able to re-create it then, or tell you if there is something not okay with the configuration.

Content Type Filter
A. Yes, if I understand correctly. The content type filter should not affect the individual post types. If there are a number of post types selected within the custom post types filter, but the custom post types is not selected in this filter, it will exclude all CPT results.

B. Unfortunately no, I don’t think this would be possible.

Feature to Disable the Isotopic Default Image
Sure, I have noted this as a feature reqest.

Date Filter Bug
You are correct, my tests approve, that should be actually included. I have noted this issue as well.

Image Caching Not Functioning in Conjunction with WP Offload S3 Plugin
That may be a bit more difficult than that. Optimally, the plugin looks for the featured image, and requests it from the WordPress core itself. However there are cases, when there is no featured image, and the images are parsed from the posts content by URL. In that case, there is no further information about the media file, as it is not possible to locate an attachment by the URL only.

The best solution in this case would be to use another filter function (just like the one with the post format title), to change the image in the results using the method that the WP Offload S3 plugin does.
The problem is, that I am not familiar with that. If you happen to know how to request via a function this cached version of the image by attachment ID or by Post ID, I think I can construct a custom code to do the trick for you.

Select All / Select None Options for Categories Filter
This is actually already possible: https://i.imgur.com/mxyaDwT.png
Taken from Taxonomy filters documentation.

Linking Attachment Search Results to Parent Posts
I looked into this, and the parent post ID is always set to the post where the attachment is attached to, I didn’t know that. This makes it possible to get the attached post URL for the attachment with a custom code:

This won’t change the attachment URL, if there is no parent, or the parent is set to 0. This may actually go to the plugin core in the next release, I would like this one as an option.

The lightbox solution is not likely to happen, given that on most WP sites there is no lightbox script availalbe, and the ones that have, there is just too many different ‘lightbox’ scripts. I wouldn’t be able to make one that works in most cases.
The only solution is to enqueue a custom lightbox script, which is again, not a good idea, as it may cause conflicts with the existing ones, if the site has any.

I may however be able to help you to initialize your lightbox script upon the results appear, most scripts are capable of that. If you can give me a wroking URL, I might be able to help.

Customizable Headers in Settings Screen for “Generic Selectors” and Date Filters
Noted, I can make this one happen in the next release.

Attachments Not Displaying When Using the Index Table
Actually, I can’t seem to re-create this one either. I feel this is again something specific. I will try to experiment with it a bit more, maybe something pops-up.

Best,
Ernest Marcinko
If you like my products, don't forget to rate them on codecanyon :)

[Michael SamsonParticipant]

Hi Ernest,

I wanted to quickly acknowledge that I saw your response and even read through it already. I’m finishing up a project here today and will fully review everything you wrote and respond in detail early to mid next week.

Thank you as always for working with me to improve the plugin. I’ll be in touch in a few short days!

~ Michael

[Michael SamsonParticipant]

Hi Ernest,

Sorry for the delay in getting back to you on everything. It’s been a busy week here.

I’ve taken the time to test the solutions you’ve provided, and I have updates on every item…

Addition of Post-Format Filter for Default Format

A. I have tried using the filter you provided to customize the name of the “Standard” post-format, but unfortunately it had no effect. I changed where it said ‘Standard term’ to ‘Article’ and nothing happened. Did I use this filter incorrectly?

B. Thank you for correcting this issue. I recently found that due to that same problem if you re-saved the settings that it would end up removing the Standard post-format because it was no longer in the selections.

C. I spent a lot of time testing this issue, and made some progress, but I also uncovered new problems at the same time. The reason I was seeing Standard post-format posts mixed in no matter what post-format was selected was due to this feature being turned on in the Front-End Search Settings -> Advanced tab: “Show posts/CPM with empty (missing) taxonomy terms?” This was the cause of that problem, and when I turned it off I no longer saw the Standard posts appearing where they shouldn’t.

I have also turned off the settings for “Ignore checkbox filters that have nothing selected?” and “Allow results with missing custom fields, when using custom field selectors?” We’re not using any custom fields right now for filters.

The setting for “Category/Taxonomy terms logic” is set to “At least one selected terms should match.” The setting for “Logic between taxonomy groups” is set to “AND (default).”

I noticed during the testing that I was seeing inconsistent search results when selecting certain combinations of post-formats. To help narrow down the problem I made sure to only search for posts (and excluded all other search types). What I found was that if I selected two post-formats simultaneously, I seemed to get correct search results. But as soon as I would add a third post-format or more, I would get inconsistent results.

An example is I select post-formats for Image and Gallery, and I see 10 results which seem to be accurate. But then I go ahead and add a third post-format (lets say “Video”) and now some of the results from the previous search with two post-formats no longer display.

I also found that this was occurring when selecting all of the post-formats. I found that lots of search results would not be displayed that would display if I limited the post-formats to just one or two. There is definitely some kind of problem going on in general when selecting different combinations of post-formats. It is extremely inconsistent, and at this point I don’t think it’s my configuration.

Btw, in case you’re wondering, I am very easily able to see this kind of issue because we have lots of test posts and they all use titles identifying their post-format. We use titles like “Article Test Post” and “Video Test Post,” etc, to make it easy to identify things during testing. So I can quickly tell when the search is not being consistent with regards to post-formats.

Related Issue: See All Search Results
During my testing I also found that the “See All Search Results” button had an incorrect number at all times. It never reflected the same number of search results seen in the ajax results screen.

Content Type Filter

A. I think you misunderstood what I meant. What I don’t understand is that in this filter I had to select “Custom Post Types” in order to add normal posts to the filter. There was no option to simply have Posts. This is what I was asking about, as I had to use Custom Post Types to have Posts be a filter option.

B. That’s unfortunate. Right now I have this labeled as “Categories & Subcategories” which is on the longer side. It also looks funny listed directly under the much shorter filter options of Posts, Media, and Members. On our site we call tags Subcategories by the way. I wish there was a way to split this into two options.

Feature to Disable the Isotopic Default Image

Thank you for adding this to the proposed features. I think you should add these options:

Show Only the Title
Show the Title and Description (I just thought of this one)

Date Filter Bug

I’m really glad you were able to confirm this issue exists, and I look forward to seeing it corrected. Thanks!

Image Caching Not Functioning in Conjunction with WP Offload S3 Plugin

There have been some key decisions made here regarding this problem.

First, just to make things clearer, the images in the search results are currently loading directly from S3 using the rewritten URLs from the WP Offload S3 plugin. This is what ASP did by default in combination with WP Offload S3. And as reported last time, the image caching system stopped functioning. It was unable to cache the images, and the images in the search results were not coming from the cache.

We started thinking about the caching of these images and realized something important. Even if the image cache was working in combination with WP Offload S3, those images would then be stored in the /uploads/bfi_thumb/ folder. This completely bypasses the entire purpose of us using WP Offload S3, which is to reduce the disk space on our servers. We will ultimately have a huge number of posts for search, and the images would take up a lot of space (even at the smaller dimensions). It makes a lot more sense for us to keep the search related images in S3 just as we’re doing for all post related images.

So rather than fix this conflict we are deactivating the image caching in ASP, and are instead making sure that our site creates a 300x375px image specifically to be used in ASP search results. These images will be stored in S3 as part of our normal media upload process.

I found this setting under General Options -> Image Options -> Featured Image Size Source

It looks like this feature was designed to be used when the image caching is turned off. Am I correct? If that’s the case then we wouldn’t need a filter here at all, and rather just select the image size we want to use. Does that sound right? We will of course have to test this to ensure that it can successfully use the correct image size from S3 (right now it is having no problem using the full size image from S3).

Select All / Select None Options for Categories Filter

Thank you for pointing out where this option was. I’m not sure how I missed it! I have it all setup and it’s working perfectly.

Linking Attachment Search Results to Parent Posts

When I wrote this last time I actually forgot there was already a related option in the Attachments settings for what to do with attachment search results. The two existing options are “Link the results to attachment page” or “Link the results to attachment file directly.”

So I installed the filter provided that looks to see if the attachment (media) search results have any “attached content” in the media library, and then to use that URL for the search result. I’m happy to say that this filter worked perfectly! I tested it with a number of media files, and in every case it used the proper post URL in the search results.

Seeing how nicely this works I have an obvious suggestion for you:

Since you already have the two options mentioned above for attachments, why not add this filter as a built-in third option. It would be “Link the results to attached content in media library.” If you build this into your plugin’s core than I can remove it from functions.php.

On our end we are going to make sure that all of the media on our site is properly attached to it’s parent post. This will be automatic. In combination with this new feature in ASP, it means all of our media search results will link to the posts it originated from.

Lastly, you may also want to add a fall-back option just case the media is somehow not attached. I don’t really need this, but it’s not a bad idea.

Regarding what I said about a light-box option… I do not need this. It was always my intention to link the media directly to it’s parent post.

Customizable Headers in Settings Screen for “Generic Selectors” and Date Filters

Thanks for including this in the next release!

Attachments Not Displaying When Using the Index Table

Some bad news here. First I tested this option again, and enabled the index table for attachment search results. I tried a variety of configurations and nothing I did made this work. In all tests I could not get a single attachment search result to display.

Here’s where things got worse…

When I turned back the setting to make attachments use a standard search, suddenly this too was not working. Now I can’t get any attachment search results either way! Yieks!

I’ve been looking this over for 30 minutes now without success. I can’t get any attachment search results any more and I don’t know why. The only thing I can think of is that I cleared the cache and regenerated the index tables a few times. But I’m not sure why that would have caused this. There is also the fact that some of the posts do predate the installation of WP Offload S3. I wonder if the media results I was seeing were only from before the installation. Perhaps this is why it’s not working, as I cleared all the caches.

If that’s the case then this means the attachment search is not working with WP Offload S3. I hope that’s not what’s going on here. I’m concerned because attachment search results are very important to our platform and they must function.

I can provide you with a copy of WP Offload S3 if we have to for testing.

…..

My continued thanks for all your help Ernest. I hope we can get everything working perfectly. Search is really a central part of this platform we’re building.

Talk soon,

~ Michael

• This reply was modified 6 years, 2 months ago by  Michael Samson.

[Ernest MarcinkoKeymaster]

Hi Michael,

Thank you for getting back to me!

A. My mistake, it should be the name attribute, not the title:

<script src=”https://gist.github.com/anag0/9c72c671fe51eb1871b9a28aea7d51b4.js”></script>

C. “An example is I select post-formats for Image and Gallery, and I see 10 results which seem to be accurate. But then I go ahead and add a third post-format (lets say “Video”) and now some of the results from the previous search with two post-formats no longer display.”

I guess you have the results limit set to 10, as it is by default. So the live results list displays 10 results at most. Whenever the ‘Video’ is added, the video post format posts should be displayed as well. That does not mean however, that those will be appended to the end of the previous results list (image and gallery), as this is a completely new search process with a different configuration – thus if there are more relevant video post format posts, those will be displayed instead, based on the ordering.

Posts with the video post format extend the potential results list within the search query. The database query will first check the restrictions, then organize the set of results, group them, then order them, then cut them off at the limit.

More results, results counter
It may actually differ, for multiple reasons. There is a limit set on this number, so when this is set to load more results it does not flood the live results list. Also, during the post-processing some results may be removed and may differ in the actual count. If the counter was to show the actual expected number of results after post-processing, it would have to post process all the results, then return the count, which may crash the server in many cases. (I have actually tested this on a somewhat bigger database, it was very accurate, but horrible performance/crashed the server) This number is the raw count of rows that matched the full query, before the post processing, adjusted by an approximation of the potentially removed results.
The results page may also have a separete query adjusting the final results count, that unfortunately can’t be affected.

Image Caching Not Functioning in Conjunction with WP Offload S3 Plugin

“I found this setting under General Options -> Image Options -> Featured Image Size Source”
This actually works both ways, if the image caching is turned off, it will simply return the image defined, otherwise it will pass this image to the crop algorithm.

I think you should simply turn off the Crop images for caching? under the Cache settings submenu. If the offload cache plugin has a filter activated on getting the attachments function, it should automatically overwrite the image urls, and return them that way.

Linking Attachment Search Results to Parent Posts
Thank you for the feedback, I will definitely add this to the core as an option. I will think about the fallback feature as well.

Attachments Not Displaying When Using the Index Table
I believe I know why. Someone else reported a similar problem with some debug information. There is some sort of issue with the table aliases whenever the attachments are selected. It appears to affect some database versions/configurations only, as it worked on my end for some reason, but it actually should not, as the table alias is indeed incorrect in the code. This will be corrected as well in the next release.

Best,
Ernest Marcinko
If you like my products, don't forget to rate them on codecanyon :)

[Michael SamsonParticipant]

Hi Ernest,

I’ve copied below most of your responses (in italics) to make this easier for you to respond to. So this is in fact shorter than it looks!

Addition of Post-Format Filter for Default Format

A.

My mistake, it should be the name attribute, not the title:

Thank you for that correction. Changing it to name made it work!

C.

I guess you have the results limit set to 10, as it is by default. So the live results list displays 10 results at most. Whenever the ‘Video’ is added, the video post format posts should be displayed as well. That does not mean however, that those will be appended to the end of the previous results list (image and gallery), as this is a completely new search process with a different configuration – thus if there are more relevant video post format posts, those will be displayed instead, based on the ordering.

Posts with the video post format extend the potential results list within the search query. The database query will first check the restrictions, then organize the set of results, group them, then order them, then cut them off at the limit.

….

You were completely right about all of this! I hadn’t really paid much attention to the limits settings, and I should have. As soon as I increased them I saw completely different results during my testing. It makes complete sense now. Why didn’t I think about relevancy before either? That’s an important function in this search, and I wasn’t even thinking about it. Between the low limits and relevancy of the search, that explains why I was seeing confusing results.

As we’re building a large informational type site where search is extremely important, it made sense to set the limits much higher. I set all the ajax limits to 50 and the search result pages to 100. This made an enormous difference when I started re-testing, in a good way! I was able to easily see that the post-format filters are now working, in any combination I chose. =)

My only question here is, do you think it’s a bad idea to have higher limits in the ajax results for any reason? (keep in mind we are using the search results caching feature) I’m a fan of the ajax results (for convenience) and prefer to see higher numbers in those results rather than having to go to the actual search result pages.

More results, Results counter

It may actually differ, for multiple reasons. There is a limit set on this number, so when this is set to load more results it does not flood the live results list. Also, during the post-processing some results may be removed and may differ in the actual count. If the counter was to show the actual expected number of results after post-processing, it would have to post process all the results, then return the count, which may crash the server in many cases. (I have actually tested this on a somewhat bigger database, it was very accurate, but horrible performance/crashed the server) This number is the raw count of rows that matched the full query, before the post processing, adjusted by an approximation of the potentially removed results. The results page may also have a separete query adjusting the final results count, that unfortunately can’t be affected.

…..

I was actually incorrect in my original question, as I said this number never matched the number in the ajax results. This is incorrect because the number in the more results button reflects the number of results on the search results page (not what’s in the ajax results). I also now understand that the limits are set differently between the ajax results and results pages (which would always make this number different from what’s in the ajax results). Am I correct?

Your explanation about how this number is generated is interesting. There’s a lot more going on here behind the scenes than I realized. Thank you for sharing this with me, as I like to have a general understanding of how everything works.

The part of your comments that stood out to me was the part about not wanting to crash the server. 😉 We’re building a very large platform here (a social media site) that will ultimately contains millions of posts and possible search results of every kind. I can easily see how the power of this search could quickly overwhelm a server. This is something I want to take measures to prevent.

We’re building our site over at AWS (Amazon Web Services), and the database is using RDS with Amazon Aurora. It’s a database cluster with one DB for reading and one for writing. The good thing about this setup is that we can increase the resources of the DB at any time. I believe we have a robust and flexible infrastructure, but I still want to be careful not to over-stress things (like overwhelming the DB with search queries). If you have any advice or tips for me along these lines please let me know.

Image Caching Not Functioning in Conjunction with WP Offload S3 Plugin

“I found this setting under General Options -> Image Options -> Featured Image Size Source”
This actually works both ways, if the image caching is turned off, it will simply return the image defined, otherwise it will pass this image to the crop algorithm.

I think you should simply turn off the Crop images for caching? under the Cache settings submenu. If the offload cache plugin has a filter activated on getting the attachments function, it should automatically overwrite the image urls, and return them that way.

…..

That’s interesting that the image source size setting works with or without the image caching. I didn’t realize that. Pretty neat!

I had actually already turned off the image caching last time. Most of what I wrote about this was explaining the reasoning behind it. Frankly, the most important reason for us not to use this is so that our images are served directly from S3 (this is an important part of our disk space saving strategy).

You’re correct about the image URLs. They are being overwritten by the WP Offload S3 plugin and the search is indeed using those URLs. This is exactly what we need it to do.

I just tested the Featured Image Size Source setting, as I wanted to see if it would work with S3. I’m happy to say that it is working! Whatever image size I select is being loaded in search, and it is retrieving it properly from S3. I even found this setting was working with the filters we’re using for returning images for categories and tags (we wrote a custom script for this, it returns the image from the most recent post in the subcategory or tag archive).

So it looks like we’re in good shape on this one!

Linking Attachment Search Results to Parent Posts

Thank you for the feedback, I will definitely add this to the core as an option. I will think about the fallback feature as well.

…..

It’s my pleasure Ernest. You’ve been so amazing working with me on issue after issue. I really appreciate it, and am glad you’ve also found my feedback and suggestions useful.

I’ll watch for when you add this feature to core, and then I’ll just remove the custom filter from functions.php.

Attachments Not Displaying When Using the Index Table

I believe I know why. Someone else reported a similar problem with some debug information. There is some sort of issue with the table aliases whenever the attachments are selected. It appears to affect some database versions/configurations only, as it worked on my end for some reason, but it actually should not, as the table alias is indeed incorrect in the code. This will be corrected as well in the next release.

…..

Well, that is good news that you found the source of the original problem I reported. I definitely want to use the index table for returning attachment search results. It’s more efficient and should greatly decrease the number of DB queries.

Unfortunately I cannot get the attachment results working at all any more! I’m not sure if you realized this from my previous message. At this moment I cannot generate attachment search results whether using the normal search engine or the index table. I think the problem may be related to our use of WP Offload S3. Perhaps it is somehow preventing the search from being able to find the attachments.

If I go into our search right now and limit it to only display attachments (we call it media), there are no results at all. This used to be working for us, and suddenly stopped working. The only thing I can think of that changed recently is that we started to use WP Offload S3. This must be the source of the problem.

Can you help me to troubleshoot this? It’s pretty easy to test with the WP Offload S3 plugin. You just need an S3 bucket to test with, and they are free for small amounts of data. You could easily test this on your end and see if it’s the problem.

I definitely need to get the attachments working. I hope we can figure it out together.

…..

Thank you again for your continued help. You’ve seriously been amazing!

~ Michael

• This reply was modified 6 years, 2 months ago by  Michael Samson.

[Ernest MarcinkoKeymaster]

Hi Michael,

Thank you for your responses, I did read them earlier, I just didn’t have enough time to respond yet.

Addition of Post-Format Filter for Default Format
C. As long as you don’t see any major performance differences, I think it is completely fine to have more results. 50/100 as limits sounds great. I usually recommend keeping them under 100/200, and that is halfway there, it should be fine.

More results, Results counter
I was actually incorrect in my original question, as I said this number never matched the number in the ajax results. This is incorrect because the number in the more results button reflects the number of results on the search results page (not what’s in the ajax results). I also now understand that the limits are set differently between the ajax results and results pages (which would always make this number different from what’s in the ajax results). Am I correct?

Yes, that is actually correct. I believe there is a minor difference whenever this is used for loading more results or redirection to the results page, I don’t actually recall exactly.

Your setup sounds really advanced, I really hope the plugin will work more-less as expected, as it has never been tested on such a huge platform.

Attachments Not Displaying When Using the Index Table
The attachments in WordPress are actually stored in the same table as the Posts, so I am convinced that this is not related to your server, as the plugin would not work with any other post types either.

I have actually two possible guesses here, as I looked up the attachments search file.

1. Either there is something not okay with the configuration
2. ..or possibly, the taxonomy term filters affecting the attachments

I feel like this is going to be no 2. in this case. To test my theory, try creating another search instance (which you can delete later) for testing only. Then configure it:
– remove the default Post and Page post type under the General Options -> Sources panel
– Enable the attachments search under the General Options -> Attachments panel

This way only attachments should be returned. Now, try a few search queries. If there are any results, then this is very likely one of the two issues. Let me know, and I will try suggesting some modification to get through it.

Best,
Ernest Marcinko
If you like my products, don't forget to rate them on codecanyon :)

[Michael SamsonParticipant]

Hi Ernest,

It’s the end of my day but I wanted to respond to you quickly since I just ran a few tests…

Addition of Post-Format Filter for Default Format

C.

As long as you don’t see any major performance differences, I think it is completely fine to have more results. 50/100 as limits sounds great. I usually recommend keeping them under 100/200, and that is halfway there, it should be fine

Thank you for letting me know this. I don’t want to overwhelm the users with search results, but it was important to have a good number show up in ajax. That’s the main reason why I chose ASP in the first place! 😉

More results, Results counter

Yes, that is actually correct. I believe there is a minor difference whenever this is used for loading more results or redirection to the results page, I don’t actually recall exactly.

I’m using the more results button to redirect to a search results page, and am then overriding the default WP results. I hadn’t considered using this to load more ajax results, and will have to look at that option. We do have some challenges related to using search result pages, but those have nothing to do with ASP.

Your setup sounds really advanced, I really hope the plugin will work more-less as expected, as it has never been tested on such a huge platform.

Yes, our setup is extremely advanced and I’m very proud of what we’ve accomplished thus far. I actually chose your search originally because it was so well developed and thought out. I was looking specifically for an ajax search, and was very impressed by what I saw in Ajax Search Pro. Once I started working with the plugin I was blown away by its sophistication and flexibility. You really have put together an amazing search. Add to that your fantastic support and willingness to make improvements; it’s a win-win scenario!

One thing to remember about our platform is that it’s new. While we intend for it to grow quickly and eventually become huge, it is not there yet. It’s not even launched yet. Once we do launch I am looking forward to showing you the site in particular. I’ve personally done the integration work for ASP, and I think you’ll be impressed. We’ll have plenty of time to test further and make refinements as the site grows, so I’m not worried. This will ultimately be a great use-case for you! (and a way to show off what ASP is really capable of)

Attachments Not Displaying When Using the Index Table

Your suggestion to remove the posts from the search made sense, so I just tested it. We actually aren’t using pages as a source in our search at all (as our content is all post based).

Unfortunately this made no difference with trying to get attachment results. I did notice one thing. I seem to be able to get attachment results for my own avatar in test comments on some posts. But I can’t get any attachment results whatsoever for items in the media library. I still believe this is related to our use of WP Offload S3.

Let me ask you this, do the attachments have to be located in the /uploads folder in order to be returned in search results?

The reason I ask is that none of our actual media files are stored in /uploads. That’s why we’re using WP Offload S3. All of our media files are stored in an S3 bucket. You can still see the files in the media library, but all of their URLs point to S3. There are no actual files stored locally in /uploads. Could this be preventing the search from returning attachment results?

I’m having difficulty reconciling why this was functioning earlier and then suddenly stopped. Something obviously changed. When the attachments were working in search I could easily test it by taking the name of an attachment file and searching for it. The attachments would immediately come up that way. But this all stopped working for some reason.

I haven’t really changed many settings in ASP either. I have tested a few settings myself to see if they were the cause, but no luck.

This is a frustrating issue. I hope we can figure it out together!

~ Michael

[Michael SamsonParticipant]

Hi Ernest,

I just wanted to check in with you to see if you had read my latest message above…

Actually, I found something small for you to fix in the plugin:

Take a look at this file:
ajax-search-pro/backend/settings/assets/upload.js

I’ve been working here normalizing line-endings in my Git repo (not a fun activity), and I found this anomalous file in your plugin. It is using the CR line-ending which is an outdated line-ending for Macs. This should be set to LF for linux.

I found this because Git’s function for normalizing line-endings doesn’t seem to recognize CF, but rather only CRLF and LF. So this caused me a little issue here and I manually fixed it on my end.

Anyway, I look forward to hearing back from you so we can get all my issues resolved. =)

~ Michael

[Ernest MarcinkoKeymaster]

Hi Michael,

Sorry about the late response!

Attachments Not Displaying When Using the Index Table
The attachments search is basically the same as the posts search, the actual file location does not matter, the plugin simply returns the locations given by wordpress – no file operations are done.

Have you tried creating a completely new search instance and enabling only the attachments search? I am asking, because as I mentioned earlier, the taxonomy term filters/exclusions affect the attachment results as well. So if you have a certain configuration, it may explain it. Changing your current search instance to search only attachments will not eleminate this.
Also, are we talking about the live search results here, or the search results page after redirection?

upload.js
Thank you for this, that file should have been deleted, it was used only for testing purposes. It will be removed within the upcoming release.

Best,
Ernest Marcinko
If you like my products, don't forget to rate them on codecanyon :)

[Michael SamsonParticipant]

Hi Ernest,

I’m sorry I didn’t reply to you earlier this week. This has been a hectic week here and I’m leaving on a trip tomorrow to see family, and won’t be back until March 3rd. I will have a computer with me however and should be able to do some limited work (like testing).

I noticed that you released a new version of ASP last night. I’ll be installing that when I return next week. I’m hoping that the update contains some of the things we previously discussed. I haven’t read the change-log yet.

Attachments Not Displaying When Using the Index Table

Have you tried creating a completely new search instance and enabling only the attachments search?

I haven’t done this yet because I didn’t want to disrupt the site. But perhaps I can create a new instance to try on a single test page.

Also, are we talking about the live search results here, or the search results page after redirection?

Yes, we’re talking about the live ajax results. In fact, I decided to alter our configuration so that we now are only using the ajax results. We are no longer pointing the more results button to a separate results page. The main reasoning here was that we present our posts in a masonry grid format which isn’t the best for viewing search results. Additionally there were many integration related problems with displaying search results for things other than posts (like users, attachments, etc.). I realized that by limiting our results to ajax only that it eliminated all these technical problems and provided a far better user experience. Simply put, I love your ajax search, and now that is all we’re using for search!

I agree that we’re looking at some kind of configuration issue most likely. That would explain why the attachments were working and then suddenly stopped working. The only other explanation is that there is a conflict of some sort, perhaps with WP Offload S3.

While I’m away on my trip I’ll do some configuration related testing to see if I can get to the bottom of this. I’ll try your suggestion with creating a second search instance. You may not hear back from me until next week, but I will get back to you, not to worry.

Have a great week and talk soon…

~ Michael

[Michael SamsonParticipant]

Hi Ernest,

I’ve returned from my trip and finally had a chance to start working on this all again.

I updated ASP to the latest version and then did some testing to try and solve the issue with our attachment results not functioning.

I took your advice and created a new search instance to test with and had a breakthrough. I found that the attachment results were indeed functioning in the test instance, proving my problem is configuration related. The difficulty now is in finding the source of the problem in my configuration. It’s like looking for a needle in a haystack. I was wondering if perhaps you could help me to figure this out?

I’m attaching here a copy of my instance from an export…

I’m hoping that you can find the problem more quickly than I can. I have tried changing many settings, but so far no luck. You’ll see that with this configuration the attachment results do not function. There’s probably some kind of bug here, but I don’t know what setting it is related to.

…..

I also wanted to follow up with you on the items below…

I don’t expect you to be able to get to all of this immediately. It’s just easier to track things in a thread by copying them down this way.

…..

Attachments Not Displaying When Using the Index Table

I believe I know why. Someone else reported a similar problem with some debug information. There is some sort of issue with the table aliases whenever the attachments are selected. It appears to affect some database versions/configurations only, as it worked on my end for some reason, but it actually should not, as the table alias is indeed incorrect in the code. This will be corrected as well in the next release.

…..

Feature to Disable the Isotopic Default Image

I almost didn’t see you had added this ability. ?
This is working very nicely, and I only have one additional request. You’ve added the following options:

Show the default image
Show the description
Don’t show that result

Can you please add an option to show only the title. The reason I’m asking for this is that the search results usually only display the title. I’d prefer to keep it that way even on results without an image.

Sure, I have noted this as a feature request.

Thank you for adding this to the proposed features. I think you should add these options:

Show Only the Title
Show the Title and Description

…..

Date Filter Bug

I just found this issue today. I have the date filters enabled and set to always start with a date range of exactly one year ago through the current date. I’m using the Relative Date feature and setting the To Date to 0 years, 0 months, 0 days before current date. That should result in any search results on today’s date being included in the search results. But this is where I found the problem. Posts that I made today were not displaying in the results (even after updating the index table manually).

I did some testing and found this was a date related problem. If I set the search to display results through tomorrow’s date then suddenly the posts from today were displaying. So for some reason when setting the date range to display results through the current date, it is not including the current date in the results.

You are correct, my tests approve, that should be actually included. I have noted this issue as well.

…..

Linking Attachment Search Results to Parent Posts

It’s easier to just read everything we wrote on this above, instead of me copying it here. 😉

Thank you for the feedback, I will definitely add this to the core as an option. I will think about the fallback feature as well.

…..

Customizable Headers in Settings Screen for “Generic Selectors” and Date Filters

I know you took care of the title for the Generic Selectors. Thank you for that! It would also be helpful to have a customizable title for the Date Filters, so I could write something like “Filter by Date”…

Noted, I can make this one happen in the next release.

…..

That’s all for now!

~ Michael

• This reply was modified 6 years, 1 month ago by  Michael Samson.
• This reply was modified 6 years, 1 month ago by  Michael Samson.

Attachments:

You must be logged in to view attached files.

[Author]

Posts