[Michael Samson]

Hi Ernest,

I’m sure this won’t come as a surprise to you, but my last few weeks have been insanely busy. I’ve been wanting to get to this for a while now, but I had to focus my attention elsewhere for a while. We’re working on a number of large infrastructure and engineering projects over here. Suffice to say things are very busy but we’re making a lot of progress towards our launch.

I’ve installed the latest version of ASP and just fully tested it. You introduced some interesting new features. Most of these don’t apply to my use-case, but neat stuff none the less!

…..

Date Filter Bug

I was disappointed to discover that this problem is in fact not fixed. I tested it extremely carefully to be sure about this. The original problem remains. I’m using the “Relative Date” options for the From Date and To Date fields. In the From Date field it is set to exactly one year ago.In the To Date field it is set to today’s date by entering in “0” into all the fields.

To reiterate the problem, any posts that are posted on today’s date simply will not show up in search results. This is despite the fact the To Date field is set to include today’s date. The results only display through yesterday’s date.

This is a really insidious problem, as it tends to cause all sorts of other search problems. It makes it very difficult for me to test ASP because none of my test results show up until the following day.

To prove this was not corrected I disabled the To Date field, and after that the problem went away. I really hope you can get this fixed.

…..

Featured Image Source Size Not Working in All Situations

I do not believe you tried to fix this yet, but I did test for it. I found this problem remained. Here are my original notes on it below:

This is a new issue I haven’t told you about until now. We’ve been working here on customizing all of our image sizes site-wide, generating specific sizes for specific locations. For ASP we are creating a 300x375px image.

I set the Featured Image Source Size to use this specific image, but it’s not working across the board. What I found is that it only seems to work for normal post search results. The attachment results are displaying the full image size, and tag results are strangely displaying the thumbnail (small) size. For tag results (we call them subcategories) we have created functionality for the search to display the image from the first post in the subcategory. This is working beautifully, but it is using the incorrect image size. The problem with the tag images may be on our end in our script, and I have brought it to the attention of our developers.

…..

Attachment (Media) Images are Not Being Automatically Added to Index Table on Post Creation

I found this issue today while testing and paying attention to attachment search results. I have our index table set to automatically index content from any new posts on creation. This is working just fine for the posts, however, it is not indexing the attachments in those posts. I found that the attachments are only indexed if the index table itself is regenerated.

…..

Intermittent Bug With Number of Search Result Columns (Cards)

I also adjusted a bit the isotopic row calculation script, I’m not sure if that changes anything.

Unfortunately this had no effect. I do have one observation about this problem. I noticed that the thumbnail sizes only decreased if you completely closed the search field and ajax results followed by running another search on the same page. Every time you repeat that process the number of thumbnail columns increases. This does not occur if you run another search without closing the ajax results.

…..

The “See More Search Results” Button is Not Using Correct Font-Color Style on Mobile Devices When First Loaded

This is a weird little issue I noticed a long time ago. It is happening in the ajax results screen on the load more button (the button to load the next set of ajax results pages). I have custom stylized this button with css. I noticed that on mobile devices those styles do not properly apply to this button when the ajax screen first loads. The font-color displays incorrectly. But as soon as you press to go to the next ajax results page the css style works properly. This is a mobile-only issue, and I do not believe it’s related to my css. It must be a JS/AJAX problem.

…..

Linking Attachment Search Results to Parent (Linked) Posts (Core Integration)

Attachment to parent is not yet implemented, I postponed it to the upcoming minor/bugfix release.

Sounds good. Please let me know when this is integrated so I can remove the custom function we added in functions.php.

Btw, we finally got the other end of this working perfectly. All of our media items now automatically link to their parent-post pages in the media library. This is working beautifully with the function you previously provided for the search.

…..

Customizable Header in Settings Screen Date Filters

I know you postponed this one. I’m just listing it here for reference.

…..

Spelling Error for Synonyms Button

Just letting you know that the button/tab for the new synonyms functionality is misspelled. 😉

…..

QUESTIONS

A. I’m curious as to how the new synonyms functionality can be used to improve the search? What is its main purpose?

B. In your change-log you mentioned adding lazy-load functionality. Where was this added exactly? Does it apply to the ajax results? Is it activated by default?

…..

Issue and Question Regarding the Search Results Caching Feature and /asp_upload/ Directory

First the issue…

I noticed a few weeks ago that our search results cache was no longer writing files to /uploads/asp_upload/. This used to work, and I’m not sure why it stopped.

Now the question…

My understanding about this feature is that it caches common search results so that those results can be reloaded again without having to access the database. If this feature was turned off it would mean that every search would run a full query as if it were the first time, obviously utilizing the database.

We’ve run into an interesting engineering problem regarding this feature. Actually, it’s a problem common to all plugins that store dynamic data such as this in /uploads/. I mentioned earlier that we are doing a large engineering project here on our infrastructure. Part of that job has to do with segregating our static content from dynamic content. Our stack is based in AWS. A major objective of our project is to eliminate as many dynamic files as possible, as those files most be shared between all servers (it’s a load-balanced environment).

This particular feature for the search caching stores its files in /uploads/asp_upload/. In addition there is a dynamic css file stored in this folder (style.instances.css).

What I am trying to do is see if this information can be stored in the DB so that we can eliminate this folder completely. So long as this folder exists it must be shared between all servers, and while we can do this it is best not do for a variety of technical reasons.

I was wondering if perhaps the search results cache could instead be stored in the database. It would still have all the benefits of a cache, but it would avoid all these files needing to be stored in /uploads/. If this information were stored in the DB it would still avoid the need to run a fresh query for a search, therefore reducing DB usage.

I’m not as sure about the dynamic css file, but this too is a problem for the same reasons. I really need to get that over to either static file storage, or instead somehow have it store the data in the DB.

As I said, we can keep all of this if necessary in /uploads/, but it would be far better if we didn’t need to. This is why I am asking you about it, as I think changing this to the DB would benefit all ASP users. Avoiding use of /uploads/ is always beneficial.

I look forward to hearing your thoughts on this!

~ Michael

• This reply was modified 8 years ago by Michael Samson.
• This reply was modified 8 years ago by Michael Samson.
• This reply was modified 8 years ago by Michael Samson.

[Michael Samson]

Hi Ernest,

I just wanted to let you know that I did see your latest response here and am aware of the new plugin version. I’ve been involved all week with a rather large project to re-engineer our AWS stack. On top of this I was away on a short trip last weekend which cost me some days as well. Suffice to say I am rather behind on many things.

I’m going to try and carve out some time next week to update ASP and evaluate the new features. It may be a few more days before you hear back from me, but I’ll be following up for sure.

Thank you keeping me posted as always. I’ll be in touch after I can test everything.

All my best!

~ Michael

[Michael Samson]

Hi Ernest,

Another late reply on my part. Sorry about that!

Attachment Search / Filter for Default Post-Format

To answer your question, yes, it is a taxonomy, with an exception that the ‘default’ term does not exist. The conflict arises from the way how the query is constructed, and that it can’t hadnle this exception.

I’m glad the solution works as it should, I will keep it noted, in case someone else needs it in the future.

Thanks for answering this. WordPress really should make the default format into an actual post-format (instead of the lack thereof). This has caused us issues on more than one occasion with filter related functions.

In any case, you found an effective way to solve this without compromising functionality. Indeed, you should definitely keep this solution for the future. You may even want to make it an option in core, as without this the filtering for the default format can’t work properly (under a configuration like ours).

Intermittent Bug With Number of Search Result Columns (Cards)

Thank you. Well, at least I know a bit more, it may help. I will go though this code section before the next update, in case I find anything suspicious.

I can also help you to test this issue. While I can’t provide direct access to our site (as it is still a private venture), I can work with you jointly to test the problem. If you have any ideas please let me know. I know how to modify scripts using developer tools to test things quickly. I just need to know what to look for.

Listing what is left for reference:

– Date Filter Bug
– Linking Attachment Search Results to Parent (Linked) Posts (Core Integration)
– Customizable Header in Settings Screen Date Filters
– Featured Image Source Size Not Working in All Situations

A major update is coming soon, the date filter issue and the attachment link will be included with it. The date filter header will stay as it is for know, until I work out a better layout.

I look forward to the update! That date filter bug is a bad one, and it will be good to see it go away finally.

Having the attachment link functionality in core will also be great. Ironically we’re having some difficulty on this end automating the linking of media in the library to the parent posts. The developers keep getting it wrong (but we’re working on it). Once that is working it will work together properly with this function in search.

The date filter header is the least important of all the above remaining items. But we’ll get to it eventually.

If you’re wondering, we’re just a few months away from our launch. There’s still a lot on the table, but we’re making great progress. We’ll also be completing some major functionality after the initial launch. I’ll keep you posted for sure! 🙂

~ Michael

• This reply was modified 8 years, 1 month ago by Michael Samson.

[Michael Samson]

Hi Ernest,

I’m sorry it took me a full week to get back to you! I wanted to respond earlier, but I’ve been deep in multiple projects here as usual.

Attachment Search / Filter for Default Post-Format

I know this wasn’t your preferred solution, but it worked! If you ever think of a better method to accomplish the same thing just let me know.

I just installed this filter and tested it pretty thoroughly. As far as I can tell this is working no matter what combination of post-format options I choose. I tested just about everything you could think of. Here are a few examples I tested for:

Selecting default post-format by itself
Selecting default post-format in combination with one other post-format
Selecting default post-format in combination with multiple other post-formats
Selecting one post-format other than the default
Selecting multiple post-formats other than default
Selecting no post-formats at all

That last one (selecting no post-formats) was a good test, as this produced no search results. Prior to this fix I would have seen the default post-format in the results when nothing was selected.

I do understand, and the issue is that that term does not actually exist – and it messes up the whole logic. If you think about it, the search is instructed to exclude items that does not belong to any of the taxonomy terms, but again somehow include the ones that does not belong anywhere either. There is no way of having both within a single query. This feature is something I have to maintain (single query), as adding/removing results after the completed query (within the core) would be a complete disaster.

Are post-formats a taxonomy term? While I don’t completely understand the mechanics of this, I get the basic idea. I also agree with you that having a single query makes the most sense. We certainly don’t want to harm the search because of this one “hole” in the logic.

The bottom line is that your filter solution works. You managed to get this all done for me. We have added the ability to filter the default post-format, took care of the default post-format showing up in results when it shouldn’t, and this now all works simultaneously with the ability to use attachment results.

Bravo! =)

…..

Intermittent Bug With Number of Search Result Columns (Cards)

Unfortunately no, and there are no other reports of it either for some reason. May I ask which browser are you testing this on, and your screen resolution? Thank you!

Yieks. I hope it isn’t one of our scripts or custom configurations that is causing this. Unfortunately this issue is still here and very much annoying.

I use Chrome for testing and my screen resolution is 1920 x 1200.

I have seen this problem happening on my mobile devices too however, such as on my iPad Pro and my iPhone 7 Plus. I don’t believe this has anything to do with my browser or the resolution. I think it’s script related.

I’m open to any ideas you have to test this and narrow down the cause. I can also tell you that this issue is fairly new. It started happening a few weeks ago. So it’s likely the result of an update. There’s probably a bug or some conflict occurring.

…..

Featured Image Source Size Not Working in All Situations

Looking at the source, this is going to be an issue with the plugin code. I see it is missing the cache section. I will fix this for the upcoming release.

Please keep in mind that I am not using the image caching function. I don’t know if that is related to the problem.

This issue is affecting our attachment and tag result images. It’s not affecting our category result images because we’re using a filter for those to bring up specific images we made.

If this is an easy fix on your side it will be one less thing for us to worry about here.

…..

– Date Filter Bug
– Linking Attachment Search Results to Parent (Linked) Posts (Core Integration)
– Customizable Header in Settings Screen Date Filters

I’m just listing these here so we don’t forget about them… 😉

…..

You and I have made a lot of progress these past weeks with refinements and corrections. I continue to be so impressed by your commitment to this plugin and helping me in particular. All I can keep saying is thank you!

~ Michael

• This reply was modified 8 years, 2 months ago by Michael Samson.

[Michael Samson]

Hi Ernest,

Like you, I had some other priorities to take care of before replying here. I also wanted to set aside enough time to review everything.

Nice job on the new index table optimization. I updated the plugin and saw what you did. Considering that we will ultimately have an enormous database, we’re going to need optimizations such as this. I’m grateful that you’re refining the search like a true engineer! =)

…..

Filter to Use Parent (Linked) Page for Attachment Search Results Not Functioning with Index Table Method

Your fix worked. I have confirmed that the attachment parent links are now functioning again when using the index table for attachment searches. Thank you for this!

Intermittent Bug With Number of Search Result Columns (Cards)

I have noted this. I am not sure what might be the issue, but I will go through the pagination generator code before the next release, maybe I can find something that does not belong there.

I can now confirm this is a real issue, as it has persisted in both our production and development environments. The search result tiles (cards) just get smaller and smaller on repetitive searches. It’s a real annoying issue. Have you seen this yet on your end?

Featured Image Source Size Not Working in All Situations

This is a new issue I haven’t told you about until now. We’ve been working here on customizing all of our image sizes site-wide, generating specific sizes for specific locations. For ASP we are creating a 300x375px image.

I set the Featured Image Source Size to use this specific image, but it’s not working across the board. What I found is that it only seems to work for normal post search results. The attachment results are displaying the full image size, and tag results are strangely displaying the thumbnail (small) size. For tag results (we call them subcategories) we have created functionality for the search to display the image from the first post in the subcategory. This is working beautifully, but it is using the incorrect image size. The problem with the tag images may be on our end in our script, and I have brought it to the attention of our developers.

Attachment Search / Filter for Default Post-Format

NOTE: This solution will only work with the non-index table version! Before editing anything, make sure to have a back-up, just in case.

The best possible solution for this is that I add an option to the back-end, just like the one regarding the custom fields. That should resolve this once and for all.

It’s important that we can use the index table for our attachment results. I also don’t like the idea of modifying the plugin, as this is really a functional problem that should be solved within core.

I just want to make sure you understand the nature of this problem… (I think you do) 😉

If I turn off the “Show posts/CPM with empty (missing) taxonomy terms” feature than the attachment search will not function. However, if I turn on this feature it is causing the Default (Article) post-format to always appear in search results even when that format is excluded in the search settings.

Obviously I do not know the inner workings of the code, but what we need here is an integrated solution. These two features, using the attachment search, and filtering out the default post-format, should not be exclusive of one another. I’m confident that you will be able to find a proper solution to this problem.

Export/Import

While I designed this one with a lot of fallback, I only recommend using the export/import feature within the same site (and same plugin versions), as a sort-of archivation tool. It should be solid, as there are many consistency checks during the process. The export gets every single option of the search instance directly from the database, then encodes it. The import does the same, but inversed.

Possible point of failures can be for example mismatches in category IDs, missing taxonomy terms, custom fields between the export and import dates – but that is resolved during the search instance save process and on other actions as well, so the data consistency is checked on multiple occasions.

Thank you for this very detailed answer. I’m using the import/export feature to backup our instances. For syncing between the two environments (production and development) I am doing this work manually 99% of the time. This kind of thing isn’t limited to the search. I’m having to keep our two environments synced manually across the board, as I can’t simply export/import the entire database (for a mix of reasons). So I usually have the admins of both sites open side by side and make sure the settings match in both. I’ll avoid using the export/import for ASP unless absolutely necessary. But if I do need to use it, it’s good to know it should be dependable.

Common Words

When using the index table engine, then on the Advanced options panel, there is a common word filter. Basically it will not index any words entered into that textarea:

Please note, that making a change on the index table options panel will require generating a new index.

Thank you for pointing this feature out. I had actually looked at this feature earlier and didn’t connect it in my mind with this problem. I’ll have to carefully consider which words are entered here, since they would not be indexed at all. I think this will solve the problem though!

Date Filter Bug
Linking Attachment Search Results to Parent (Linked) Posts (Core Integration)
Customizable Header in Settings Screen Date Filters

I’m just listing these here so we don’t forget about them.

…..

Thanks as always Ernest!

~ Michael

• This reply was modified 8 years, 2 months ago by Michael Samson.

[Michael Samson]

Hi Ernest,

Did you see my post above from a few days ago? No rush, just want to make sure you saw it…

I have an important issue to bring to your attention:

Filter to Use Parent (Linked) Page for Attachment Search Results Not Functioning with Index Table Method

If you recall you had provided me with a filter that is able to detect when an attachment has a parent (linked) page, and to then use that URL for the attachment search result. This is a feature that you said you’re soon integrating into core.

I’ve been working here with my own development team to ensure that all of our attachments are properly linked to their parent post page in the media library. The idea being that the search will then see that linked post page and use it for the attachment link in search results.

During testing today I was having difficulty getting the filter to function. I was confused as to why it wasn’t working because I had tested it when you first provided it to me. I figured out that the filter does not function if the attachments are set to use the index table method. As soon as I turned it back to the normal search method the filter worked, and the attachments used the linked post page.

I wanted to bring this issue to your attention so that when you build this feature into core (you said it would be in the next release), that it will function when attachments are using the index table.

…..

Just a note that the date filter bug is driving me crazy here with testing. What is happening is that the search cannot see any posts from today’s date when the date filter is active. I have it set by default to display the last year of results. So anything posted on the current day is not being seen. This is troublesome especially when you’re testing stuff on the site and need it to come up in search. Anyway, I look forward to this one being fixed too. 😉

~ Michael

[Michael Samson]

Hi Ernest,

Attachment Search

I can’t thank you enough for finding the source of my attachment problem. I had a feeling that if I provided you with my configuration you’d be able to pinpoint the source of the problem much faster than me.

So this actually makes a lot of sense, because I had purposefully turned off the “Show posts/CPM with empty (missing) taxonomy terms” setting a few weeks ago. This is why my attachment search suddenly stopped functioning. I turned this setting back on and like magic the attachments are displaying again.

Here’s the issue…

The reason I had turned off this setting in the first place was because it was causing problems with post-format filters. The problem is that when this setting is turned on that the default post-format will display in search results even when it is not selected. So for example, if I select to only show image posts in the search results, the default posts (we call them articles) will also be displayed. A few weeks back I found that if I turned this above setting off it solved this problem. What I didn’t realize at the time was that this is what broke the attachment search.

So the question is, can you fix this problem with the default post-format in the filters?

Obviously we don’t want the default post-format to display in search results if it isn’t selected. I’m thinking this is something you just missed when first introducing this to the filters.

Btw, the attachment search results are working beautifully now, and there are no problems fetching the images from the S3 bucket (via the WP Offload S3 plugin).

…..

Attachments Not Displaying When Using the Index Table

I have turned on the index table option for attachments and can confirm that it is now working properly. Thanks for getting this one fixed!

…..

Feature to Disable the Isotopic Default Image

Today I decided to take advantage of the new feature you added to display the background color in the search result cards if no image is present. I really like this feature and it is working perfectly. We needed to turn this on because without it the result cards were only the height of the title. I want to maintain a perfect grid, so this was an elegant solution.

I did have to do a little custom css however as the title was displaying at the top of the card when no image was present. It was a simple css fix.

I wanted to mention to you that it seems the default behavior of the isotopic results is to only display the title. This is exactly what I want, as there’s not enough room to show more than the title. I had previously requested to add these options when no image is present:

Show Only the Title
Show the Title and Description

What I realized however is that the title is already the only thing being displayed. In our case we’re displaying the title over the background color when no image is present. So technically I don’t need these other options. If you did add more options, I would need something like “Show Title and Background Color.” But as I said, this is already what it’s doing.

…..

Date Filter Bug and Linking Attachment Search Results to Parent Posts

coming in next release

Sounds great! Thank you!

When you do integrate the attachment results functionality I’ll need to remove the filter that you originally provided.

…..

Customizable Header in Settings Screen Date Filters

I purposely left out this one out for now, as there is a header text for both inputs separately: https://i.imgur.com/wrtsLod.png
I think I will have to make a bigger change there, to somehow display these labels before the inputs, then introduce a 3rd label, that would be above them.

That makes sense. There just needs to be a main header above the entire container to match the other sections. I’ll end up making the sub-titles for the two date fields a little smaller, so they don’t compete with the top header.

…..

Intermittent Bug With Number of Search Result Columns (Cards)

I ran into this very strange issue today during testing. It’s not something I’ve ever seen before, so the problem may be on our end somehow. I wanted to bring it to your attention just in case.

What I was seeing was that the number of search result columns kept increasing every time I would perform a search. It would start off with the normal number of columns, and then on subsequent searches there would be one additional column. This kept repeating over and over again, and each time there would be more columns and the result cards would be smaller and smaller. As I said, I’ve never seen this before. The problem is intermittent and seemed to go away if I would refresh the page and start over.

…..

Quick Questions

1. I started using the export/import function today to transfer my configuration from the production environment to development. I’m using the export/import of settings feature (not the instance). I didn’t have any problems, but was wondering afterwards if it really worked perfectly. That’s my question… how perfect is the export/import of settings? Does it really get every single setting?

2. While doing a search today I noticed that using a common word like “the” could actually produce non-relevant results. A word like this can be used at the beginning of many search terms, when the real relevant word(s) come after the word “the.” So I was wondering if there was some kind of solution to this? Perhaps there’s a feature that already exists for this kind of tweak.

…..

My thanks as always for your continued help and support. You’ve honestly provided the best support I’ve ever received for a premium plugin. I’m very impressed by you, and grateful. I’m looking forward to showing you our platform when it finally goes live (in a few months still). 😉

~ Michael

• This reply was modified 8 years, 2 months ago by Michael Samson.

[Michael Samson]

Hi Ernest,

I’ve returned from my trip and finally had a chance to start working on this all again.

I updated ASP to the latest version and then did some testing to try and solve the issue with our attachment results not functioning.

I took your advice and created a new search instance to test with and had a breakthrough. I found that the attachment results were indeed functioning in the test instance, proving my problem is configuration related. The difficulty now is in finding the source of the problem in my configuration. It’s like looking for a needle in a haystack. I was wondering if perhaps you could help me to figure this out?

I’m attaching here a copy of my instance from an export…

I’m hoping that you can find the problem more quickly than I can. I have tried changing many settings, but so far no luck. You’ll see that with this configuration the attachment results do not function. There’s probably some kind of bug here, but I don’t know what setting it is related to.

…..

I also wanted to follow up with you on the items below…

I don’t expect you to be able to get to all of this immediately. It’s just easier to track things in a thread by copying them down this way.

…..

Attachments Not Displaying When Using the Index Table

I believe I know why. Someone else reported a similar problem with some debug information. There is some sort of issue with the table aliases whenever the attachments are selected. It appears to affect some database versions/configurations only, as it worked on my end for some reason, but it actually should not, as the table alias is indeed incorrect in the code. This will be corrected as well in the next release.

…..

Feature to Disable the Isotopic Default Image

I almost didn’t see you had added this ability. ?
This is working very nicely, and I only have one additional request. You’ve added the following options:

Show the default image
Show the description
Don’t show that result

Can you please add an option to show only the title. The reason I’m asking for this is that the search results usually only display the title. I’d prefer to keep it that way even on results without an image.

Sure, I have noted this as a feature request.

Thank you for adding this to the proposed features. I think you should add these options:

Show Only the Title
Show the Title and Description

…..

Date Filter Bug

I just found this issue today. I have the date filters enabled and set to always start with a date range of exactly one year ago through the current date. I’m using the Relative Date feature and setting the To Date to 0 years, 0 months, 0 days before current date. That should result in any search results on today’s date being included in the search results. But this is where I found the problem. Posts that I made today were not displaying in the results (even after updating the index table manually).

I did some testing and found this was a date related problem. If I set the search to display results through tomorrow’s date then suddenly the posts from today were displaying. So for some reason when setting the date range to display results through the current date, it is not including the current date in the results.

You are correct, my tests approve, that should be actually included. I have noted this issue as well.

…..

Linking Attachment Search Results to Parent Posts

It’s easier to just read everything we wrote on this above, instead of me copying it here. 😉

Thank you for the feedback, I will definitely add this to the core as an option. I will think about the fallback feature as well.

…..

Customizable Headers in Settings Screen for “Generic Selectors” and Date Filters

I know you took care of the title for the Generic Selectors. Thank you for that! It would also be helpful to have a customizable title for the Date Filters, so I could write something like “Filter by Date”…

Noted, I can make this one happen in the next release.

…..

That’s all for now!

~ Michael

• This reply was modified 8 years, 3 months ago by Michael Samson.
• This reply was modified 8 years, 3 months ago by Michael Samson.

[Michael Samson]

Hi Ernest,

I’m sorry I didn’t reply to you earlier this week. This has been a hectic week here and I’m leaving on a trip tomorrow to see family, and won’t be back until March 3rd. I will have a computer with me however and should be able to do some limited work (like testing).

I noticed that you released a new version of ASP last night. I’ll be installing that when I return next week. I’m hoping that the update contains some of the things we previously discussed. I haven’t read the change-log yet.

Attachments Not Displaying When Using the Index Table

Have you tried creating a completely new search instance and enabling only the attachments search?

I haven’t done this yet because I didn’t want to disrupt the site. But perhaps I can create a new instance to try on a single test page.

Also, are we talking about the live search results here, or the search results page after redirection?

Yes, we’re talking about the live ajax results. In fact, I decided to alter our configuration so that we now are only using the ajax results. We are no longer pointing the more results button to a separate results page. The main reasoning here was that we present our posts in a masonry grid format which isn’t the best for viewing search results. Additionally there were many integration related problems with displaying search results for things other than posts (like users, attachments, etc.). I realized that by limiting our results to ajax only that it eliminated all these technical problems and provided a far better user experience. Simply put, I love your ajax search, and now that is all we’re using for search!

I agree that we’re looking at some kind of configuration issue most likely. That would explain why the attachments were working and then suddenly stopped working. The only other explanation is that there is a conflict of some sort, perhaps with WP Offload S3.

While I’m away on my trip I’ll do some configuration related testing to see if I can get to the bottom of this. I’ll try your suggestion with creating a second search instance. You may not hear back from me until next week, but I will get back to you, not to worry.

Have a great week and talk soon…

~ Michael

[Michael Samson]

Hi Ernest,

I just wanted to check in with you to see if you had read my latest message above…

Actually, I found something small for you to fix in the plugin:

Take a look at this file:
ajax-search-pro/backend/settings/assets/upload.js

I’ve been working here normalizing line-endings in my Git repo (not a fun activity), and I found this anomalous file in your plugin. It is using the CR line-ending which is an outdated line-ending for Macs. This should be set to LF for linux.

I found this because Git’s function for normalizing line-endings doesn’t seem to recognize CF, but rather only CRLF and LF. So this caused me a little issue here and I manually fixed it on my end.

Anyway, I look forward to hearing back from you so we can get all my issues resolved. =)

~ Michael

[Michael Samson]

Hi Ernest,

It’s the end of my day but I wanted to respond to you quickly since I just ran a few tests…

Addition of Post-Format Filter for Default Format

C.

As long as you don’t see any major performance differences, I think it is completely fine to have more results. 50/100 as limits sounds great. I usually recommend keeping them under 100/200, and that is halfway there, it should be fine

Thank you for letting me know this. I don’t want to overwhelm the users with search results, but it was important to have a good number show up in ajax. That’s the main reason why I chose ASP in the first place! 😉

More results, Results counter

Yes, that is actually correct. I believe there is a minor difference whenever this is used for loading more results or redirection to the results page, I don’t actually recall exactly.

I’m using the more results button to redirect to a search results page, and am then overriding the default WP results. I hadn’t considered using this to load more ajax results, and will have to look at that option. We do have some challenges related to using search result pages, but those have nothing to do with ASP.

Your setup sounds really advanced, I really hope the plugin will work more-less as expected, as it has never been tested on such a huge platform.

Yes, our setup is extremely advanced and I’m very proud of what we’ve accomplished thus far. I actually chose your search originally because it was so well developed and thought out. I was looking specifically for an ajax search, and was very impressed by what I saw in Ajax Search Pro. Once I started working with the plugin I was blown away by its sophistication and flexibility. You really have put together an amazing search. Add to that your fantastic support and willingness to make improvements; it’s a win-win scenario!

One thing to remember about our platform is that it’s new. While we intend for it to grow quickly and eventually become huge, it is not there yet. It’s not even launched yet. Once we do launch I am looking forward to showing you the site in particular. I’ve personally done the integration work for ASP, and I think you’ll be impressed. We’ll have plenty of time to test further and make refinements as the site grows, so I’m not worried. This will ultimately be a great use-case for you! (and a way to show off what ASP is really capable of)

Attachments Not Displaying When Using the Index Table

Your suggestion to remove the posts from the search made sense, so I just tested it. We actually aren’t using pages as a source in our search at all (as our content is all post based).

Unfortunately this made no difference with trying to get attachment results. I did notice one thing. I seem to be able to get attachment results for my own avatar in test comments on some posts. But I can’t get any attachment results whatsoever for items in the media library. I still believe this is related to our use of WP Offload S3.

Let me ask you this, do the attachments have to be located in the /uploads folder in order to be returned in search results?

The reason I ask is that none of our actual media files are stored in /uploads. That’s why we’re using WP Offload S3. All of our media files are stored in an S3 bucket. You can still see the files in the media library, but all of their URLs point to S3. There are no actual files stored locally in /uploads. Could this be preventing the search from returning attachment results?

I’m having difficulty reconciling why this was functioning earlier and then suddenly stopped. Something obviously changed. When the attachments were working in search I could easily test it by taking the name of an attachment file and searching for it. The attachments would immediately come up that way. But this all stopped working for some reason.

I haven’t really changed many settings in ASP either. I have tested a few settings myself to see if they were the cause, but no luck.

This is a frustrating issue. I hope we can figure it out together!

~ Michael

[Michael Samson]

Hi Ernest,

I’ve copied below most of your responses (in italics) to make this easier for you to respond to. So this is in fact shorter than it looks!

Addition of Post-Format Filter for Default Format

A.

My mistake, it should be the name attribute, not the title:

Thank you for that correction. Changing it to name made it work!

C.

I guess you have the results limit set to 10, as it is by default. So the live results list displays 10 results at most. Whenever the ‘Video’ is added, the video post format posts should be displayed as well. That does not mean however, that those will be appended to the end of the previous results list (image and gallery), as this is a completely new search process with a different configuration – thus if there are more relevant video post format posts, those will be displayed instead, based on the ordering.

Posts with the video post format extend the potential results list within the search query. The database query will first check the restrictions, then organize the set of results, group them, then order them, then cut them off at the limit.

….

You were completely right about all of this! I hadn’t really paid much attention to the limits settings, and I should have. As soon as I increased them I saw completely different results during my testing. It makes complete sense now. Why didn’t I think about relevancy before either? That’s an important function in this search, and I wasn’t even thinking about it. Between the low limits and relevancy of the search, that explains why I was seeing confusing results.

As we’re building a large informational type site where search is extremely important, it made sense to set the limits much higher. I set all the ajax limits to 50 and the search result pages to 100. This made an enormous difference when I started re-testing, in a good way! I was able to easily see that the post-format filters are now working, in any combination I chose. =)

My only question here is, do you think it’s a bad idea to have higher limits in the ajax results for any reason? (keep in mind we are using the search results caching feature) I’m a fan of the ajax results (for convenience) and prefer to see higher numbers in those results rather than having to go to the actual search result pages.

More results, Results counter

It may actually differ, for multiple reasons. There is a limit set on this number, so when this is set to load more results it does not flood the live results list. Also, during the post-processing some results may be removed and may differ in the actual count. If the counter was to show the actual expected number of results after post-processing, it would have to post process all the results, then return the count, which may crash the server in many cases. (I have actually tested this on a somewhat bigger database, it was very accurate, but horrible performance/crashed the server) This number is the raw count of rows that matched the full query, before the post processing, adjusted by an approximation of the potentially removed results. The results page may also have a separete query adjusting the final results count, that unfortunately can’t be affected.

…..

I was actually incorrect in my original question, as I said this number never matched the number in the ajax results. This is incorrect because the number in the more results button reflects the number of results on the search results page (not what’s in the ajax results). I also now understand that the limits are set differently between the ajax results and results pages (which would always make this number different from what’s in the ajax results). Am I correct?

Your explanation about how this number is generated is interesting. There’s a lot more going on here behind the scenes than I realized. Thank you for sharing this with me, as I like to have a general understanding of how everything works.

The part of your comments that stood out to me was the part about not wanting to crash the server. 😉 We’re building a very large platform here (a social media site) that will ultimately contains millions of posts and possible search results of every kind. I can easily see how the power of this search could quickly overwhelm a server. This is something I want to take measures to prevent.

We’re building our site over at AWS (Amazon Web Services), and the database is using RDS with Amazon Aurora. It’s a database cluster with one DB for reading and one for writing. The good thing about this setup is that we can increase the resources of the DB at any time. I believe we have a robust and flexible infrastructure, but I still want to be careful not to over-stress things (like overwhelming the DB with search queries). If you have any advice or tips for me along these lines please let me know.

Image Caching Not Functioning in Conjunction with WP Offload S3 Plugin

“I found this setting under General Options -> Image Options -> Featured Image Size Source”
This actually works both ways, if the image caching is turned off, it will simply return the image defined, otherwise it will pass this image to the crop algorithm.

I think you should simply turn off the Crop images for caching? under the Cache settings submenu. If the offload cache plugin has a filter activated on getting the attachments function, it should automatically overwrite the image urls, and return them that way.

…..

That’s interesting that the image source size setting works with or without the image caching. I didn’t realize that. Pretty neat!

I had actually already turned off the image caching last time. Most of what I wrote about this was explaining the reasoning behind it. Frankly, the most important reason for us not to use this is so that our images are served directly from S3 (this is an important part of our disk space saving strategy).

You’re correct about the image URLs. They are being overwritten by the WP Offload S3 plugin and the search is indeed using those URLs. This is exactly what we need it to do.

I just tested the Featured Image Size Source setting, as I wanted to see if it would work with S3. I’m happy to say that it is working! Whatever image size I select is being loaded in search, and it is retrieving it properly from S3. I even found this setting was working with the filters we’re using for returning images for categories and tags (we wrote a custom script for this, it returns the image from the most recent post in the subcategory or tag archive).

So it looks like we’re in good shape on this one!

Linking Attachment Search Results to Parent Posts

Thank you for the feedback, I will definitely add this to the core as an option. I will think about the fallback feature as well.

…..

It’s my pleasure Ernest. You’ve been so amazing working with me on issue after issue. I really appreciate it, and am glad you’ve also found my feedback and suggestions useful.

I’ll watch for when you add this feature to core, and then I’ll just remove the custom filter from functions.php.

Attachments Not Displaying When Using the Index Table

I believe I know why. Someone else reported a similar problem with some debug information. There is some sort of issue with the table aliases whenever the attachments are selected. It appears to affect some database versions/configurations only, as it worked on my end for some reason, but it actually should not, as the table alias is indeed incorrect in the code. This will be corrected as well in the next release.

…..

Well, that is good news that you found the source of the original problem I reported. I definitely want to use the index table for returning attachment search results. It’s more efficient and should greatly decrease the number of DB queries.

Unfortunately I cannot get the attachment results working at all any more! I’m not sure if you realized this from my previous message. At this moment I cannot generate attachment search results whether using the normal search engine or the index table. I think the problem may be related to our use of WP Offload S3. Perhaps it is somehow preventing the search from being able to find the attachments.

If I go into our search right now and limit it to only display attachments (we call it media), there are no results at all. This used to be working for us, and suddenly stopped working. The only thing I can think of that changed recently is that we started to use WP Offload S3. This must be the source of the problem.

Can you help me to troubleshoot this? It’s pretty easy to test with the WP Offload S3 plugin. You just need an S3 bucket to test with, and they are free for small amounts of data. You could easily test this on your end and see if it’s the problem.

I definitely need to get the attachments working. I hope we can figure it out together.

…..

Thank you again for your continued help. You’ve seriously been amazing!

~ Michael

• This reply was modified 8 years, 3 months ago by Michael Samson.

[Michael Samson]

Hi Ernest,

Sorry for the delay in getting back to you on everything. It’s been a busy week here.

I’ve taken the time to test the solutions you’ve provided, and I have updates on every item…

Addition of Post-Format Filter for Default Format

A. I have tried using the filter you provided to customize the name of the “Standard” post-format, but unfortunately it had no effect. I changed where it said ‘Standard term’ to ‘Article’ and nothing happened. Did I use this filter incorrectly?

B. Thank you for correcting this issue. I recently found that due to that same problem if you re-saved the settings that it would end up removing the Standard post-format because it was no longer in the selections.

C. I spent a lot of time testing this issue, and made some progress, but I also uncovered new problems at the same time. The reason I was seeing Standard post-format posts mixed in no matter what post-format was selected was due to this feature being turned on in the Front-End Search Settings -> Advanced tab: “Show posts/CPM with empty (missing) taxonomy terms?” This was the cause of that problem, and when I turned it off I no longer saw the Standard posts appearing where they shouldn’t.

I have also turned off the settings for “Ignore checkbox filters that have nothing selected?” and “Allow results with missing custom fields, when using custom field selectors?” We’re not using any custom fields right now for filters.

The setting for “Category/Taxonomy terms logic” is set to “At least one selected terms should match.” The setting for “Logic between taxonomy groups” is set to “AND (default).”

I noticed during the testing that I was seeing inconsistent search results when selecting certain combinations of post-formats. To help narrow down the problem I made sure to only search for posts (and excluded all other search types). What I found was that if I selected two post-formats simultaneously, I seemed to get correct search results. But as soon as I would add a third post-format or more, I would get inconsistent results.

An example is I select post-formats for Image and Gallery, and I see 10 results which seem to be accurate. But then I go ahead and add a third post-format (lets say “Video”) and now some of the results from the previous search with two post-formats no longer display.

I also found that this was occurring when selecting all of the post-formats. I found that lots of search results would not be displayed that would display if I limited the post-formats to just one or two. There is definitely some kind of problem going on in general when selecting different combinations of post-formats. It is extremely inconsistent, and at this point I don’t think it’s my configuration.

Btw, in case you’re wondering, I am very easily able to see this kind of issue because we have lots of test posts and they all use titles identifying their post-format. We use titles like “Article Test Post” and “Video Test Post,” etc, to make it easy to identify things during testing. So I can quickly tell when the search is not being consistent with regards to post-formats.

Related Issue: See All Search Results
During my testing I also found that the “See All Search Results” button had an incorrect number at all times. It never reflected the same number of search results seen in the ajax results screen.

Content Type Filter

A. I think you misunderstood what I meant. What I don’t understand is that in this filter I had to select “Custom Post Types” in order to add normal posts to the filter. There was no option to simply have Posts. This is what I was asking about, as I had to use Custom Post Types to have Posts be a filter option.

B. That’s unfortunate. Right now I have this labeled as “Categories & Subcategories” which is on the longer side. It also looks funny listed directly under the much shorter filter options of Posts, Media, and Members. On our site we call tags Subcategories by the way. I wish there was a way to split this into two options.

Feature to Disable the Isotopic Default Image

Thank you for adding this to the proposed features. I think you should add these options:

Show Only the Title
Show the Title and Description (I just thought of this one)

Date Filter Bug

I’m really glad you were able to confirm this issue exists, and I look forward to seeing it corrected. Thanks!

Image Caching Not Functioning in Conjunction with WP Offload S3 Plugin

There have been some key decisions made here regarding this problem.

First, just to make things clearer, the images in the search results are currently loading directly from S3 using the rewritten URLs from the WP Offload S3 plugin. This is what ASP did by default in combination with WP Offload S3. And as reported last time, the image caching system stopped functioning. It was unable to cache the images, and the images in the search results were not coming from the cache.

We started thinking about the caching of these images and realized something important. Even if the image cache was working in combination with WP Offload S3, those images would then be stored in the /uploads/bfi_thumb/ folder. This completely bypasses the entire purpose of us using WP Offload S3, which is to reduce the disk space on our servers. We will ultimately have a huge number of posts for search, and the images would take up a lot of space (even at the smaller dimensions). It makes a lot more sense for us to keep the search related images in S3 just as we’re doing for all post related images.

So rather than fix this conflict we are deactivating the image caching in ASP, and are instead making sure that our site creates a 300x375px image specifically to be used in ASP search results. These images will be stored in S3 as part of our normal media upload process.

I found this setting under General Options -> Image Options -> Featured Image Size Source

It looks like this feature was designed to be used when the image caching is turned off. Am I correct? If that’s the case then we wouldn’t need a filter here at all, and rather just select the image size we want to use. Does that sound right? We will of course have to test this to ensure that it can successfully use the correct image size from S3 (right now it is having no problem using the full size image from S3).

Select All / Select None Options for Categories Filter

Thank you for pointing out where this option was. I’m not sure how I missed it! I have it all setup and it’s working perfectly.

Linking Attachment Search Results to Parent Posts

When I wrote this last time I actually forgot there was already a related option in the Attachments settings for what to do with attachment search results. The two existing options are “Link the results to attachment page” or “Link the results to attachment file directly.”

So I installed the filter provided that looks to see if the attachment (media) search results have any “attached content” in the media library, and then to use that URL for the search result. I’m happy to say that this filter worked perfectly! I tested it with a number of media files, and in every case it used the proper post URL in the search results.

Seeing how nicely this works I have an obvious suggestion for you:

Since you already have the two options mentioned above for attachments, why not add this filter as a built-in third option. It would be “Link the results to attached content in media library.” If you build this into your plugin’s core than I can remove it from functions.php.

On our end we are going to make sure that all of the media on our site is properly attached to it’s parent post. This will be automatic. In combination with this new feature in ASP, it means all of our media search results will link to the posts it originated from.

Lastly, you may also want to add a fall-back option just case the media is somehow not attached. I don’t really need this, but it’s not a bad idea.

Regarding what I said about a light-box option… I do not need this. It was always my intention to link the media directly to it’s parent post.

Customizable Headers in Settings Screen for “Generic Selectors” and Date Filters

Thanks for including this in the next release!

Attachments Not Displaying When Using the Index Table

Some bad news here. First I tested this option again, and enabled the index table for attachment search results. I tried a variety of configurations and nothing I did made this work. In all tests I could not get a single attachment search result to display.

Here’s where things got worse…

When I turned back the setting to make attachments use a standard search, suddenly this too was not working. Now I can’t get any attachment search results either way! Yieks!

I’ve been looking this over for 30 minutes now without success. I can’t get any attachment search results any more and I don’t know why. The only thing I can think of is that I cleared the cache and regenerated the index tables a few times. But I’m not sure why that would have caused this. There is also the fact that some of the posts do predate the installation of WP Offload S3. I wonder if the media results I was seeing were only from before the installation. Perhaps this is why it’s not working, as I cleared all the caches.

If that’s the case then this means the attachment search is not working with WP Offload S3. I hope that’s not what’s going on here. I’m concerned because attachment search results are very important to our platform and they must function.

I can provide you with a copy of WP Offload S3 if we have to for testing.

…..

My continued thanks for all your help Ernest. I hope we can get everything working perfectly. Search is really a central part of this platform we’re building.

Talk soon,

~ Michael

• This reply was modified 8 years, 4 months ago by Michael Samson.

[Michael Samson]

Hi Ernest,

I wanted to quickly acknowledge that I saw your response and even read through it already. I’m finishing up a project here today and will fully review everything you wrote and respond in detail early to mid next week.

Thank you as always for working with me to improve the plugin. I’ll be in touch in a few short days!

~ Michael

[Michael Samson]

Hi Ernest,

Thank you for letting me know that you saw everything. There’s no rush here at all, and I’d much rather you take your time carefully examining everything that I reported. I simply wanted to make sure you had seen my post.

While it’s true there are many items above, I did my very best to organize it clearly for you. With that said, I know it’s going to take some time to fully review everything.

I want you to know how much I appreciate your responsiveness and willingness to improve your plugin from this kind of user feedback. I’m very impressed both by you and Ajax Search Pro. My compliments!

Our platform is making great use of your plugin, and I’m really looking forward to showing you our implementation when we launch in a few more months!

Talk soon,

~ Michael

[Author]

Posts