View Issue Details

IDProjectCategoryView StatusLast Update
5025Composrcore_database_driverspublic2022-11-23 18:38
ReporterAdam Edington Assigned ToChris Graham  
PrioritynormalSeveritytrivial 
Status resolvedResolutionfixed 
Product Version10.0.43 
Fixed in Version10.0.44 
Summary5025: Searching title/meta tags in a Gallery category fails to populate results
DescriptionEven following the links from the meta tags which match the title yields nothing.
TagsNo tags attached.
Attach Tags
Attached Files
Time estimation (hours)
Sponsorship open

Sponsor

Date Added Member Amount Sponsored

Activities

Adam Edington

2022-11-14 16:41

administrator   ~7644

Last edited: 2022-11-15 00:31

To clarify further, it's the title of the gallery that doesn't return a result, despite suggesting the title as I type it. Adding a meta tag matching gallery title has the same effect, "Your search yielded no results. Please try a less specific search. Other category meta tags are returning results so not sure if this is a bug or just localhost weirdness.

Chris Graham

2022-11-21 08:26

administrator   ~7679

Is this any meta tag, or just specific ones? If specific ones can you give an example?

Adam Edington

2022-11-22 18:13

administrator   ~7701

The title of the gallery category is Your Last Wish (autocompletes when searching but doesn't return anything)
Meta tags on the category such as Canada, Female Fronted Metal, Melodic Death Metal return results
Adding Your Last Wish as a meta tag weirdly also fails to return anything (but I'd prefer a return on the title than adding an additional tag for the band name anyway).

Other category titles are returning results in search, so maybe this is stop words or something?

Chris Graham

2022-11-23 16:55

administrator   ~7716

Last edited: 2022-11-23 17:09

I can reproduce. Actually YOUR, LAST, and WISH, are all stopwords...

https://dev.mysql.com/doc/refman/8.0/en/fulltext-stopwords.html

admin

2022-11-23 17:18

administrator   ~7717

Automated response: Workaround MySQL aggressive fulltext stoplist

MySQL's fulltext stoplist includes the word "wish" and possibly other reasonably distinctive words, which can break some perfectly reasonable queries working.

We already have support for switching out a stopword search to a simple string comparison. Extend that to detect a compound-stopword search. Also allow it to do a prefix-locked LIKE query if the search is at least 8 characters; which will be indexable and reasonably distinct, so not too much of a performance penalty.

admin

2022-11-23 17:18

administrator   ~7718

Fixed in git commit 86351f4c0e (https://gitlab.com/composr-foundation/composr/commit/86351f4c0e - link will become active once code pushed to GitLab)

A hotfix (a TAR of files to upload) has been uploaded to this issue. These files are made to the latest intra-version state (i.e. may roll in earlier fixes too if made to the same files) - so only upload files newer than what you have already. If there are files in a hot-fix that you don't have then they probably relate to addons that you don't have installed and should be skipped. Always take backups of files you are replacing or keep a copy of the manual installer for your version, and only apply fixes you need. These hotfixes are not necessarily reliable or well supported. Not sure how to extract TAR files to your Windows computer? Try 7-zip (http://www.7-zip.org/).

Chris Graham

2022-11-23 17:20

administrator   ~7719

Oh yes and the autocomplete just represents common past searches, not what searches will work.

Adam Edington

2022-11-23 17:26

administrator   ~7720

Quite a few of those stopwords exist in band names, 'Yours' Truly, Devil 'Within', 'Be Under' Arms, etc. Is there a way to turn them off or are they serving a useful purpose?

Adam Edington

2022-11-23 17:42

administrator   ~7721

Actually I've got a German band called Isn't I need to add. If I change the 8 characters to 4 would that be acceptable? Or is that too low for other things?

Chris Graham

2022-11-23 17:53

administrator   ~7722

I believe MySQL code has to be edited then it has to be recompiled on the server, then the fulltext indexes in Composr would need deleting and re-adding. i.e. long and complex even for me.

Likely MySQL found those words by statistical analysis of common words in text. The idea is that indexing wastes space and searching wastes time and relevancy suffers unless common words are first taken out of the equation.

Chris Graham

2022-11-23 17:56

administrator   ~7723

There's 2 reasons you would not want to do that:
1) Performance, as now any short search matching stopwords becomes a LIKE query which is more intense. It's a possible DOS-attack vector. This is all based on my perceptions, every site will have different performance based on the nature and quantity of content, and server resources,
2) You presumably would only want "Isn't" to match (which will work with my change), but not "Isn't It Wonderful" or whatever. The character limit of 8 was also intended as a tradeoff to resolve this problem.

Adam Edington

2022-11-23 18:12

administrator   ~7725

Last edited: 2022-11-23 18:13

Okay, wasn't sure if Isn't was covered in the change being 4 chars because I wrongly assumed 8 was the minimum. Thanks for the fix :)

Issue History

Date Modified Username Field Change
2022-11-13 16:34 Adam Edington New Issue
2022-11-14 16:41 Adam Edington Note Added: 0007644
2022-11-15 00:31 Adam Edington Note Edited: 0007644
2022-11-21 08:26 Chris Graham Note Added: 0007679
2022-11-22 18:13 Adam Edington File Added: Screenshot 2022-11-22 at 18-06-57 Search results.png
2022-11-22 18:13 Adam Edington Note Added: 0007701
2022-11-23 16:55 Chris Graham Note Added: 0007716
2022-11-23 17:09 Chris Graham Note Edited: 0007716
2022-11-23 17:20 Chris Graham Note Added: 0007719
2022-11-23 17:26 Adam Edington Note Added: 0007720
2022-11-23 17:42 Adam Edington Note Added: 0007721
2022-11-23 17:53 Chris Graham Note Added: 0007722
2022-11-23 17:56 Chris Graham Note Added: 0007723
2022-11-23 18:12 Adam Edington Note Added: 0007725
2022-11-23 18:13 Adam Edington Note Edited: 0007725
2022-11-23 18:38 Adam Edington Summary Searching meta tags in a Gallery category fails to populate results => Searching title/meta tags in a Gallery category fails to populate results
2022-11-23 18:38 Adam Edington Description Updated