We are building a site that uses products and other data with naming using Swedish/Scandinavian characters like åäö but we have an international audience that can not easily type these characters so we require the search functionality to have an understanding of the data so the users can search for “sodermalm” and get results using “södermalm”. This is not currently working for us with a default search configuration.
What options are available for us to support this? Lucene has filters for “Scandinavian Folding”: https://lucene.apache.org/core/7_5_0/analyzers-common/org/apache/lucene/analysis/miscellaneous/ScandinavianFoldingFilter.html
Can we utilize this in Litium in some way?
Based on the documentation I can see that we can specify an Analyzer when creating the search queries but I guess we also need to control the analyzer used when indexing to get matching results? Is this possible?
Should we hook into the indexing process using a IIndexingProviderPreProcessor or something similar to change the results? Or is there a better alternative that I’m not seeing?
You are in the right direction about how you can fine-tune Lucene.NET with better control about how it should behave. Please see Lucene stemming where you have example how to change analyzers for index and searching.
Note: Litium uses Lucene.Net 3.0.3 that will match Java version of Lucene 3.0.3 and your reference above is for Java Lucene 7.5.0 so it can be some differences in existing functionality.
Great to hear. We have implemented a custom IIndexingProviderPreProcessor that adds a new field for the products which uses our own Analyzer with the ASCIIFoldingFilter from Lucene. This gives us the desired searching behavior for our use case.
It would be great if the documentation at https://docs.litium.com/documentation/architecture/search/analyzers could be expanded a bit with the information you provided about Analyzers and IIndexingProviderPreProcessor. It’s nice to know that it’s possible to extend the default functionality and see which options are available.