Elasticsearch fails with production database

[Robin.w]

Hi, I have set up Elasticsearch and it works fine with the accelerator. But if I switch to our production database I get his error when trying to view a category. The index seems to work just fine when i pay with kibana.

This is the debug output query:

{ "query":{ "bool":{ "filter":[ { "term":{ "channelSystemId":{ "value":"baa772ef-f705-4765-82ad-686560a35e6a", "boost":1.0 } } }, { "term":{ "assortments":{ "value":"15d7b6d8-fd0b-4e5c-5ef9-08d6e1f51a46", "boost":1.0 } } }, { "term":{ "organizations":{ "value":"00000000-0000-0000-0000-000000000000", "boost":1.0 } } }, { "term":{ "parentCategories":{ "value":"55207ca2-be87-4ab7-a587-541c2ae35445", "boost":1.0 } } }, { "bool":{ "should":[ { "term":{ "permissions":{ "value":"Group:5b521f8c-a7a2-44e0-ab76-1c60d740bded", "boost":1.0 } } }, { "term":{ "permissions":{ "value":"Group:3bb00cc4-a86f-4d5d-b43a-aed35d4ecab5", "boost":1.0 } } }, { "term":{ "permissions":{ "value":"Function/Websites/Content", "boost":1.0 } } }, { "term":{ "permissions":{ "value":"Function/Media/Content", "boost":1.0 } } }, { "term":{ "permissions":{ "value":"Function/Products/Content", "boost":1.0 } } } ], "adjust_pure_negative":true, "boost":1.0 } } ], "adjust_pure_negative":true, "boost":1.0 } }, "sort":[ { "mostSold.quantity":{ "order":"desc", "missing":7.922816251426434E28, "nested":{ "path":"mostSold", "filter":{ "term":{ "mostSold.systemId":{ "value":"51888d35-529f-4aa3-8e1b-d0d813d89daf", "boost":1.0 } } } } } }, { "name.keyword":{ "order":"asc" } }, { "articleNumber":{ "order":"asc" } } ] }

And this is the error:

"stacktrace": ["org.elasticsearch.transport.RemoteTransportException: [bca31517332e][172.18.0.2:9300][indices:data/read/search[phase/query]]",

"Caused by: org.elasticsearch.search.aggregations.AggregationExecutionException: [nested] nested path [tags] is not nested",

"stacktrace": ["org.elasticsearch.search.aggregations.AggregationExecutionException: [nested] nested path [tags] is not nested",

{"type": "server", "timestamp": "2020-04-22T18:09:40,077Z", "level": "WARN", "component": "r.suppressed", "cluster.name": "docker-cluster", "node.name": "bca31517332e", "message": "path: /litiumeducationproductdocument.sv-se/_search, params: {typed_keys=true, index=litiumeducationproductdocument.sv-se}", "cluster.uuid": "cbGNjVEGRfy-m_WD1K0KRA", "node.id": "Op7yCWW8RUKexpk_Hc5Luw" ,

Any suggestions?

Litium version: 7.4.2

[patric.forsgard]

Check that you have the settings and index mapping on the index. An easy way to see that is to look at the index mapping and if mostly all properties are indexed as text with an multi field as keyword, then you know that the field was not defined when the index was created.

To correct this you need to stop Litium and then remove the index in Elasticsearch again. When Litium now try to reach the index it is created correctly again with the correct settings and index mapping.

[Robin.w]

We are getting closer The query workes now, but we get hundreds of prices on every product, is this single result normal? Its seams massive for one hit

https://pastebin.com/g5hDtzCc

[patric.forsgard]

Elasticsearch is as default returning the content for the document that is indexed except if the _source parameter is included to minimize the data that is returned.

To set the fields that should be returned you can use the fluent way with nest as below and only specify the columns that we using and want back.

                     .Source(s => s
                        .Includes(i => i
                            .Fields(
                                f => f.IsBaseProduct,
                                f => f.BaseProductSystemId,
                                f => f.VariantSystemIds
                            )
                        )
                     )

From ProductSeaarchServiceDecorator

[Robin.w]

Yes, but what are all this prices? Why does a variant document have 210 prices connected to it?

[patric.forsgard]

That’s depend on how many prices you have for your variant and how many countries you have in your system.

If we take this part

          "prices" : [
            {
              "systemId" : "3e252dcf-957f-471d-adad-bf824c4701a6",
              "countrySystemId" : "cb7048a5-0afa-48be-8995-533865c452ca",
              "isCampaignPrice" : false,
              "price" : 529.0
            },

the systemId is the price list system id and then we can see that we have the country and the price also. If we take all records that have the same price list system id and converting them into a table you will get the following result

systemId                             countrySystemId                      isCampaignPrice   price
--------                             ---------------                      ---------------   -----
3e252dcf-957f-471d-adad-bf824c4701a6 cb7048a5-0afa-48be-8995-533865c452ca           False     529
3e252dcf-957f-471d-adad-bf824c4701a6 cc0aba23-2948-4b03-b82d-3435918277cc           False     529
3e252dcf-957f-471d-adad-bf824c4701a6 2a927839-c78b-43d7-a00e-078b3a67d045           False 524,768
3e252dcf-957f-471d-adad-bf824c4701a6 a71bb310-d9ac-4d68-a744-3c0e65930005           False 524,768
3e252dcf-957f-471d-adad-bf824c4701a6 eb991f56-4a70-4d35-8f9c-50cf85c1b44d           False 524,768
3e252dcf-957f-471d-adad-bf824c4701a6 a1678b89-0903-4880-9891-5ea4dcf31b6c           False 524,768
3e252dcf-957f-471d-adad-bf824c4701a6 e83f7f2d-ff6d-44ce-9da8-85064f039d96           False 524,768
3e252dcf-957f-471d-adad-bf824c4701a6 2a2e959f-e1d1-4e79-962b-85541eee3f81           False 524,768
3e252dcf-957f-471d-adad-bf824c4701a6 9e9e51f8-192f-42d8-875d-aa82dbae9ec6           False 524,768
3e252dcf-957f-471d-adad-bf824c4701a6 5101ef09-589a-4561-8f66-af98f92c3bf0           False 524,768
3e252dcf-957f-471d-adad-bf824c4701a6 04aa69ae-1538-4272-9993-d3952a10d391           False 524,768
3e252dcf-957f-471d-adad-bf824c4701a6 31418f63-5108-440d-a07a-daea8d5296fa           False 524,768
3e252dcf-957f-471d-adad-bf824c4701a6 c6c883f8-c679-423a-9af0-8d3837ea0b88           False     529

We can from this see that we have 13 countries for each price in the price list. You can also see that the same price list will result in different prices depend on country and the VAT

The reason that all prices is stored inside the index is that different users at different times have access to different price lists. When we filtering on the prices we also pointing out the price list and the country that should be used to get correct result back.