Multiple DELETE requests with long response times

[vlowe85]

The search logs show that the plugin is making multiple DELETE requestd for the same articles ID, seems on a 5 min interval.

These sometimes take 7 seconds to respond

Description
[2022-12-13T13:05:54Z INFO actix_web::middleware::logger] 172.19.16.209 "DELETE /indexes/article/documents/article-686 HTTP/1.1" 202 130 "-" "node-fetch/1.0 (+https://github.com/bitinn/node-fetch)" 7.700711

article-686 has not been deleted in strapi or updated, but it is not published so not sure how it was indexed in first place

Expected behavior
Dont repeatedly send DELETE calls

Environment (please complete the following information):

• OS: getmeili/meilisearch:v0.29
• Meilisearch version: 0.29.3
• strapi-plugin-meilisearch version: 0.8.2
• Strapi version: 4.5.3

[bidoubiwa]

Hey @vlowe85
Thanks a lot for raising this potential bug. When are these deletes triggered?

[vlowe85]

Seems every 5 mins

[vlowe85]

I have no idea what is so special about these 6 articles, they have not been modified recently but they all seem to be drafts

Correction: The item is updated via the strapi API every 5 mins, but the data is unchanged and the publication state is unchanged.

More info on this:
Ther eis a script running that will poll a few different API's like Vimeo and Podbean and get the last 10 items, If the item does not exist in strapi it is created, if it exists it is updated. The script never updates the publication state but it will update the title, or image etc if it has changed on the external API side to keep strapi in sync.

[bidoubiwa]

Hey @vlowe85, I'll come back to you tomorrow, I'm off for today. Thanks a lot for the additional context

[vlowe85]

No problem, so every 5 mins it will be updating probably about 50 strapi articles with a PUT API call.

I guess it thinks these 6 are in meilisearch and need to be deleted. But if i search for any of these articles in meilisearch directly they don't seem to exist.

[bidoubiwa]

Hey @vlowe85,

My bad for the late reply, it flew under the radar. So after investigating a bit, this is what's happening.

• You update 50 articles.
• All of them triggers the afterUpdate of afterUpdateMany hook.
• A validator functions ensures these are supposed to be present in Meilisearch. For example if one of the entries is a draft, it should not be in meilisearch.
• If the entry should not be in Meilisearch, but might already be present, we delete it.

This is because we are not aware what the entries value were before the update, thus we cannot guarantee for example that the change is not publish -> unpublished.
We have two choices here:

1. We fetch the entry in Meilisearch, and if it is present we delete it
2. We delete it in any case. Since the deletion is an asynchronous task, if the entry does not exist in Meilisearch it will not throw an error.

The advantages of 1 is that we do not spam the tasks list with failed document deletion. The disadvantage is that we are now possibly doing two requests to delete one entry: get document + delete document.

Now, in any case, we can batch the document deletion in case you are updating multiple entries in one request. Which actually is way better than the current implementation.

[bidoubiwa]

See #632