Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle multiple index names in strapi config #916

Merged
merged 26 commits into from
May 16, 2024
Merged

Conversation

L-Weisz
Copy link
Contributor

@L-Weisz L-Weisz commented Apr 4, 2024

Pull Request

Related issue

Fixes #651

What does this PR do?

This Pull Request introduces the capability to associate a single collection with multiple index names within Strapi's configuration. Previously, the indexName parameter was limited to a single string value, specifying one index per collection. Now, indexName can be defined as an array, enabling a collection to be indexed under multiple names simultaneously.

For example, the configuration can now be specified as:

Before: indexName: "my_restaurant"
After: indexName: "my_restaurant" (after review, found a way to avoid the breaking change)
After: indexName: ["my_restaurant"]
After: indexName: ["my_restaurant", "all_restaurants"]
Additionally, this update includes the introduction of new tests to ensure that the expanded functionality does not disrupt existing features. An existing test was also corrected.

Important Changes:

Limitation: While indexName can now accept multiple values, all other configuration attributes remain uniform across the specified indexes. This means that any update to one index will apply identically to all indexes associated with a given collection, reflecting the same settings and data.

This enhancement provides greater flexibility in managing index names for collections within Strapi, streamlining the process of data organization and retrieval across multiple indexes.

Result

image
image

PR checklist

Please check if your PR fulfills the following requirements:

  • Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
  • Have you read the contributing guidelines?
  • Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!

L-Weisz and others added 20 commits April 3, 2024 10:50
we have to use restaurant as content type to really testthis function
this will allow to link a content type to multiples indexes

BREAKING CHANGE: in strapi config indexName will have to be an array
… array index name

BREAKING CHANGE: indexName needs to be an array in strapi config
to handle multiple indexes

we will have one row per index name
to handle multiples index names
…exNames

Feat/index name/handle multiple index names
feat(configuration-validation): test that indexName is an array
@brunoocasali brunoocasali added the breaking-change The related changes are breaking for the users label Apr 4, 2024
Copy link
Member

@brunoocasali brunoocasali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @L-Weisz!

What a high-quality PR! Thanks a lot for your effort here!

I would like to discuss some of the decisions I made from the diff. Let me know what you think about them.

Apart from them, I want to know if you considered a way to avoid the breaking change?

@@ -113,7 +113,7 @@ describe('Test plugin configuration', () => {
expect(strapiMock.log.error).toHaveBeenCalledTimes(0)
})

test('Test indexName with empty string', async () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch!

server/configuration-validation.js Outdated Show resolved Hide resolved
server/services/meilisearch/config.js Outdated Show resolved Hide resolved
server/services/meilisearch/connector.js Outdated Show resolved Hide resolved
contentType,
entryId: entry.id,
const indexUids = config.getIndexNamesOfContentType({ contentType })
await Promise.all(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use allSettled here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And in the other places where we make calls for all the indexes...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It depends on how we want to handle the rejection of the promises generated by
contentTypeConfig.transformEntry.

Use Promise.all if: You want the entire operation to fail as soon as one of the transformEntry calls fails. This is useful when it's critical that all entries must be successfully transformed; if any single transformation fails, the whole batch should be considered failed, and you might want to handle that error specifically, perhaps by logging or retrying.

Use Promise.allSettled if: You want to wait for all promises to settle, regardless of whether they are fulfilled or rejected. This approach is useful when you want to attempt to transform all entries and then deal with them individually, depending on whether each promise was fulfilled or rejected. It allows you to handle each entry's transformation success or failure case separately.

which one do you think we should use ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we had a way to retry the failed entries would be cool I guess. But we can do it later :)

server/services/meilisearch/connector.js Outdated Show resolved Hide resolved
@L-Weisz
Copy link
Contributor Author

L-Weisz commented Apr 5, 2024

Hi @brunoocasali thanks a lot for your feedback! Really appreciated your insights

I’ve pushed some updates to reflect your suggestions.

On the topic of the breaking change, I found the issue was with how the validator treated the indexName. I made sure it now accepts indexName as a string, avoiding any breaking changes ! 🎉 (As the function that retrieves indexNames already transforms a simple string into an array)
Also, I’ve checked it with the playground connected to my MeiliSearch instance, the old indexNames worked well

@L-Weisz
Copy link
Contributor Author

L-Weisz commented Apr 19, 2024

Hello @brunoocasali were you able to check the changes ?

@curquiza
Copy link
Member

Hello here @brunoocasali is still on Holidays and check when he'll come back

Copy link
Member

@brunoocasali brunoocasali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing work here @L-Weisz ❤️

bors merge

Thank you for your contribution!

Copy link
Contributor

meili-bors bot commented May 16, 2024

@L-Weisz L-Weisz closed this May 16, 2024
@meili-bors meili-bors bot merged commit ffcf43c into meilisearch:main May 16, 2024
6 checks passed
@L-Weisz
Copy link
Contributor Author

L-Weisz commented May 16, 2024

Thanks a lot for your feedback @brunoocasali ! so happy to contribute
Could the tag breaking change be removed since it's not a breaking change anymore, it could be confusing at first sight

@brunoocasali brunoocasali removed the breaking-change The related changes are breaking for the users label May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Possibility to use a content-type in multiple indexes
3 participants