Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove runtime patterns after run #4429

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

ElenaKhaustova
Copy link
Contributor

Description

Solves #4340

Development notes

Added a method to remove runtime patters after the run so they live only within the run and do not affect other runs.

This solution originally was suggested in #4236 but was replaced with another one which solved the original problem but not catalog mutation.

Developer Certificate of Origin

We need all contributions to comply with the Developer Certificate of Origin (DCO). All commits must be signed off by including a Signed-off-by line in the commit message. See our wiki for guidance.

If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.

Checklist

  • Read the contributing guidelines
  • Signed off each commit with a Developer Certificate of Origin (DCO)
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added a description of this change in the RELEASE.md file
  • Added tests to cover my changes
  • Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team

Copy link
Member

@merelcht merelcht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This solution originally was suggested in #4236 but was replaced with another one which solved the original problem but not catalog mutation.

Why was this solution replaced in the first PR? I couldn't quite figure it out from the thread there.

kedro/io/catalog_config_resolver.py Show resolved Hide resolved
tests/io/test_kedro_data_catalog.py Show resolved Hide resolved
@ElenaKhaustova
Copy link
Contributor Author

This solution originally was suggested in #4236 but was replaced with another one which solved the original problem but not catalog mutation.

Why was this solution replaced in the first PR? I couldn't quite figure it out from the thread there.

There were actually two bugs and both of them were related to the way we process MemoryDataset's and how we define free outputs. The first reported by the user could be resolved by removing patterns but the second could not, so another solution was implemented to handle them both. But mutation remained as we didn't go with the patterns removing solution.

Here is the comment about the implemented solution #4236 (comment)

The previous solution also covers the current bug, but we still think mutation looks confusing and should not happen, so in this PR, we remove patterns added at runtime.

Copy link
Member

@merelcht merelcht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants