Skip to content

feat: Cache Labels for Taxonomy #7383

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: develop
Choose a base branch
from

Conversation

sbatchelder
Copy link

See issue #7382

This fix allows for the value(s) of Taxonomy annotations to be displayed by the experimental Cache Labels feature.
Prior to this PR, Cache Labels would create an all-blank column when applied to a Control Tag for a Taxonomy annotation.
With this PR, values selected in a Taxonomy annotation are correctly displayed.

The issue arose from Taxonomy annotation values being returned as a list, as opposed to a str. For Taxonomy type annotations, a list is required for each selected element so as to capture the hierarchical path taken for any given value in a taxonomy tree. In this PR, consideration was taken so as to only show the selected value, and not capture parts of the path leading up to the selected value. One or more selected values, as well as leaf and intermediate/branch values are supported but this fix.

image

image

This is a very simple, very small, low risk PR. It is also my first PR with Label Studio, yay!
My thanks to @makseq who pointed me towards the cache_labels.py file lines most likely responsible for the blank-taxonomy-cache-labels bug.

Copy link

netlify bot commented Apr 16, 2025

👷 Deploy request for heartex-docs pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit d7c6168

Copy link

netlify bot commented Apr 16, 2025

👷 Deploy request for label-studio-docs-new-theme pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit d7c6168

Copy link

netlify bot commented Apr 16, 2025

Deploy Preview for label-studio-storybook canceled.

Name Link
🔨 Latest commit d7c6168
🔍 Latest deploy log https://app.netlify.com/sites/label-studio-storybook/deploys/681d515f868e620008c11bc8

@sbatchelder sbatchelder changed the title [fix] for Taxonomy Cache Labels fix: for Taxonomy Cache Labels Apr 16, 2025
@sbatchelder
Copy link
Author

In discussions with Max, it was determined that displaying just the leaf node of a selected taxonomy was not always desirable, as important information about taxonomic hierarchy paths can be lost.

I've made an update such that cache_labels will match the formatting schema of the given Taxonomy tag's labeling interface showFullPath and pathSeparator attributes. This was achieved by instantiating a label_studio_sdk.label_interface.LabelInterface object with config of the given project. label_studio/data_manager/actions/cache_labels.extract_labels(...) now optionally accepts this label_interface object.

Now <Taxonomy name="Type" toName="image" showFullPath="true" pathSeparator="."> leads to:
image

Defaults are respected, so <Taxonomy name="Type" toName="image" showFullPath="true"> becomes:
image

And all defaults, ie a basic <Taxonomy name="Type" toName="image"> becomes:
image

Defaults are hardcoded and come from Label Interface Taxonomy Params .
If default taxonomy parameters for showFullPath and pathSeparator are every updated they will need to be manually updated here as well, as I am unaware of where default values for control tags are stored.

If the taxonomy control tag is a Custom control tag, naturally it will not have any Labeling Interface configuration to specify its formatting. In this case, showFullPath='true' and pathSeparator='/' are used. If a Labeling Interface object is not provided to extract_labels(...) (or is None) these defaults are also used. Although showFullPath='true' differs from the default for Label Interface Taxonomy Params, I think here it made more sense to err on the side of displaying/transcribing more information (the full path) to the cache_label than not.

@makseq makseq requested a review from Copilot April 17, 2025 18:15
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

label_studio/data_manager/actions/cache_labels.py:73

  • [nitpick] Consider renaming 'showFullPath' to 'show_full_path' and 'pathSeparator' to 'path_separator' for consistency with Python naming conventions.
showFullPath = 'true'

@makseq makseq changed the title fix: for Taxonomy Cache Labels feat: Cache Labels for Taxonomy Apr 17, 2025
@github-actions github-actions bot added the feat label Apr 17, 2025
@sbatchelder
Copy link
Author

new commit: I've pre-loaded the label interface tags prior to the tasks and annotations loops. The project's label interface control tags are easily accessible via dict now, should be orders of magnitude more efficient.
Also corrected the "delim-" typo :)

@sbatchelder
Copy link
Author

Hey @makseq ,

I tried following your steps above to setup the situation you described above, but I find myself unable to remove Choices name="a" using the UI, after having submitted a couple tasks containing the name=a Choices. I get a Created annotations are incompatible with provided labeling schema, we found: 2 with from_name=a, to_name=image, type=choices error when attempting to remove the name=a Choices field; the new config does not save. Note that I am using an Image data object instead of Text, because that's what I have on-hand for my project but this I'd image shouldn't affect this step.

image

So unfortunately I am unable to test the situation you describe.

That said, I think my implementation still works as intended and I am not convinced the label interface config needs to be rechecked on-the-fly. If an annotation is not present in the label_interface_tags = {tag.name:tag for tag in label_interface.find_tags('control')} dict but is present in the annotation results, then currently my code treats the value like a Custom Control Tag and it would get included in a cache_all column with default formatting (showFullPath='true' and pathSeparator='/').

Have you tested my code against the situation you describe and found it to fail? If yes, I'd love a snapshot of the issue because I can't reproduce your scenario.
I am open to stepping through the code with you in a live call to test or discuss if you like.

Also, I'm not sure how the situation you describe above relates to rechecking the label interface config on-the-fly.
Is that still a concern you hold?

@makseq
Copy link
Member

makseq commented Apr 30, 2025

@sbatchelder yes, this situation is not very often and it's a bit tricky to reproduce it. Maybe we can ignore it.

I am going to run pytests for it tomorrow. If they pass, we can merge it.

@makseq
Copy link
Member

makseq commented May 8, 2025

Could you please merge the latest develop to your branch? It should fix pytests

@makseq
Copy link
Member

makseq commented May 8, 2025

Also, you need to fix Linter. To do this, run:

pip install pre-commit
make fmt-all

@makseq makseq enabled auto-merge (squash) May 13, 2025 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants