Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1848 add permission rights to async jobs #7262

Open
wants to merge 81 commits into
base: master
Choose a base branch
from

Conversation

bisgaard-itis
Copy link
Contributor

@bisgaard-itis bisgaard-itis commented Feb 21, 2025

What do these changes do?

  • Add permission check to export data endpoint in storage before submitting job to worker
  • Change AsyncJobId from uuid.UUID to strin order to support "embedding" metadata into task name

Related issue/s

How to test

Dev-ops checklist

@bisgaard-itis bisgaard-itis self-assigned this Feb 21, 2025
@bisgaard-itis bisgaard-itis added the a:storage issue related to storage service label Feb 21, 2025
@bisgaard-itis bisgaard-itis requested a review from GitHK February 21, 2025 16:04
@bisgaard-itis bisgaard-itis added this to the The Unknown Elefant milestone Feb 21, 2025
Copy link

codecov bot commented Feb 21, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.91%. Comparing base (b084481) to head (089d2bf).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7262      +/-   ##
==========================================
+ Coverage   86.97%   87.91%   +0.94%     
==========================================
  Files        1686     1176     -510     
  Lines       65340    50684   -14656     
  Branches     1115      253     -862     
==========================================
- Hits        56829    44560   -12269     
+ Misses       8193     6041    -2152     
+ Partials      318       83     -235     
Flag Coverage Δ
integrationtests 72.41% <ø> (+6.91%) ⬆️
unittests 86.68% <100.00%> (+0.40%) ⬆️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library ∅ <ø> (∅)
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library ∅ <ø> (∅)
pkg_notifications_library ∅ <ø> (∅)
pkg_postgres_database ∅ <ø> (∅)
pkg_service_integration 70.03% <ø> (ø)
pkg_service_library ∅ <ø> (∅)
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 85.08% <ø> (ø)
agent 96.46% <ø> (ø)
api_server 90.56% <ø> (ø)
autoscaling 96.08% <ø> (ø)
catalog 91.71% <ø> (ø)
clusters_keeper 99.24% <ø> (ø)
dask_sidecar 91.25% <ø> (ø)
datcore_adapter 93.19% <ø> (ø)
director 76.59% <ø> (-0.10%) ⬇️
director_v2 91.29% <ø> (-0.02%) ⬇️
dynamic_scheduler 97.33% <ø> (ø)
dynamic_sidecar 89.74% <ø> (ø)
efs_guardian 90.25% <ø> (ø)
invitations 93.28% <ø> (ø)
osparc_gateway_server ∅ <ø> (∅)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.08% <ø> (+0.10%) ⬆️
storage 86.88% <100.00%> (+0.23%) ⬆️
webclient ∅ <ø> (∅)
webserver 84.70% <ø> (+0.01%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b084481...089d2bf. Read the comment docs.

Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx!


from models_library.users import UserID
from pydantic import BaseModel, model_validator
from typing_extensions import Self

from ..progress_bar import ProgressReport

AsyncJobId: TypeAlias = UUID
AsyncJobId: TypeAlias = str
Copy link
Member

@pcrespov pcrespov Feb 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIP: I recommend you using IDStr. It is a constraint string designed for string-based identifiers. In term of strictness it is between str (weak) and a UUID (strong)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not gonna work. I will unify the identifier with a more properly typed when I do the final integration


from models_library.users import UserID
from pydantic import BaseModel, model_validator
from typing_extensions import Self

from ..progress_bar import ProgressReport

AsyncJobId: TypeAlias = UUID
AsyncJobId: TypeAlias = str
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not gonna work. I will unify the identifier with a more properly typed when I do the final integration

Comment on lines +35 to +36
for _id in data_export_start.file_and_folder_ids:
_ = await dsm.get_file(user_id=data_export_start.user_id, file_id=_id)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: could you please check if this works with a file within a folder? since the user can select their virtualenv folder (for example) to be exported. I don't recall if the permissions check will work.

Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking good! thanks

),
)
assert isinstance(result, AsyncJobGet)


async def test_start_data_export_fail(
rpc_client: RabbitMQRPCClient, user_id: UserID, faker: Faker
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have extensively refactored also the testing. I can show you how to do the testing @GitHK is asking for.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:storage issue related to storage service
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants