Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 catalog-api: Fixes validation errors on /services entrypoint #7268

Merged

Conversation

pcrespov
Copy link
Member

@pcrespov pcrespov commented Feb 24, 2025

What do these changes do?

This PR fixes a validation issue caused by invalid values in the database.

A request to the catalog service was failing due to an invalid value in the database:

curl http://master_catalog:8000/v0/services/simcore%2Fservices%2Fdynamic%2Fjupyter-math/3.0.3?user_id=530
{"errors":["1 validation errors:\n  {'type': 'url_parsing', 'loc': ('response', 'icon'), 'msg': 'Input should be a valid URL, input is empty', 'input': '', 'ctx': {'error': 'input is empty'}}\n"]}

The issue occurred because the icon column in the second row was set to an empty string instead of either a valid URL or NULL.

image

This could have been introduced manually in the database or as a result of an unintended transformation between models in the system.

To address this, we implemented the following changes:

  • Database Migration: Replaces existing empty strings with NULL in nullable string columns.
  • Validation Mechanism: Prevents writing empty strings to the database at the model level.

Since these values should not be modified in the database directly, we consider additional constraints at the database level unnecessary, as the existing code-level checks are sufficient.

Related issue/s

  • Issue reported by @GitHK in master deploy

How to test

Dev-ops

None

@pcrespov pcrespov self-assigned this Feb 24, 2025
@pcrespov pcrespov added a:database associated to postgres service and postgres-database package a:catalog catalog service labels Feb 24, 2025
Copy link

codecov bot commented Feb 24, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.22%. Comparing base (29816b1) to head (52fc53f).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7268      +/-   ##
==========================================
+ Coverage   86.97%   88.22%   +1.24%     
==========================================
  Files        1686     1454     -232     
  Lines       65340    57900    -7440     
  Branches     1115      616     -499     
==========================================
- Hits        56832    51084    -5748     
+ Misses       8190     6644    -1546     
+ Partials      318      172     -146     
Flag Coverage Δ
integrationtests 65.47% <ø> (+0.04%) ⬆️
unittests 87.15% <100.00%> (+1.15%) ⬆️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library ∅ <ø> (∅)
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library 91.56% <100.00%> (ø)
pkg_notifications_library 84.57% <ø> (ø)
pkg_postgres_database 88.28% <ø> (ø)
pkg_service_integration 70.03% <ø> (ø)
pkg_service_library ∅ <ø> (∅)
pkg_settings_library ∅ <ø> (∅)
pkg_simcore_sdk 85.08% <ø> (ø)
agent 96.46% <ø> (ø)
api_server 90.56% <ø> (ø)
autoscaling 96.08% <ø> (ø)
catalog 91.73% <100.00%> (+0.01%) ⬆️
clusters_keeper 99.24% <ø> (ø)
dask_sidecar 91.25% <ø> (ø)
datcore_adapter 93.19% <ø> (ø)
director 76.59% <ø> (-0.19%) ⬇️
director_v2 91.30% <ø> (ø)
dynamic_scheduler 97.33% <ø> (ø)
dynamic_sidecar 89.74% <ø> (ø)
efs_guardian 90.25% <ø> (ø)
invitations 93.28% <ø> (ø)
osparc_gateway_server ∅ <ø> (∅)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.08% <ø> (-0.06%) ⬇️
storage 86.65% <ø> (-0.06%) ⬇️
webclient ∅ <ø> (∅)
webserver 84.68% <ø> (ø)

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 29816b1...52fc53f. Read the comment docs.

Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏 thanks a lot!

@odeimaiz
Copy link
Member

Does this mean that an empty string is not a valid value?

@pcrespov pcrespov force-pushed the fix/catalog-validation-errors branch from 68121ed to 114204e Compare February 24, 2025 13:25
@pcrespov pcrespov marked this pull request as ready for review February 24, 2025 13:26
@pcrespov
Copy link
Member Author

pcrespov commented Feb 24, 2025

Does this mean that an empty string is not a valid value?

@odeimaiz in the database is not an expected value when the column is nullable. At least for the columns we are speaking about in this PR

  • the front-end should not be affected by this constraint because there is a mechanism in place for these cases that will replace empty strings by null values.

@pcrespov pcrespov enabled auto-merge (squash) February 24, 2025 13:31
@pcrespov pcrespov disabled auto-merge February 24, 2025 14:10
@pcrespov pcrespov added this to the The Awakening milestone Feb 24, 2025
@pcrespov pcrespov merged commit e3fdf00 into ITISFoundation:master Feb 24, 2025
92 of 95 checks passed
@pcrespov pcrespov deleted the fix/catalog-validation-errors branch February 24, 2025 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:catalog catalog service a:database associated to postgres service and postgres-database package
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants