Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(BA-502): Add the skeleton interface of vfolder CRUD handlers in storage-proxy (#3434) #3516

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

MintCat98
Copy link

@MintCat98 MintCat98 commented Jan 22, 2025

resolves #3434 (BA-502)

Add skeletion interface of storage-proxy handlers for refactoring.

Files were added in ./src/ai/backend/storage/api/vfolder

[25-01-31] Updated with applying the feedback
manager_handler.py (updated)
manager_service.py (deleted)
types.py (updated)
response_model.py (added)
conftest.py (deleted)
test_handler.py (updated)

Checklist: (if applicable)

  • Milestone metadata specifying the target backport version
  • Mention to the original issue
  • Installer updates including:
    • Fixtures for db schema changes
    • New mandatory config options
  • Update of end-to-end CLI integration tests in ai.backend.test
  • API server-client counterparts (e.g., manager API -> client SDK)
  • Test case(s) to:
    • Demonstrate the difference of before/after
    • Demonstrate the flow of abstract/conceptual models with a concrete implementation
  • Documentation
    • Contents in the docs directory
    • docstrings in public interfaces and type annotations

@github-actions github-actions bot added comp:storage-proxy Related to Storage proxy component size:XL 500~ LoC labels Jan 22, 2025
src/ai/backend/storage/api/vfolder/manager_handler.py Outdated Show resolved Hide resolved
src/ai/backend/storage/api/vfolder/types.py Outdated Show resolved Hide resolved
tests/storage-proxy/vfolder/conftest.py Outdated Show resolved Hide resolved
src/ai/backend/storage/api/vfolder/manager_handler.py Outdated Show resolved Hide resolved
@HyeockJinKim
Copy link
Collaborator

Please check the reason for the CI failure as well.

@MintCat98
Copy link
Author

Please check the reason for the CI failure as well.

I made some changes before committing, but I found that I only ran pants check :: and forgot to run pants test. I'll fix the test errors along with other modifications and ping you again.

class BaseModel(PydanticBaseModel):
"""Base model for all models in this module"""

model_config = {"arbitrary_types_allowed": True}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
model_config = {"arbitrary_types_allowed": True}
model_config = ConfigDict(arbitrary_types_allowed = True)
  1. How about using pydantic.ConfigDict?
  2. Why is the config needed for all models in this module?
  3. Let's rename this class to _BaseModel to represent that it is private if it is used only in this types module


# Common fields for VolumeID and VFolderID
VOLUME_ID_FIELD = Field(
...,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are those ellipsis used in pydantic field definition?

Comment on lines +80 to +98
# For `get_vfolder_info`: mount
subpath: Optional[PurePosixPath] = Field(
default=None,
description="For `get_vfolder_info`\n\
The subpath inside the virtual folder to be queried.",
)
# For `clone_vfolder`
# You can use volume_id and vfolder_id as src_volume and src_vfolder_id.
dst_vfolder_id: Optional[VFolderID] = Field(
default=None,
validation_alias=AliasChoices(
"dst_vfid",
"dstvfolderid",
"dst_vfolder_id",
"dstVfolderId",
),
description="For `clone_vfolder`\n\
The destination virtual folder ID to clone to.",
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the models in this module are used to serialize and validate input of API handlers.
This VFolderIDModel seems to be used in too many cases.
Let's divide this into separate models.

@fregataa fregataa requested a review from HyeockJinKim February 7, 2025 04:56
@@ -0,0 +1,3 @@
python_test_utils()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp:storage-proxy Related to Storage proxy component size:XL 500~ LoC
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add the skeleton interface of VFolder CRUD APIs using the new layered architecture in storage-proxy
3 participants