Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DIY-OOD uninstall performance issue with deleting NameObject files. #192

Open
lizhihongTest opened this issue Apr 11, 2023 · 4 comments
Open
Assignees
Labels
bug Something isn't working Performance about performance issues Tools CYFS related tools

Comments

@lizhihongTest
Copy link
Collaborator

Describe the bug
I run uninst.exe to uninstall my DIY-OOD.It took 30 minutes and the uninstallation is still not complete

To Reproduce

  • I stored 500,000 objects via put_object.
  • Excute C:\cyfs\uninst.exe

System information
System :windows10
DIY-OOD : CYFSOOD-x86-64-1.1.1.30-beta.exe

Additional context

@lizhihongTest lizhihongTest added the bug Something isn't working label Apr 11, 2023
@lurenpluto
Copy link
Member

During the uninstall process of DIY OOD, in most cases, the local data (objects and chunks) will be retained, as many of these are critical user data. Only some binary files and cache files will be unloaded. Is your issue related to the data being cleared during the unloading process as well?

Currently, the noc's object storage uses blob mode. In the case of having a large number of objects, I'm not sure if this step is causing the slow uninstall process. However, even so, it shouldn't take half an hour to finish.

@lurenpluto lurenpluto added the Performance about performance issues label Apr 11, 2023
@lizhihongTest
Copy link
Collaborator Author

I confirm that the current uninstallation program does delete local data (objects and chunks), and it traverses all subdirectories under the C:\cyfs\data\named-object-cache\objects directory to delete files. However, it only deletes one file at a time, so the uninstallation speed is very slow.This could be due to the uninstallation program not deleting the folders and instead using a recursive algorithm to delete one file at a time.

@lurenpluto
Copy link
Member

Currently, object blobs are stored in a two-level directory structure with single files, under the named-object-cache\objects directory. When the total number of objects is large, this directory will have a large number of first and second-level sub directories, so some traversal operations on this directory can be slow.

A few suggestions when uninstalling:

  • If the uninstall operation does not need to delete data (including objects and chunks), there is no need for additional processing of the objects directory during deletion, and it can be omitted.
  • If the uninstall operation does indeed need to delete data, it is better to handle the objects directory by deleting the top-level directory directly, rather than attempting to traverse these operations.

@lurenpluto lurenpluto added the Tools CYFS related tools label Apr 11, 2023
@lurenpluto lurenpluto moved this to To Discussion in CYFS Tools Apr 12, 2023
@weiqiushi
Copy link
Member

We used NSIS's built-in RMDir function to delete the CYFS directory, and I'm afraid there's very little we can do if it's the implementation of this function that's faulty

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Performance about performance issues Tools CYFS related tools
Projects
Status: To Discussion
Development

No branches or pull requests

4 participants
@lurenpluto @weiqiushi @lizhihongTest and others