Release v2.14.1 #7986
Merged
Release v2.14.1 #7986
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![cvat-bot[bot]](https://avatars.githubusercontent.com/in/406724?s=60&v=4){kind=small}
Update develop after v2.14.0
Add the serviceName field to the kvrocks StatefulSet as per the Kubernetes specification. This change ensures that the service name is correctly associated with the StatefulSet pods, allowing for proper DNS resolution and service discovery within the cluster. Fixes #7741 ### Motivation and context The Helm installation is currently failing as reported in issue #7741 ### How has this been tested? ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - [x] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/cvat-ai/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/cvat-ai/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/cvat-ai/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/cvat-ai/cvat/tree/develop/cvat-ui#versioning)) ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Resolved the issue of a missing `serviceName` field in `kvrocks`, ensuring proper configuration and improved stability. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
…ated after invitation to some org (#7906) <!-- Raise an issue to propose your change (https://github.com/cvat-ai/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://docs.cvat.ai/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - [ ] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/cvat-ai/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/cvat-ai/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/cvat-ai/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/cvat-ai/cvat/tree/develop/cvat-ui#versioning)) ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Improved email creation process to ensure the use of the normalized email from the user object, enhancing data consistency and reducing errors. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Raise an issue to propose your change (https://github.com/cvat-ai/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://docs.cvat.ai/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> This PR addresses several problems: - when requesting a dataset download, it's possible to get the 500 error with the message "The result file does not exist in export cache", which isn't expected for this request - when downloading the dataset the same error can be obtained if the file is requested right before the cache expiration - there are several [TOCTOU](https://en.wikipedia.org/wiki/Time-of-check_to_time-of-use) bugs related to dataset cache file existence checks - under some conditions, it's possible that the export job is never started - the finished RQ jobs were removed automatically on result reporting (after the client requested the result). This made it hard to debug problems for admins, as the jobs were often removed already This PR fixes the problems by the following: - introduced dataset cache file locking (via redis) during reading, writing, and removal - the 500 error is changed to automatic reexporting attempt on export status request - the 500 error is changed to 404 when the file is not available for downloading - the exported files are now have different names for each instance update time - the lifetime of the exported files is now automatically prolonged on each export request for the file (given the export is still valid) - the deferred export jobs are now checked to have ghost dependencies. If so, the requested job is restarted - added several environment variables for configuration - <s>finished RQ export jobs are not removed automatically on result retrieval. Now, they just use the export cache lifetime instead (should be continued in another PR)</s> ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [ ] I submit my changes into the `develop` branch - [ ] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/cvat-ai/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/cvat-ai/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/cvat-ai/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/cvat-ai/cvat/tree/develop/cvat-ui#versioning)) ### License - [ ] I submit _my code changes_ under the same [MIT License]( https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Improved reliability of file handling during export and cleanup processes. - Introduced new functionality for managing export cache locks and directories. - **Bug Fixes** - Addressed race conditions in concurrent export and cleanup operations. - **Dependencies** - Updated multiple packages to their latest versions for enhanced security and performance: - `cryptography` to `42.0.7` - `django` to `4.2.13` - `django-health-check` to `3.18.2` - `freezegun` to `1.5.1` - `jinja2` to `3.1.4` - `limits` to `3.12.0` - `lxml` to `5.2.2` - `orjson` to `3.10.3` - Added `pottery` version `3.0.0` - Updated `tqdm` to `4.66.4` <!-- end of auto-generated comment: release notes by coderabbit.ai -->
`test_two_project_webhooks_intersection` is supposed to trigger each webhook once. However, the first one of these webhooks actually gets triggered twice, because creating a task causes the project's `updated_date` to be bumped, which triggers an `update:project` event. The test still passes a lot of the time (I guess because the second delivery doesn't appear immediately?), but sometimes it fails. It's very easy to make it fail consistently, though - just add a `sleep(5)` before the `get_deliveries` calls. Fix this by changing the first webhook's second event to something that will not be triggered.
The recent changes enhance the dataset import functionality across various dataset formats in the CVAT application by integrating specific importers from the Datumaro library. The updates streamline the detection of datasets, improve error handling, and introduce new tests to ensure robustness against incorrect file structures during import operations.
Fix memory consumption when exporting to azure blob storage
<!-- Raise an issue to propose your change (https://github.com/cvat-ai/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://docs.cvat.ai/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> @coderabbitai summary ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - [x] I have created a changelog fragment <!-- see top comment in CHANGELOG.md --> - [ ] I have updated the documentation accordingly - [x] I have added tests to cover my changes (*partially*) - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/cvat-ai/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/cvat-ai/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/cvat-ai/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/cvat-ai/cvat/tree/develop/cvat-ui#versioning)) ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Improved media download performance with parallel downloading. - Enhanced file handling with the new `NamedBytesIO` class. - Added support for specifying stop frames in task manifest generation. - Enhanced `DatasetImagesReader` to handle generator sources. - **Performance Improvements** - Optimized image download methods to use threading for faster processing. - **Configuration** - Introduced new settings for maximum threads and files per thread in cloud data downloading. These updates enhance the flexibility, performance, and configurability of media handling and downloading in the application. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Parsing JSON payloads to get `working_time` in general leads to low performance in Clickhouse requests. This patch will not fix it right now, but with this patch, after a period of time we may switch to new quick approach to calculate working time. - There will not be a lot of `send:working_time` events, we may store this scope of events for a longer time (e.g. 5 years instead of one by default). - Finally storing working time in such events like `click:element` or `send:exception`, or `debug:info` seems not logical. - Also, the history showed, that as result in different bugs, these events may sometime lose information about `job_id`, `task_id`, etc. Resolved #7884
cvat-bot
bot
requested review from
azhavoro,
zhiltsov-max,
bsekachev,
mdacoca,
SpecLad,
Marishka17 and
nmanovic
as code owners
|
Important Review skippedBot user detected. To trigger a single review, invoke the You can disable this status message by setting the Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
|
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #7986 +/- ##
==========================================
+ Coverage 83.35% 83.47% +0.11%
==========================================
Files 380 384 +4
Lines 40034 40320 +286
Branches 3769 3772 +3
==========================================
+ Hits 33371 33656 +285
- Misses 6663 6664 +1
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added
DatasetNotFounderror message #7923)Changed
Upgraded React and Antd dependencies, it leads to stylistic changes in the user interface (Upgrade React and Antd till the latest version #7466)
CVAT now stores users' working time in events of a dedicated type (Using dedicated event to store working time #7958)
Fixed
The 500 / "The result file does not exist in export cache" error on dataset export request (Fix dataset downloading #7864)
Fix missing serviceName field in kvrocks (issue Helm upgrade fails due to missing serviceName field in kvrocks deployment #7741) (Fix missing serviceName field in kvrocks (issue #7741) #7924)
UI crash on hovering conflict related to hidden objects (Fixed conflicts highlight crash in case of hidden by
zOrderobjects #7917)Login when the domain of a user email contains capital symbols and a user was created after being invited to an org (Fix login when email domain contains capital symbols and user was created after invitation to some org #7906)
Exception "Cannot set properties of undefined (setting 'serverID')" occurs when attempting to save a job after removing the first keyframe of a track (Cannot set properties of undefined (setting 'serverID') #7949)
Spent working time for a user may not be counted in analytics (Fixed working time lost in click:element events #7942)
A classifier model can not be used on annotation view (unknown object shape error) (Fix automatic
tagannotation support #7839)Optimized memory usage by not keeping all downloaded images/part of images in memory while creating a manifest file (Fixed several issues related to creating tasks with cloud data #7969)
Optimized the number of requests to CS providers by downloading only images from a specified range (
use_cache==False) (Fixed several issues related to creating tasks with cloud data #7969)Task creation with random sorting and cloud data (Fixed several issues related to creating tasks with cloud data #7969)