ci: Limit unit-test duration #534
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: oliver könig <[email protected]>
sarahyurick
reviewed
Feb 10, 2025
Comment on lines
+43
to
+44
| # Unit tests shouldn't take longer than 30minutes | ||
| timeout-minutes: 30 |
There was a problem hiding this comment.
This time is only for running the PyTests and does not include the time to build the container?
There was a problem hiding this comment.
Yes, for container build this doesn't impose a constraint. As long as we're not experiencing hangs during the build, I wouldn't suggest constraining it
sarahyurick
approved these changes
Feb 10, 2025
philm001
pushed a commit
to philm001/NeMo-Curator
that referenced
this pull request
Feb 10, 2025
Signed-off-by: oliver könig <[email protected]> Signed-off-by: Phillip Mobley <[email protected]>
philm001
pushed a commit
to philm001/NeMo-Curator
that referenced
this pull request
Feb 10, 2025
Signed-off-by: oliver könig <[email protected]> Signed-off-by: Phillip Mobley <[email protected]>
ryantwolf
added a commit
that referenced
this pull request
Feb 11, 2025
* ci: Pin twine in release workflow (#512) * ci: Pin twine in release workflow Signed-off-by: oliver könig <[email protected]> * maybe fix? Signed-off-by: oliver könig <[email protected]> * fix Signed-off-by: oliver könig <[email protected]> --------- Signed-off-by: oliver könig <[email protected]> Signed-off-by: Phillip Mobley <[email protected]> * ci: Version bump to 0.7.0rc1.dev0 (#513) Signed-off-by: oliver könig <[email protected]> Co-authored-by: oliver könig <[email protected]> Signed-off-by: Phillip Mobley <[email protected]> * Enforce Dataframe Backend Checks (#514) * Add module and to backend Signed-off-by: Ryan Wolf <[email protected]> * Add backend tests Signed-off-by: Ryan Wolf <[email protected]> * Fix tests Signed-off-by: Ryan Wolf <[email protected]> * Add switch backend tests Signed-off-by: Ryan Wolf <[email protected]> * Update modules to use module interface Signed-off-by: Ryan Wolf <[email protected]> * Directly invoke module init Signed-off-by: Ryan Wolf <[email protected]> * Fix call method Signed-off-by: Ryan Wolf <[email protected]> * Fix shuffle call method Signed-off-by: Ryan Wolf <[email protected]> * Add docs and more tests Signed-off-by: Ryan Wolf <[email protected]> * Fix list formatting in docs Signed-off-by: Ryan Wolf <[email protected]> * Address Sarah and Praateek's reviews Signed-off-by: Ryan Wolf <[email protected]> * Fix modifier get_backend to backend Signed-off-by: Ryan Wolf <[email protected]> * Address Ayush's review Signed-off-by: Ryan Wolf <[email protected]> --------- Signed-off-by: Ryan Wolf <[email protected]> Signed-off-by: Phillip Mobley <[email protected]> * Updated documentation to include packaging requirements Signed-off-by: Phillip Mobley <[email protected]> * Fixed formatting issues. Signed-off-by: Phillip Mobley <[email protected]> Signed-off-by: Phillip Mobley <[email protected]> * Enable ADD ID to work with CPU/GPU both (#479) * Enable ADD ID to work with CPU/GPU both Signed-off-by: Vibhu Jawa <[email protected]> * Make Test runable in a CPU only environment Signed-off-by: Vibhu Jawa <[email protected]> * Fix pytest skipping behavior in CPU/GPU environment Signed-off-by: Vibhu Jawa <[email protected]> * Raise error instead of skipping test Signed-off-by: Vibhu Jawa <[email protected]> --------- Signed-off-by: Vibhu Jawa <[email protected]> Signed-off-by: Phillip Mobley <[email protected]> * Add Pooling Strategy Option for embedding creation (#491) * Add pooling stratedgy Signed-off-by: Vibhu Jawa <[email protected]> * Ensure pytest is importable in a CPU only environment Signed-off-by: Vibhu Jawa <[email protected]> * Fix last token based on Avinash's feedback Signed-off-by: Vibhu Jawa <[email protected]> * Fix indexing issues Signed-off-by: Vibhu Jawa <[email protected]> * Merge in main Signed-off-by: Vibhu Jawa <[email protected]> * Fix Doc-string Signed-off-by: Vibhu Jawa <[email protected]> * Address Sarah's reviews Signed-off-by: Vibhu Jawa <[email protected]> --------- Signed-off-by: Vibhu Jawa <[email protected]> Signed-off-by: Phillip Mobley <[email protected]> * Add Partition On Logic (#519) * add partition_on logic Signed-off-by: Vibhu Jawa <[email protected]> * Add Docstring based on Sarah's review Signed-off-by: Vibhu Jawa <[email protected]> * Apply Praateek's suggestion and skip test with using pytest.mark.gpu Signed-off-by: Vibhu Jawa <[email protected]> * Apply Praateek's suggestion and force index=False Signed-off-by: Vibhu Jawa <[email protected]> --------- Signed-off-by: Vibhu Jawa <[email protected]> Signed-off-by: Phillip Mobley <[email protected]> * Add improved cleaning methods from Nemotron-CC (#517) * Add improved cleaning features Signed-off-by: Ryan Wolf <[email protected]> * Fix cleaning tests Signed-off-by: Ryan Wolf <[email protected]> * Update documentation and CLI scripts Signed-off-by: Ryan Wolf <[email protected]> * Address Sarah and Lawrence's reviews Signed-off-by: Ryan Wolf <[email protected]> --------- Signed-off-by: Ryan Wolf <[email protected]> Signed-off-by: Phillip Mobley <[email protected]> * Update model nomenclature (#497) * Update model nomenclature Signed-off-by: Sarah Yurick <[email protected]> * minor notebook grammar Signed-off-by: Sarah Yurick <[email protected]> * add lawrence's suggestion Signed-off-by: Sarah Yurick <[email protected]> --------- Signed-off-by: Sarah Yurick <[email protected]> Signed-off-by: Phillip Mobley <[email protected]> * small add_id backend fix (#525) Signed-off-by: Vibhu Jawa <[email protected]> Signed-off-by: Phillip Mobley <[email protected]> * benchmark readme updates (#508) * benchmark readme updates Signed-off-by: Lawrence Lane <[email protected]> * benchmark image update Signed-off-by: Lawrence Lane <[email protected]> * benchmark text update Signed-off-by: Lawrence Lane <[email protected]> --------- Signed-off-by: Lawrence Lane <[email protected]> Signed-off-by: Phillip Mobley <[email protected]> * Removal logic for fuzzy / exact (no class abstraction) (#509) Signed-off-by: Phillip Mobley <[email protected]> * ci: Limit unit-test duration (#534) Signed-off-by: oliver könig <[email protected]> Signed-off-by: Phillip Mobley <[email protected]> * Enforce Dataframe Backend Checks (#514) * Add module and to backend Signed-off-by: Ryan Wolf <[email protected]> * Add backend tests Signed-off-by: Ryan Wolf <[email protected]> * Fix tests Signed-off-by: Ryan Wolf <[email protected]> * Add switch backend tests Signed-off-by: Ryan Wolf <[email protected]> * Update modules to use module interface Signed-off-by: Ryan Wolf <[email protected]> * Directly invoke module init Signed-off-by: Ryan Wolf <[email protected]> * Fix call method Signed-off-by: Ryan Wolf <[email protected]> * Fix shuffle call method Signed-off-by: Ryan Wolf <[email protected]> * Add docs and more tests Signed-off-by: Ryan Wolf <[email protected]> * Fix list formatting in docs Signed-off-by: Ryan Wolf <[email protected]> * Address Sarah and Praateek's reviews Signed-off-by: Ryan Wolf <[email protected]> * Fix modifier get_backend to backend Signed-off-by: Ryan Wolf <[email protected]> * Address Ayush's review Signed-off-by: Ryan Wolf <[email protected]> --------- Signed-off-by: Ryan Wolf <[email protected]> * small add_id backend fix (#525) Signed-off-by: Vibhu Jawa <[email protected]> Signed-off-by: Phillip Mobley <[email protected]> * Removal logic for fuzzy / exact (no class abstraction) (#509) Signed-off-by: Phillip Mobley <[email protected]> --------- Signed-off-by: oliver könig <[email protected]> Signed-off-by: Phillip Mobley <[email protected]> Signed-off-by: Ryan Wolf <[email protected]> Signed-off-by: Vibhu Jawa <[email protected]> Signed-off-by: Sarah Yurick <[email protected]> Signed-off-by: Lawrence Lane <[email protected]> Co-authored-by: oliver könig <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Ryan Wolf <[email protected]> Co-authored-by: Vibhu Jawa <[email protected]> Co-authored-by: Sarah Yurick <[email protected]> Co-authored-by: L.B. <[email protected]> Co-authored-by: Praateek Mahajan <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Usage
# Add snippet demonstrating usageChecklist