Update for D4Science dev release #7
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Remove tasks from Elasticsearch when workflow is removed (#1505) * Add removeTask method for IndexDAO Implement removeTask and asyncRemoveTask methods to the IndexDAO interface. Implemented in ElasticSearchDAOV6, ElasticSearchRestDAOV6, and NoopIndexDAO. * Implement removal of tasks for removeWorkflow - Update ExecutionDAOFacade#removeWorkflow with two new parameters: removeTasks and archiveTasks - Added ExecutionDAOFacade#removeTaskIndex - Added methods updateTask and asyncUpdateWorkflow to IndexDAO - Added methods updateTask and asyncUpdateWorkflow to ElasticSearchDAOV6 and ElasticSearchRestDAOV6 * Add tests for removing workflow through /workflow/remove - Introduced tests for WorkflowServiceTest - Introduced tests for WorkflowResourceTest - Introduced test for ExecutionDAOFacadeTest - Introduced test for QueueResilienceSpec.groovy * Update Javadoc to include task archival when workflow archived and removeTasks true * Implement fix for when tasks are not removed (#1505) - Now tasks are search and associated with workflows through a search query directly embedded rather than constructed - If a task can not be removed it will error and return - Monitors are updated automatically - Updated TestElasticSearchRestDAOV6 and TestElasticSearchDAOV6 * Removed rawJSON field from task archival from index DAO (#1505) * Removed removeTasks boolean parameter, now always removing tasks (#1505) * Update QueueResilienceSpec tests to include archive workflow * Update ExecutionDAOFacade to fix removeTask Currently ExecutionDAOFacade would call removeTask double the amount of times, this has been fixed. Now the only bug still present is that tasks are not terminated once removed and with that the index may also not be removed. * Remove archival of workflow tasks from QueueResiliencySpec * Merged loops for removing task index and queue removal (#1505)
Fix typo in docs: Desiging → Designing
fix a spell mistake
* Update workflow_sdk.md Fixed the 404 error for the existing task input link and some minor updates on the content. * fixed 404 link and some minor updates Fixed 404 error link of WorkflowExecutor. Apart from that, few update on the content. * content update Few content updates
Rephased some text. Added appropriate punctuation to the file.
* Heal the Java Worker doc page * Fixed MD formatting issues * Fixed broken stylig * Improved spelling/grammar * Add missing 'conductor-common' dependency The `Task`-related classes are located in the 'conductor-common' lib. * Bumped the lib version
Signed-off-by: Simon Misencik <[email protected]> Signed-off-by: Simon Misencik <[email protected]>
* Fix unused version in Running Workflows metric * spotless * Fix for sorting by version and adding test * Running spotless with java 11 * undo spotless
Event.path is deprecated in chrome as of the last update https://chromestatus.com/feature/5726124632965120. This is causing a bug when clicking on a task in the UI. This is also see in safari: #3330. Updating to use composedPath() instead per guidance. This is compatible with all browsers: https://developer.mozilla.org/en-US/docs/Web/API/Event/composedPath
Bumps [decode-uri-component](https://github.com/SamVerschueren/decode-uri-component) from 0.2.0 to 0.2.2. - [Release notes](https://github.com/SamVerschueren/decode-uri-component/releases) - [Commits](SamVerschueren/decode-uri-component@v0.2.0...v0.2.2) --- updated-dependencies: - dependency-name: decode-uri-component dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]>
* Allow for an upgrade of log4j2 versions * Update lock
Fixed spelling in documentation files.
Corrected the spellings
Corrected spellings
Bumps [com.thoughtworks.xstream:xstream](https://github.com/x-stream/xstream) from 1.4.19 to 1.4.20. - [Release notes](https://github.com/x-stream/xstream/releases) - [Commits](https://github.com/x-stream/xstream/commits) --- updated-dependencies: - dependency-name: com.thoughtworks.xstream:xstream dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>
1. Add annotation support for pollingInterval 2. Allow configuration using spring 3. Tests
Fix one word.
Updated to make it work on Apple Mac M1
Introducing a Noop task to allow for switch decision cases to have "no" tasks
* Add support for JDK17 in the docker containers * Docker compose for redis, postgres and mysql with Elasticsearch7 co-author: @manan164
* remove previous index of event handlers by event if event handler's event gets updated * run ./gradlew spotlessApply
…3836) Issue Summary: There's a race condition in the system involving async system tasks and the WorkflowRepairService. For example, when a SUB_WORKFLOW task starts, the WorkflowRepairService sometimes erroneously reinserts the task into the processing queue because it perceives the task as out-of-sync between the ExecutorDAO and the queueDAO. This issue stems from the AsyncSystemTaskExecutor updating a task's status only after it removes it from the queue, creating a window where the WorkflowRepairService can wrongly assess the task state. This leads to duplicate subworkflows/http/… tasks being executed concurrently, which complicates maintaining idempotency of Tasks. Proposed Solution: To resolve the issue, it's suggested that the AsyncSystemTaskExecutor should update the status of tasks before removing them from the queue. This should close the window where the WorkflowRepairService can misidentify the task state and prevent unnecessary re-queuing of tasks. An edge case we’ve considered is if the process crashes after the task is updated but before it's removed from the queue. If that happens, the executor will simply remove the task from the queue the next time it runs, thereby not affecting system correctness. Co-authored-by: Jaim Silva <[email protected]>
* Version updates
* Fix: include monaco sources into bundle
* Optimizing reading from datastore during WorkflowSweeper#sweep * Added javadoc to decideWithLock method --------- Co-authored-by: Boyan Georgiev <[email protected]>
Fix Conductor users broken links
`provies` -> `provides`
…ername for ACL. (#3847) Co-authored-by: Anup Chatterjee <[email protected]>
Co-authored-by: Robert Karlsson <[email protected]>
* ISICO-15108: javascript validation done on updating w/d * ISICO-14902: NPE while checking EvaluatorType is fixed * ISICO-14902: unused variables removed * ISICO-15108: javascript validation added in the constraint violation part * ISICO-15108: getExpression() is used for switch javascript code
* Upgrading OSS libraries with known open CVEs * Updating the protobuf library. * Resolve the remaining build issues. --------- Co-authored-by: Scott Carter <[email protected]>
* Enable hosting app in any route, with easy config * Review Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request type
Changes in this PR
Describe the new behavior from this PR, and why it's needed
Issue #
Alternatives considered
Describe alternative implementation you have considered