Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate Elmer Stacky Service, remove bsshFastqCopy as a 'WorkflowRunStateChange' service #785

Open
alexiswl opened this issue Dec 18, 2024 · 1 comment
Assignees
Labels
feature New feature pipeline Workflow/Pipeline Manager

Comments

@alexiswl
Copy link
Member

alexiswl commented Dec 18, 2024

Currently bsshFastqCopy runs through the WorkflowManager with READY + SUCCEDED events, however we want to nest the ORA Compression Pipeline (which definitely needs to run through the workflow manager) inside the bsshFastqCopy workflow, which makes things a little hairy. Instead the bsshFastqCopy manager should just listen to the bclConvert Manager success event (which is what Elmer stacky service does), and then

  1. Copy over the interops
  2. Run the ORA Compression service

Then generate a BSSH Fastq Copy Complete event that doesn't need to have all the bells and whistles of a WorkflowRunStateChange event, downstream listeners, clag (bsshFastqCopy to fastq list row) and gorilla (glue from bsshFastqCopy to BCLConvertInteropQC) will need updates as a result

image

@alexiswl alexiswl added the feature New feature label Dec 18, 2024
@alexiswl alexiswl self-assigned this Dec 18, 2024
@victorskl victorskl added the pipeline Workflow/Pipeline Manager label Dec 18, 2024
@alexiswl
Copy link
Member Author

If we are to 'start from fastqs' as a base, it might be worth merging the bsshFastqCopy service and the bclconvert manager services.

So instead of having multiple events between the bclconvert manager and the bssh fastq copy steps prior to fastqs even existing in our byob service. An 'all-in-one' would mean we have one service that listens to bclconvert manager completing and then performs the following:

  • Runs the ORA compression pipeline (and copies over the interops)
  • Generate library exists events, and fastq exists events

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature pipeline Workflow/Pipeline Manager
Projects
None yet
Development

No branches or pull requests

2 participants