Multiple custom scorefiles on the cloud

[geocarvalho]

Hi, I am running pgsc_calc on AWS HealthOmics and I am not sure how to set up the multiple custom scorfiles. What is the recommendation to run multiple scorefiles without using glob? For example using the bash:

nextflow run main.nf --format json --input /home/ssm-user/gvcn/pgsc_calc/parameter-input.json --hg38_chain /home/ssm-user/gvcn/hg38ToHg19.over.chain.gz --scorefile /home/ssm-user/gvcn/PGS000004.txt --scorefile /home/ssm-user/gvcn/PGS003725.txt --target_build GRCh38 --run_ancestry /home/ssm-user/gvcn/pgsc_HGDP+1kGP_v1.tar.zst -profile docker

Thanks!

[nebfield]

The only supported method of working with multiple custom scoring files is by using quoted wildcards:

--scorefile "/home/ssm-user/gvcn/PGS*.txt"

Nextflow handles the globs to set up the file paths.

[cocademolletes]

I'm using
--scorefile /home/ssm-user/gvcn/PGS\*.txt
on my ubuntu server machine and it works fine.

[nebfield]

Yes, any method that prevents your shell from expanding the glob (i.e. escaping with \, quoting it) means that Nextflow is able to find the files.

[geocarvalho]

In my case, I am running on AWS HealthOmics. So, I need to specify the s3 file in a parameter because it doesn't work with the glob. Right now the only solution I see is running one score at a time.

[smlmbrt]

@geocarvalho, you could pre-combine scorefiles into a single file containing multiples scores (https://pgsc-calc.readthedocs.io/en/latest/how-to/multiple.html#setting-multiple-scores-in-one-custom-scoring-file) and then run the pipeline on that single file.

[nebfield]

I need to double check but I think the docs are a bit out of date with the latest version of pgscatalog-combine (I don't think it will work).

Multiple custom scoring files in a bucket is an interesting use case I hadn't considered! @geocarvalho As a temporary workaround I would suggest mounting a bucket that contains your scoring files using something like:

• https://docs.aws.amazon.com/AmazonS3/latest/userguide/mountpoint.html
• https://cloud.google.com/storage/docs/cloud-storage-fuse/overview

That way you can get globs working with files stored in S3.

[geocarvalho]

There is any recomendation on how to combine custom scores so I can set just one file? --scorefile /home/ssm-user/gvcn/PGS_combined.txt

Thanks!

[nebfield]

Not currently! It's important that each scoring file is separate. The calculator combines them internally for parallel calculation.

If putting the custom files in a single directory is a problem perhaps creating symlinks might be a useful workaround.