Skip to content

megSAP in AWS #364

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
4 of 11 tasks
KilianIlius opened this issue Mar 20, 2025 · 1 comment
Open
4 of 11 tasks

megSAP in AWS #364

KilianIlius opened this issue Mar 20, 2025 · 1 comment
Assignees
@KilianIlius

Comments

@KilianIlius
Copy link
Collaborator

KilianIlius commented Mar 20, 2025 •
edited
Loading

  • Install megSAP on AWS instance
  • Get NGSD access
  • Test pipeline with small sample (Panel?)
  • Test pipeline with container and data_folder on s3
  • Implement on demand workflow:
    • EC2 instance is spawned
    • Apptainer is installed on the instance
    • Data_folder, apptainer container and sample folder is mounted into the instance from s3
    • Pipeline execution
    • Output files are stored on s3
    • Instance is deleted
@KilianIlius KilianIlius self-assigned this Mar 20, 2025
@KilianIlius
Copy link
Collaborator Author

KilianIlius commented Apr 9, 2025 •
edited
Loading

Runtime comparison on Panel DX191956_01

Container, data_folder and sample_folder on s3:

  • Elapsed time: 53:14 (mm:ss)
  • Maximum resident set size: 19.13 GB

Container, data_folder and sample_folder on EC2 instanc:

  • Elapsed time: 25:44 (mm:ss)
  • Maximum resident set size: 19.3 GB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@KilianIlius KilianIlius

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@KilianIlius