Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for AWS Batch #456

Closed
delagoya opened this issue Feb 26, 2024 · 7 comments
Closed

Add support for AWS Batch #456

delagoya opened this issue Feb 26, 2024 · 7 comments
Labels
enhancement New feature or request

Comments

@delagoya
Copy link
Contributor

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

What is the outcome that you are trying to reach?

AWS Batch is a managed service for handling batch and high-performance compute workloads. It works with existing EKS clusters to launch EC2 instances within a specific namespace, and places pods onto these instances to handle these types of work requests.

AWS Batch requires your EKS cluster to be preconfigured to allow the managed services to communicate with the head node to add compute nodes and create pod specs that are placed on the launched nodes. For exact details refer to https://docs.aws.amazon.com/batch/latest/userguide/getting-started-eks.html

Describe the solution you would like

A blueprint that allows customers easy deployment of the necessary EKS and Batch components to allow for Batch to run asynchronous, high performance and batch workloads on EKS using AWS Batch managed services

Describe alternatives you have considered

All other batch schedulers require customer-deployed and managed components (Kueue, Volcano, etc).

Additional context

n/a
@delagoya
Copy link
Contributor Author

We have a set of contributors identified and are targeting an initial PR for review by 3/26

@github-actions GitHub Actions
Copy link
Contributor

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

@github-actions github-actions bot added the stale label Apr 11, 2024
@delagoya
Copy link
Contributor Author

Comment to keep alive. Progress is being made.

@askulkarni2 askulkarni2 added enhancement New feature or request and removed stale labels Apr 11, 2024
@jrussell9000
Copy link

Any update on this? Comprehensive, annotated code describing (in full) how to programmatically set up Batch on EKS seems non-existent on the interwebs.

@delagoya
Copy link
Contributor Author

Should have a PR by end of week.

@jrussell9000
Copy link

Bumping this again. I'm also curious if it's possible (and how) to use Bottlerocket on the batch nodes.

delagoya pushed a commit to delagoya/data-on-eks that referenced this issue Aug 26, 2024
Adding support for AWS Batch, addresses issue awslabs#456
cd64960 
* Add the AWS Batch blueprint under `schedulers/terraform`
* Adds tutorial for running an example AWS Batch job in the created EKS cluster
* Added AWS Batch to Job Schedulers index page.
@delagoya delagoya mentioned this issue Aug 26, 2024
Merged
5 tasks
@delagoya
Copy link
Contributor Author

This feature was implemented and merged in PR #620. Closing the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@delagoya @askulkarni2 @jrussell9000