Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Optimum Neuron Containers support(0.0.25 with Neuron 2.20.0) #713

Open
cszhz opened this issue Oct 8, 2024 · 1 comment
Open

Add Optimum Neuron Containers support(0.0.25 with Neuron 2.20.0) #713

cszhz opened this issue Oct 8, 2024 · 1 comment

Comments

@cszhz
Copy link

cszhz commented Oct 8, 2024

Feature request

So far the optimum version is 0.0.24, is there any plan to upgrade to 0.0.25 with Neuron 2.20.0?
https://huggingface.co/docs/optimum-neuron/en/containers

Motivation

AWS Neuron SDK 2.20.0 has released in September, is there any plan to support Neuron 2.20.0 for SageMaker inference container?
Thank You.

Your contribution

N/A

@dacorvo
Copy link
Collaborator

dacorvo commented Oct 8, 2024

The SageMaker 0.0.25 inference image is currently being reviewed: aws/deep-learning-containers#4308.
The NeuronX TGI 0.0.25 SageMaker image has already been generated at the end of last week and is currently being deployed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants