[TensorFlow][Inference][Sagemaker] TensorFlow 2.19.0 Currency Release #4883

bhanutejagk · 2025-06-03T23:08:46Z

GitHub Issue #, if available:

Note:

If merging this PR should also close the associated Issue, please also add that Issue # to the Linked Issues section on the right.
All PR's are checked weekly for staleness. This PR will be closed if not updated in 30 days.

Description

Tests run

NOTE: By default, docker builds are disabled. In order to build your container, please update dlc_developer_config.toml and specify the framework to build in "build_frameworks"

I have run builds/tests on commit for my changes.

Confused on how to run tests? Try using the helper utility...

Assuming your remote is called origin (you can find out more with git remote -v)...

Run default builds and tests for a particular buildspec - also commits and pushes changes to remote; Example:

python src/prepare_dlc_dev_environment.py -b </path/to/buildspec.yml> -cp origin

Enable specific tests for a buildspec or set of buildspecs - also commits and pushes changes to remote; Example:

python src/prepare_dlc_dev_environment.py -b </path/to/buildspec.yml> -t sanity_tests -cp origin

Restore TOML file when ready to merge

python src/prepare_dlc_dev_environment.py -rcp origin

NOTE: If you are creating a PR for a new framework version, please ensure success of the standard, rc, and efa sagemaker remote tests by updating the dlc_developer_config.toml file:

Expand

sagemaker_remote_tests = true
sagemaker_efa_tests = true
sagemaker_rc_tests = true

Additionally, please run the sagemaker local tests in at least one revision:

sagemaker_local_tests = true

Formatting

I have run black -l 100 on my code (formatting tool: https://black.readthedocs.io/en/stable/getting_started.html)

DLC image/dockerfile

Builds to Execute

Expand

Fill out the template and click the checkbox of the builds you'd like to execute

Note: Replace with <X.Y> with the major.minor framework version (i.e. 2.2) you would like to start.

Additional context

PR Checklist

Expand

I've prepended PR tag with frameworks/job this applies to : [mxnet, tensorflow, pytorch] | [ei/neuron/graviton] | [build] | [test] | [benchmark] | [ec2, ecs, eks, sagemaker]
If the PR changes affects SM test, I've modified dlc_developer_config.toml in my PR branch by setting sagemaker_tests = true and efa_tests = true
If this PR changes existing code, the change fully backward compatible with pre-existing code. (Non backward-compatible changes need special approval.)
(If applicable) I've documented below the DLC image/dockerfile this relates to
(If applicable) I've documented below the tests I've run on the DLC image
(If applicable) I've reviewed the licenses of updated and new binaries and their dependencies to make sure all licenses are on the Apache Software Foundation Third Party License Policy Category A or Category B license list. See https://www.apache.org/legal/resolved.html.
(If applicable) I've scanned the updated and new binaries to make sure they do not have vulnerabilities associated with them.

NEURON/GRAVITON Testing Checklist

When creating a PR:

I've modified dlc_developer_config.toml in my PR branch by setting neuron_mode = true or graviton_mode = true

Benchmark Testing Checklist

When creating a PR:

I've modified dlc_developer_config.toml in my PR branch by setting ec2_benchmark_tests = true or sagemaker_benchmark_tests = true

Pytest Marker Checklist

Expand

(If applicable) I have added the marker @pytest.mark.model("<model-type>") to the new tests which I have added, to specify the Deep Learning model that is used in the test (use "N/A" if the test doesn't use a model)
(If applicable) I have added the marker @pytest.mark.integration("<feature-being-tested>") to the new tests which I have added, to specify the feature that will be tested
(If applicable) I have added the marker @pytest.mark.multinode(<integer-num-nodes>) to the new tests which I have added, to specify the number of nodes used on a multi-node test
(If applicable) I have added the marker @pytest.mark.processor(<"cpu"/"gpu"/"eia"/"neuron">) to the new tests which I have added, if a test is specifically applicable to only one processor type

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Yadan-Wei · 2025-06-04T18:18:13Z

tensorflow/inference/buildspec-2-19-sm.yml

+    dockerd-entrypoint:
+      source: docker/build_artifacts/dockerd-entrypoint.py
+      target: dockerd-entrypoint.py
+    dockerd_ec2_entrypoint:


We don't need this for inference.

tensorflow/inference/docker/2.19/py3/cu122/Dockerfile.gpu

Yadan-Wei · 2025-06-04T18:19:49Z

tensorflow/inference/docker/2.19/py3/cu122/Dockerfile.gpu

+ARG TFS_SHORT_VERSION=2.19
+ARG CUDA_DASH=12-2
+
+ARG TF_SERVING_VERSION_GIT_COMMIT=HEAD


We shouldn't use HEAD here for TF2.19, we can use a specific commit for TF2.19 can refer to https://hub.docker.com/layers/tensorflow/serving/2.19.0-gpu/images/sha256-d32afbfadf8cc3fdc14ce3e613badc109317dd952cef50dd0c899253f6373809

Yadan-Wei · 2025-06-05T17:12:43Z

tensorflow/inference/docker/2.19/py3/cu122/Dockerfile.gpu

+# Specify accept-bind-to-port LABEL for inference pipelines to use SAGEMAKER_BIND_TO_PORT
+# https://docs.aws.amazon.com/sagemaker/latest/dg/inference-pipeline-real-time.html
+LABEL com.amazonaws.sagemaker.capabilities.accept-bind-to-port=true
+LABEL com.amazonaws.sagemaker.inference.cuda.verified_versions=12.5


We use 12.2 not 12.5 here

Yadan-Wei · 2025-06-05T23:32:20Z

test/dlc_tests/conftest.py

-    "bashrc": {"pytorch": ["2.4.0", "2.5.1", "2.6.0"], "tensorflow": ["2.18.0"]},
-    "framework": {"pytorch": [""], "tensorflow": [""]},
+    "entrypoint": {"pytorch": ["2.4.0", "2.5.1", "2.6.0"], "tensorflow": ["2.18.0", "2.19.0"]},
+    "bashrc": {"pytorch": ["2.4.0", "2.5.1", "2.6.0"], "tensorflow": ["2.18.0", "2.19.0"]},


We have bashrc and entrypoint telemetry in TF2.19 inference, we shouldn't skip it.

Yadan-Wei · 2025-06-07T00:28:32Z

You can also try to pin openssl >= a version to see if that works.

Yadan-Wei · 2025-06-10T17:09:25Z

tensorflow/inference/docker/2.19/py3/Dockerfile.cpu

+
+#we use different installation method other than this
+#RUN curl $TFS_URL -o /usr/bin/tensorflow_model_server \
+# && chmod 555 /usr/bin/tensorflow_model_server


You can remove these lines.

Yadan-Wei

You can revert toml file before submitting for PR review.

…ing reporitories

…libreadline-dev in gpu file

… also skipped telemetry tests

…pping of bashrc and entrypoint telemtry

…lities & checking only security tests

…error

…rors

* Add license file content test * use short verison * test no build * print string * enable build * fix allowlist * rebuild * buildtest ec2 * test arm * build test inference * disable arm64 mode * disable build * revert toml

* update EFA to 1.41.0 vllm to 0.9.0.1

Yadan-Wei · 2025-06-10T18:12:02Z

tensorflow/inference/docker/2.19/py3/Dockerfile.cpu

+ARG PYTHON_PIP=python3-pip
+ARG PIP=pip3
+ARG PYTHON_VERSION=3.12.10
+#ARG TFS_URL=https://framework-binaries.s3.us-west-2.amazonaws.com/tensorflow_serving/r2.18_aws/cpu/2025-01-15-19-48/tensorflow_model_server


You can also remove this line.

Yadan-Wei · 2025-06-10T18:12:52Z

tensorflow/inference/docker/2.19/py3/cu122/Dockerfile.gpu

+ARG PYTHON_PIP=python3-pip
+ARG PIP=pip3
+ARG PYTHON_VERSION=3.12.10
+#ARG TFS_URL=https://framework-binaries.s3.us-west-2.amazonaws.com/tensorflow_serving/r2.18_aws/gpu/2025-01-17-21-54/tensorflow_model_server


Remove this line too.

bhanutejagk requested a review from a team as a code owner June 3, 2025 23:08

aws-deep-learning-containers-ci bot added authorized build Reflects file change in build folder Size:S Determines the size of the PR tensorflow Reflects file change in tensorflow folder labels Jun 3, 2025

bhanutejagk changed the title ~~Building inf image~~ [TensorFlow][Inference][Sagemaker] TensorFlow 2.19.0 Currency Release Jun 3, 2025

Yadan-Wei reviewed Jun 4, 2025

View reviewed changes

tensorflow/inference/docker/2.19/py3/cu122/Dockerfile.gpu Show resolved Hide resolved

Yadan-Wei reviewed Jun 4, 2025

View reviewed changes

Yadan-Wei reviewed Jun 5, 2025

View reviewed changes

bhanutejagk requested a review from a team as a code owner June 9, 2025 18:05

aws-deep-learning-containers-ci bot added ec2 Reflects file change in dlc_tests/ec2 folder pytorch Reflects file change in pytorch folder sagemaker_tests sanity Reflects file change in dlc_tests/sanity folder test Reflects file change in test folder labels Jun 9, 2025

bhanutejagk force-pushed the tf_inf branch 2 times, most recently from 58ad489 to 03faa70 Compare June 9, 2025 23:07

Yadan-Wei reviewed Jun 10, 2025

View reviewed changes

Bhanu Teja Goshikonda added 8 commits June 10, 2025 10:52

Building inf image

226ec55

added dockerd_ec2_entrypoint.sh file

54a73c2

fixed syntax

4978578

modified env variable with values directly

abc8ca7

rearranged build_image creation line to top of file

b0313d4

added setup.sources.sh file to add NVIDIA's package & Tensorflow serv…

4d48b21

…ing reporitories

removed libssl1.1 line and modified libreadline-gplv2-dev

aa0ae5c

removed libssl1.1 installation and modified libreadline-gplv2-dev to …

42f96e6

…libreadline-dev in gpu file

Bhanu Teja Goshikonda and others added 23 commits June 10, 2025 10:52

added allowlist to remove vulnerability on gpu image

701bf30

removed logic for build to only test

1c65e62

setup nvidia repositories through cuda-keyring and building image and…

006fe82

… also skipped telemetry tests

modified wget installation code after installing wget

27d04b0

added 2.19 to skip framework telemetry

037cc56

checking with openssl/libssl1.1.1p to check for CVE's and removed ski…

eda307c

…pping of bashrc and entrypoint telemtry

Upgraded OpenSSL to a newer version that doesn't have these vulnerabi…

f556d09

…lities & checking only security tests

building the image

cb568cc

upgraded libssl1.1 to 1.1.1t

a65aebc

replace 1.1.1t with 1.1.1o for libssl1.1

2137960

upgraded libssl1.1 version to 1.1.1f-1ubuntu2.24

9529855

changed libssl version to libssl1.1_1.1.1-1ubuntu2.1~18.04.23_amd64.deb

8ad25a2

upgraded libssl1.1 to 1.1.1-1ubuntu2.1~18.04.23

006529e

removed libssl1.1 installation to check if it even needed to remove …

142a4f4

…error

uninstalling libssl1.1 after installing cuda packages to check for er…

7d5d8f0

…rors

Add license file content test (aws#4890)

4a25155

* Add license file content test * use short verison * test no build * print string * enable build * fix allowlist * rebuild * buildtest ec2 * test arm * build test inference * disable arm64 mode * disable build * revert toml

update EFA to 1.41.0 vllm to 0.9.0.1 (aws#4898)

5a76ec0

* update EFA to 1.41.0 vllm to 0.9.0.1

upgraded base image to cuda 12.2.2 to check for errors of openssl

b40aefc

changed cuda base image version to 12.2.0 from 12.2.2

a719b41

added allowlist.json file for the openssl CVE's

2601603

enabled some more tests

e117bbb

removed some commented lines

b643786

reverted back toml file

e95cb61

bhanutejagk force-pushed the tf_inf branch from 4b23655 to e95cb61 Compare June 10, 2025 17:52

Yadan-Wei reviewed Jun 10, 2025

View reviewed changes

Removed all the commented lines for cleaner code

4ae8f12

Yadan-Wei approved these changes Jun 10, 2025

View reviewed changes

sallyseok approved these changes Jun 10, 2025

View reviewed changes

bhanutejagk merged commit ee5db79 into aws:master Jun 10, 2025
33 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[TensorFlow][Inference][Sagemaker] TensorFlow 2.19.0 Currency Release #4883

[TensorFlow][Inference][Sagemaker] TensorFlow 2.19.0 Currency Release #4883

Uh oh!

bhanutejagk commented Jun 3, 2025

Uh oh!

Yadan-Wei Jun 4, 2025

Uh oh!

Uh oh!

Yadan-Wei Jun 4, 2025

Uh oh!

Yadan-Wei Jun 5, 2025

Uh oh!

bhanutejagk Jun 6, 2025

Uh oh!

Yadan-Wei Jun 5, 2025

Uh oh!

bhanutejagk Jun 6, 2025

Uh oh!

Yadan-Wei commented Jun 7, 2025

Uh oh!

Yadan-Wei Jun 10, 2025

Uh oh!

Yadan-Wei left a comment

Uh oh!

Yadan-Wei Jun 10, 2025

Uh oh!

Yadan-Wei Jun 10, 2025

Uh oh!

Uh oh!

Uh oh!

[TensorFlow][Inference][Sagemaker] TensorFlow 2.19.0 Currency Release #4883

[TensorFlow][Inference][Sagemaker] TensorFlow 2.19.0 Currency Release #4883

Uh oh!

Conversation

bhanutejagk commented Jun 3, 2025

Description

Tests run

Formatting

DLC image/dockerfile

Builds to Execute

Additional context

PR Checklist

NEURON/GRAVITON Testing Checklist

Benchmark Testing Checklist

Pytest Marker Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Yadan-Wei commented Jun 7, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Yadan-Wei left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!