Skip to content

Commit

Permalink
Merge branch 'main' into release-1.9
Browse files Browse the repository at this point in the history
  • Loading branch information
claytonparnell committed Jun 20, 2024
2 parents e0403b5 + 9695c95 commit 5bef0fe
Show file tree
Hide file tree
Showing 13 changed files with 104 additions and 5 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
## Amazon SageMaker Distribution

Amazon SageMaker Distribution is a set of Docker images that include popular frameworks for machine learning, data
science and visualization.
science and visualization. For the list of supported SageMaker Distributions images, see [SageMaker Distributions Images](supported_images.md#supported-image-versions).

These images come in two variants, CPU and GPU, and include deep learning frameworks like PyTorch, TensorFlow and
Keras; popular Python packages like numpy, scikit-learn and pandas; and IDEs like Jupyter Lab. The distribution contains
Expand Down
11 changes: 11 additions & 0 deletions build_artifacts/v1/v1.9/v1.9.0/CHANGELOG-cpu.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,23 @@

Package | Previous Version | Current Version
---|---|---
boto3|1.34.51|1.34.106
ipython|8.22.2|8.25.0
jupyter-ai|2.14.1|2.17.0
jupyter-scheduler|2.5.2|2.7.1
amazon-sagemaker-jupyter-scheduler|3.0.11|3.1.0
amazon-sagemaker-sql-magic|0.1.1|0.1.3
amazon_sagemaker_sql_editor|0.1.7|0.1.9
ipywidgets|8.1.2|8.1.3
aws-glue-sessions|1.0.5|1.0.6
jupyter-server-proxy|4.1.2|4.2.0
sagemaker-python-sdk|2.219.0|2.223.0
uvicorn|0.29.0|0.30.1

## What's new:

Package | Version
---|---
amazon-q-developer-jupyterlab-ext|3.1.0
amazon-sagemaker-jupyter-ai-q-developer|1.0.4
langchain-aws|0.1.6
11 changes: 11 additions & 0 deletions build_artifacts/v1/v1.9/v1.9.0/CHANGELOG-gpu.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,25 @@

Package | Previous Version | Current Version
---|---|---
boto3|1.34.51|1.34.106
ipython|8.22.2|8.25.0
jupyter-ai|2.14.1|2.17.0
jupyter-scheduler|2.5.2|2.7.1
amazon-sagemaker-jupyter-scheduler|3.0.11|3.1.0
amazon-sagemaker-sql-magic|0.1.1|0.1.3
amazon_sagemaker_sql_editor|0.1.7|0.1.9
ipywidgets|8.1.2|8.1.3
aws-glue-sessions|1.0.5|1.0.6
jupyter-server-proxy|4.1.2|4.2.0
keras|2.15.0|2.14.0
sagemaker-python-sdk|2.219.0|2.223.0
uvicorn|0.29.0|0.30.1

## What's new:

Package | Version
---|---
amazon-q-developer-jupyterlab-ext|3.1.0
amazon-sagemaker-jupyter-ai-q-developer|1.0.4
langchain-aws|0.1.6
tensorflow-gpu|2.14.0
4 changes: 2 additions & 2 deletions build_artifacts/v1/v1.9/v1.9.0/cpu.env.out
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ https://conda.anaconda.org/conda-forge/linux-64/python_abi-3.10-4_cp310.conda#26
https://conda.anaconda.org/conda-forge/linux-64/numpy-1.26.4-py310hb13e2d6_0.conda#6593de64c935768b6bad3e19b3e978be
https://conda.anaconda.org/conda-forge/noarch/packaging-23.2-pyhd8ed1ab_0.conda#79002079284aa895f883c6b7f3f88fd6
https://conda.anaconda.org/conda-forge/linux-64/psutil-5.9.8-py310h2372a71_0.conda#bd19b3096442ea342c4a5208379660b1
https://conda.anaconda.org/conda-forge/noarch/filelock-3.15.1-pyhd8ed1ab_0.conda#ca4149866d80007713ff47906bba8cb3
https://conda.anaconda.org/conda-forge/noarch/filelock-3.15.3-pyhd8ed1ab_0.conda#eae681f708bd52d9d172bd5c9af23898
https://conda.anaconda.org/conda-forge/linux-64/markupsafe-2.1.5-py310h2372a71_0.conda#f6703fa0214a00bf49d1bef6dc7672d0
https://conda.anaconda.org/conda-forge/noarch/jinja2-3.1.4-pyhd8ed1ab_0.conda#7b86ecb7d3557821c649b3c31e3eb9f2
https://conda.anaconda.org/conda-forge/linux-64/libabseil-20230802.1-cxx17_h59595ed_0.conda#2785ddf4cb0e7e743477991d64353947
Expand Down Expand Up @@ -598,7 +598,7 @@ https://conda.anaconda.org/conda-forge/linux-64/lame-3.100-h166bdaf_1003.tar.bz2
https://conda.anaconda.org/conda-forge/noarch/langchain-aws-0.1.6-pyhd8ed1ab_0.conda#7250b13961b3b4b8179752fbd22ec950
https://conda.anaconda.org/conda-forge/linux-64/libcap-2.69-h0f662aa_0.conda#25cb5999faa414e5ccb2c1388f62d3d5
https://conda.anaconda.org/conda-forge/linux-64/libclang-cpp15-15.0.7-default_h127d8a8_5.conda#d0a9633b53cdc319b8a1a532ae7822b8
https://conda.anaconda.org/conda-forge/linux-64/libllvm18-18.1.7-hc9dba70_1.conda#d321aff45652d36ced0fa1a2a71af4b7
https://conda.anaconda.org/conda-forge/linux-64/libllvm18-18.1.8-hc9dba70_0.conda#f94ed0c5953c78dcca7adb953f4c5bfb
https://conda.anaconda.org/conda-forge/linux-64/libclang13-18.1.7-default_h087397f_0.conda#536526073c2e7f9056fdce8584da779e
https://conda.anaconda.org/conda-forge/linux-64/libcups-2.3.3-h4637d8d_4.conda#d4529f4dff3057982a7617c7ac58fde3
https://conda.anaconda.org/conda-forge/linux-64/libflac-1.4.3-h59595ed_0.conda#ee48bf17cc83a00f59ca1494d5646869
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,6 @@
module = __import__("amazon_sagemaker_sql_editor")
module_location = os.path.dirname(module.__file__)
c.LanguageServerManager.extra_node_roots = [f"{module_location}/sql-language-server"]
c.AiExtension.default_language_model = "amazon-q:q-developer"
except:
pass
4 changes: 2 additions & 2 deletions build_artifacts/v1/v1.9/v1.9.0/gpu.env.out
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ https://conda.anaconda.org/conda-forge/linux-64/psutil-5.9.8-py310h2372a71_0.con
https://conda.anaconda.org/conda-forge/linux-64/cudatoolkit-11.8.0-h4ba93d1_13.conda#eb43f5f1f16e2fad2eba22219c3e499b
https://conda.anaconda.org/conda-forge/noarch/cuda-version-11.8-h70ddcb2_3.conda#670f0e1593b8c1d84f57ad5fe5256799
https://conda.anaconda.org/conda-forge/linux-64/cudnn-8.9.7.29-hbc23b4c_3.conda#4a2d5fab2871d95544de4e1752948d0f
https://conda.anaconda.org/conda-forge/noarch/filelock-3.15.1-pyhd8ed1ab_0.conda#ca4149866d80007713ff47906bba8cb3
https://conda.anaconda.org/conda-forge/noarch/filelock-3.15.3-pyhd8ed1ab_0.conda#eae681f708bd52d9d172bd5c9af23898
https://conda.anaconda.org/conda-forge/linux-64/markupsafe-2.1.5-py310h2372a71_0.conda#f6703fa0214a00bf49d1bef6dc7672d0
https://conda.anaconda.org/conda-forge/noarch/jinja2-3.1.4-pyhd8ed1ab_0.conda#7b86ecb7d3557821c649b3c31e3eb9f2
https://conda.anaconda.org/conda-forge/linux-64/libmagma-2.7.1-hc72dce7_6.conda#afd44491bd564d8dd6cda5d9aecaa452
Expand Down Expand Up @@ -581,7 +581,7 @@ https://conda.anaconda.org/conda-forge/noarch/langchain-aws-0.1.6-pyhd8ed1ab_0.c
https://conda.anaconda.org/conda-forge/linux-64/libcap-2.69-h0f662aa_0.conda#25cb5999faa414e5ccb2c1388f62d3d5
https://conda.anaconda.org/conda-forge/linux-64/libllvm15-15.0.7-hb3ce162_4.conda#8a35df3cbc0c8b12cc8af9473ae75eef
https://conda.anaconda.org/conda-forge/linux-64/libclang-cpp15-15.0.7-default_h127d8a8_5.conda#d0a9633b53cdc319b8a1a532ae7822b8
https://conda.anaconda.org/conda-forge/linux-64/libllvm18-18.1.7-hc9dba70_1.conda#d321aff45652d36ced0fa1a2a71af4b7
https://conda.anaconda.org/conda-forge/linux-64/libllvm18-18.1.8-hc9dba70_0.conda#f94ed0c5953c78dcca7adb953f4c5bfb
https://conda.anaconda.org/conda-forge/linux-64/libclang13-18.1.7-default_h087397f_0.conda#536526073c2e7f9056fdce8584da779e
https://conda.anaconda.org/conda-forge/linux-64/libcups-2.3.3-h4637d8d_4.conda#d4529f4dff3057982a7617c7ac58fde3
https://conda.anaconda.org/conda-forge/linux-64/libflac-1.4.3-h59595ed_0.conda#ee48bf17cc83a00f59ca1494d5646869
Expand Down
2 changes: 2 additions & 0 deletions build_artifacts/v2/v2.0/v2.0.0/cpu.additional_packages_env.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#Add any additional packages here
conda-forge::mlflow[version='>=2.13.0,<3.0']
2 changes: 2 additions & 0 deletions build_artifacts/v2/v2.0/v2.0.0/gpu.additional_packages_env.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#Add any additional packages here
conda-forge::mlflow[version='>=2.13.2,<3.0']
25 changes: 25 additions & 0 deletions supported_images.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
## Supported Image Versions

The tables below list the supported SageMaker Distribution image versions. SageMaker will only release functionality and security fixes for images versions present in the tables. When you are building images or pulling images from the ECR repository, it is highly recommended for you to choose the supported image versions from the tables below.

### CPU Images

| Image Major Version | Image Minor Version | ECR Image URI |
|---------------------|---------------------|-----------------------|
|1 |1.8 |public.ecr.aws/sagemaker/sagemaker-distribution:1.8-cpu |
|1 |1.7 |public.ecr.aws/sagemaker/sagemaker-distribution:1.7-cpu |
|1 |1.6 |public.ecr.aws/sagemaker/sagemaker-distribution:1.6-cpu |
|1 |1.5 |public.ecr.aws/sagemaker/sagemaker-distribution:1.5-cpu |
|1 |1.4 |public.ecr.aws/sagemaker/sagemaker-distribution:1.4-cpu |
|0 |0.12 |public.ecr.aws/sagemaker/sagemaker-distribution:0.12-cpu |

### GPU Images

| Image Major Version | Image Minor Version | ECR Image URI |
|---------------------|---------------------|-----------------------|
|1 |1.8 |public.ecr.aws/sagemaker/sagemaker-distribution:1.8-gpu |
|1 |1.7 |public.ecr.aws/sagemaker/sagemaker-distribution:1.7-gpu |
|1 |1.6 |public.ecr.aws/sagemaker/sagemaker-distribution:1.6-gpu |
|1 |1.5 |public.ecr.aws/sagemaker/sagemaker-distribution:1.5-gpu |
|1 |1.4 |public.ecr.aws/sagemaker/sagemaker-distribution:1.4-gpu |
|0 |0.12 |public.ecr.aws/sagemaker/sagemaker-distribution:0.12-gpu |
1 change: 1 addition & 0 deletions template/v1/dirs/etc/jupyter/jupyter_server_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,6 @@
module = __import__("amazon_sagemaker_sql_editor")
module_location = os.path.dirname(module.__file__)
c.LanguageServerManager.extra_node_roots = [f"{module_location}/sql-language-server"]
c.AiExtension.default_language_model = "amazon-q:q-developer"
except:
pass
16 changes: 16 additions & 0 deletions test/test_artifacts/v2/mlflow.test.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
ARG SAGEMAKER_DISTRIBUTION_IMAGE
FROM $SAGEMAKER_DISTRIBUTION_IMAGE

ARG MAMBA_DOCKERFILE_ACTIVATE=1

RUN python -c "import mlflow"

RUN sudo apt-get update && sudo apt-get install -y git && \
git clone --recursive https://github.com/mlflow/mlflow.git && \
:

WORKDIR "mlflow/"
COPY --chown=$MAMBA_USER:$MAMBA_USER scripts/run_mlflow_tests.sh .
RUN chmod +x run_mlflow_tests.sh
# Run tests in run_matplotlib_tests.sh
CMD ["./run_mlflow_tests.sh"]
28 changes: 28 additions & 0 deletions test/test_artifacts/v2/scripts/run_mlflow_tests.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
#!/bin/bash

# Run examples for keras, pytorch, sklearn, tensorflow
cd examples

# keras
cd keras/
python train.py
cd -

# pytorch
cd pytorch/
python mnist_tensorboard_artifact.py
cd -

# sklearn
for folder in "sklearn_autolog/" "sklearn_elasticnet_diabetes/linux/" "sklearn_elasticnet_wine/" "sklearn_logistic_regression/"; do
cd ${folder}
for file in *.py; do
python "$file" || exit $?
done
cd -
done

# tensorflow
cd tensorflow/
python train.py
cd -
2 changes: 2 additions & 0 deletions test/test_dockerfile_based_harness.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@
("amazon_sagemaker_sql_editor.test.Dockerfile", ["amazon_sagemaker_sql_editor"]),
("serve.test.Dockerfile", ["serve-langchain"]),
("langchain-aws.test.Dockerfile", ["langchain-aws"]),
("mlflow.test.Dockerfile", ["mlflow"]),
],
)
def test_dockerfiles_for_cpu(
Expand Down Expand Up @@ -83,6 +84,7 @@ def test_dockerfiles_for_cpu(
("amazon_sagemaker_sql_editor.test.Dockerfile", ["amazon_sagemaker_sql_editor"]),
("serve.test.Dockerfile", ["serve-langchain"]),
("langchain-aws.test.Dockerfile", ["langchain-aws"]),
("mlflow.test.Dockerfile", ["mlflow"]),
],
)
def test_dockerfiles_for_gpu(
Expand Down

0 comments on commit 5bef0fe

Please sign in to comment.