UnixPB: Include Pre-Req Packages For SSL Test (adoptium#3169)

* Include SSL Packages Include pre-requisite packages for running SSL client tests. * Add VPC SSL tests for Linux * Fix missing bracket. * Revert debian URL change * Add twisty section * Fix twisty section * Update ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/Alpine.yml Co-authored-by: Stewart X Addison <[email protected]> * Update DockerStatic Files For SSL Tests * Fix errant comment. * Seperate ssl packages for alpine for consistency --------- Co-authored-by: Stewart X Addison <[email protected]>
steelhead31 · Oct 12, 2023 · fcb6021 · fcb6021
1 parent 87b659e
commit fcb6021
Show file tree

Hide file tree

Showing 28 changed files with 211 additions and 6 deletions.
diff --git a/FAQ.md b/FAQ.md
@@ -211,6 +211,66 @@ should have an underscore `_` prepended to it.
 (For the last one, that makes use of the system.custom target added via
 [this PR](https://github.com/AdoptOpenJDK/openjdk-tests/pull/2234))
 
+## Running The SSL Test Suites
+<details>
+<summary>Quick Guide To Running The SSL Test Suites</summary>
+
+As part of the fix for infrastructure [issue 3059](https://github.com/adoptium/infrastructure/issues/3059) several new pre-requisite packages have been added to the Unix playbooks, usually things such as (gnutls, gnutls-utils, libnss3.so, libnssutil3.so, nss-devel, nss-tools) or their O/S specific variants. In order to validate that these tests can run following any changes, the following process can be followed once the playbooks have been run successfully:
+
+N.B. Currently the integration testing for other clients is currently not enabed on non-Linux platforms.
+
+1) Clone The Open JDK ssl test suites
+
+```
+git clone https://github.com/rh-openjdk/ssl-tests
+
+```
+
+2) Download and install the JDK to be tested, and export the TESTJAVA environment variable.
+```
+export TESTJAVA=/home/user/jdk17
+```
+
+3) Execute The 3 Test Suites To Test External clients, from the directory the git clone of the openjdk ssl test suites was carried out:
+```
+cd ssl-tests/jtreg-wrappers
+
+Run each of the following test suites:
+
+./ssl-tests-gnutls-client.sh
+./ssl-tests-nss-client.sh
+./ssl-tests-openssl-client.sh
+```
+
+Each script should produce output similar to the below, with some tests being completed, and others skipped, but as long as the tests run without errors, this can be considered a success.
+
+```
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_DHE_RSA_WITH_AES_256_CBC_SHA
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_DHE_DSS_WITH_AES_256_CBC_SHA
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_DHE_RSA_WITH_AES_128_CBC_SHA
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_DHE_DSS_WITH_AES_128_CBC_SHA
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_ECDH_RSA_WITH_AES_256_CBC_SHA
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_ECDH_RSA_WITH_AES_128_CBC_SHA
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_RSA_WITH_AES_256_GCM_SHA384
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_RSA_WITH_AES_128_GCM_SHA256
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_RSA_WITH_AES_256_CBC_SHA256
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_RSA_WITH_AES_128_CBC_SHA256
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_RSA_WITH_AES_256_CBC_SHA
+PASSED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_RSA_WITH_AES_128_CBC_SHA
+IGNORED: SunJSSE/TLSv1.3: TLSv1.2 + TLS_EMPTY_RENEGOTIATION_INFO_SCSV
+
+```
+
+N.B. Due to a missing pre-requisite binary(tstclnt) not being available in the nss packages on Alpine, OpenSuse or SLES, the ssl-tests-nss-client.sh tests can not be run.
+
+</details>
+
 ## Testing changes
 
 If you are making a change which might have a negative effect on the
@@ -251,7 +311,7 @@ the Adoptium projects, and people can be given "contributor" or
 [the wiki](https://github.com/adoptium/adoptium/wiki/Working-with-Eclipse) for
 the processes around this) to the repositories which are under each Adoptium
 project as per
-[this comment](https://github.com/adoptium/infrastructure/issues/2549#issuecomment-1178903957). 
+[this comment](https://github.com/adoptium/infrastructure/issues/2549#issuecomment-1178903957).
 Most of the relevant ones are under the
 [temurin](https://projects.eclipse.org/projects/adoptium.temurin/who)
 or [aqavit](https://projects.eclipse.org/projects/adoptium.aqavit) projects.

diff --git a/ansible/pbTestScripts/testJDK.sh b/ansible/pbTestScripts/testJDK.sh
@@ -36,3 +36,26 @@ else
  $MAKE_COMMAND compile
  $MAKE_COMMAND _MBCS_Tests_pref_ja_JP_linux_0
 fi
+
+# Run SSL Client Tests Linux Only ( Not Solaris / FreeBSD )
+if [[ "$(uname)" == "FreeBSD" ]] || [["$(uname)" == "SunOS"]]; then
+ echo "Skipping SSL Tests As Not Supported"
+else
+ export TESTJAVA=$TEST_JDK_HOME
+ echo DEBUG: TESTJAVA = $TEST_JDK_HOME
+ mkdir -p $HOME/testLocation
+ [ ! -d $HOME/testLocation/ssl-tests ] && git clone https://github.com/rh-openjdk/ssl-tests $HOME/testLocation/ssl-tests
+ cd $HOME/testLocation/ssl-tests/jtreg-wrappers
+ ls -l
+ # Reduce Tests For Alpine/Sles/OpenSuse
+ if [[ "$(uname -v)" =~ .*"Alpine"*. ]] || [[ `cat /etc/os-release|grep -i opensuse|wc -l` -gt 0 ]] || [[ `cat /etc/os-release|grep -i SLES|wc -l` -gt 0 ]] ; then
+ echo "Run Alpine/OpenSuse/Sles SSL Client Tests"
+ ./ssl-tests-gnutls-client.sh
+ ./ssl-tests-openssl-client.sh
+ else
+ echo "Run Full Set Of SSL Client Tests"
+ ./ssl-tests-gnutls-client.sh
+ ./ssl-tests-nss-client.sh
+ ./ssl-tests-openssl-client.sh
+ fi
+fi
diff --git a/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/tasks/Solaris.yml b/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/tasks/Solaris.yml
@@ -443,7 +443,7 @@
  - not liberica11_installed.stat.exists
  tags: build_tools
 
-- name: Check for /usr/lib/jbm/fallocate.so
+- name: Check for /usr/lib/jvm/fallocate.so
  stat:
  path: /usr/lib/jvm/fallocate.so
  register: fallocate_installed

diff --git a/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/Alpine.yml b/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/Alpine.yml
@@ -20,6 +20,9 @@ Build_Tool_Packages:
  - freetype-dev
  - grep
  - gnupg
+ - gnutls # OpenSSL tests
+ - gnutls-dev # OpenSSL tests
+ - gnutls-utils # OpenSSL tests
  - libdwarf # OpenJ9
  - libdwarf-dev # OpenJ9
  - libx11
@@ -35,6 +38,9 @@ Build_Tool_Packages:
  - libxtst
  - libxtst-dev
  - linux-headers
+ - nss
+ - nss-dev
+ - nss-tools
  - numactl
  - numactl-dev # OpenJ9
  - pigz # Used in preference to gzip for tar.gz'ing
@@ -59,3 +65,9 @@ Test_Tool_Packages:
  - xauth
  - xorg-server
  - xvfb
+ - gnutls
+ - gnutls-dev
+ - gnutls-utils
+ - nss
+ - nss-dev
+ - nss-tools
diff --git a/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/CentOS.yml b/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/CentOS.yml
@@ -26,8 +26,12 @@ Build_Tool_Packages:
  - glibc-common
  - glibc-devel
  - gmp-devel
+ - gnutls
+ - gnutls-utils
  - java-1.8.0-openjdk-devel
  - libcurl-devel
+ - libnss3.so
+ - libnssutil3.so
  - libpng-devel
  - libXext-devel
  - libXi-devel # JDK12+ compilation
@@ -38,6 +42,8 @@ Build_Tool_Packages:
  - make
  - mesa-libGL-devel
  - mpfr-devel
+ - nss-devel
+ - nss-tools
  - numactl-devel # OpenJ9
  - openssh-clients # IBM: cloning over SSH
  - openssl-devel
@@ -105,3 +111,9 @@ Test_Tool_Packages:
  - xorg-x11-xauth
  - xorg-x11-server-Xvfb
  - fakeroot
+ - gnutls
+ - gnutls-utils
+ - libnss3.so
+ - libnssutil3.so
+ - nss-devel
+ - nss-tools
diff --git a/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/Debian.yml b/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/Debian.yml
@@ -16,6 +16,7 @@ Build_Tool_Packages:
  - g++
  - gcc
  - gettext
+ - gnutls-bin
  - libasound2-dev
  - libcapstone-dev
  - libcups2-dev
@@ -29,6 +30,9 @@ Build_Tool_Packages:
  - libgmp3-dev
  - libmpfr-dev
  - libmpfr-doc
+ - libnss3
+ - libnss3-dev
+ - libnss3-tools
  - libssl-dev
  - libwww-perl
  - libx11-dev
@@ -98,6 +102,10 @@ Test_Tool_Packages:
  - xvfb
  - binfmt-support
  - qemu-user-static
+ - gnutls-bin
+ - libnss3
+ - libnss3-dev
+ - libnss3-tools
 
 Test_Tool_Packages_x86_64:
  - pulseaudio

diff --git a/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/Fedora.yml b/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/Fedora.yml
@@ -27,8 +27,11 @@ Build_Tool_Packages:
  - glibc-common
  - glibc-devel
  - gmp-devel
+ - gnutls
+ - gnutls-utils
  - libcurl-devel
  - libffi-devel
+ - libnss3.so
  - libpng-devel
  - libXext-devel
  - libXi-devel # JDK12+ compilation
@@ -39,6 +42,8 @@ Build_Tool_Packages:
  - make
  - mesa-libGL-devel
  - mpfr-devel
+ - nss-devel
+ - nss-tools
  - openssl-devel
  - perl-devel
  - pkgconfig
@@ -128,3 +133,8 @@ Test_Tool_Packages:
  - expat-devel
  - libcurl-devel
  - mercurial
+ - gnutls
+ - gnutls-utils
+ - libnss3.so
+ - nss-devel
+ - nss-tools
diff --git a/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/RedHat.yml b/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/RedHat.yml
@@ -20,6 +20,8 @@ Build_Tool_Packages:
  - fontconfig-devel
  - freetype-devel
  - gnupg
+ - gnutls
+ - gnutls-utils
  - gcc
  - gcc-c++
  - gettext
@@ -29,6 +31,7 @@ Build_Tool_Packages:
  - gmp-devel
  - libcurl-devel
  - libffi-devel
+ - libnss3.so
  - libpng-devel
  - libXext-devel
  - libXi-devel # JDK12+ compilation
@@ -39,6 +42,8 @@ Build_Tool_Packages:
  - make
  - mesa-libGL-devel
  - mpfr-devel
+ - nss-devel
+ - nss-tools
  - openssl-devel
  - perl-devel
  - perl-IPC-Cmd # required for openssl v3 compiles
@@ -107,3 +112,8 @@ Test_Tool_Packages:
  - expat-devel
  - libcurl-devel
  - mercurial
+ - gnutls
+ - gnutls-utils
+ - libnss3.so
+ - nss-devel
+ - nss-tools
diff --git a/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/SLES.yml b/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/SLES.yml
@@ -16,9 +16,13 @@ Build_Tool_Packages:
  - gcc-c++
  - glibc
  - glibc-devel
+ - gnutls
  - libdw1
  - libelf1
  - make
+ - mozilla-nss
+ - mozilla-nss-devel
+ - mozilla-nss-tools
  - pkg-config
  - unzip
  - wget
@@ -92,3 +96,7 @@ Test_Tool_Packages:
  - xorg-x11-server
  - xorg-x11-server-extra
  - glibc-locale # Internationalization tests
+ - gnutls
+ - mozilla-nss
+ - mozilla-nss-devel
+ - mozilla-nss-tools
diff --git a/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/Ubuntu.yml b/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/Ubuntu.yml
@@ -18,6 +18,7 @@ Build_Tool_Packages:
  - gettext
  - git
  - gnupg
+ - gnutls-bin
  - libasound2-dev
  - libcapstone-dev
  - libcups2-dev
@@ -31,6 +32,9 @@ Build_Tool_Packages:
  - libgmp3-dev
  - libmpfr-dev
  - libmpfr-doc
+ - libnss3
+ - libnss3-tools
+ - libnss3-dev
  - libssl-dev
  - libwww-perl
  - libx11-dev
@@ -106,6 +110,10 @@ Test_Tool_Packages:
  - libexpat1-dev
  - libcurl4-openssl-dev
  - fakeroot
+ - gnutls-bin
+ - libnss3
+ - libnss3-tools
+ - libnss3-dev
 
 Test_Tool_Packages_x86_64:
  - pulseaudio
diff --git a/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/openSUSE.yml b/ansible/playbooks/AdoptOpenJDK_Unix_Playbook/roles/Common/vars/openSUSE.yml
@@ -18,12 +18,14 @@ Build_Tool_Packages:
  - gcc-c++
  - glibc
  - glibc-devel
+ - gnutls
  - libcurl-devel
  - libdw1
  - libdwarf-devel
  - libelf-devel
  - libelf0
  - libelf1
+ - libnss3.so
  - libnuma-devel
  - libpng15-devel
  - libXext-devel
@@ -33,6 +35,7 @@ Build_Tool_Packages:
  - libXt-devel
  - libXtst-devel
  - make
+ - mozilla-nss
  - ntp
  - numactl
  - pkg-config
@@ -63,3 +66,6 @@ Test_Tool_Packages:
  - xorg-x11
  - xorg-x11-devel
  - glibc-locale # Internationalization tests
+ - gnutls
+ - libnss3.so
+ - mozilla-nss
diff --git a/...ble/playbooks/AdoptOpenJDK_Unix_Playbook/roles/DockerStatic/Dockerfiles/Dockerfile.alp311 b/...ble/playbooks/AdoptOpenJDK_Unix_Playbook/roles/DockerStatic/Dockerfiles/Dockerfile.alp311
@@ -2,6 +2,10 @@ FROM alpine:3.11
 
 RUN apk --update add bash shadow openssh-server openssh-client unzip zip wget git curl make gcc perl xvfb \
  libxrender libxi libxtst procps musl-dev perl-doc alsa-lib libx11 msttcorefonts-installer fontconfig libxext freetype zlib fakeroot gnupg
+
+# Add SSL Test packages
+RUN apk --update add openssl gnutls gnutls-dev gnutls-utils nss nss-dev nss-tools
+
 RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -P ""
 
 ## Ensure Fonts Are Updated (Issue https://github.com/adoptium/infrastructure/issues/3039)

diff --git a/...ble/playbooks/AdoptOpenJDK_Unix_Playbook/roles/DockerStatic/Dockerfiles/Dockerfile.alp312 b/...ble/playbooks/AdoptOpenJDK_Unix_Playbook/roles/DockerStatic/Dockerfiles/Dockerfile.alp312
@@ -2,6 +2,10 @@ FROM alpine:3.12
 
 RUN apk --update add bash shadow openssh-server openssh-client unzip zip wget git curl make gcc perl xvfb \
  libxrender libxi libxtst procps musl-dev perl-doc alsa-lib libx11 msttcorefonts-installer fontconfig libxext freetype zlib fakeroot gnupg
+
+# Add SSL Test packages
+RUN apk --update add openssl gnutls gnutls-dev gnutls-utils nss nss-dev nss-tools
+
 RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -P ""
 
 ## Ensure Fonts Are Updated (Issue https://github.com/adoptium/infrastructure/issues/3039)

diff --git a/...ble/playbooks/AdoptOpenJDK_Unix_Playbook/roles/DockerStatic/Dockerfiles/Dockerfile.alp313 b/...ble/playbooks/AdoptOpenJDK_Unix_Playbook/roles/DockerStatic/Dockerfiles/Dockerfile.alp313
@@ -2,6 +2,10 @@ FROM alpine:3.13
 
 RUN apk --update add bash shadow openssh-server openssh-client unzip zip wget git curl make gcc perl xvfb \
  libxrender libxi libxtst procps musl-dev perl-doc alsa-lib libx11 msttcorefonts-installer fontconfig libxext freetype zlib fakeroot gnupg
+
+# Add SSL Test packages
+RUN apk --update add openssl gnutls gnutls-dev gnutls-utils nss nss-dev nss-tools
+
 RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -P ""
 
 ## Ensure Fonts Are Updated (Issue https://github.com/adoptium/infrastructure/issues/3039)

diff --git a/...ble/playbooks/AdoptOpenJDK_Unix_Playbook/roles/DockerStatic/Dockerfiles/Dockerfile.alp314 b/...ble/playbooks/AdoptOpenJDK_Unix_Playbook/roles/DockerStatic/Dockerfiles/Dockerfile.alp314
@@ -2,6 +2,10 @@ FROM alpine:3.14
 
 RUN apk --update add bash shadow openssh-server openssh-client unzip zip wget git curl make gcc perl xvfb \
  libxrender libxi libxtst procps musl-dev perl-doc alsa-lib libx11 msttcorefonts-installer fontconfig libxext freetype zlib fakeroot gnupg
+
+# Add SSL Test packages
+RUN apk --update add openssl gnutls gnutls-dev gnutls-utils nss nss-dev nss-tools
+
 RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -P ""
 
 ## Ensure Fonts Are Updated (Issue https://github.com/adoptium/infrastructure/issues/3039)