[SW-2789] Remove Python 2.7 support (#5625)

krasinski · web-flow · commit 1123672512f6 · 2023-06-06T14:16:46.000+02:00
* [SW-2789] Remove Python 2.7 support
diff --git a/bin/sparkling-env.sh b/bin/sparkling-env.sh
@@ -67,7 +67,6 @@ function checkPythonPackages() {
     error=0
     checkPythonPackage "$packages" "requests"
     checkPythonPackage "$packages" "tabulate"
-    checkPythonPackage "$packages" "future" "0.4.0"
 
     if [ $error == -1 ]; then
         exit -1
diff --git a/booklet/src/sections/deployment.tex b/booklet/src/sections/deployment.tex
@@ -195,15 +195,14 @@ \subsubsection{Running PySparkling}
 \item PySparkling zip file
 \item Python Module: request
 \item Python Module: tabulate
-\item Python Module: future
 \end{itemize}
 
 \textbf{Steps}:
 \begin{enumerate}
 \item Create a new Python library containing the PySparkling zip file.
 \item Download the selected Sparkling Water version from \url{https://www.h2o.ai/download/}.
 \item The PySparkling zip file is located in the sparkling water zip file at the following location: `py/h2o\_pysparkling\_*.zip.`
-\item Create libraries for the following python modules: request, tabulate and future.
+\item Create libraries for the following python modules: request, tabulate.
 \item Attach the PySparkling library and python modules to the cluster.
 \item Create a new python notebook.
 \item Create an H2O cluster inside the Spark cluster:
diff --git a/booklet/src/sections/intro.tex b/booklet/src/sections/intro.tex
@@ -145,14 +145,14 @@
 
     \subsection{Sparkling Water Requirements}
 
-    Sparkling Water supports Spark 2.1, 2.2, 2.3, 2.4 (except Spark 2.4.2) and 3.0. In specific examples of this
+    Sparkling Water supports Spark 2.3, 2.4 (except Spark 2.4.2) and 3.0. In specific examples of this
     booklet we refer to artifacts for Spark 3.0 and use Sparkling Water 3.30.0.7, however, the Sparkling Water code is
     consistent across versions for each Spark.
 
     \begin{itemize}
         \item Linux/OS X/Windows
-        \item Java 1.8 or higher
-        \item Python 2.7+ For Python version of Sparkling Water (PySparkling)
+        \item Java 8 or higher
+        \item Python SUBST_MIN_SUPPORTED_PYTHON+ For Python version of Sparkling Water (PySparkling)
         \item R 3.4+ for R version of Sparkling Water (RSparkling)
         \item Installed Spark and have SPARK\_HOME environmental variable pointing to its home.
     \end{itemize}
diff --git a/booklet/src/sections/starting.tex b/booklet/src/sections/starting.tex
@@ -14,7 +14,6 @@ \subsection{Setting up the Environment}
 \begin{itemize}
     \item requests
     \item tabulate
-    \item future
 \end{itemize}.
 Also please make sure that your Python environment is set-up to run regular Spark applications.
 
diff --git a/doc/src/site/sphinx/deployment/azure_hdi.rst b/doc/src/site/sphinx/deployment/azure_hdi.rst
@@ -15,7 +15,7 @@ The H2O Artificial Intelligence for Azure HDInsight is an application you can in
 Create the H2O AI for Azure HDInsight
 '''''''''''''''''''''''''''''''''''''
 
-**Requirement**: Python 2.7 or 3.
+**Requirements**: Python SUBST_MIN_SUPPORTED_PYTHON+.
 
 Follow the steps below to create a new H2O Artificial Intelligence for Azure HDInsight.
 
diff --git a/doc/src/site/sphinx/deployment/sw_google_cloud_dataproc.rst b/doc/src/site/sphinx/deployment/sw_google_cloud_dataproc.rst
@@ -19,7 +19,7 @@ In this tutorial we will use Dataproc image version 2.0-debian10 which has Spark
     --region $GCP_REGION \
     --image-version 2.0-debian10 \
     --num-workers 3 \
-    --properties='^#^dataproc:pip.packages=tabulate==0.8.3,requests==2.21.0,future==0.17.1'
+    --properties='^#^dataproc:pip.packages=tabulate==0.8.3,requests==2.21.0'
 
 .. content-tabs::
 
diff --git a/doc/src/site/sphinx/pysparkling.rst b/doc/src/site/sphinx/pysparkling.rst
@@ -71,7 +71,7 @@ For example, to install PySparkling for Spark SUBST_SPARK_MAJOR_VERSION, the com
 Dependencies
 ------------
 
-Supported Python versions are Python 2.7 or Python 3+.
+Supported Python versions are Python SUBST_MIN_SUPPORTED_PYTHON+.
 
 The major dependency is Spark. Please make sure that your Python environment has functional Spark with all its
 dependencies.
@@ -80,7 +80,6 @@ dependencies.
 
   $ pip install requests
   $ pip install tabulate
-  $ pip install future
 
 These dependencies are installed automatically in case PySparkling is installed from PyPI.
 
diff --git a/gradle-spark2.1.properties b/gradle-spark2.1.properties
@@ -6,4 +6,4 @@ scalaVersion=2.11.12
 fabricK8sClientVersion=4.6.4
 executorOverheadMemoryOption=spark.yarn.executor.memoryOverhead
 driverOverheadMemoryOption=spark.yarn.driver.memoryOverhead
-supportedPythonVersions=2.7 3.6
+supportedPythonVersions=3.6
diff --git a/gradle-spark2.3.properties b/gradle-spark2.3.properties
@@ -6,4 +6,4 @@ scalaVersion=2.11.12
 fabricK8sClientVersion=4.6.4
 executorOverheadMemoryOption=spark.executor.memoryOverhead
 driverOverheadMemoryOption=spark.driver.memoryOverhead
-supportedPythonVersions=2.7 3.6 3.7
+supportedPythonVersions=3.6 3.7
diff --git a/gradle-spark2.4.properties b/gradle-spark2.4.properties
@@ -7,4 +7,4 @@ databricksVersion=6.6.x-cpu-ml-scala2.11
 fabricK8sClientVersion=4.6.4
 executorOverheadMemoryOption=spark.executor.memoryOverhead
 driverOverheadMemoryOption=spark.driver.memoryOverhead
-supportedPythonVersions=2.7 3.6 3.7
+supportedPythonVersions=3.6 3.7
diff --git a/gradle-spark3.0.properties b/gradle-spark3.0.properties
@@ -7,4 +7,4 @@ databricksVersion=7.3.x-cpu-ml-scala2.12
 fabricK8sClientVersion=4.9.2
 executorOverheadMemoryOption=spark.executor.memoryOverhead
 driverOverheadMemoryOption=spark.driver.memoryOverhead
-supportedPythonVersions=2.7 3.6 3.7 3.8 3.9
+supportedPythonVersions=3.6 3.7 3.8 3.9
diff --git a/gradle.properties b/gradle.properties
@@ -25,7 +25,7 @@ isNightlyBuild=false
 # Supported Major Spark Versions
 supportedSparkVersions=2.3 2.4 3.0 3.1 3.2 3.3
 # The list of python environments used in automated tests
-pythonEnvironments=2.7 3.6 3.7 3.8 3.9
+pythonEnvironments=3.6 3.7 3.8 3.9
 # Select for which Spark version is Sparkling Water built by default
 spark=3.3
 # Sparkling Water Version
diff --git a/py-scoring/src/setup.cfg b/py-scoring/src/setup.cfg
@@ -1,7 +1 @@
 [bdist_wheel]
-# This flag says that the code is written to work on both Python 2 and Python
-# 3.
-# I.e.:
-#   1. Your project runs on Python 2 and 3 with no changes (i.e. it does not require 2to3).
-#   2. Your project does not have any C extensions.
-universal=1
diff --git a/py/build.gradle b/py/build.gradle
@@ -35,7 +35,6 @@ python {
   pip "pytest:4.6.9" // For running tests
   pip "tabulate:0.8.3"
   pip "requests:2.21.0"
-  pip "future:0.17.1"
   pip "numpy:${numpyVersion}"
   pip "pyspark:${sparkVersion}"
   if (project.hasProperty("pythonEnvBasePath")) {
diff --git a/py/conda/h2o_pysparkling_SUBST_SPARK_MAJOR_VERSION/meta.yaml b/py/conda/h2o_pysparkling_SUBST_SPARK_MAJOR_VERSION/meta.yaml
@@ -10,7 +10,6 @@ requirements:
     - python
     - pip >=9.0.1
     - setuptools
-    - future >=0.15.2
     - tabulate >=0.7.5
     - requests >=2.10
 
diff --git a/py/src/setup.cfg b/py/src/setup.cfg
@@ -1,7 +1 @@
 [bdist_wheel]
-# This flag says that the code is written to work on both Python 2 and Python
-# 3.
-# I.e.:
-#   1. Your project runs on Python 2 and 3 with no changes (i.e. it does not require 2to3).
-#   2. Your project does not have any C extensions.
-universal=1
diff --git a/py/src/setup.py b/py/src/setup.py
@@ -40,8 +40,7 @@
     # run-time dependencies
     install_requires=[
         'requests',
-        'tabulate',
-        'future'],
+        'tabulate'],
 
     # bundled binary packages
     package_data={'sparkling_water': ['*.jar'],
diff --git a/templates/src/aws/install_sparkling_water.sh b/templates/src/aws/install_sparkling_water.sh
@@ -15,7 +15,6 @@ sudo ln -sf /usr/local/bin/pip2.7 /usr/bin/pip
 # Install PySparkling Dependencies
 sudo python -m pip install -U requests
 sudo python -m pip install -U tabulate
-sudo python -m pip install -U future
 sudo python -m pip install -U six
 
 # Install Scikit Learn
diff --git a/templates/src/terraform/aws/modules/emr_deployment/main.tf b/templates/src/terraform/aws/modules/emr_deployment/main.tf
@@ -38,16 +38,9 @@ set -x -e
 
 sudo python3 -m pip install -U requests
 sudo python3 -m pip install -U tabulate
-sudo python3 -m pip install -U future
 sudo python3 -m pip install -U six
 sudo python3 -m pip install -U scikit-learn
 
-sudo python2.7 -m pip install -U requests
-sudo python2.7 -m pip install -U tabulate
-sudo python2.7 -m pip install -U future
-sudo python2.7 -m pip install -U six
-sudo python2.7 -m pip install -U scikit-learn
-
 mkdir -p /home/hadoop/h2o
 cd /home/hadoop/h2o
 
@@ -57,7 +50,6 @@ unzip -o sparkling-water-${var.sw_version}.zip 1> /dev/null & wait
 
 PYSPARKLING_ZIP=$(find /home/hadoop/h2o/ -name h2o_pysparkling_*.zip)
 sudo python3 -m pip install $PYSPARKLING_ZIP
-sudo python2.7 -m pip install $PYSPARKLING_ZIP
 
 export MASTER="yarn-client"
 EOF
