unparser: implement operator precedence algorithm for unparser

Backport operator precedence algorithm from here:
    python/cpython@397b96f

This eliminates unnecessary parentheses from our unparsed output and makes Spack's unparser
consistent with the one in upstream Python 3.9+, with one exception.

Our parser normalizes argument order when `py_ver_consistent` is set, so that star arguments
in function calls come last.  We have to do this because Python 2's AST doesn't have information
about their actual order.

If we ever support only Python 3.9 and higher, we can easily switch over to `ast.unparse`, as
the unparsing is consistent except for this detail (modulo future changes to `ast.unparse`)

COPYRIGHT

@@ -42,6 +42,10 @@ PackageName: argparse
 PackageHomePage: https://pypi.python.org/pypi/argparse
 PackageLicenseDeclared: Python-2.0
 
+PackageName: astunparse
+PackageHomePage: https://github.com/simonpercivall/astunparse
+PackageLicenseDeclared: Python-2.0
+
 PackageName: attrs
 PackageHomePage: https://github.com/python-attrs/attrs
 PackageLicenseDeclared: MIT
@@ -101,7 +105,3 @@ PackageLicenseDeclared: Apache-2.0 OR MIT
 PackageName: six
 PackageHomePage: https://pypi.python.org/pypi/six
 PackageLicenseDeclared: MIT
-
-PackageName: spack_astunparse
-PackageHomePage: https://github.com/simonpercivall/astunparse
-PackageLicenseDeclared: Python-2.0

lib/spack/external/__init__.py

@@ -31,6 +31,22 @@
   vendored copy ever needs to be updated again:
   https://github.com/spack/spack/pull/6786/commits/dfcef577b77249106ea4e4c69a6cd9e64fa6c418
 
+astunparse
+----------------
+
+* Homepage: https://github.com/simonpercivall/astunparse
+* Usage: Unparsing Python ASTs for package hashes in Spack
+* Version: 1.6.3 (plus modifications)
+* Note: This is in ``spack.util.unparse`` because it's very heavily
+  modified, and we want to track coverage for it.
+  Specifically, we have modified this library to generate consistent unparsed ASTs
+  regardless of the Python version. It is based on:
+    1. The original ``astunparse`` library;
+    2. Modifications for consistency;
+    3. Backports from the ``ast.unparse`` function in Python 3.9 and later
+  The unparsing is now mostly consistent with upstream ``ast.unparse``, so if
+  we ever require Python 3.9 or higher, we can drop this external package.
+
 attrs
 ----------------
 
@@ -138,21 +154,4 @@
 * Usage: Python 2 and 3 compatibility utilities.
 * Version: 1.16.0
 
-
-spack_astunparse
-----------------
-
-* Homepage: https://github.com/simonpercivall/astunparse
-* Usage: Unparsing Python ASTs for package hashes in Spack
-* Version: 1.6.3 (plus modifications)
-* Note: We have modified this library to generate consistent unparsed ASTs
-  regardless of the Python version. It contains the original ``astunparse``
-  library, as well as modifications for consistency. It also contains
-  backports from the ``ast.unparse`` function in Python 3.9 and later, so
-  that it will generate output consistent with the builtin ``ast.unparse``
-  function, in case we ever want to drop astunparse as an external
-  dependency.  Because we have modified the parsing (potentially at the
-  cost of round-trippability of the code), we call this ``spack_astunparse``
-  to avoid confusion with ``astunparse``.
-
 """

lib/spack/spack/cmd/license.py

@@ -34,8 +34,8 @@
     r'^bin/spack$',
     r'^bin/spack-python$',
 
-    # all of spack core
-    r'^lib/spack/spack/.*\.py$',
+    # all of spack core except unparse
+    r'^lib/spack/spack/(?!util/unparse).*\.py$',
     r'^lib/spack/spack/.*\.sh$',
     r'^lib/spack/spack/.*\.lp$',
     r'^lib/spack/llnl/.*\.py$',

lib/spack/spack/util/unparse/__init__.py

@@ -0,0 +1,19 @@
+# Copyright (c) 2014-2021, Simon Percivall and Spack Project Developers.
+#
+# SPDX-License-Identifier: Python-2.0
+# coding: utf-8
+
+from __future__ import absolute_import
+
+from six.moves import cStringIO
+
+from .unparser import Unparser
+
+__version__ = '1.6.3'
+
+
+def unparse(tree, py_ver_consistent=False):
+    v = cStringIO()
+    unparser = Unparser(py_ver_consistent=py_ver_consistent)
+    unparser.visit(tree, v)
+    return v.getvalue().strip() + "\n"