Running ara-manage is complaining about 'future feature annotations is not defined'

[Thulium-Drake]

What is the issue ?

• ara-manage (ara-api)
• RHEL 8
  • python 3.6.8 in the venv
  • ansible 2.14.2 (running on python 3.11.2)
• ARA 1.6.1 installed via the Collection (and PyPI) running on sqlite

Running ara-manage (anything, but the stack trace if from the SQL migrations task in the collection) results in the following stack trace:

TASK [recordsansible.ara.ara_api : Run SQL migrations] ****************************************************************************fatal: [lnx-av-m-a05-01.idm.svb.org]: FAILED! => changed=true 
  cmd:
  - ara-manage
  - migrate
  delta: '0:00:00.243187'
  end: '2023-11-20 12:25:35.494624'
  msg: non-zero return code
  rc: 1
  start: '2023-11-20 12:25:35.251437'
  stderr: |-
    Traceback (most recent call last):
      File "/opt/ara/api/virtualenv/bin/ara-manage", line 11, in <module>
        sys.exit(main())
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/ara/server/__main__.py", line 27, in main
        if not os.path.exists(settings.ARA_SETTINGS):
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/django/conf/__init__.py", line 82, in __getattr__
        self._setup(name)
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/django/conf/__init__.py", line 69, in _setup
        self._wrapped = Settings(settings_module)
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/django/conf/__init__.py", line 170, in __init__
        mod = importlib.import_module(self.SETTINGS_MODULE)
      File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 994, in _gcd_import
      File "<frozen importlib._bootstrap>", line 971, in _find_and_load
      File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 678, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/ara/server/settings.py", line 10, in <module>
        from dynaconf import LazySettings
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/dynaconf/__init__.py", line 1, in <module>
        from dynaconf.base import LazySettings  # noqa
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/dynaconf/base.py", line 11, in <module>
        from dynaconf import default_settings
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/dynaconf/default_settings.py", line 10, in <module>
        from dynaconf.utils.parse_conf import parse_conf_data
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/dynaconf/utils/parse_conf.py", line 11, in <module>
        from dynaconf.utils.boxing import DynaBox
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/dynaconf/utils/boxing.py", line 7, in <module>
        from dynaconf.vendor.box import Box
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/dynaconf/vendor/box/__init__.py", line 3, in <module>
        from .box import Box
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/dynaconf/vendor/box/box.py", line 33, in <module>
        from .converters import _to_json,_from_json,_from_toml,_to_toml,_from_yaml,_to_yaml,BOX_PARAMETERS
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/dynaconf/vendor/box/converters.py", line 10, in <module>
        import dynaconf.vendor.ruamel.yaml as yaml
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/dynaconf/vendor/ruamel/yaml/__init__.py", line 8, in <module>    
        try:from .cyaml import *;__with_libyaml__=True
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/dynaconf/vendor/ruamel/yaml/cyaml.py", line 3, in <module>       
        from _ruamel_yaml import CParser,CEmitter
      File "_ruamel_yaml.pyx", line 17, in init ruamel.yaml.clib._ruamel_yaml
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/ruamel/yaml/__init__.py", line 50, in <module>
        from .cyaml import *  # NOQA
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/ruamel/yaml/cyaml.py", line 5, in <module>
        from ruamel.yaml.constructor import Constructor, BaseConstructor, SafeConstructor
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/ruamel/yaml/constructor.py", line 16, in <module>
        from ruamel.yaml.compat import (builtins_module, # NOQA
      File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/ruamel/yaml/compat.py", line 3
        from __future__ import annotations
        ^
    SyntaxError: future feature annotations is not defined
  stderr_lines: <omitted>
  stdout: ''
  stdout_lines: <omitted>

Note, the reported version in the ARA UI is 1.6.1, so the application itself seems to run just fine, but the CLI tool has borked out.

What should be happening ?

Not barf out errors and run migrations if they are needed :-)

[flowerysong]

This does not appear to be an ARA issue. You have installed a version of ruamel.yaml that is not compatible with Python 3.6.

     File "/opt/ara/api/virtualenv/lib64/python3.6/site-packages/ruamel/yaml/compat.py", line 3
        from __future__ import annotations

[dmsimard]

@Thulium-Drake I have not come across this issue before. Are you still able to reproduce it ?

Could you also share an output of a pip freeze ?

It would also be interesting to know if you can reproduce the issue on the latest version (1.7.0) -- note that it bumped the python requirement to >=3.8.

Thanks.

[Thulium-Drake]

I will get back to you in the new year as I don't have access to the environment due to the holidays :)

[martl20]

Thanks for the tip. I checked https://pypi.org/project/ruamel.yaml/ and python3.6 is indeed not compatible with the latest ruamel version. This has to be not newer then 0.17.21 .The downgrade worked for now, but for sure we need upgrading.

[Thulium-Drake]

Sorry for the late reply... This was still on my todo, as promised! This is the output for the same user that runs the service.

[ansible@lnx-av-m-a05-01 ~]$ pip freeze
ansible-merge-vars==5.0.0
ara==1.5.8
attrs==22.1.0
autopage==0.5.1
Babel==2.5.1
cffi==1.11.5
chardet==3.0.4
cliff==3.10.1
cmd2==2.4.2
cryptography==3.2.1
dbus-python==1.2.4
decorator==4.2.1
dnspython==1.15.0
elementpath==2.4.0
ethtool==0.14
file-magic==0.3.0
gpg==1.13.1
gssapi==1.5.1
idna==2.5
importlib-metadata==4.8.3
iniparse==0.4
ipaclient==4.9.12
ipalib==4.9.12
ipaplatform==4.9.12
ipapython==4.9.12
isc==2.0
Jinja2==2.10.1
json2html==1.3.0
jwcrypto==0.5.0
katello-host-tools==4.2.3
libcomps==0.1.18
MarkupSafe==0.23
netaddr==0.7.19
netifaces==0.10.6
nftables==0.1
passlib==1.7.2
pbr==5.10.0
perf==0.1
pexpect==4.3.1
ply==3.9
prettytable==2.5.0
ptyprocess==0.5.2
pyasn1==0.3.7
pyasn1-modules==0.1.5
pycparser==2.14
PyGObject==3.28.3
pyinotify==0.9.6
pyparsing==3.0.9
pyperclip==1.8.2
PySocks==1.6.8
python-augeas==0.5.0
python-dateutil==2.6.1
python-ldap==3.3.1
python-linux-procfs==0.7.0
python-yubico==1.3.2
pytz==2017.2
pyudev==0.21.0
pyusb==1.0.0
PyYAML==3.12
qrcode==5.1
requests==2.20.0
rpm==4.14.3
ruamel.yaml==0.17.20
ruamel.yaml.clib==0.2.8
selinux==2.9
sepolicy==1.1
setools==4.3.0
six==1.16.0
slip==0.6.4
slip.dbus==0.6.4
sos==4.6.0
SSSDConfig==2.9.1
stevedore==3.5.0
subscription-manager==1.28.40
syspurpose==1.28.40
systemd-python==234
typing_extensions==4.1.1
urllib3==1.24.2
wcwidth==0.2.5
xmlschema==1.9.2
zipp==3.6.0

And as my colleague mentioned above, this is most likely caused by ruamel.yaml being too new for RHEL 8.8's Python 3.6.8.

Though I have to say, I never tried switching to an AppStream for that... I'll give that a shot :)

EDIT: Ah, I found why that doesn't work, the role uses the default /usr/bin/python3 as the interpreter for all venv commands. I'm working on a PR that will make that variable