ERROR:root:module 'pyparsing' has no attribute 'downcaseTokens'

[mattiasr]

Tried to install misp via our chef installation 2 days ago, noticed it didn't work. after some log digging i found out that this is the problem.

misp-modules[156554]: ERROR:root:module 'pyparsing' has no attribute 'downcaseTokens'
misp-modules[156554]: Traceback (most recent call last):
misp-modules[156554]:   File "/opt/misp/install/venv/lib/python3.8/site-packages/misp_modules/__init__.py", line 41, in <module>
misp-modules[156554]:     from .modules import *  # noqa
misp-modules[156554]:   File "/opt/misp/install/venv/lib/python3.8/site-packages/misp_modules/modules/__init__.py", line 1, in <module>
misp-modules[156554]:     from .expansion import *  # noqa
misp-modules[156554]:   File "/opt/misp/install/venv/lib/python3.8/site-packages/misp_modules/modules/expansion/vulndb.py", line 9, in <module>
misp-modules[156554]:     import oauth2 as oauth
misp-modules[156554]:   File "/opt/misp/install/venv/lib/python3.8/site-packages/oauth2/__init__.py", line 31, in <module>
misp-modules[156554]:     import httplib2
misp-modules[156554]:   File "/opt/misp/install/venv/lib/python3.8/site-packages/httplib2/__init__.py", line 52, in <module>
misp-modules[156554]:     from . import auth
misp-modules[156554]:   File "/opt/misp/install/venv/lib/python3.8/site-packages/httplib2/auth.py", line 20, in <module>
misp-modules[156554]:     auth_param_name = token.copy().setName("auth-param-name").addParseAction(pp.downcaseTokens)
misp-modules[156554]: AttributeError: module 'pyparsing' has no attribute 'downcaseTokens'

It seems that pyparsing has released a 3.x major version and misp-modules doesn't work with that.

pip install pyparsing==2.4.7 works as a workaround

Installing collected packages: pyparsing
  Attempting uninstall: pyparsing
    Found existing installation: pyparsing 3.0.6
    Uninstalling pyparsing-3.0.6:
      Successfully uninstalled pyparsing-3.0.6
Successfully installed pyparsing-2.4.7

[adulau]

Nice catch. I suppose replacing it with

https://pythonhosted.org/pyparsing/pyparsing.pyparsing_common-class.html#downcaseTokens

from downcaseTokens to pyparsing_common.downcaseTokens might work out of the box.

I'll check to do a variable import depending of the version found.

[adulau]

Looking a bit deeper into it, it's related to the oauth2 library but the current requirements file is already with pyparsing==2.4.7 - did you use the requirement file? or another one?

[mattiasr]

I installed with the following lines

    sudo git clone https://github.com/MISP/misp-modules.git
    sudo git clone https://github.com/stricaud/faup.git
    sudo git clone https://github.com/stricaud/gtcaca.git gtcaca
    sudo mkdir -p gtcaca/build
    sudo mkdir -p faup/build
    sudo chown -R misp:misp faup gtcaca
    cd gtcaca/build
    sudo cmake ..
    sudo make
    sudo make install
    cd ../../faup/build
    sudo cmake ..
    sudo make
    sudo make install
    sudo ldconfig
    cd /opt/misp/install/misp-modules
    sudo apt install libpq5 libjpeg-dev tesseract-ocr libpoppler-cpp-dev imagemagick libopencv-dev zbar-tools libzbar0 libzbar-dev libfuzzy-dev -y
    sudo chgrp www-data .
    sudo chmod og+w .
    sudo -u www-data /opt/misp/install/venv/bin/pip install -I -r REQUIREMENTS
    sudo chgrp staff .
    sudo -u www-data /opt/misp/install/venv/bin/pip install -I .
    sudo -u www-data /opt/misp/install/venv/bin/pip install censys pyfaup
    ```
    
    Might be that some other deps has a non-version deps?

[mattiasr]

It might be related to installing faup when installing misp using the INSTALL.sh script https://raw.githubusercontent.com/MISP/MISP/2.4/INSTALL/INSTALL.sh

Collecting pyparsing!=3.0.5,>=2.0.2

[adulau]

I'll close the issue as a new deployment/installation is available in misp-modules using pypi or poetry. Feel free to reopen, if it doesn't solve the issue.