[Bug]: ImportError: /usr/lib/python3.9/site-packages/wolfcrypt/_ffi.abi3.so: undefined symbol: wc_DerToPemEx

[stanislaw]

Contact Details

[email protected]

Version

Cloned from GitHub. Commit: 03ed52b

When I switch to v5.6.6-stable, I still get the same import error.

Description

Following up on a conversation with a wolfSSL representative at Embedded World 2024, I am trying to improve a performance of our encryption code that uses the wolfcrypt Python package.

We are using PetaLinux 2022.1 which is running on Xilinx UltraScale+. I am compiling wolfssl using the default instructions (provided below) and getting an import error (see the relevant logs below).

I tried to build both the latest version of wolfssl repository as well as the version that is used by the Pip package, which is v5.6.6-stable. The result is the same, I am getting the import error.

At the same time, when I install a Pip package wolfcrypt, the latest version 5.6.6.post0, without manually building wolfssl myself, I confirm that the library is working correctly.

I am new to building wolfssl, so maybe I am missing something trivial. I tried adding --enable-asn with my configure step because I thought that the missing symbol wc_DerToPemEx had to do with a missing ASN support but I still got the same import error.

Thanks for reading this. I would be thankful for any advice.

Reproduction steps

git clone https://github.com/wolfssl/wolfssl.git --depth=1
cd wolfssl
./autogen.sh
./configure
make && make install
USE_LOCAL_WOLFSSL=/usr/local/ pip install wolfcrypt

I run my Python like this:

LD_LIBRARY_PATH=/usr/local/lib/ python my_script.py

Relevant log output

The `main` branch's latest commit that I am building from:


commit 03ed52bd816af7a8b5d80384c2ce05b182d51528 (grafted, HEAD -> master, origin/master, origin/HEAD)
Author: JacobBarthelmeh <[email protected]>
Date:   Tue Apr 16 17:56:55 2024 -0600

    Merge pull request #7297 from SparkiDev/ssl_move_5
    
    ssl.c: Move functions out to separate files

The import error that I am getting:

  File "<REDACTED.py>", line 6, in <module>
    from wolfcrypt.ciphers import AesGcmStream, RsaPublic
  File "/usr/lib/python3.9/site-packages/wolfcrypt/__init__.py", line 47, in <module>
    from wolfcrypt._ffi import ffi as _ffi
ImportError: /usr/lib/python3.9/site-packages/wolfcrypt/_ffi.abi3.so: undefined symbol: wc_DerToPemEx

[stanislaw]

I have diffed the configurations used for the manual step (left side) and by the Python package (right side) and I am not experienced enough to guess what could be causing my import error:

$ diff ~/SSL_LEFT_MANUAL_IMPORT_ERROR.txt ~/SSL_RIGHT_WORKING_PYTHON.txt 
3c3
< * Installation prefix:        /usr/local
---
> * Installation prefix:        /tmp/pip-install-dkn94k5z/wolfcrypt_8bb0ec0fb3c84dc8a38c484f47a2afab/lib/wolfssl/linux-aarch64/v5.6.6-stable
8,14c8,14
< * C++ Compiler:               
< * C++ Flags:                  
< * CPP Flags:                  
< * CCAS Flags:                   
< * LD Flags:                   
< * LIB Flags:                   -pie -z relro -z now -Werror 
< * Library Suffix:             
---
> * C++ Compiler:
> * C++ Flags:
> * CPP Flags:
> * CCAS Flags:
> * LD Flags:
> * LIB Flags:                   -pie -z relro -z now -Werror
> * Library Suffix:
17c17
< * Coverage enabled:           
---
> * Coverage enabled:
22c22
< Features 
---
> Features
29c29
< * Old Names:                  yes
---
> * Old Names:                  no
47c47
< * AES-GCM streaming:          no
---
> * AES-GCM streaming:          yes
49c49
< * AES-CTR:                    no
---
> * AES-CTR:                    yes
56c56
< * DES3:                       no
---
> * DES3:                       yes
66c66
< * MD5:                        yes
---
> * MD5:                        no
69c69
< * SHA-224:                    yes
---
> * SHA-224:                    no
74c74
< * SHAKE256:                   no
---
> * SHAKE256:                   yes
80c80
< * keygen:                     no
---
> * keygen:                     yes
92c92
< * wolfCrypt Only:             no
---
> * wolfCrypt Only:             yes
95c95
< * X9.63 KDF:                  no
---
> * X9.63 KDF:                  yes
98c98
< * Poly1305:                   yes
---
> * Poly1305:                   no
104c104
< * DH:                         yes
---
> * DH:                         no
113,114c113,114
< * CURVE25519:                 no
< * ED25519:                    no
---
> * CURVE25519:                 yes
> * ED25519:                    yes
117c117
< * ED448:                      no
---
> * ED448:                      yes
154c154
< * ERROR_STRINGS:              yes
---
> * ERROR_STRINGS:              no
164,165c164,165
< * TLS v1.2:                   yes
< * TLS v1.3:                   yes
---
> * TLS v1.2:                   no
> * TLS v1.3:                   no
191c191
< * Extended Master Secret:     yes
---
> * Extended Master Secret:     no
198c198
< * PKCS#7:                     no
---
> * PKCS#7:                     yes
212,213c212,213
< * Examples:                   yes
< * Crypt tests:                yes
---
> * Examples:                   no
> * Crypt tests:                no
226c226
< * AES Key Wrap:               no
---
> * AES Key Wrap:               yes
244a245
> 

[kareem-wolfssl]

Hi @stanislaw ,

wc_DerToPemEx requires that WOLFSSL_DER_TO_PEM is defined, and NO_ASN/NO_CERTS are not defined. Using configure, you can enable either certgen or keygen: --enable-certgen or --enable-keygen to enable this flag.

[stanislaw]

Hi @kareem-wolfssl,

thanks a lot for your answer. Right now, I have tried the options:

./configure --enable-certgen --enable-aesgcm --enable-aesgcm-stream --enable-aesctr --enable-aeskeywrap

The previous error has gone but now I am getting:

  File "<...>", line 6, in <module>
    from wolfcrypt.ciphers import AesGcmStream, RsaPublic
  File "/usr/lib/python3.9/site-packages/wolfcrypt/__init__.py", line 47, in <module>
    from wolfcrypt._ffi import ffi as _ffi
ImportError: /usr/lib/python3.9/site-packages/wolfcrypt/_ffi.abi3.so: undefined symbol: wc_Sha512Final

I have already tried the option of --enable-sha512 but the result is still the same.
Please advise.

---

UPDATED LATER:

Following this issue #7116, I have switched to the latest master and the undefined symbol: wc_Sha512Final is gone now. The above command is confirmed to work on UltraScale PetaLinux 22.1 when run against this commit:

Merge: 1f61ed353 8e9810e87
Author: JacobBarthelmeh <[email protected]>
Date:   Tue Apr 16 17:56:55 2024 -0600

    Merge pull request #7297 from SparkiDev/ssl_move_5
    
    ssl.c: Move functions out to separate files

The issue that I am facing now is that the encryption is still very slow but I will open a separate issue to track that.

Before I close this issue, is there anything that I can do to support documenting the missing information as per your comment? It would have been great to see this documented somewhere.

Thanks.