Testsuite fails on in a chroot on qemu/kvm VM

[xnox]

make  check-TESTS
make[3]: Entering directory '/<<PKGBUILDDIR>>/test'
make[4]: Entering directory '/<<PKGBUILDDIR>>/test'
FAIL: des-cbc-test.pl
FAIL: des-ofb-test.pl
FAIL: des-cfb-test.pl
FAIL: des-ecb-test.pl
FAIL: 3des-cbc-test.pl
FAIL: 3des-ecb-test.pl
FAIL: 3des-cfb-test.pl
FAIL: 3des-ofb-test.pl
FAIL: aes-128-ecb-test.pl
FAIL: aes-128-cbc-test.pl
FAIL: aes-128-cfb-test.pl
FAIL: aes-192-ecb-test.pl
FAIL: aes-128-ofb-test.pl
FAIL: aes-192-cbc-test.pl
FAIL: aes-192-cfb-test.pl
FAIL: aes-192-ofb-test.pl
FAIL: aes-256-cbc-test.pl
FAIL: aes-256-ecb-test.pl
FAIL: aes-256-ofb-test.pl
PASS: aes-256-cfb-test.pl
==============================================
   openssl-ibmca 2.0.0: test/test-suite.log
==============================================

# TOTAL: 20
# PASS:  1
# SKIP:  0
# XFAIL: 0
# FAIL:  19
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: des-ecb-test.pl
=====================

unable to write 'random state'
unable to write 'random state'
FAIL des-ecb-test.pl (exit status: 1)

FAIL: des-cbc-test.pl
=====================

unable to write 'random state'
bad decrypt
4396773508896:error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt:../crypto/evp/evp_enc.c:541:
FAIL des-cbc-test.pl (exit status: 1)

FAIL: des-cfb-test.pl
=====================

unable to write 'random state'
FAIL des-cfb-test.pl (exit status: 1)

FAIL: des-ofb-test.pl
=====================

unable to write 'random state'
FAIL des-ofb-test.pl (exit status: 1)

FAIL: 3des-ecb-test.pl
======================

unable to write 'random state'
bad decrypt
4396134926112:error:0606506D:digital envelope routines:EVP_DecryptFinal_ex:wrong final block length:../crypto/evp/evp_enc.c:525:
unable to write 'random state'
bad decrypt
4396630378272:error:0606506D:digital envelope routines:EVP_DecryptFinal_ex:wrong final block length:../crypto/evp/evp_enc.c:525:
FAIL 3des-ecb-test.pl (exit status: 1)

FAIL: 3des-cbc-test.pl
======================

unable to write 'random state'
unable to write 'random state'
bad decrypt
4396428003104:error:0606506D:digital envelope routines:EVP_DecryptFinal_ex:wrong final block length:../crypto/evp/evp_enc.c:525:
FAIL 3des-cbc-test.pl (exit status: 1)

FAIL: 3des-cfb-test.pl
======================

unable to write 'random state'
unable to write 'random state'
FAIL 3des-cfb-test.pl (exit status: 1)

FAIL: 3des-ofb-test.pl
======================

unable to write 'random state'
unable to write 'random state'
cmp: EOF on data.in which is empty
FAIL 3des-ofb-test.pl (exit status: 1)

FAIL: aes-128-ecb-test.pl
=========================

unable to write 'random state'
bad decrypt
4396508219168:error:0606506D:digital envelope routines:EVP_DecryptFinal_ex:wrong final block length:../crypto/evp/evp_enc.c:525:
FAIL aes-128-ecb-test.pl (exit status: 1)

FAIL: aes-128-cbc-test.pl
=========================

unable to write 'random state'
cmp: EOF on data.dec which is empty
FAIL aes-128-cbc-test.pl (exit status: 1)

FAIL: aes-128-cfb-test.pl
=========================

unable to write 'random state'
unable to write 'random state'
FAIL aes-128-cfb-test.pl (exit status: 1)

FAIL: aes-128-ofb-test.pl
=========================

unable to write 'random state'
unable to write 'random state'
cmp: EOF on data.dec which is empty
FAIL aes-128-ofb-test.pl (exit status: 1)

FAIL: aes-192-ecb-test.pl
=========================

unable to write 'random state'
cmp: EOF on data.dec which is empty
FAIL aes-192-ecb-test.pl (exit status: 1)

FAIL: aes-192-cbc-test.pl
=========================

unable to write 'random state'
cmp: EOF on data.dec which is empty
FAIL aes-192-cbc-test.pl (exit status: 1)

FAIL: aes-192-cfb-test.pl
=========================

unable to write 'random state'
unable to write 'random state'
FAIL aes-192-cfb-test.pl (exit status: 1)

FAIL: aes-192-ofb-test.pl
=========================

unable to write 'random state'
FAIL aes-192-ofb-test.pl (exit status: 1)

FAIL: aes-256-ecb-test.pl
=========================

unable to write 'random state'
unable to write 'random state'
FAIL aes-256-ecb-test.pl (exit status: 1)

FAIL: aes-256-cbc-test.pl
=========================

unable to write 'random state'
bad decrypt
4395925210912:error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt:../crypto/evp/evp_enc.c:536:
FAIL aes-256-cbc-test.pl (exit status: 1)

FAIL: aes-256-ofb-test.pl
=========================

unable to write 'random state'
FAIL aes-256-ofb-test.pl (exit status: 1)

============================================================================
Testsuite summary for openssl-ibmca 2.0.0
============================================================================
# TOTAL: 20
# PASS:  1
# SKIP:  0
# XFAIL: 0
# FAIL:  19
# XPASS: 0
# ERROR: 0
============================================================================
See test/test-suite.log
Please report to [email protected]
============================================================================

This is on Ubuntu Cosmic (to become 18.10) as built in launchpad PPA. Note anybody can use PPAs on launchpad and activate builds for s390x.

When doing a similar build, but in chroot on a z/VM, things are slightly better:

============================================================================
Testsuite summary for openssl-ibmca 2.0.0
============================================================================
# TOTAL: 20
# PASS:  0
# SKIP:  20
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================

So no tests are actually run, and everything is skipped.

Regular user build (non-chrooted) on z/VM goes fine and results in full test suite pass:

============================================================================
Testsuite summary for openssl-ibmca 2.0.0
============================================================================
# TOTAL: 20
# PASS:  20
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================

[p-steuer]

The test cases a simple script (test.pm) that use the openssl command line utility to perform pairwise-consistency tests: 1. Enc/Dec a tmp file configured to use the ibmca openssl-engine (which in turn uses s390x crypto hw via libica). 2. Dec/Enc the result using openssl (w/o engine configuration), 3. compare results. At the moment, the testcases operate on the same temporary input/output files, so they cant be run concurrently.

Regrading the test case failures:

unable to write 'random state'

is an openssl error ( https://www.openssl.org/docs/faq.html#USER2 ).

Regarding the skipped test cases:

A test is only skipped, if the engine could not be configured. This is checked by parsing the output of

OPENSSL_CONF=${srcdir}/openssl-test.cnf openssl engine -c

output must contain "ibmca", srcdir is from the autotools test framwork

[xnox]

thank you for details. I shall check what exactly why those things are failing here.

[ifranzki]

Is this one still relevant?

[xnox]

I think this was resolved some time ago:

============================================================================
Testsuite summary for openssl-ibmca 2.4.1
============================================================================
# TOTAL: 13
# PASS:  10
# SKIP:  3
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0

As seen in build log of https://launchpad.net/ubuntu/+source/openssl-ibmca/2.4.1-0ubuntu1/+build/27674210 at https://launchpadlibrarian.net/710728827/buildlog_ubuntu-noble-s390x.openssl-ibmca_2.4.1-0ubuntu1_BUILDING.txt.gz which is the current build of openssl-ibmca since Noble onwards.