[Bug] Test failed: when running make installcheck-world

[tuhaihe]

Apache Cloudberry version

main

What happened

When after building the Cloudberry based on the main branch, running the PGCLIENTENCODING=UTF8 PGOPTIONS='-c optimizer=off' make installcheck-world command, returned:

=======================
 1 of 16 tests failed.
=======================

The differences that caused some tests to fail can be viewed in the
file "/home/gpadmin/cloudberry/src/test/isolation2/regression.diffs".  A copy of the test summary that you see
above is saved in the file "/home/gpadmin/cloudberry/src/test/isolation2/regression.out".

make[2]: *** [Makefile:79: installcheck-parallel-retrieve-cursor] Error 1
make[2]: Leaving directory '/home/gpadmin/cloudberry/src/test/isolation2'
make[1]: *** [Makefile:57: installcheck-isolation2-recurse] Error 2
make[1]: Leaving directory '/home/gpadmin/cloudberry/src/test'
make: *** [GNUmakefile:186: installcheck-world-src/test-recurse] Error 2

[gpadmin@cdw cloudberry]$ cat /home/gpadmin/cloudberry/src/test/isolation2/regression.diffs
diff -I HINT: -I CONTEXT: -I GP_IGNORE: -U3 /home/gpadmin/cloudberry/src/test/isolation2/expected/parallel_retrieve_cursor/explain.out /home/gpadmin/cloudberry/src/test/isolation2/results/parallel_retrieve_cursor/explain.out
--- /home/gpadmin/cloudberry/src/test/isolation2/expected/parallel_retrieve_cursor/explain.out	2024-12-15 19:47:54.624363110 -0800
+++ /home/gpadmin/cloudberry/src/test/isolation2/results/parallel_retrieve_cursor/explain.out	2024-12-15 19:47:54.636363222 -0800
@@ -113,40 +113,40 @@
 QUERY PLAN
 ___________
  Seq Scan on pg_catalog.pg_class
-   Output: oid, relname, relnamespace, reltype, reloftype, relowner, relam, relfilenode, reltablespace, relpages, reltuples, relallvisible, reltoastrelid, relhasindex, relisshared, relpersistence, relkind, relnatts, relchecks, relhasrules, relhastriggers, relhassubclass, relrowsecurity, relforcerowsecurity, relispopulated, relreplident, relispartition, relisivm, relrewrite, relfrozenxid, relminmxid, relacl, reloptions, relpartbound
-GP_IGNORE:(3 rows)
+   Output: oid, relname, relnamespace, reltype, reloftype, relowner, relam, relfilenode, reltablespace, relpages, reltuples, relallvisible, reltoastrelid, relhasindex, relisshared, relpersistence, relkind, relnatts, relchecks, relhasrules, relhastriggers, relhassubclass, relrowsecurity, relforcerowsecurity, relispopulated, relreplident, relispartition, relisivm, relisdynamic, relrewrite, relfrozenxid, relminmxid, relacl, reloptions, relpartbound
+GP_IGNORE:(4 rows)

 -- Test: explain output: Endpoint info (on coordinator/on some segments/on all segments)
 EXPLAIN (VERBOSE, COSTS false) DECLARE c1 PARALLEL RETRIEVE CURSOR FOR SELECT * FROM pg_class;
 QUERY PLAN
 ___________
  Seq Scan on pg_catalog.pg_class
-   Output: oid, relname, relnamespace, reltype, reloftype, relowner, relam, relfilenode, reltablespace, relpages, reltuples, relallvisible, reltoastrelid, relhasindex, relisshared, relpersistence, relkind, relnatts, relchecks, relhasrules, relhastriggers, relhassubclass, relrowsecurity, relforcerowsecurity, relispopulated, relreplident, relispartition, relisivm, relrewrite, relfrozenxid, relminmxid, relacl, reloptions, relpartbound
+   Output: oid, relname, relnamespace, reltype, reloftype, relowner, relam, relfilenode, reltablespace, relpages, reltuples, relallvisible, reltoastrelid, relhasindex, relisshared, relpersistence, relkind, relnatts, relchecks, relhasrules, relhastriggers, relhassubclass, relrowsecurity, relforcerowsecurity, relispopulated, relreplident, relispartition, relisivm, relisdynamic, relrewrite, relfrozenxid, relminmxid, relacl, reloptions, relpartbound
  Endpoint: "on coordinator"
-GP_IGNORE:(4 rows)
+GP_IGNORE:(5 rows)
 EXPLAIN (VERBOSE, COSTS false) DECLARE c1 PARALLEL RETRIEVE CURSOR FOR SELECT * FROM pg_class ORDER BY relname;
 QUERY PLAN
 ___________
  Index Scan using pg_class_relname_nsp_index on pg_catalog.pg_class
-   Output: oid, relname, relnamespace, reltype, reloftype, relowner, relam, relfilenode, reltablespace, relpages, reltuples, relallvisible, reltoastrelid, relhasindex, relisshared, relpersistence, relkind, relnatts, relchecks, relhasrules, relhastriggers, relhassubclass, relrowsecurity, relforcerowsecurity, relispopulated, relreplident, relispartition, relisivm, relrewrite, relfrozenxid, relminmxid, relacl, reloptions, relpartbound
+   Output: oid, relname, relnamespace, reltype, reloftype, relowner, relam, relfilenode, reltablespace, relpages, reltuples, relallvisible, reltoastrelid, relhasindex, relisshared, relpersistence, relkind, relnatts, relchecks, relhasrules, relhastriggers, relhassubclass, relrowsecurity, relforcerowsecurity, relispopulated, relreplident, relispartition, relisivm, relisdynamic, relrewrite, relfrozenxid, relminmxid, relacl, reloptions, relpartbound
  Endpoint: "on coordinator"
 GP_IGNORE:(5 rows)
 EXPLAIN (VERBOSE, COSTS false) DECLARE c1 PARALLEL RETRIEVE CURSOR FOR SELECT * FROM pg_class WHERE gp_segment_id=1;
 QUERY PLAN
 ___________
  Seq Scan on pg_catalog.pg_class
-   Output: oid, relname, relnamespace, reltype, reloftype, relowner, relam, relfilenode, reltablespace, relpages, reltuples, relallvisible, reltoastrelid, relhasindex, relisshared, relpersistence, relkind, relnatts, relchecks, relhasrules, relhastriggers, relhassubclass, relrowsecurity, relforcerowsecurity, relispopulated, relreplident, relispartition, relisivm, relrewrite, relfrozenxid, relminmxid, relacl, reloptions, relpartbound
+   Output: oid, relname, relnamespace, reltype, reloftype, relowner, relam, relfilenode, reltablespace, relpages, reltuples, relallvisible, reltoastrelid, relhasindex, relisshared, relpersistence, relkind, relnatts, relchecks, relhasrules, relhastriggers, relhassubclass, relrowsecurity, relforcerowsecurity, relispopulated, relreplident, relispartition, relisivm, relisdynamic, relrewrite, relfrozenxid, relminmxid, relacl, reloptions, relpartbound
    Filter: (pg_class.gp_segment_id = 1)
  Endpoint: "on coordinator"
-GP_IGNORE:(5 rows)
+GP_IGNORE:(6 rows)
 EXPLAIN (VERBOSE, COSTS false) DECLARE c1 PARALLEL RETRIEVE CURSOR FOR SELECT * FROM pg_class WHERE gp_segment_id=1 OR gp_segment_id=2;
 QUERY PLAN
 ___________
  Seq Scan on pg_catalog.pg_class
-   Output: oid, relname, relnamespace, reltype, reloftype, relowner, relam, relfilenode, reltablespace, relpages, reltuples, relallvisible, reltoastrelid, relhasindex, relisshared, relpersistence, relkind, relnatts, relchecks, relhasrules, relhastriggers, relhassubclass, relrowsecurity, relforcerowsecurity, relispopulated, relreplident, relispartition, relisivm, relrewrite, relfrozenxid, relminmxid, relacl, reloptions, relpartbound
+   Output: oid, relname, relnamespace, reltype, reloftype, relowner, relam, relfilenode, reltablespace, relpages, reltuples, relallvisible, reltoastrelid, relhasindex, relisshared, relpersistence, relkind, relnatts, relchecks, relhasrules, relhastriggers, relhassubclass, relrowsecurity, relforcerowsecurity, relispopulated, relreplident, relispartition, relisivm, relisdynamic, relrewrite, relfrozenxid, relminmxid, relacl, reloptions, relpartbound
    Filter: ((pg_class.gp_segment_id = 1) OR (pg_class.gp_segment_id = 2))
  Endpoint: "on coordinator"
-GP_IGNORE:(5 rows)
+GP_IGNORE:(6 rows)

 -- Test for UDF which can be executed on coordinator
 EXPLAIN (VERBOSE, COSTS false) DECLARE c1 PARALLEL RETRIEVE CURSOR FOR SELECT * FROM generate_series(1,10) as F;

What you think should happen instead

It should run without the failed tests.

How to reproduce

1. Download the build docker image.

docker run -it --rm -h cdw --shm-size=30gb apache/incubator-cloudberry:cbdb-build-rocky9-latest

2. Then run the following commands in the build env.

    1  git clone https://github.com/apache/cloudberry.git ~/cloudberry
    2  #cd cloudberrydb/
    3  #git branch -a
    4  sudo dnf install -y apr-devel autoconf bison bzip2 bzip2-devel cmake3 createrepo_c ed flex gcc gcc-c++ git glibc-langpack-en initscripts iproute java-11-openjdk java-11-openjdk-devel krb5-devel less libcurl-devel libevent-devel libuuid-devel libxml2-devel libzstd-devel lz4 lz4-devel m4 nc net-tools openldap-devel openssh-clients openssh-server openssl-devel pam-devel passwd perl perl-Env perl-ExtUtils-Embed perl-Test-Simple perl-core pinentry python3-devel python3-lxml python3-psutil python3-pytest python3-pyyaml readline-devel rpm-build rpm-sign rpmdevtools rsync tar unzip util-linux-ng wget which zlib-devel
    5   sudo dnf install -y --enablerepo=crb libuv-devel libyaml-devel perl-IPC-Run
    6
    7  XERCES_LATEST_RELEASE=3.3.0
    8  INSTALL_PREFIX="/usr/local/xerces-c"
    9  wget -nv "https://dlcdn.apache.org//xerces/c/3/sources/xerces-c-${XERCES_LATEST_RELEASE}.tar.gz"
   10  echo "$(curl -sL https://dlcdn.apache.org//xerces/c/3/sources/xerces-c-${XERCES_LATEST_RELEASE}.tar.gz.sha256)" | sha256sum -c -
   11  tar xf "xerces-c-${XERCES_LATEST_RELEASE}.tar.gz"
   12  rm "xerces-c-${XERCES_LATEST_RELEASE}.tar.gz"
   13  cd xerces-c-${XERCES_LATEST_RELEASE}
   14  ./configure --prefix="${INSTALL_PREFIX}-${XERCES_LATEST_RELEASE}" | tee configure-$(date "+%Y.%m.%d-%H.%M.%S").log
   15  make -j$(nproc) | tee make-$(date "+%Y.%m.%d-%H.%M.%S").log
   16  #make check | tee make-check-$(date "+%Y.%m.%d-%H.%M.%S").log
   17  sudo make install | tee make-install-$(date "+%Y.%m.%d-%H.%M.%S").log
   18  sudo ln -s ${INSTALL_PREFIX}-${XERCES_LATEST_RELEASE} ${INSTALL_PREFIX}
   19  sudo rm -rf /usr/local/cloudberry-db
   20  sudo chmod a+w /usr/local
   21  mkdir -p /usr/local/cloudberry-db/lib
   22  sudo cp -v /usr/local/xerces-c/lib/libxerces-c.so            /usr/local/xerces-c/lib/libxerces-c-3.*.so            /usr/local/cloudberry-db/lib
   23  sudo chown -R gpadmin.gpadmin /usr/local/cloudberry-db
   24  export LD_LIBRARY_PATH=/usr/local/cloudberry-db/lib:LD_LIBRARY_PATH
   25  cd ~/cloudberry
   26  ./configure --prefix=/usr/local/cloudberry-db             --disable-external-fts             --enable-debug             --enable-cassert             --enable-debug-extensions             --enable-gpcloud             --enable-ic-proxy             --enable-mapreduce             --enable-orafce             --enable-orca             --enable-pxf             --enable-tap-tests             --with-gssapi             --with-ldap             --with-libxml             --with-lz4             --with-openssl             --with-pam             --with-perl             --with-pgport=5432             --with-python             --with-pythonsrc-ext             --with-ssl=openssl             --with-openssl             --with-uuid=e2fs             --with-includes=/usr/local/xerces-c/include             --with-libraries=/usr/local/cloudberry-db/lib | tee ~/cloudberry/configure-$(date "+%Y.%m.%d-%H.%M.%S").log
   27  make -j$(nproc) -C ~/cloudberry | tee ~/cloudberry/make-$(date "+%Y.%m.%d-%H.%M.%S").log && make -j$(nproc) -C ~/cloudberry/contrib | tee ~/cloudberry/make-contrib-$(date "+%Y.%m.%d-%H.%M.%S").log &&  make install -C ~/cloudberry | tee ~/cloudberry/make-install-$(date "+%Y.%m.%d-%H.%M.%S").log &&  make install -C ~/cloudberry/contrib | tee ~/cloudberry/make-contrib-install-$(date "+%Y.%m.%d-%H.%M.%S").log
   28  /usr/local/cloudberry-db/bin/postgres --gp-version
   29  /usr/local/cloudberry-db/bin/postgres --version
   30  ldd /usr/local/cloudberry-db/bin/postgres
   31  source /usr/local/cloudberry-db/greenplum_path.sh
   32  make create-demo-cluster -C ~/cloudberry
   33  #make destroy-demo-cluster -C ~/cloudberry
   34  source ~/cloudberry/gpAux/gpdemo/gpdemo-env.sh
   35  gpstate
   36  psql -P pager=off template1 -c 'SELECT * from gp_segment_configuration'
   37  PGCLIENTENCODING=UTF8 PGOPTIONS='-c optimizer=off' make installcheck-world

Operating System

Rocky Linux 9

Anything else

No response

Are you willing to submit PR?

• Yes, I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct.

[my-ship-it]

@avamingli please help on it, thanks!

[tuhaihe]

I tested again on one virtual machine on the Huawei Cloud, following this guide (https://github.com/edespino/cloudberry/blob/rocky9-dev-readme/deploy/build/README-rockylinux9.md) to build and install Cloudberry.

The PGOPTIONS='-c optimizer=off' make installcheck-world command still returned the same result.

$ cat /etc/os-release

NAME="Rocky Linux"
VERSION="9.5 (Blue Onyx)"
...

[avamingli]

Isolation2 test is being brought back to CI, see #782 (comment)

It's the same diff.