Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linking issue with static libpq compilation #1123

Open
Progdrasil opened this issue Jun 25, 2020 · 6 comments
Open

Linking issue with static libpq compilation #1123

Progdrasil opened this issue Jun 25, 2020 · 6 comments

Comments

@Progdrasil
Copy link

Progdrasil commented Jun 25, 2020

I'm currently building an app using buildozer, i've patched it to compile the version 10.12 of libpq.

Their recipe for compiling psycopg2 compiles against the static lib of libpq which works fine.
Instead it breaks at runtime with the error:

ImportError: undefined symbol: PQencryptPasswordConn

I'm running with psycopg2 2.8.4
I've read the issues #1093 and #1018 but i've confirmed that the libpq version really is 10.

@dvarrazzo
Copy link
Member

Hello @Progdrasil

Because the problem comes your environment I don't really have a way to know what happens there. You can see that the only use of that function is guarded at compile time.

#if PG_VERSION_NUM >= 100000
if (!conn) {
PyErr_SetString(ProgrammingError,
"password encryption (other than 'md5' algorithm)"
" requires a connection or cursor");
goto exit;
}
/* TODO: algo = None will block: forbid on async/green conn? */
encrypted = PQencryptPasswordConn(conn->pgconn,
Bytes_AS_STRING(password), Bytes_AS_STRING(user),
algorithm != Py_None ? Bytes_AS_STRING(algorithm) : NULL);
#else
PyErr_SetString(NotSupportedError,
"password encryption (other than 'md5' algorithm)"
" requires libpq 10");
goto exit;
#endif

If you provide a docker image reproducing the issue in isolation we can try taking a look, thank you.

@Progdrasil
Copy link
Author

Funny thing is I was actually getting this issue in docker yesterday fairly systematically. But today it seems to not want to reproduce...

Thats annoying. I made a repo here which should reproduce the issue as it would happen as soon as psycopg2 was imported, but it does not.
https://github.com/Progdrasil/buildozer-psycopg2-issue-1123

@grubberr
Copy link

I have similar problem
I cannot compile psycopg2 with static linking to libpq

Run inside AWS Lambda container

docker run -ti --rm --entrypoint /bin/bash --platform linux/amd64 public.ecr.aws/lambda/python:3.11

Install all needed rpm packages

yum -y install wget tar gzip gcc make flex bison perl patch

Download and compile postgresql-14.9

wget https://ftp.postgresql.org/pub/source/v14.9/postgresql-14.9.tar.gz
tar xfz postgresql-14.9.tar.gz
pushd postgresql-14.9
./configure --prefix /opt/postgresql-14.9 --without-readline --without-zlib
make
make install
popd

Download and compile psycopg2-2.9.7 with static linking to libpq

wget https://files.pythonhosted.org/packages/f7/fa/6e6bb7a7bbe4e02b35aa2fc009fb53221663a5e07e333b72cb5a85e4dbb0/psycopg2-2.9.7.tar.gz
tar xfz psycopg2-2.9.7.tar.gz
pushd psycopg2-2.9.7
sed -i 's@pg_config =.*@pg_config = /opt/postgresql-14.9/bin/pg_config@' setup.cfg
sed -i 's@static_libpq =.*@static_libpq = 1@' setup.cfg
python3 setup.py build
python3 setup.py install
popd

Import compile psycopg2 module

bash-4.2# python3 -m psycopg2
Traceback (most recent call last):
  File "<frozen runpy>", line 189, in _run_module_as_main
  File "<frozen runpy>", line 148, in _get_module_details
  File "<frozen runpy>", line 112, in _get_module_details
  File "/var/lang/lib/python3.11/site-packages/psycopg2-2.9.7-py3.11-linux-x86_64.egg/psycopg2/__init__.py", line 51, in <module>
    from psycopg2._psycopg import (                     # noqa
ImportError: /var/lang/lib/python3.11/site-packages/psycopg2-2.9.7-py3.11-linux-x86_64.egg/psycopg2/_psycopg.cpython-311-x86_64-linux-gnu.so: undefined symbol: scram_SaltedPassword

Any idea how can we eliminate this problem ?

@minhio
Copy link

minhio commented Jul 15, 2024

I have similar problem I cannot compile psycopg2 with static linking to libpq

Run inside AWS Lambda container

docker run -ti --rm --entrypoint /bin/bash --platform linux/amd64 public.ecr.aws/lambda/python:3.11

Install all needed rpm packages

yum -y install wget tar gzip gcc make flex bison perl patch

Download and compile postgresql-14.9

wget https://ftp.postgresql.org/pub/source/v14.9/postgresql-14.9.tar.gz
tar xfz postgresql-14.9.tar.gz
pushd postgresql-14.9
./configure --prefix /opt/postgresql-14.9 --without-readline --without-zlib
make
make install
popd

Download and compile psycopg2-2.9.7 with static linking to libpq

wget https://files.pythonhosted.org/packages/f7/fa/6e6bb7a7bbe4e02b35aa2fc009fb53221663a5e07e333b72cb5a85e4dbb0/psycopg2-2.9.7.tar.gz
tar xfz psycopg2-2.9.7.tar.gz
pushd psycopg2-2.9.7
sed -i 's@pg_config =.*@pg_config = /opt/postgresql-14.9/bin/pg_config@' setup.cfg
sed -i 's@static_libpq =.*@static_libpq = 1@' setup.cfg
python3 setup.py build
python3 setup.py install
popd

Import compile psycopg2 module

bash-4.2# python3 -m psycopg2
Traceback (most recent call last):
  File "<frozen runpy>", line 189, in _run_module_as_main
  File "<frozen runpy>", line 148, in _get_module_details
  File "<frozen runpy>", line 112, in _get_module_details
  File "/var/lang/lib/python3.11/site-packages/psycopg2-2.9.7-py3.11-linux-x86_64.egg/psycopg2/__init__.py", line 51, in <module>
    from psycopg2._psycopg import (                     # noqa
ImportError: /var/lang/lib/python3.11/site-packages/psycopg2-2.9.7-py3.11-linux-x86_64.egg/psycopg2/_psycopg.cpython-311-x86_64-linux-gnu.so: undefined symbol: scram_SaltedPassword

Any idea how can we eliminate this problem ?

@grubberr were you able to solve this?

@grubberr
Copy link

grubberr commented Aug 3, 2024

@minhio no, we did some work-around :(

@jarias1
Copy link

jarias1 commented Sep 9, 2024

I have similar problem I cannot compile psycopg2 with static linking to libpq

Run inside AWS Lambda container

docker run -ti --rm --entrypoint /bin/bash --platform linux/amd64 public.ecr.aws/lambda/python:3.11

Install all needed rpm packages

yum -y install wget tar gzip gcc make flex bison perl patch

Download and compile postgresql-14.9

wget https://ftp.postgresql.org/pub/source/v14.9/postgresql-14.9.tar.gz
tar xfz postgresql-14.9.tar.gz
pushd postgresql-14.9
./configure --prefix /opt/postgresql-14.9 --without-readline --without-zlib
make
make install
popd

Download and compile psycopg2-2.9.7 with static linking to libpq

wget https://files.pythonhosted.org/packages/f7/fa/6e6bb7a7bbe4e02b35aa2fc009fb53221663a5e07e333b72cb5a85e4dbb0/psycopg2-2.9.7.tar.gz
tar xfz psycopg2-2.9.7.tar.gz
pushd psycopg2-2.9.7
sed -i 's@pg_config =.*@pg_config = /opt/postgresql-14.9/bin/pg_config@' setup.cfg
sed -i 's@static_libpq =.*@static_libpq = 1@' setup.cfg
python3 setup.py build
python3 setup.py install
popd

Import compile psycopg2 module

bash-4.2# python3 -m psycopg2
Traceback (most recent call last):
  File "<frozen runpy>", line 189, in _run_module_as_main
  File "<frozen runpy>", line 148, in _get_module_details
  File "<frozen runpy>", line 112, in _get_module_details
  File "/var/lang/lib/python3.11/site-packages/psycopg2-2.9.7-py3.11-linux-x86_64.egg/psycopg2/__init__.py", line 51, in <module>
    from psycopg2._psycopg import (                     # noqa
ImportError: /var/lang/lib/python3.11/site-packages/psycopg2-2.9.7-py3.11-linux-x86_64.egg/psycopg2/_psycopg.cpython-311-x86_64-linux-gnu.so: undefined symbol: scram_SaltedPassword

Any idea how can we eliminate this problem ?

same issue here, I compiled it with option static_libpq=1 but receive error that says: undefined symbol: pg_tolower

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants