Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to install jep-4.2.0 on Python 3.6.13 with Java 11 #516

Open
pvb05 opened this issue Jan 25, 2024 · 2 comments
Open

Unable to install jep-4.2.0 on Python 3.6.13 with Java 11 #516

pvb05 opened this issue Jan 25, 2024 · 2 comments

Comments

@pvb05
Copy link

pvb05 commented Jan 25, 2024

Describe the problem
We are unable to install jep-4.2.0 on Python 3.6.13 (pyenv environment) and it throws errors when compiling the SubInterpreterOptions.java.

/usr/lib/jvm/java-11-amazon-corretto.x86_64/bin/javac -deprecation -d build/java -h build/include -classpath src src/main/java/jep/JepConfig.java src/main/java/jep/ClassList.java src/main/java/jep/Jep.java src/main/java/jep/Interpreter.java src/main/java/jep/LibraryLocator.java src/main/java/jep/DirectNDArray.java src/main/java/jep/SharedInterpreter.java src/main/java/jep/NamingConventionClassEnquirer.java src/main/java/jep/MainInterpreter.java src/main/java/jep/PyConfig.java src/main/java/jep/PyMethod.java src/main/java/jep/AbstractNDArray.java src/main/java/jep/SubInterpreterOptions.java src/main/java/jep/ClassEnquirer.java src/main/java/jep/SubInterpreter.java src/main/java/jep/NDArray.java src/main/java/jep/JepAccess.java src/main/java/jep/Run.java src/main/java/jep/Proxy.java src/main/java/jep/JepException.java src/main/java/jep/python/PyObject.java src/main/java/jep/python/MemoryManager.java src/main/java/jep/python/InvocationHandler.java src/main/java/jep/python/PyPointer.java src/main/java/jep/python/PyCallable.java src/main/java/jep/python/PyBuiltins.java
    src/main/java/jep/SubInterpreterOptions.java:61: error: unmappable character (0xE2) for encoding US-ASCII
         * allocator state. Otherwise it will use (share) the main interpreter???s.
                                                                              ^
    src/main/java/jep/SubInterpreterOptions.java:61: error: unmappable character (0x80) for encoding US-ASCII
         * allocator state. Otherwise it will use (share) the main interpreter???s.
                                                                               ^
    src/main/java/jep/SubInterpreterOptions.java:61: error: unmappable character (0x99) for encoding US-ASCII
         * allocator state. Otherwise it will use (share) the main interpreter???s.
                                                                                ^
    src/main/java/jep/SubInterpreterOptions.java:114: error: unmappable character (0xE2) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                    ^
    src/main/java/jep/SubInterpreterOptions.java:114: error: unmappable character (0x80) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                     ^
    src/main/java/jep/SubInterpreterOptions.java:114: error: unmappable character (0x99) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                      ^
    src/main/java/jep/SubInterpreterOptions.java:114: error: unmappable character (0xE2) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                                             ^
    src/main/java/jep/SubInterpreterOptions.java:114: error: unmappable character (0x80) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                                              ^
    src/main/java/jep/SubInterpreterOptions.java:114: error: unmappable character (0x99) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                                               ^
    src/main/java/jep/SubInterpreterOptions.java:128: error: unmappable character (0xE2) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                    ^
    src/main/java/jep/SubInterpreterOptions.java:128: error: unmappable character (0x80) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                     ^
    src/main/java/jep/SubInterpreterOptions.java:128: error: unmappable character (0x99) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                      ^
    src/main/java/jep/SubInterpreterOptions.java:128: error: unmappable character (0xE2) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                                             ^
    src/main/java/jep/SubInterpreterOptions.java:128: error: unmappable character (0x80) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                                              ^
    src/main/java/jep/SubInterpreterOptions.java:128: error: unmappable character (0x99) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                                               ^
    src/main/java/jep/SubInterpreterOptions.java:162: error: unmappable character (0xE2) for encoding US-ASCII
         * this is false use (share) the main interpreter???s GIL. When this is true
                                                         ^
    src/main/java/jep/SubInterpreterOptions.java:162: error: unmappable character (0x80) for encoding US-ASCII
         * this is false use (share) the main interpreter???s GIL. When this is true
                                                          ^
    src/main/java/jep/SubInterpreterOptions.java:162: error: unmappable character (0x99) for encoding US-ASCII
         * this is false use (share) the main interpreter???s GIL. When this is true
                                                           ^
    src/main/java/jep/SubInterpreterOptions.java:163: error: unmappable character (0xE2) for encoding US-ASCII
         * use the sub-interpreter???s own GIL.
                                  ^
    src/main/java/jep/SubInterpreterOptions.java:163: error: unmappable character (0x80) for encoding US-ASCII
         * use the sub-interpreter???s own GIL.
                                   ^
    src/main/java/jep/SubInterpreterOptions.java:163: error: unmappable character (0x99) for encoding US-ASCII
         * use the sub-interpreter???s own GIL.
                                    ^
    21 errors
    error: command '/usr/lib/jvm/java-11-amazon-corretto.x86_64/bin/javac' failed with exit status 1
    ----------------------------------------

Environment (please complete the following information):

  • OS Platform, Distribution, and Version: Amazon Linux 2.0.20240109.0
  • Python Distribution and Version: 3.6.13 built and installed via Pyenv with shared modules options.
  • Java Distribution and Version: Java 11 - java-11-amazon-corretto-headless-1:11.0.22+7-1.amzn2.x86_64
  • Jep Version: 4.2.0
  • Python packages used (e.g. numpy, pandas, tensorflow): Numpy 1.19.5, Scipy 1.5.4

Logs

[INFO] DOCKER> Collecting numpy==1.19.5
[INFO] DOCKER> Downloading numpy-1.19.5-cp36-cp36m-manylinux2010_x86_64.whl (14.8 MB)
[INFO] DOCKER> Collecting scipy==1.5.4
[INFO] DOCKER> Downloading scipy-1.5.4-cp36-cp36m-manylinux1_x86_64.whl (25.9 MB)
[INFO] DOCKER> Collecting scikit-learn==0.24.2
[INFO] DOCKER> Downloading scikit_learn-0.24.2-cp36-cp36m-manylinux2010_x86_64.whl (22.2 MB)
[INFO] DOCKER> Collecting dill==0.3.4
[INFO] DOCKER> Downloading dill-0.3.4-py2.py3-none-any.whl (86 kB)
[INFO] DOCKER> Collecting joblib==1.1.1
[INFO] DOCKER> Downloading joblib-1.1.1-py2.py3-none-any.whl (309 kB)
[INFO] DOCKER> Collecting jep==4.2.0
[INFO] DOCKER> Downloading jep-4.2.0.tar.gz (3.0 MB)
[INFO] DOCKER> Preparing metadata (setup.py): started
[INFO] DOCKER> Preparing metadata (setup.py): finished with status 'done'
[INFO] DOCKER> Collecting threadpoolctl>=2.0.0
[INFO] DOCKER> Downloading threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
[INFO] DOCKER> Using legacy 'setup.py install' for jep, since package 'wheel' is not installed.
[INFO] DOCKER> Installing collected packages: numpy, threadpoolctl, scipy, joblib, scikit-learn, jep, dill
[INFO] DOCKER> Running setup.py install for jep: started
[INFO] DOCKER> Running setup.py install for jep: finished with status 'error'
[INFO] DOCKER> [91m    ERROR: Command errored out with exit status 1:
     command: /prereqs/pyenv/versions/3.6.13/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_waf9dr3/jep_1f9fe8d5a48043f2a1c1e7aae859f4b9/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_waf9dr3/jep_1f9fe8d5a48043f2a1c1e7aae859f4b9/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-ftu2lizj/install-record.txt --single-version-externally-managed --compile --install-headers /prereqs/pyenv/versions/3.6.13/include/python3.6m/jep
         cwd: /tmp/pip-install-_waf9dr3/jep_1f9fe8d5a48043f2a1c1e7aae859f4b9/
    Complete output (72 lines):
    numpy include found at /prereqs/pyenv/versions/3.6.13/lib/python3.6/site-packages/numpy/core/include
    running install
    running build
    running setup_java
    Using JAVA_HOME: /usr/lib/jvm/java-11-amazon-corretto.x86_64/
    running build_java
    /usr/lib/jvm/java-11-amazon-corretto.x86_64/bin/javac -deprecation -d build/java -h build/include -classpath src src/main/java/jep/JepConfig.java src/main/java/jep/ClassList.java src/main/java/jep/Jep.java src/main/java/jep/Interpreter.java src/main/java/jep/LibraryLocator.java src/main/java/jep/DirectNDArray.java src/main/java/jep/SharedInterpreter.java src/main/java/jep/NamingConventionClassEnquirer.java src/main/java/jep/MainInterpreter.java src/main/java/jep/PyConfig.java src/main/java/jep/PyMethod.java src/main/java/jep/AbstractNDArray.java src/main/java/jep/SubInterpreterOptions.java src/main/java/jep/ClassEnquirer.java src/main/java/jep/SubInterpreter.java src/main/java/jep/NDArray.java src/main/java/jep/JepAccess.java src/main/java/jep/Run.java src/main/java/jep/Proxy.java src/main/java/jep/JepException.java src/main/java/jep/python/PyObject.java src/main/java/jep/python/MemoryManager.java src/main/java/jep/python/InvocationHandler.java src/main/java/jep/python/PyPointer.java src/main/java/jep/python/PyCallable.java src/main/java/jep/python/PyBuiltins.java
    src/main/java/jep/SubInterpreterOptions.java:61: error: unmappable character (0xE2) for encoding US-ASCII
         * allocator state. Otherwise it will use (share) the main interpreter???s.
                                                                              ^
    src/main/java/jep/SubInterpreterOptions.java:61: error: unmappable character (0x80) for encoding US-ASCII
         * allocator state. Otherwise it will use (share) the main interpreter???s.
                                                                               ^
    src/main/java/jep/SubInterpreterOptions.java:61: error: unmappable character (0x99) for encoding US-ASCII
         * allocator state. Otherwise it will use (share) the main interpreter???s.
                                                                                ^
    src/main/java/jep/SubInterpreterOptions.java:114: error: unmappable character (0xE2) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                    ^
    src/main/java/jep/SubInterpreterOptions.java:114: error: unmappable character (0x80) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                     ^
    src/main/java/jep/SubInterpreterOptions.java:114: error: unmappable character (0x99) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                      ^
    src/main/java/jep/SubInterpreterOptions.java:114: error: unmappable character (0xE2) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                                             ^
    src/main/java/jep/SubInterpreterOptions.java:114: error: unmappable character (0x80) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                                              ^
    src/main/java/jep/SubInterpreterOptions.java:114: error: unmappable character (0x99) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                                               ^
    src/main/java/jep/SubInterpreterOptions.java:128: error: unmappable character (0xE2) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                    ^
    src/main/java/jep/SubInterpreterOptions.java:128: error: unmappable character (0x80) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                     ^
    src/main/java/jep/SubInterpreterOptions.java:128: error: unmappable character (0x99) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                      ^
    src/main/java/jep/SubInterpreterOptions.java:128: error: unmappable character (0xE2) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                                             ^
    src/main/java/jep/SubInterpreterOptions.java:128: error: unmappable character (0x80) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                                              ^
    src/main/java/jep/SubInterpreterOptions.java:128: error: unmappable character (0x99) for encoding US-ASCII
         * If this is false then the sub-interpreter???s threading module won???t be
                                                                               ^
    src/main/java/jep/SubInterpreterOptions.java:162: error: unmappable character (0xE2) for encoding US-ASCII
         * this is false use (share) the main interpreter???s GIL. When this is true
                                                         ^
    src/main/java/jep/SubInterpreterOptions.java:162: error: unmappable character (0x80) for encoding US-ASCII
         * this is false use (share) the main interpreter???s GIL. When this is true
                                                          ^
    src/main/java/jep/SubInterpreterOptions.java:162: error: unmappable character (0x99) for encoding US-ASCII
         * this is false use (share) the main interpreter???s GIL. When this is true
                                                           ^
    src/main/java/jep/SubInterpreterOptions.java:163: error: unmappable character (0xE2) for encoding US-ASCII
         * use the sub-interpreter???s own GIL.
                                  ^
    src/main/java/jep/SubInterpreterOptions.java:163: error: unmappable character (0x80) for encoding US-ASCII
         * use the sub-interpreter???s own GIL.
                                   ^
    src/main/java/jep/SubInterpreterOptions.java:163: error: unmappable character (0x99) for encoding US-ASCII
         * use the sub-interpreter???s own GIL.
                                    ^
    21 errors
    error: command '/usr/lib/jvm/java-11-amazon-corretto.x86_64/bin/javac' failed with exit status 1
    ----------------------------------------

[INFO] DOCKER> [91mERROR: Command errored out with exit status 1: /prereqs/pyenv/versions/3.6.13/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_waf9dr3/jep_1f9fe8d5a48043f2a1c1e7aae859f4b9/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_waf9dr3/jep_1f9fe8d5a48043f2a1c1e7aae859f4b9/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-ftu2lizj/install-record.txt --single-version-externally-managed --compile --install-headers /prereqs/pyenv/versions/3.6.13/include/python3.6m/jep Check the logs for full command output.
@pvb05
Copy link
Author

pvb05 commented Jan 25, 2024 •
edited

It looks like non US-ASCII apostrophe used in the SubIntepreterOptions.java javadocs. Can you please suggest options to resolve it?

@bsteffensmeier
Copy link
Member

You should be able to resolve this by setting the JAVA_TOOL_OPTIONS env var to include -Dfile.encoding=UTF-8.

I will convert the apostrophe to U+0027 in the next jep release so this won't be an issue in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@pvb05 @bsteffensmeier