Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pyre fails to run in Docker container on m1 host #744

Open
jepify opened this issue Jun 14, 2023 · 7 comments
Open

Pyre fails to run in Docker container on m1 host #744

jepify opened this issue Jun 14, 2023 · 7 comments

Comments

@jepify
Copy link

jepify commented Jun 14, 2023

Pyre Bug

Bug description
Running pyre inside a Linux Docker container fails.

Reproduction steps

docker run -it ubuntu /bin/bash

inside container:

apt update && apt upgrade -y
apt install python3 python3-pip
pip install pyre-check
pyre init 
pyre

Expected behavior
Once .pyre_configuration has been created the binary fails to execute
Expected type checking to start

Logs
Please include any relevant logs here:

ƛ No watchman binary found.
To enable pyre incremental, you can install watchman: https://facebook.github.io/watchman/docs/install
ƛ Defaulting to non-incremental check.
ƛ [Errno 8] Exec format error: '/usr/local/bin/pyre.bin'
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/pyre_check/client/pyre.py", line 1489, in main
    return_code = pyre(argv, auto_envvar_prefix="PYRE", standalone_mode=False)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1636, in invoke
    return _process_result([] if self.chain else rv)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1626, in _process_result
    value = ctx.invoke(self._result_callback, value, **ctx.params)
  File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/pyre_check/client/pyre.py", line 1477, in run_default_command
    return _run_default_command(command_argument)
  File "/usr/local/lib/python3.10/dist-packages/pyre_check/client/pyre.py", line 139, in _run_default_command
    return _run_check_command(arguments)
  File "/usr/local/lib/python3.10/dist-packages/pyre_check/client/pyre.py", line 94, in _run_check_command
    return commands.check.run(configuration, check_arguments)
  File "/usr/local/lib/python3.10/dist-packages/pyre_check/client/commands/check.py", line 227, in run
    return run_check(frontend_configuration.OpenSource(configuration), check_arguments)
  File "/usr/local/lib/python3.10/dist-packages/pyre_check/client/commands/check.py", line 218, in run_check
    return _run_check_command(
  File "/usr/local/lib/python3.10/dist-packages/pyre_check/client/commands/check.py", line 171, in _run_check_command
    result = subprocess.run(
  File "/usr/lib/python3.10/subprocess.py", line 501, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.10/subprocess.py", line 969, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.10/subprocess.py", line 1845, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/usr/local/bin/pyre.bin'

Please run your reproduction steps followed by pyre rage > pyre_rage.log, and upload the file here:

pyre_rage.log:

Versions
========
Client version: 0.9.18
Could not determine binary version: [Errno 8] Exec format error: '/usr/local/bin/pyre.bin'

Configuration
=============
{
  "global_root": "/",
  "dot_pyre_directory": "/.pyre",
  "only_check_paths": [],
  "excludes": [],
  "extensions": [],
  "ignore_all_errors": [],
  "other_critical_files": [],
  "search_path": [],
  "site_package_search_strategy": "pep561",
  "site_roots": [],
  "source_directories": [
    "SimpleRawElement(root='/')"
  ],
  "strict": false,
  "taint_models_path": [],
  "typeshed": "/",
  "use_buck2": false
}

Client Log
==========
2023-06-14 06:43:55,878 [PID 4242] INFO No binary specified, looking for `pyre.bin` in PATH
2023-06-14 06:43:55,878 [PID 4242] INFO Could not determine the number of Pyre workers from configuration. Auto-set the value to 4.
2023-06-14 06:43:55,880 [PID 4242] INFO Writing arguments into /tmp/pyre_arguments_n27ls92u.json...
2023-06-14 06:43:55,880 [PID 4242] DEBUG Arguments:
{
  "source_paths": {
    "kind": "simple",
    "paths": [
      "/"
    ]
  },
  "search_paths": [
    "/usr/local/lib/python3.10/dist-packages$testslide",
    "/usr/local/lib/python3.10/dist-packages$dataclasses_json",
    "/usr/local/lib/python3.10/dist-packages$libcst",
    "/usr/local/lib/python3.10/dist-packages$packaging",
    "/usr/local/lib/python3.10/dist-packages$marshmallow",
    "/usr/lib/python3/dist-packages$pip",
    "/usr/local/lib/python3.10/dist-packages$typeguard",
    "/usr/local/lib/python3.10/dist-packages$click"
  ],
  "excludes": [],
  "checked_directory_allowlist": [
    "/"
  ],
  "checked_directory_blocklist": [],
  "extensions": [],
  "log_path": "/.pyre",
  "global_root": "/",
  "debug": false,
  "python_version": {
    "major": 3,
    "minor": 10,
    "micro": 6
  },
  "shared_memory": {},
  "parallel": true,
  "number_of_workers": 4,
  "additional_logging_sections": [],
  "show_error_traces": false,
  "strict": false
}
2023-06-14 06:43:55,884 [PID 4242] ERROR [Errno 8] Exec format error: '/usr/local/bin/pyre.bin'


Client Log
==========
2023-06-14 06:43:55,878 [PID 4242] INFO No binary specified, looking for `pyre.bin` in PATH
2023-06-14 06:43:55,878 [PID 4242] INFO Could not determine the number of Pyre workers from configuration. Auto-set the value to 4.
2023-06-14 06:43:55,880 [PID 4242] INFO Writing arguments into /tmp/pyre_arguments_n27ls92u.json...
2023-06-14 06:43:55,880 [PID 4242] DEBUG Arguments:
{
  "source_paths": {
    "kind": "simple",
    "paths": [
      "/"
    ]
  },
  "search_paths": [
    "/usr/local/lib/python3.10/dist-packages$testslide",
    "/usr/local/lib/python3.10/dist-packages$dataclasses_json",
    "/usr/local/lib/python3.10/dist-packages$libcst",
    "/usr/local/lib/python3.10/dist-packages$packaging",
    "/usr/local/lib/python3.10/dist-packages$marshmallow",
    "/usr/lib/python3/dist-packages$pip",
    "/usr/local/lib/python3.10/dist-packages$typeguard",
    "/usr/local/lib/python3.10/dist-packages$click"
  ],
  "excludes": [],
  "checked_directory_allowlist": [
    "/"
  ],
  "checked_directory_blocklist": [],
  "extensions": [],
  "log_path": "/.pyre",
  "global_root": "/",
  "debug": false,
  "python_version": {
    "major": 3,
    "minor": 10,
    "micro": 6
  },
  "shared_memory": {},
  "parallel": true,
  "number_of_workers": 4,
  "additional_logging_sections": [],
  "show_error_traces": false,
  "strict": false
}
2023-06-14 06:43:55,884 [PID 4242] ERROR [Errno 8] Exec format error: '/usr/local/bin/pyre.bin'

Additional context

root@649ced2c5e40:/# uname -a
Linux 649ced2c5e40 5.15.49-linuxkit #1 SMP PREEMPT Tue Sep 13 07:51:32 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

file /usr/local/bin/pyre.bin
/usr/local/bin/pyre.bin: Mach-O 64-bit x86_64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|WEAK_DEFINES|BINDS_TO_WEAK|PIE>
@connernilsen
Copy link
Contributor

connernilsen commented Jun 21, 2023 •
edited

Hi @jepify, it looks like Watchman hasn't been installed according to the logs and your setup steps. You can try to install it with brew install watchman or with the official installation instructions. Sorry, I misread what was going on here. We're discussing this internally to get a better idea of what's going on here.

@Peteling
Copy link

Peteling commented Nov 6, 2023

When will there be progress?

@connernilsen
Copy link
Contributor

Sorry, this totally fell off my radar. Let me bring this back up to the team and see what they think. In the meantime, we released a new version of Pyre, so try updating to that and let me know if it helps.

@Peteling
Copy link

Peteling commented Nov 9, 2023

Sorry, this totally fell off my radar. Let me bring this back up to the team and see what they think. In the meantime, we released a new version of Pyre, so try updating to that and let me know if it helps.

I try it in new version of Pyre and it still occurs error.

@connernilsen
Copy link
Contributor

connernilsen commented Nov 9, 2023 •
edited

So it looks like the issue is that an x86_64 executable is being run in an arm64 architecture, which are incompatible. The best case solution here is that we need to push a new release with an arm64 build, but we unfortunately don't have the infrastructure to do that at the moment. We'll begin to look into doing that so we can start publishing for more platforms.

My recommendation at the moment is to try building from source in Docker yourself. Let me know if you have any questions or issues with that, and sorry that we don't have an easier solution for you at the moment.

@bughou
Copy link

bughou commented Apr 9, 2024 •
edited

when building from source in an linux/arm64 Docker container，I encountered the following error, any help is appreciated.

$ ./scripts/setup.sh --local
[2024-04-09 02:48:31,252] [INFO] ['opam', 'switch', 'set', '4.14.0', '--root', '/home/opam/.opam']
[2024-04-09 02:48:32,900] [INFO] Activating opam
[2024-04-09 02:48:32,901] [INFO] ['opam', 'env', '--yes', '--switch', '4.14.0', '--root', '/home/opam/.opam', '--set-root', '--set-switch']
[2024-04-09 02:48:32,929] [INFO] OPAMROOT="/home/opam/.opam"
[2024-04-09 02:48:32,929] [INFO] OPAMSWITCH="4.14.0"
[2024-04-09 02:48:32,929] [INFO] OPAM_SWITCH_PREFIX="/home/opam/.opam/4.14.0"
[2024-04-09 02:48:32,929] [INFO] CAML_LD_LIBRARY_PATH="/home/opam/.opam/4.14.0/lib/stublibs:/home/opam/.opam/4.14.0/lib/ocaml/stublibs:/home/opam/.opam/4.14.0/lib/ocaml"
[2024-04-09 02:48:32,929] [INFO] OCAML_TOPLEVEL_PATH="/home/opam/.opam/4.14.0/lib/toplevel"
[2024-04-09 02:48:32,930] [INFO] MANPATH=":/home/opam/.opam/4.14.0/man"
[2024-04-09 02:48:32,930] [INFO] PATH="/home/opam/.opam/4.14.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
[2024-04-09 02:48:32,930] [INFO] ['opam', 'install', '--yes', 'base64.3.5.0', 'core.v0.15.1', 'core_unix.v0.15.2', 're2.v0.15.0', 'dune.3.4.1', 'yojson.2.0.2', 'ppx_deriving_yojson.3.7.0', 'ounit.2.2.6', 'menhir.20220210', 'lwt.5.6.1', 'lwt_ppx.2.1.0', 'ounit2-lwt.2.2.6', 'pyre-ast.0.1.8', 'mtime.1.4.0']
[ERROR] The compilation of ocaml_intrinsics failed at "/home/opam/.opam/4.14.0/bin/dune build -p ocaml_intrinsics -j 63".
#=== ERROR while compiling ocaml_intrinsics.v0.15.2 ===========================#
# context              2.0.10 | linux/arm64 | ocaml-base-compiler.4.14.0 | git+https://mirrors.sjtug.sjtu.edu.cn/git/opam-repository.git
# path                 ~/.opam/4.14.0/.opam-switch/build/ocaml_intrinsics.v0.15.2
# command              ~/.opam/4.14.0/bin/dune build -p ocaml_intrinsics -j 63
# exit-code            1
# env-file             ~/.opam/log/ocaml_intrinsics-22-041a[19](https://scsdev.antiy.cn/sast/pyre-check/-/jobs/35015#L19).env
# output-file          ~/.opam/log/ocaml_intrinsics-22-041a19.out
### output ###
# File "src/dune", line 2, characters 46-55:
# 2 |  (c_names int_stubs float_stubs perfmon_stubs crc_stubs ext_pointer_stubs
#                                                   ^^^^^^^^^
# (cd _build/default/src && /usr/bin/gcc -std=gnu99 -O2 -fno-strict-aliasing -fwrapv -fno-builtin-memcmp -pthread -fPIC -D_FILE_OFFSET_BITS=64 -std=gnu99 -O2 -fno-strict-aliasing -fwrapv -fno-builtin-memcmp -pthread -fPIC -g -I /home/opam/.opam/4.14.0/lib/ocaml -o crc_stubs.o -c crc_stubs.c)
# crc_stubs.c: In function 'crc64':
# crc_stubs.c:16:2: error: #error "Target not supported"
#  #error "Target not supported"
#   ^
# (cd _build/default/src && /usr/bin/gcc -std=gnu99 -O2 -fno-strict-aliasing -fwrapv -fno-builtin-memcmp -pthread -fPIC -D_FILE_OFFSET_BITS=64 -std=gnu99 -O2 -fno-strict-aliasing -fwrapv -fno-builtin-memcmp -pthread -fPIC -g -I /home/opam/.opam/4.14.0/lib/ocaml -o prefetch_stubs.o -c prefetch_stubs.c)
# prefetch_stubs.c: In function 'caml_pause_hint':
# prefetch_stubs.c:271:2: warning: #warning "This target does not support PAUSE hints, emit NOP instead." [-Wcpp]
#  #warning "This target does not support PAUSE hints, emit NOP instead."
#   ^
The former state can be restored with:
    opam switch import "/home/opam/.opam/4.14.0/.opam-switch/backup/state-[20](https://scsdev.antiy.cn/sast/pyre-check/-/jobs/35015#L20)240409024834.export"
Traceback (most recent call last):
  File "/builds/sast/pyre-check/./scripts/setup.py", line 347, in <module>
    setup(Setup)
  File "/builds/sast/pyre-check/./scripts/setup.py", line 339, in setup
    runner.full_setup(
  File "/builds/sast/pyre-check/./scripts/setup.py", line 241, in full_setup
    ] = self.set_opam_switch_and_install_dependencies()
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builds/sast/pyre-check/./scripts/setup.py", line 225, in set_opam_switch_and_install_dependencies
    self.run(
  File "/builds/sast/pyre-check/./scripts/setup.py", line 279, in run
    output = subprocess.check_output(
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['opam', 'install', '--yes', 'base64.3.5.0', 'core.v0.15.1', 'core_unix.v0.15.2', 're2.v0.15.0', 'dune.3.4.1', 'yojson.2.0.2', 'ppx_deriving_yojson.3.7.0', 'ounit.2.2.6', 'menhir.20220[21](https://scsdev.antiy.cn/sast/pyre-check/-/jobs/35015#L21)0', 'lwt.5.6.1', 'lwt_ppx.2.1.0', 'ounit2-lwt.2.2.6', 'pyre-ast.0.1.8', 'mtime.1.4.0']' returned non-zero exit status 31.

@cyounkins
Copy link

Set the container platform to be amd64: https://stackoverflow.com/questions/65612411/forcing-docker-to-use-linux-amd64-platform-by-default-on-macos

And you can set the "Use Rosetta for x86_64/amd64 emulation on Apple Silicon" option under in docker desktop -> settings -> general, which hopefully makes it faster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@cyounkins @bughou @connernilsen @Peteling @jepify