Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

iOS support? #1170

Open
adamfeuer opened this issue Jan 22, 2024 · 12 comments
Open

iOS support? #1170

adamfeuer opened this issue Jan 22, 2024 · 12 comments
Assignees
@samuelcolvin
Labels
unconfirmed

Comments

@adamfeuer
Copy link

adamfeuer commented Jan 22, 2024 •
edited

pydantic and pydantic-core do not appear to build on iOS. Do you plan to add support for it?

Or if it is supported, where can I find documentation for it?

I am using the Beeware mobile-forge and Briefcase and can't generate a binary wheel for pydantic-core, which means I can't install it on iOS.

Many libraries use Pydantic, which blocks them from being used on iOS. For example, spaCy, Presidio, thinc, and others.
@adamfeuer
Copy link
Author

Here's my mobile-forge recipe:

package:
  name: pydantic-core
  version: 2.14.6

Here's the output from the forge build:

$ forge iphoneos:12.0:arm64 pydantic-core
================================================================================
Building pydantic-core 2.14.6 for ios_12_0_iphoneos_arm64
================================================================================

[venv3.10-ios_12_0_iphoneos_arm64] Unpack sources
Unpacking downloads/pydantic-core-2.14.6.tar.gz...

[venv3.10-ios_12_0_iphoneos_arm64] Apply patches
No patches to apply.

[venv3.10-ios_12_0_iphoneos_arm64] Create clean build environment
Creating venv3.10-ios_12_0_iphoneos_arm64...

Verifying cross-platform environment...
done.

Cross platform-environment venv3.10-ios_12_0_iphoneos_arm64 created.

[venv3.10-ios_12_0_iphoneos_arm64] Install forge host requirements
No host requirements.

[venv3.10-ios_12_0_iphoneos_arm64] Install forge build requirements
No build requirements.

[venv3.10-ios_12_0_iphoneos_arm64] Install pyproject.toml build requirements

Looking in links: /Users/adam/src/mobile-forge/dist
Collecting build
Using cached build-1.0.3-py3-none-any.whl (18 kB)
Collecting wheel
Using cached wheel-0.42.0-py3-none-any.whl (65 kB)
Processing ./dist/maturin-1.4.0-cp310-cp310-ios_12_0_iphoneos_arm64.whl
Collecting typing-extensions!=4.7.0,>=4.6.0
Using cached typing_extensions-4.9.0-py3-none-any.whl (32 kB)
Collecting pyproject_hooks
Using cached pyproject_hooks-1.0.0-py3-none-any.whl (9.3 kB)
Collecting tomli>=1.1.0
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting packaging>=19.0
Using cached packaging-23.2-py3-none-any.whl (53 kB)
Installing collected packages: wheel, typing-extensions, tomli, packaging, pyproject_hooks, maturin, build
Successfully installed build-1.0.3 maturin-1.4.0 packaging-23.2 pyproject_hooks-1.0.0 tomli-2.0.1 typing-extensions-4.9.0 wheel-0.42.0

Looking in links: /Users/adam/src/mobile-forge/dist
Collecting build
Using cached build-1.0.3-py3-none-any.whl (18 kB)
Collecting wheel
Using cached wheel-0.42.0-py3-none-any.whl (65 kB)
Collecting maturin<2,>=1
Downloading maturin-1.4.0-py3-none-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl (15.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.7/15.7 MB 33.5 MB/s eta 0:00:00
Collecting typing-extensions!=4.7.0,>=4.6.0
Using cached typing_extensions-4.9.0-py3-none-any.whl (32 kB)
Collecting pyproject_hooks
Using cached pyproject_hooks-1.0.0-py3-none-any.whl (9.3 kB)
Collecting tomli>=1.1.0
Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting packaging>=19.0
Using cached packaging-23.2-py3-none-any.whl (53 kB)
Installing collected packages: wheel, typing-extensions, tomli, packaging, pyproject_hooks, maturin, build
Successfully installed build-1.0.3 maturin-1.4.0 packaging-23.2 pyproject_hooks-1.0.0 tomli-2.0.1 typing-extensions-4.9.0 wheel-0.42.0

* Getting build dependencies for wheel...
* Building wheel...
Running `maturin pep517 build-wheel -i /Users/adam/src/mobile-forge/build/cp310/pydantic-core/2.14.6/venv3.10-ios_12_0_iphoneos_arm64/venv3.10-ios_12_0_iphoneos_arm64/bin/python --compatibility off`
💥 maturin failed
Caused by: Cargo metadata failed. Do you have cargo in your PATH?
Caused by: No such file or directory (os error 2)
Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/Users/adam/src/mobile-forge/build/cp310/pydantic-core/2.14.6/venv3.10-ios_12_0_iphoneos_arm64/venv3.10-ios_12_0_iphoneos_arm64/bin/python', '--compatibility', 'off'] returned non-zero exit status 1

ERROR Backend subprocess exited when trying to invoke build_wheel
********************************************************************************
Failed build: pydantic-core 2.14.6 for iphoneos 12.0 on arm64
********************************************************************************
Traceback (most recent call last):
  File "/Users/adam/src/mobile-forge/src/forge/build.py", line 267, in build
    self._build()
  File "/Users/adam/src/mobile-forge/src/forge/build.py", line 518, in _build
    self.cross_venv.run(
  File "/Users/adam/src/mobile-forge/src/forge/cross.py", line 356, in run
    return subprocess.run(logfile, *args, **self.cross_kwargs(kwargs))
  File "/Users/adam/src/mobile-forge/src/forge/subprocess.py", line 49, in run
    raise stdlib_subprocess.CalledProcessError(return_code, args)
subprocess.CalledProcessError: Command '(['python', '-m', 'build', '--no-isolation', '--wheel', '--outdir', '/Users/adam/src/mobile-forge/dist'],)' returned non-zero exit status 1.
Failed builds for:
 * pydantic-core (default version) (ios_12_0_iphoneos_arm64)

@adamfeuer
Copy link
Author

It appears that this is because maturin does not support iOS. If I run the command that the bdist_wheel script is trying to run, I get this:

$ maturin pep517 build-wheel -i /Users/adam/src/mobile-forge/build/cp310/pydantic-core/2.14.6/venv3.10-ios_12_0_iphoneos_arm64/venv3.10-ios_12_0_iphoneos_arm64/bin/python --compatibility off --target aarch64-apple-ios
📦 Including license file "/Users/adam/src/mobile-forge/build/cp310/pydantic-core/2.14.6/LICENSE"
🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings
💥 maturin failed
  Caused by: The operating system Ios is not supported

I looked at trying to add iOS support to maturin, but my changes don't seem to work. Here's the branch with my iOS changes:

https://github.com/adamfeuer/maturin/tree/feature/ios-support

When I use my changed maturin, I get the following output:

$ ~/src/maturin/target/release/maturin pep517 build-wheel -i python3.10 --compatibility off --target aarch64-apple-ios
📦 Including license file "/Users/adam/src/mobile-forge/build/cp310/pydantic-core/2.14.6/LICENSE"
🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings
🐍 Found CPython 3.10 at /Users/adam/src/mobile-forge/venv3.10/bin/python3.10
📡 Using build options features, bindings from pyproject.toml
   Compiling autocfg v1.1.0
   Compiling proc-macro2 v1.0.69
   Compiling quote v1.0.29
   Compiling unicode-ident v1.0.10
   Compiling target-lexicon v0.12.9
   Compiling python3-dll-a v0.2.9
   Compiling once_cell v1.18.0
   Compiling libc v0.2.147
   Compiling static_assertions v1.1.0
   Compiling heck v0.4.1
   Compiling version_check v0.9.4
   Compiling cfg-if v1.0.0
   Compiling lexical-util v0.8.5
   Compiling rustversion v1.0.13
   Compiling parking_lot_core v0.9.8
   Compiling tinyvec_macros v0.1.1
   Compiling smallvec v1.11.1
   Compiling scopeguard v1.1.0
   Compiling tinyvec v1.6.0
   Compiling ahash v0.8.6
   Compiling num-traits v0.2.16
   Compiling lock_api v0.4.10
   Compiling num-integer v0.1.45
   Compiling memoffset v0.9.0
   Compiling num-bigint v0.4.4
   Compiling lexical-parse-integer v0.8.6
   Compiling lexical-write-integer v0.8.5
   Compiling memchr v2.6.3
   Compiling pyo3-build-config v0.20.0
   Compiling serde v1.0.190
   Compiling syn v2.0.38
   Compiling aho-corasick v1.0.2
   Compiling lexical-write-float v0.8.5
   Compiling lexical-parse-float v0.8.5
   Compiling unicode-normalization v0.1.22
   Compiling getrandom v0.2.10
   Compiling parking_lot v0.12.1
   Compiling equivalent v1.0.1
   Compiling percent-encoding v2.3.0
   Compiling serde_json v1.0.108
   Compiling hashbrown v0.14.0
   Compiling indoc v2.0.4
   Compiling unicode-bidi v0.3.13
   Compiling regex-syntax v0.8.2
   Compiling zerocopy v0.7.20
   Compiling unindent v0.2.3
   Compiling idna v0.4.0
   Compiling indexmap v2.0.0
   Compiling form_urlencoded v1.2.0
   Compiling lexical-core v0.8.5
   Compiling ryu v1.0.14
   Compiling itoa v1.0.8
   Compiling base64 v0.21.5
   Compiling url v2.4.1
   Compiling uuid v1.5.0
   Compiling pyo3-ffi v0.20.0
   Compiling pyo3 v0.20.0
   Compiling pydantic-core v2.14.6 (/Users/adam/src/mobile-forge/build/cp310/pydantic-core/2.14.6)
   Compiling pyo3-macros-backend v0.20.0
error: failed to run custom build command for `pyo3-ffi v0.20.0`

Caused by:
  process didn't exit successfully: `/Users/adam/src/mobile-forge/build/cp310/pydantic-core/2.14.6/target/release/build/pyo3-ffi-b9fbb289a993548c/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-env-changed=PYO3_CROSS
  cargo:rerun-if-env-changed=PYO3_CROSS_LIB_DIR
  cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_VERSION
  cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_IMPLEMENTATION
  cargo:rerun-if-env-changed=PYO3_NO_PYTHON

  --- stderr
  error: PYO3_CROSS_PYTHON_VERSION or an abi3-py3* feature must be specified when cross-compiling and PYO3_CROSS_LIB_DIR is not set.
warning: build failed, waiting for other jobs to finish...
💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_ENVIRONMENT_SIGNATURE="cpython-3.10-64bit" PYO3_PYTHON="/Users/adam/src/mobile-forge/venv3.10/bin/python3.10" PYTHON_SYS_EXECUTABLE="/Users/adam/src/mobile-forge/venv3.10/bin/python3.10" "cargo" "rustc" "--features" "pyo3/extension-module" "--target" "aarch64-apple-ios" "--message-format" "json-render-diagnostics" "--manifest-path" "/Users/adam/src/mobile-forge/build/cp310/pydantic-core/2.14.6/Cargo.toml" "--release" "--lib" "--crate-type" "cdylib"

Do you know where the problem lies? In maturin, in pydantic-core, or both? I'm happy to work on this, I'm just not sure where to go from here.

@samuelcolvin
Copy link
Member

samuelcolvin commented Jan 22, 2024 •
edited

If rust/maturin/pyo3 can build for iOS, we would absolutely support it. Until then there's nothing we can do.

When I checked last, there were virtually no (10s out of 250m) of installs of pydantic v1 on iOS. So it's very rarely used.

You might try the wasm build which should work.

@davidhewitt
Copy link
Contributor

@adamfeuer I think the best course of action is to start with an issue / PR in maturin to understand how it should detect the mobile-forge build configuration, and then we can proceed from there to land changes in PyO3 also if needed.

@adamfeuer
Copy link
Author

@samuelcolvin @davidhewitt Thanks for getting back to me so fast!

I'll pursue the maturin modifications– open a PR and try to get someone with my iOS modifications. I'll report back here.

@samuelcolvin In the meantime I could use pydantic v1 because the other Python modules spacy can use v1. It looks like v1 is built by cargo... I can cross compile other rust modules, so I'll give that a shot, if it works it could solve my immediate problem.

@davidhewitt
Copy link
Contributor

@adamfeuer v1 doesn't have Rust at all ;)

@adamfeuer
Copy link
Author

@davidhewitt Ah! Thank you! That is great! If I need to see the source code, is it on the v1 branch?

Do you know how the other iOS users installed pydantic v1?

@davidhewitt
Copy link
Contributor

v1 is pure python. (It was optimized by Cython at build time, but you can skip that if needed.)

I believe the maintenance branch is at https://github.com/pydantic/pydantic/tree/1.10.X-fixes

Note that we don't intend to support v1 much longer, and we only merge occasional bugfixes now.

@samuelcolvin
Copy link
Member

V1 is on the 1.10.X-fixes branch. Pydantic-core didn't exist until V2.

I've updated my comment above to say.

You might try the wasm build which should work.

As you'll see here we already have a preview of running pydantic-core with webassembly, if that runs for you in iOS, it might be the easiest solution.

@adamfeuer
Copy link
Author

adamfeuer commented Jan 22, 2024 •
edited

@davidhewitt Pure python, perfect. I just built it for iOS using the mobile-forge, so it looks like that will work. I have to rebuild the other stack of software that uses it now.

Re: the 1.10.x branch not being supported soon, I understand. If I can get this working with 1.10.x, it will solve my immediate problem, and then I can devote some effort to fixing maturin for iOS.

@samuelcolvin I'll try the wasm if the pure python 1.10.x doesn't work.

Thank you both so much for your fast help! I'll post here if I get this working, and also let you know what's happening with maturin.

@messense
Copy link
Contributor

@adamfeuer See PyO3/maturin#1742

@adamfeuer
Copy link
Author

@messense Thank you for letting me know about this! Wonderful news!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@samuelcolvin samuelcolvin

Labels
unconfirmed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@adamfeuer @messense @davidhewitt @samuelcolvin