Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add WASI (WebAssembly) cross compile support #550

Merged
merged 3 commits into from
May 21, 2024

Conversation

agoode
Copy link
Contributor

@agoode agoode commented Feb 13, 2024

Fixes #549.

See https://github.com/agoode/mlton/blob/wascally/doc/guide/src/RunningOnWASI.adoc for instructions.

I've only tested with macOS, but I could try Linux also.

This makes a bunch of improvements to cross compilation support in general, since WASI is really always going to be a cross target.

Potentially controversial is leaving unimplemented POSIX functions missing from wasi.c. I think it works well to have this become a linker error instead of a runtime error, but I am open to suggestions.

Thanks to @jcreedcmu for tricking me into doing this.

@jcreedcmu
Copy link

jcreedcmu commented Feb 13, 2024

I've also successfully tested this PR on linux. For what it's worth, here is the detailed script of what I ran to build (although I think @agoode 's existing documentation in this PR suffices)

Additionally, I was able to build a fairly nontrivial project, namely the twelf logical framework, and it executed fine on a WASI runtime. A demo is available at https://jcreedcmu.github.io/twelf-wasm/ --- this was exactly the thing that I hoped this PR would enable. This exercises at least file IO as a proof of concept.

######### wasi-sdk
cd $BUILD
git clone --recursive https://github.com/WebAssembly/wasi-sdk.git
cd wasi-sdk
sudo apt install cmake
sudo apt install clang
sudo apt install ninja-build
NINJA_FLAGS=-v make package

######### gmp
cd $BUILD
export WASISDK=$BUILD/wasi-sdk/build/install/opt/wasi-sdk
wget https://gmplib.org/download/gmp/gmp-6.3.0.tar.xz
tar xvf gmp-6.3.0.tar.xz
cd gmp-6.3.0
./configure --host=wasm32-unknown-wasi \
            CC=$WASISDK/bin/clang \
            RANLIB=$WASISDK/bin/ranlib \
            CFLAGS=-D_WASI_EMULATED_SIGNAL LDFLAGS=-lwasi-emulated-signal \
            --prefix=$BUILD/gmp-wasi-INSTALL
make

######### wasmtime
curl https://wasmtime.dev/install.sh -sSf | bash

######### mlton binary
cd $BUILD
git clone https://github.com/agoode/mlton agoode-mlton
cd agoode-mlton
export MLTON=$BUILD/mlton-INSTALL
make all
make PREFIX=$MLTON install

######### mlton wasm runtime
make clean
make CC=$WASISDK/bin/clang \
     AR=$WASISDK/bin/ar \
     RANLIB=$WASISDK/bin/ranlib \
     TARGET_OS=wasi \
     TARGET_ARCH=wasm32 \
     TARGET=wasm32-unknown-wasi \
     WITH_GMP_DIR=$BUILD/gmp-wasi-INSTALL \
     PREFIX=$MLTON \
     dirs runtime install-runtime

######### twelf
cd $BUILD/twelf
git checkout jcreed/mlton-wasm
$MLTON/bin/mlton -target wasm32-unknown-wasi build/twelf-server-mlton-wasm.mlb

######### run twelf
wasmtime ./build/twelf-server-mlton-wasm
# Twelf 1.7.1+ (r, built 02/03/24 at 15:21:14 on system76-pc)
# %% OK %%

@MatthewFluet
Copy link
Member

This looks very nice; I'm pleasantly surprised at how minimal the PR is to support WebAssembly/WASI.

@agoode agoode force-pushed the wascally branch 3 times, most recently from ce17fa2 to af7f6ea Compare February 25, 2024 18:17
jcreedcmu pushed a commit to jcreedcmu/twelf that referenced this pull request Feb 29, 2024
[This pull request against
MLton](MLton/mlton#550) enables compiling to
WebAssembly, so that twelf can be run from, e.g. inside a web browser.

A demo can be seen at:
https://jcreedcmu.github.io/twelf-wasm/

Test plan:
- ensure that the make target `twelf-server-mlton` still produces a
twelf-server binary
- ensure that the make target `wasi` produces
`bin/twelf.wasm`, when an appropriate version of `mlton` is installed.
Detailed instructions for building follow. Comments on the above pull
request have more details pertaining to build and install of GMP.

Checked out
https://github.com/agoode/mlton/tree/wasm2
at commit
MLton/mlton@d2b9e5d
put into
`$BUILD/dev-mlton`

and then did the following:

```
MLTON=$BUILD/mlton-wasm2-INSTALL

cd $BUILD/dev-mlton
make clean
make CC=$WASISDK/bin/clang \
     AR=$WASISDK/bin/ar \
     RANLIB=$WASISDK/bin/ranlib \
     TARGET_OS=wasi \
     TARGET_ARCH=wasm32 \
     TARGET=wasm32-unknown-wasi \
     WITH_GMP_DIR=$BUILD/gmp-wasi-INSTALL \
     PREFIX=$MLTON \
     dirs runtime install-runtime

cd $BUILD/dev-mlton
make clean
make all
make PREFIX=$MLTON install

cd $BUILD/twelf
mlton=$BUILD/mlton-wasm2-INSTALL/bin/mlton make wasi
```
robsimmons pushed a commit to standardml/twelf that referenced this pull request Feb 29, 2024
[This pull request against
MLton](MLton/mlton#550) enables compiling to
WebAssembly, so that twelf can be run from, e.g. inside a web browser.

A demo can be seen at:
https://jcreedcmu.github.io/twelf-wasm/

Test plan:
- ensure that the make target `twelf-server-mlton` still produces a
twelf-server binary
- ensure that the make target `wasi` produces
`bin/twelf.wasm`, when an appropriate version of `mlton` is installed.
Detailed instructions for building follow. Comments on the above pull
request have more details pertaining to build and install of GMP.

Checked out
https://github.com/agoode/mlton/tree/wasm2
at commit
MLton/mlton@d2b9e5d
put into
`$BUILD/dev-mlton`

and then did the following:

```
MLTON=$BUILD/mlton-wasm2-INSTALL

cd $BUILD/dev-mlton
make clean
make CC=$WASISDK/bin/clang \
     AR=$WASISDK/bin/ar \
     RANLIB=$WASISDK/bin/ranlib \
     TARGET_OS=wasi \
     TARGET_ARCH=wasm32 \
     TARGET=wasm32-unknown-wasi \
     WITH_GMP_DIR=$BUILD/gmp-wasi-INSTALL \
     PREFIX=$MLTON \
     dirs runtime install-runtime

cd $BUILD/dev-mlton
make clean
make all
make PREFIX=$MLTON install

cd $BUILD/twelf
mlton=$BUILD/mlton-wasm2-INSTALL/bin/mlton make wasi
```

Co-authored-by: Jason Reed <[email protected]>
@agoode
Copy link
Contributor Author

agoode commented Mar 4, 2024

There are some improvements coming to the wasi-sdk (notably WebAssembly/wasi-sdk#388), so I would maybe avoid merging this PR right now in favor of a more simple one that may come soon.

@agoode agoode marked this pull request as draft March 4, 2024 21:32
jcreedcmu pushed a commit to jcreedcmu/twelf-wasm that referenced this pull request Mar 8, 2024
My goal is to set down once and for all a reproducible build process
that shows how the `twelf.wasm` was constructed, instead of it
existing ad hoc across my memory and comments on the
PR MLton/mlton#550 . Already I discovered a
couple of crucial omissions in those comments.

This still deserves some cleanup, as the ordering of steps is not
really a coherent narrative, because I added some as I discovered
them. A cleaner one is forthcoming.
@agoode agoode force-pushed the wascally branch 2 times, most recently from ce57803 to dc9e1c5 Compare March 15, 2024 18:55
@MatthewFluet
Copy link
Member

@agoode is this PR ready to be merged? It seems that it is working well for the Twelf project.

@agoode
Copy link
Contributor Author

agoode commented May 13, 2024

I'm going to split it up into a few smaller PRs. There are some good changes that can go in now, and a few others I want to revisit for the updated wasi-sdk-22. I'll send some new PRs when they are ready.

@agoode agoode marked this pull request as ready for review May 19, 2024 18:36
@MatthewFluet
Copy link
Member

Looks good. (Either I missed it or changing to "ready for review" didn't re-raise the issue in my notifications.)
In runtime/Makefile, I will probably change the WASMTIME variable to something less WASM-specific, since it could be useful for other wrappers that allow one to emulate a target system on the build system.

@MatthewFluet MatthewFluet merged commit 359be75 into MLton:master May 21, 2024
12 checks passed
@agoode agoode deleted the wascally branch May 21, 2024 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support wasm target
3 participants