Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker build for multi-architecture amd/arm #1165

Open
wants to merge 18 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 15 additions & 11 deletions .github/workflows/ci-build-manual-crf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,17 +32,21 @@ jobs:
- name: Create more disk space
run: sudo rm -rf /usr/share/dotnet && sudo rm -rf /opt/ghc && sudo rm -rf "/usr/local/share/boost" && sudo rm -rf "$AGENT_TOOLSDIRECTORY"
- uses: actions/checkout@v4
- name: Build and push
id: docker_build
uses: mr-smithers-excellent/docker-build-push@v6
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub
uses: docker/login-action@v1
with:
dockerfile: Dockerfile.crf
username: ${{ secrets.DOCKERHUB_USERNAME_LFOPPIANO }}
password: ${{ secrets.DOCKERHUB_TOKEN_LFOPPIANO }}
image: lfoppiano/grobid
registry: docker.io
pushImage: true
tags: |
latest-develop, ${{ github.event.inputs.custom_tag}}
- name: Image digest
run: echo ${{ steps.docker_build.outputs.digest }}
- name: Build and push
uses: docker/build-push-action@v3
with:
file: Dockerfile.crf
platforms: linux/amd64,linux/arm64
push: true
tags: |
lfoppiano/grobid:latest-develop, lfoppiano/grobid:${{ github.event.inputs.custom_tag}}
17 changes: 14 additions & 3 deletions Dockerfile.crf
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# -------------------
# build builder image
# -------------------
FROM openjdk:17-jdk-slim as builder
FROM eclipse-temurin:17-jdk-focal as builder

USER root

Expand Down Expand Up @@ -63,7 +63,7 @@ RUN rm -rf grobid-source
# -------------------
# build runtime image
# -------------------
FROM openjdk:17-slim
FROM eclipse-temurin:17-jre-focal

RUN apt-get update && \
apt-get -y upgrade && \
Expand All @@ -80,7 +80,18 @@ WORKDIR /opt/grobid

COPY --from=builder /opt/grobid .

ENV GROBID_SERVICE_OPTS "-Djava.library.path=grobid-home/lib/lin-64:grobid-home/lib/lin-64/jep --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/sun.nio.ch=ALL-UNNAMED --add-opens java.base/java.io=ALL-UNNAMED"
ARG TARGETARCH
ARG PROCESS_MECHANISM

RUN if [ "$TARGETARCH" = "arm64" ]; then \
PROCESS_MECHANISM="-Djdk.lang.Process.launchMechanism=vfork "; \
else \
PROCESS_MECHANISM=""; \
fi

RUN echo "Selected process spawning mechanism: $PROCESS_MECHANISM"

ENV GROBID_SERVICE_OPTS "$PROCESS_MECHANISM-Djava.library.path=grobid-home/lib/lin-64:grobid-home/lib/lin-64/jep --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/sun.nio.ch=ALL-UNNAMED --add-opens java.base/java.io=ALL-UNNAMED"

CMD ["./grobid-service/bin/grobid-service"]

Expand Down
20 changes: 19 additions & 1 deletion doc/Grobid-docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ It is then possible to select other Deep Learning models for other processing st

## CRF-only image

The process for retrieving and running the image is as follow:
The process for retrieving and running the image is as follows:

- Pull the image from docker HUB (check the [latest version number](https://hub.docker.com/r/lfoppiano/grobid/tags)):

Expand Down Expand Up @@ -115,6 +115,24 @@ Access the service:

Grobid web services are then available as described in the [service documentation](https://grobid.readthedocs.io/en/latest/Grobid-service/).

### ARM64

Since version 0.8.1 the CRF-only image is available for both x86_64 and arm64.
The `lfoppiano/grobid:{latest_grobid_version}` support both architectures by default.

To run it on arm64, use the following command:

```bash
docker run --ulimit core=0 --platform linux/amd64 --init -p 8070:8070 --name grobid lfoppiano/grobid:{latest_grobid_version}
```

As discussed [here](https://github.com/kermitt2/grobid/issues/1119#issuecomment-2362204879), it is mandatory to provide the parameter `--platform linux/amd64` to force the usage of amd64 architecture and avoid the raising of the rosetta error:

```
rosetta error: failed to open elf at /lib64/ld-linux-x86-64.so.2
```

**NOTE**: The arm64 image has been tested only on `macOs` (not linux `arm64`).

## Configure using the yaml config file

Expand Down
Loading