Skip to content

Commit e81b6ae

Browse files
committed
support for docker buildx, for debian12/ubuntu2204/ubuntu2404/ol7/ol8/ol9/amzn2023/alpine321, on arm64 and amd64
- multiarch build (arm64/amd64) - parallel builds - supported platforms: - oraclelinux:7,oraclelinux:8,oraclelinux:9 - debian:12 - amazonlinux:2023 - ubuntu:22.04,ubuntu:24.04 - alpine:3.21
1 parent fca1883 commit e81b6ae

12 files changed

+1519
-1
lines changed

.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,12 @@ Makefile.in
3030
/*.ign/
3131
*.log
3232

33+
# from buildx
34+
*.rpm
35+
*.deb
36+
*.tar.gz
37+
*.apk
38+
3339
# from pkcs11-tools
3440
.pkcs11rc
3541
pkcs11rc

CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,9 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [UNRELEASED]
9+
- support for building the project - multiplatform builds using docker
10+
811
## [1.8.0] - 2024-11-27
912
### Added
1013
- custom mechanisms are printed

README.md

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,15 @@
11
# libpkcs11shim : a shim library for PKCS#11
22

33
## Introduction
4+
45
`libpkcs11shim` is a shim library that you insert between an application and a target PKCS#11 library. This project is actually a fork on a small part of [OpenSC project on GitHub](https://github.com/OpenSC/OpenSC), called [`pkcs11-spy`](src/pkcs11/pkcs11-spy.c). In addition to `pkcs11-spy`, `libpkcs11shim` adds some capabilities:
6+
57
- cleaner log output
68
- ability to capture logs in a multithreaded environment
79
- ability to carry on capture upon fork of the calling process
810
- provides a deferred logging capability, reducing significantly the impact on performance on a library being logged (at the cost of extra memory allocation)
911
- microsecond resolution for API call, allowing to identify library performance problems
10-
- hides passphrase information by default (that can be overriden by an environment variable, see options below)
12+
- hides passphrase information by default (that can be overriden by an environment variable)
1113

1214
## Download
1315
Releases are hosted on Github: https://github.com/Mastercard/libpkcs11shim/releases/
@@ -44,6 +46,27 @@ To build the library:
4446
4. execute `make`
4547
5. optionally, `make install`. The library is named `libpkcs11shim.so` and is deployed by default to `/usr/local/lib`.
4648

49+
## Building with docker
50+
51+
It is possible to build the library artifacts using docker. there is a script called `./buildx.sh` that can perform multi-arch builds, for the following platforms:
52+
53+
- `oraclelinux:7`, `oraclelinux:8` and `oraclelinux:9` ( RPM and tar.gz )
54+
- `amazonlinux:2023` ( RPM and tar.gz )
55+
- `debian:12` ("bookworm") ( DEB and tar.gz )
56+
- `ubuntu:22.04`,`ubuntu24:04` ( DEB and tar.gz )
57+
- `alpinelinux:3.21` ( APK with a dummy signature and tar.gz )
58+
59+
You need to have a working docker environment.
60+
61+
- To build ubuntu:24.04 artifacts for the host architecture, use `./buildx.sh ubuntu2404`
62+
- To build ubuntu:24.04 artifacts in verbose mode, use `./buildx.sh -v ubuntu2404`
63+
- To build ubuntu:24.04 artifacts in very verbose mode, use `./buildx.sh -v ubuntu2404`
64+
- To build ubuntu:24.04 artifacts for amd64, use `./buildx.sh ubuntu2404/amd64` (you must have docker configured properly if this is not your host architecture)
65+
- To build ubuntu:24.04 artifacts for arm64, use `./buildx.sh ubuntu2404/arm64` (you must have docker configured properly if this is not your host architecture)
66+
- To build ubuntu:24.04 artifacts for arm64 and amd64, use `./buildx.sh ubuntu2404/all` (you must have docker configured properly if this is not your host architecture)
67+
- execute the script without arguments for further help.
68+
69+
4770
## Output format
4871

4972
### Fields description

buildx.sh

Lines changed: 239 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,239 @@
1+
#!/usr/bin/env bash
2+
#
3+
#
4+
# pkcs11shim : a PKCS#11 shim library
5+
#
6+
# This work is based upon OpenSC pkcs11spy (https://github.com/OpenSC/OpenSC.git)
7+
#
8+
# Copyright (C) 2020 Mastercard
9+
#
10+
# This library is free software; you can redistribute it and/or
11+
# modify it under the terms of the GNU Lesser General Public
12+
# License as published by the Free Software Foundation; either
13+
# version 2.1 of the License, or (at your option) any later version.
14+
#
15+
# This library is distributed in the hope that it will be useful,
16+
# but WITHOUT ANY WARRANTY; without even the implied warranty of
17+
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
18+
# Lesser General Public License for more details.
19+
#
20+
# You should have received a copy of the GNU Lesser General Public
21+
# License along with this library; if not, write to the Free Software
22+
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
23+
#
24+
##############################################################################
25+
# This script builds the libpkcs11-shim tarball for the given distro and arch.
26+
# The script uses Docker Buildx to build the tarball in a container.
27+
# The tarball is output to the current directory.
28+
#
29+
set -e
30+
31+
PACKAGE="libpkcs11shim"
32+
GITHUB_REPO="https://github.com/Mastercard/libpkcs11shim"
33+
SUPPORTED_ARCHS="amd64 arm64"
34+
SUPPORTED_DISTROS="ol7 ol8 ol9 deb12 ubuntu2204 ubuntu2404 amzn2023 alpine321"
35+
36+
# Declare an associative array, needed by docker buildx --platform option
37+
declare -A rev_arch_map
38+
rev_arch_map["x86_64"]="amd64"
39+
rev_arch_map["aarch64"]="arm64"
40+
41+
#
42+
# Usage: buildx.sh [--repo URL | -r URL] [--verbose | -v] [--max-procs N | -j N] [distro[/arch]|all[/all]] [...]
43+
#
44+
function usage() {
45+
echo "Usage: $0 [--repo URL | -r URL] [--verbose | -v] [--max-procs N | -j N] [distro[/arch]|all[/all]] [...]"
46+
echo "Supported distros: $SUPPORTED_DISTROS"
47+
echo "Supported archs: $SUPPORTED_ARCHS"
48+
echo ""
49+
echo "Options:"
50+
echo " --repo URL, -r URL Specify the repository URL"
51+
echo " --verbose, -v Increase verbosity (can be specified multiple times)"
52+
echo " --max-procs N, -j N Specify the maximum number of processes"
53+
exit 1
54+
}
55+
56+
#
57+
# Get the current directory
58+
#
59+
function get_current_dir() {
60+
current_dir="$(pwd)"
61+
echo "${current_dir}"
62+
}
63+
64+
#
65+
# Get the directory of the script
66+
#
67+
function get_script_dir() {
68+
script_dir="$(cd "$(dirname "$0")" && pwd)"
69+
echo "${script_dir}"
70+
}
71+
72+
#
73+
# Generate a random container name
74+
#
75+
function gen_random_container_name() {
76+
random_docker_name=$(head -c 16 /dev/urandom | base64 | tr -dc 'a-z0-9' | head -c 12)
77+
echo -n "container-$PACKAGE-$random_docker_name"
78+
}
79+
80+
#
81+
# Get the current git tag or commit hash if current commit is not tagged
82+
#
83+
function get_git_tag_or_hash() {
84+
# Get the current tag if it exists, otherwise get the short commit hash
85+
git describe --tags --abbrev=0 2>/dev/null || git rev-parse --short HEAD
86+
}
87+
88+
#
89+
# Build the tarball for the given distro and arch
90+
#
91+
# $1 - distro
92+
# $2 - arch
93+
# $3 - verbose: 0 or 1
94+
# $4 - repo_url (default: $GITHUB_REPO)
95+
function create_build() {
96+
local distro="$1"
97+
local arch="$2"
98+
local verbose="$3"
99+
local repo_url="$4"
100+
101+
local verbosearg="--quiet"
102+
103+
if [ "$verbose" -eq 1 ]; then
104+
verbosearg="--progress=auto"
105+
elif [ "$verbose" -eq 2 ]; then
106+
verbosearg="--progress=plain"
107+
fi
108+
109+
# TODO: keep this outside of this function, should be a global variable
110+
declare -A arch_map
111+
arch_map["amd64"]="x86_64"
112+
arch_map["arm64"]="aarch64"
113+
114+
local platformarch="${arch_map[$arch]:-$arch}"
115+
116+
117+
118+
119+
echo "Building artifacts for $distro on arch $arch (platform: $platformarch)..."
120+
121+
local containername=$(gen_random_container_name)
122+
123+
docker buildx build $verbosearg --platform linux/$platformarch --build-arg REPO_URL=$repo_url -t libpkcs11shim-build-$distro-$arch -f $(get_script_dir)/buildx/Dockerfile.$distro $(get_script_dir)/buildx
124+
125+
local artifacts=$(docker run --platform linux/$platformarch --name $containername libpkcs11shim-build-$distro-$arch)
126+
127+
for artifact in $artifacts; do
128+
docker cp --quiet $containername:$artifact $(get_current_dir)/
129+
done
130+
docker rm -f $containername > /dev/null 2>&1
131+
echo "Done with for $distro on $arch, produced artifacts:"
132+
for artifact in $artifacts; do
133+
echo " $(get_current_dir)/$(basename $artifact)"
134+
done
135+
}
136+
137+
# main function.
138+
139+
#
140+
# Parse the arguments and execute the builds
141+
#
142+
function parse_and_build() {
143+
local repo_url="$GITHUB_REPO"
144+
local verbose=0
145+
local args=()
146+
local numprocs=$(nproc)
147+
148+
# Parse optional -repo and -verbose arguments
149+
while [[ "$1" == --* || "$1" == -* ]]; do
150+
case "$1" in
151+
--repo|-r)
152+
shift
153+
repo_url="$1"
154+
;;
155+
--verbose|-v)
156+
if [ "$verbose" -lt 2 ]; then
157+
verbose=$(($verbose + 1))
158+
fi
159+
;;
160+
-vv)
161+
verbose=2
162+
;;
163+
--max-procs|-j)
164+
shift
165+
numprocs="$1"
166+
# Validate the number of processes:
167+
# - Must be a positive integer
168+
# - Must be less than or equal to the number of CPUs
169+
if ! [[ "$numprocs" =~ ^[0-9]+$ ]] || [ "$numprocs" -le 0 ] || [ "$numprocs" -gt "$(nproc)" ]; then
170+
echo "Invalid number of processes: $numprocs"
171+
usage
172+
fi
173+
;;
174+
*)
175+
echo "Unknown option: $1"
176+
usage
177+
;;
178+
esac
179+
shift
180+
done
181+
182+
# Collect remaining arguments
183+
local args=("$@")
184+
185+
local build_args=()
186+
187+
for arg in "${args[@]}"; do
188+
if [[ "$arg" == "all/all" ]]; then
189+
for distro in $SUPPORTED_DISTROS; do
190+
for arch in $SUPPORTED_ARCHS; do
191+
build_args+=("$distro $arch $verbose $repo_url")
192+
done
193+
done
194+
elif [[ "$arg" == "all" ]]; then
195+
local host_arch=$(uname -m)
196+
for distro in $SUPPORTED_DISTROS; do
197+
build_args+=("$distro $host_arch $verbose $repo_url")
198+
done
199+
elif [[ "$arg" == */* ]]; then
200+
IFS='/' read -r distro arch_list <<< "$arg"
201+
if [[ "$arch_list" == "all" ]]; then
202+
for arch in $SUPPORTED_ARCHS; do
203+
build_args+=("$distro $arch $verbose $repo_url")
204+
done
205+
else
206+
IFS=',' read -ra archs <<< "$arch_list"
207+
for arch in "${archs[@]}"; do
208+
build_args+=("$distro $arch $verbose $repo_url")
209+
done
210+
fi
211+
else
212+
IFS=',' read -ra distros <<< "$arg"
213+
local host_arch=${rev_arch_map[$(uname -m)]:-$(uname -m)}
214+
for distro in "${distros[@]}"; do
215+
build_args+=("$distro $host_arch $verbose $repo_url")
216+
done
217+
fi
218+
done
219+
220+
export -f create_build
221+
export -f get_current_dir
222+
export -f get_script_dir
223+
export -f gen_random_container_name
224+
225+
# Run builds in parallel, limiting to the number of jobs specified by the user
226+
#printf "%s\n" "${build_args[@]}" | xargs -P $numprocs -I {} bash -c 'echo "BUILD {}" && sleep 2'
227+
printf "%s\n" "${build_args[@]}" | xargs -P $numprocs -I {} bash -c 'create_build {}'
228+
}
229+
230+
#
231+
# Main logic
232+
#
233+
if [[ "$#" -lt 1 ]]; then
234+
usage
235+
fi
236+
237+
parse_and_build "$@"
238+
239+
# EOF

0 commit comments

Comments
 (0)