Recommendation: Place this file in source control.
Auto-generated documentation by
./dk dksdk.project.new
of SquirrelScout.
In the src/
directory we have subdirectories for:
HelloLib
- a simple libraryMainCLI
- a command line interface (CLI) executable that uses the library
The dependencies/
contains all the instructions to download and build the third-party
dependencies.
Visit https://diskuv.com/cmake/help/latest/guide/releases for the latest list of known issues ("Errata") and how to fix them.
Email support AT diskuv.com if you encounter unreported issues.
You will need WSL2 on your Windows system before you can follow this
section. Typically it is just running wsl --install
in an Administrative
Command Prompt or PowerShell. If that does not work, full instructions
are available at https://learn.microsoft.com/en-us/windows/wsl/install.
When you run the commands below, you may be asked to:
- Setup a UNIX username and password. This will be used to create a login for a new Debian Linux installation of WSL2. Do not re-use your Windows password. If you already have Debian, it will not be re-installed.
- Told about
Running command: sudo ...
and asked for your password. This will be your Debian Linux (WSL2) password, not your Windows password.
In Powershell or Command Prompt, run the following command:
.\ci\wsl2-setup.cmd
wsl -d Debian ./dk user.dev.gcm.install
wsl -d Debian git config --global credential.credentialStore secretservice
# You do not have to clone the source code to ~/source. But you must make sure to
# use a directory that is on a native Linux filesystem. If you don't know what
# that means, just use the commands below!
install -d ~/source
git -C ~/source clone https://github.com/diskuv/scoutapps.git
- Follow https://learn.microsoft.com/en-us/windows/wsl/tutorials/gui-apps#install-support-for-linux-gui-apps
- Follow https://wslutiliti.es/wslu/install.html#debian in a
wsl -d Debian
login. Use the Debian 11 instructions.
Using CLion?
-
In Powershell or Command Prompt, run the following command:
wsl -d Debian ./dk user.dev.clion.install
./dk dksdk.ninja.link
./dk dksdk.cmake.link
./dk dksdk.android.ndk.download
./dk dksdk.project.get
# Do configure with CMake 3.25.2 to avoid https://discourse.cmake.org/t/cmake-exception/2240/2
# with CLion's 3.24.
/opt/diskuv/usr/share/dktool/cmake-3.25.2/bin/cmake --preset=ci-linux_x86_64_X_android_x86_64
# Using CLion?
./dk user.dev.clion.run
# After running CLion:
# 1. Go to Settings > Build, Execution, Deployment > CMake and disable _all_ the profiles.
# 2. Delete the `build/`, `_dn/` and `_build/` directories if present.
# 3. Go to Settings > Build, Execution, Deployment > CMake and enable the `linux_x86_X_android_x86_64` profile.
./dk dksdk.ninja.link
./dk dksdk.cmake.link
./dk dksdk.android.ndk.download
./dk dksdk.project.get
# Do configure with CMake 3.25.2 to avoid https://discourse.cmake.org/t/cmake-exception/2240/2
# with CLion's 3.24.
/opt/diskuv/usr/share/dktool/cmake-3.25.2/bin/cmake --preset=ci-linux_x86_X_android_arm32v7a
# Using CLion?
./dk user.dev.clion.run
# After running CLion:
# 1. Go to Settings > Build, Execution, Deployment > CMake and disable _all_ the profiles.
# 2. Delete the `build/`, `_dn/` and `_build/` directories if present.
# 3. Go to Settings > Build, Execution, Deployment > CMake and enable the `linux_x86_X_android_arm32v7a` profile.
You will need either DkML installed or a MSYS2 installation.
- There is nothing to do if you have DkML installed. It has an embedded MSYS2 installation that will be re-used for DkSDK.
- If you have a standalone MSYS2 installation, you must set the CMake variable
DKSDK_MSYS2_DIR
.
The Ninja generator, used in CMakePresets.json
and recommended by
Microsoft, needs
to be "external"-ly setup. That means you run vcvarsall.bat
before running CMake.
You will need to have selected:
- The architecture
x86
orx64
. Be aware that Ninja usesx86
for 32-bit, while Visual Studio traditionally has usedWin32
. - The VC version 14.25 or 14.26
- The Windows SDK version 10.0.18362.0
For example, using the Command Prompt (not Powershell!):
vcvarsall.bat x64 10.0.18362.0 -vcvars_ver=14.26
cmake --preset THE_PRESET
cmake --build --preset THE_BUILD_PRESET
In Settings > Build, Execution, Deployment > CMake > Toolchains
, make sure there is at least one
"Visual Studio" toolchain that has a Toolset with:
- a directory that is a Visual Studio 2019 installation (also known as v142, or Version 16)
- has
-vcvars_ver=14.26
(or-vcvars_ver=14.25
) in Version field
This is usually very slow because there will be a slow link between the Windows filesystem and the Linux filesystem. Use at your own risk.
Prerequisite: Setup WSL2
In Powershell or Command Prompt, run the following commands:
./dk dksdk.project.get
wsl -d Debian -- sh -cx './dk dksdk.ninja.copy && dksdk.cmake.copy && ./dk dksdk.android.ndk.download'
wsl -d Debian
You will now be inside the WSL2 Debian installation with all of the SquirrelScout
source code available in the current directory. Run the following to build the source
code into the build/
build directory:
export PATH="$PWD/.ci/wsl2/bin:$PATH"
export OPAMROOTISOK=1
cmake --version
> cmake version 3.25.2
>
> CMake suite maintained and supported by Kitware (kitware.com/cmake).
sh ci/exec.sh linux_x86_64 cmake --preset=wsl2-android_x86_64
sh ci/exec.sh linux_x86_64 cmake --build --preset=wsl2-linux_x86_64-main
sh ci/exec.sh linux_x86_64 ctest --preset=wsl2-linux_x86_64-test
There are over 600 active Linux distributions.
DkSDK supports:
- Debian "stable" is the supported "modern" Linux production server.
- Debian "stable" is the supported "modern" Linux build server.
- The latest Ubuntu LTS, which is derived from Debian, is the supported Linux developer desktop.
- Debian "stable-slim" is the supported "modern" Linux Docker container image.
manylinux_2_28
2_28 is the supported "backwards-compatible, old GLIBC" Linux Docker container image.glibc
-basedmanylinux_2_28
is compatible with Linux distributions released after 2018 with the notable exception of Alpine. You may also see some support formanylinux2014
which is compatible with most versions of most major distributions released after 2014; however, Diskuv is actively deprecatingmanylinux2014
.
If and when you need to distribute your DkSDK-produced binaries to Linux customers, you would use the "backwards-compatible, old GLIBC" Linux Docker container image. That way almost all Linux distributions that your customers use will be supported.
This old GLIBC technique has been adopted from the Python community, where they use it to distribute binary "wheels". An alternative approach from the Go community is to distribute statically linked executables. DkSDK does not support statically linked executables for two reasons:
- Statically linked executables are not conventional for OCaml (even though there are some hacky techniques to do static linking).
- Static linking can violate LGPL/GPL and other restrictively licensed libraries.
./dk dksdk.project.get
./dk dksdk.ninja.link
./dk dksdk.cmake.link
./dk dksdk.android.ndk.download
Then use the ci-linux_x86_64_X_android_x86_64
CMake profile in your favorite
CMake IDE.
./dk dksdk.project.get
./dk dksdk.ninja.link
./dk dksdk.cmake.link
./dk dksdk.android.ndk.download
Then use the ci-darwin_arm64_X_android_arm64v8a
CMake profile in your favorite
CMake IDE.
You can compile this project's code by directly using Android NDK or (TBD) using Android Studio.
Android NDK builds will use NDK r23 and target a minimum API of 21.
Type: STRING
Applies to: Windows with Visual Studio compiler
What: This variable can be set to Visual Studio C version numbers like 14.25
or 14.26
.
These version numbers correspond to the Visual Studio components that can be seen
in the Visual Studio Installer like:
Microsoft.VisualStudio.Component.VC.14.25
Microsoft.VisualStudio.Component.VC.14.26
Only versions 14.25
and 14.26
are compatible with DkSDK.
When to use: If you have multiple Visual Studio installations, and some of those installations are not compatible versions with DkSDK, this variable can let you tell DkSDK to pick a compatible installation. The variable will be ignored, however, if you have hardcoded the CMAKE_C_COMPILER cache variable.
Type: FILEPATH
Applies to: Windows
What: Tells DkSDK to re-use the specified MSYS2 system. Without this variable, DkSDK locates and expects the embedded MSYS2 system inside a pre-installed DkML.
When to use:
- If you don't want to install DkML, just install MSYS2 from its website and point
the
DKSDK_MSYS2_DIR
directory to your newly installed MSYS2 system. - Many CI services like GitHub Actions and GitLab CI already come bundled with MSYS2, or have easy ways to install MSYS2. If you are in CI, use this variable.
For example, cmake -G Ninja -D DKSDK_MSYS2_DIR=msys64
if MSYS2 is installed in the subdirectory
msys64
of the project source code, or cmake -G Ninja -D DKSDK_MSYS2_DIR=C:\msys64
if installed
to C:\msys64
.
Type: FILEPATH
What: Tells DkSDK to use the specified opam root. Without this variable, DkSDK creates its own opam root inside the CMake binary directory including somewhat time-consuming downloads of the central opam repository.
When to use:
- The OCaml CI actions
setup-ocaml
andsetup-dkml
already provide an opam root. If you are using either of them, use this variable to save time.
For example, cmake -G Ninja -D DKSDK_OPAM_ROOT=msys64
if the opam root is in the subdirectory
.ci/o
under the project source code, or cmake -G Ninja -D DKSDK_OPAM_ROOT=C:\opam
if the
opam root is C:\opam
.
DKSDK_OPAM_ROOT_USER
takes precedence over this variable.
Type: BOOL
What: Tells DkSDK to use the standard opam root on your machine. Without this variable, DkSDK creates its own opam root inside the CMake binary directory including somewhat time-consuming downloads of the central opam repository.
When to use:
- If you have already installed opam on your development machine and are fine with DkSDK switches being visible to other opam-aware processes on your machine (like Visual Studio Code with the OCaml Language Server), use this variable to save time.
This variable takes precedence over DKSDK_OPAM_ROOT
.
Type: BOOL
What: Tells DkSDK that your host ABI (the build machine) does not support precise C99 float operations.
When to use: If your build machine is VirtualBox or some other emulator, and you see the error
configure: error: C99 float ops unavailable, enable replacements with --enable-imprecise-c99-float-ops
,
then set this variable to ON
.
In the ocaml
toplevel do:
OCaml version 4.14.0
Enter #help;; for help.
# #use_output "dune ocaml top src";;
Running from Windows?
- You will need a POSIX shell. If you have DkML on your machine,
with-dkml bash
will start a POSIX shell. Other alternatives are a standalone MSYS2 or Cygwin installation.
Docker container running slow?
- You may need more memory available in your Docker container.
- Docker Desktop on Windows, when its
Settings > General
has enabledUse the WSL 2 based engine
, will have slow file operations. The recommendation for performance is to use the WSL filesystem, but sincedockcross
mounts Windows directories into Docker you don't have that option.
In a POSIX compatible shell (ex. bash
on Linux and macOS) do the following:
sh ci/setup-dkml/pc/setup-dkml-linux_x86.sh --SKIP_OPAM_MODIFICATIONS=true
./dk dksdk.project.get
.ci/sd4/opamrun/cmdrun sh -cx './dk dksdk.ninja.copy && dksdk.cmake.copy && ./dk dksdk.android.ndk.download'
.ci/sd4/opamrun/cmdrun -it bash
You will now be inside the Linux dockcross container with all of the SquirrelScout
source code available in the /work
directory. Run the following to build the source
code into the build/
build directory:
export PATH="$PWD/.ci/dockcross/bin:$PATH"
export OPAMROOTISOK=1
cmake --version
> cmake version 3.25.2
>
> CMake suite maintained and supported by Kitware (kitware.com/cmake).
# Note: On some platforms `sudo` is not required for the following commands.
sudo sh ci/exec.sh linux_x86 cmake --preset=dbg-linux_x86
sudo sh ci/exec.sh linux_x86 cmake --build --preset=dbg-main
sudo sh ci/exec.sh linux_x86 ctest --preset=dbg-test
In a POSIX compatible shell (ex. bash
on Linux and macOS) do the following:
sh ci/setup-dkml/pc/setup-dkml-linux_x86_64.sh --SKIP_OPAM_MODIFICATIONS=true --dockcross_image=dockcross/manylinux_2_28-x64
./dk dksdk.project.get
.ci/sd4/opamrun/cmdrun sh -cx './dk dksdk.ninja.copy && dksdk.cmake.copy && ./dk dksdk.android.ndk.download'
.ci/sd4/opamrun/cmdrun -it bash
You will now be inside the Linux dockcross container with all of the SquirrelScout
source code available in the /work
directory. Run the following to build the source
code into the build/
build directory:
export PATH="$PWD/.ci/dockcross/bin:$PATH"
export OPAMROOTISOK=1
cmake --version
> cmake version 3.25.2
>
> CMake suite maintained and supported by Kitware (kitware.com/cmake).
# Note: On some platforms `sudo` is not required for the following commands.
sudo sh ci/exec.sh linux_x86_64 cmake --preset=dbg-linux_x86_64
sudo sh ci/exec.sh linux_x86_64 cmake --build --preset=dbg-main
sudo sh ci/exec.sh linux_x86_64 ctest --preset=dbg-test
In a POSIX compatible shell (ex. bash
on Linux and macOS) do the following:
sh ci/setup-dkml/pc/setup-dkml-linux_x86_64.sh --SKIP_OPAM_MODIFICATIONS=true --dockcross_image=dockcross/manylinux_2_28-x64
./dk dksdk.project.get
.ci/sd4/opamrun/cmdrun sh -cx './dk dksdk.ninja.copy && dksdk.cmake.copy && ./dk dksdk.android.ndk.download'
.ci/sd4/opamrun/cmdrun -it bash
You will now be inside the Linux dockcross container with all of the SquirrelScout
source code available in the /work
directory. Run the following to build the source
code into the build/
build directory:
export PATH="$PWD/.ci/dockcross/bin:$PATH"
export OPAMROOTISOK=1
cmake --version
> cmake version 3.25.2
>
> CMake suite maintained and supported by Kitware (kitware.com/cmake).
# Note: On some platforms `sudo` is not required for the following commands.
sudo sh ci/exec.sh linux_x86_64 cmake --preset=ci-linux_x86_64_X_android_x86_64
sudo sh ci/exec.sh linux_x86_64 cmake --build --preset=ci-main
sudo sh ci/exec.sh linux_x86_64 ctest --preset=ci-test
All compilation (bytecode or native) and host/cross-compiling is handled by Dune.
The CMAKE_OCamlDune_COMPILER
is the host ABI ocamlopt.opt
(or
ocamlopt
). That is sufficient for Dune to locate ocamlfind
and ocamlc.opt
which are in the same directory.
Unless -D CMAKE_OCamlDune_COMPILER_EXTERNAL_REQUIRED=ON
(or any
truthy value), DKSDK will automatically build OCaml, findlib
and Dune in what we call the OCaml sub-build.
Debugging, optimization and other flags that are set for the assembler or C compiler strongly influence how OCaml code compiles. That is because OCaml delegates to the assembler and C compiler for native code generation.
In CMake those flags are controlled by CMAKE_<LANG>_FLAGS
and its variants CMAKE_<LANG>_FLAGS_<CONFIG>
. For example,
CMAKE_C_FLAGS_DEBUG
are the flags given to the C compiler during a
Debug build.
When you change those flags in the CMake cache, the OCaml sub-build
is not automatically rebuilt. You will need to remove the _ocaml
directory completely in your build directory, and then re-run the
CMake generator.
The source code for the OCaml sub-build uses:
- https://github.com/ocaml/ocaml.git
- https://github.com/alainfrisch/flexdll.git (only for Windows)
- https://github.com/diskuv/dkml-compiler.git
- https://github.com/diskuv/dkml-runtime-common.git
See All IDEs: CMakeUserPresets.json
for the FETCHCONTENT_SOURCE_DIR_*
variables you would use to point
to your own custom OCaml code.
When you first run the CMake generator (cmake -G ...
, or
"Reload CMake Project" in the CLion IDE),
CMake will use the FETCHCONTENT_SOURCE_DIR_*
variables you have
set.
After the first run of the CMake generator any changes
to the source code in your ocaml
, flexdll
, dkml-compiler
and
dkml-runtime-common
will be ignored. To get CMake to see your
changes you will have to:
- Remove the
DkSDKFiles/o/160-dune/exports_COMMON.cmake
file from your binary (aka. build) directory. - Re-run the CMake generator.
This has yet to be implemented.
Most of the Lang=OCamlDune
modules can be re-used as-is.
Generally only the CMakeOCamlHostBytecodeInformation.cmake
(etc.) module will need to be written; that will give CMake the
compiler and linking command lines to build libraries and executables.
This is optional but highly recommended on Unix, especially if you use Visual Studio Code
On Unix systems direnv will let you configure your environment when your interactive shell enters your project directory.
The setup below will configure:
dune
to be in your PATHcmake
to be in your PATHopam
to be in your PATH
First, install direnv.
Second, you will need to run CMake once (just the -G
configure phase).
Third, in your project source directory, create the file .envrc
:
#!/bin/sh
# If you need to modify the rules, do not edit this file,
# but place your changes in `user.envrc` in this directory.
for _dksdk_build_dir in build_dev build; do
if [ -e "$_dksdk_build_dir/DkSDKFiles/ocaml_project.source.sh" ]; then
# shellcheck disable=SC1090
dotenv "$_dksdk_build_dir/DkSDKFiles/ocaml_project.source.sh"
break
fi
done
unset _dksdk_build_dir
if [ -n "${CMAKE_COMMAND_DIR:-}" ] && [ -x "$CMAKE_COMMAND_DIR/cmake" ]; then
PATH_add "$CMAKE_COMMAND_DIR"
fi
if [ -n "${CMAKE_DUNE_DIR:-}" ] && [ -x "$CMAKE_DUNE_DIR/dune" ]; then
PATH_add "$CMAKE_DUNE_DIR"
fi
if [ -n "${CMAKE_OCAMLDUNE_OPAM_HOME:-}" ] && [ -x "$CMAKE_OCAMLDUNE_OPAM_HOME/bin/opam" ]; then
PATH_add "$CMAKE_OCAMLDUNE_OPAM_HOME/bin"
fi
# Lets you add your own modifications
source_env_if_exists user.envrc
# Advanced: If you uncomment, your IDEs won't work, but you can inspect your project with `opam list`.
# if [ -n "${CMAKE_OCAMLDUNE_OPAM_ROOT:-}" ] && [ -e "$CMAKE_OCAMLDUNE_OPAM_ROOT/config" ]; then
# export OPAMROOT="$CMAKE_OCAMLDUNE_OPAM_ROOT"
# fi
# if [ -e "_dn/_opam/.opam-switch/switch-config" ]; then
# export OPAMSWITCH="$(expand_path _dn)"
# fi
Finally, run:
cd YOUR_PROJECT_DIRECTORY
direnv allow
We suggest you make your own CMakeUserPresets.json
file. A good starting
point is CMakeUserPresets-SUGGESTED.json,
which you can copy into your own CMakeUserPresets.json
.
Customize it according to the rules:
- Any developer overrides of source code go into the
dev-source-dirs
hidden preset. In the template above, you have checked out the source code fordksdk-cmake
in the sibling directory../dksdk-cmake
so the standard Git checkout ofdksdk-cmake
will not happen. Add or remove as many source code overrides as you need. - Each source code override directory is copied in its entirety. Often
CMake can't copy a local
_opam
directory but will silently continue. - The same CMakeUserPresets.json can be used for all your OCaml packages, as long as you check out all of your OCaml packages in the same source tree as siblings of each other.
The source code for the OCaml sub-build uses:
- https://github.com/ocaml/ocaml.git
- https://github.com/alainfrisch/flexdll.git
- https://github.com/diskuv/dkml-compiler.git
- https://github.com/diskuv/dkml-runtime-common.git
The source code needed by the DkSDK_OpamPackages
target
needs dkml-compiler
and dkml-runtime-common
above, and also:
You can use your own source code and edits by checking out any or all of the git repositories above, and setting the corresponding CMake variables:
FETCHCONTENT_SOURCE_DIR_OCAML
FETCHCONTENT_SOURCE_DIR_FLEXDLL
FETCHCONTENT_SOURCE_DIR_DKML-COMPILER
FETCHCONTENT_SOURCE_DIR_DKML-RUNTIME-COMMON
FETCHCONTENT_SOURCE_DIR_DKML-RUNTIME-DISTRIBUTION
For example, if you have checked out dkml-compiler
and
dkml-runtime-common
and dkml-runtime-distribution
as sibling directories to your project code,
then you can use the following in your project's CMakeUserPresets.json
:
{
// The full documentation is at https://cmake.org/cmake/help/latest/manual/cmake-presets.7.html.
// You CANNOT include comments like this one in your .json files though!
version: 3,
// ... any other fields ...
"configurePresets": [
{
"name": "some_name",
// ... any other fields ...
"cacheVariables": [
{
"FETCHCONTENT_SOURCE_DIR_DKML-RUNTIME-COMMON": {
"type": "FILEPATH",
"value": "${sourceParentDir}/dkml-runtime-common"
},
"FETCHCONTENT_SOURCE_DIR_DKML-RUNTIME-DISTRIBUTION": {
"type": "FILEPATH",
"value": "${sourceParentDir}/dkml-runtime-distribution"
},
"FETCHCONTENT_SOURCE_DIR_DKML-COMPILER": {
"type": "FILEPATH",
"value": "${sourceParentDir}/dkml-compiler"
}
}
]
}
]
}
When you first run the CMake generator (cmake -G ...
, or
"Reload CMake Project" in the CLion IDE),
CMake will use those FETCHCONTENT_SOURCE_DIR_*
variables you have
set.
Changes to those directories should automatically reconfigure CMake and rebuild affected CMake targets. If not, you can manually configure CMake again.
The exception is OCaml sub-build; see the link for how it behaves differently.
Use the OCaml Platform
plugin to edit your OCaml source code:
- Press Cmd-Shift-P on macOS, or Windows-Shift-P on Windows, to open the Visual Studio commands dialog
- Choose
OCaml: Select a Sandbox for this Workspace
- Choose
dkml
(Windows) ordksdk-util
(macOS/Linux)
- Choose
If you use a CMake cross-compiling compiler, then CMake will not
build the host ABI files that are expected in _build/default
.
You will instead need to run: dune build
from the command line
to get OCaml Platform
working.
On Windows you will need to install DkML. Nothing else is needed.
Do the following:
opam switch create dksdk-util 4.14.0
opam install --switch dksdk-util ocaml-lsp-server ocamlformat.0.24.1
sudo install "$(opam var --switch dksdk-util bin)/ocamllsp" /usr/local/bin/
sudo install "$(opam var --switch dksdk-util bin)/ocamlformat" /usr/local/bin/
sudo install "$(opam var --switch dksdk-util bin)/ocamlformat-rpc" /usr/local/bin/
Use the ReasonML plugin because it is the best supported OCaml (and Reason and ReScript) in IntelliJ. But it doesn't use Dune RPC or even the OCaml Language Server, so it won't work with modern OCaml.
You'll find it best to use Visual Studio Code.