pkl packaged UTM virtual machines for macOS

[mobileskyfi]

Tried pkl for a more modest deployment target than k8s:
desktop virtual machines using macOS UTM app

I posted it on GitHub here: https://github.com/tikoci/mikropkl

Nothing fancy, but thought I share since there are few pkl examples in the world.   To me, pkl-lang seems useful in apps as a kinda “SwiftScript™”.  But the first problem is you have to learn pkl.  And despite reading the manual a few months back, thought it best to try pkl-lang on something smaller, before trying to integrate pkl into some Swift-based app at some point. 

Since I use UTM-based virtual machines for testing, I had an idea to try pkl-lang to automate UTM  machine bundle creation on GitHub CI. The underlying problem is .plist file modifications are tough outside macOS (i.e. no “PlistBuddy” etc).  So, one main advantage of pkl-lang here is it works on Linux and via the PlistRenderer, and allows “mikropkl” code to converts some .pkl into a .plist on a cross-platform basis (e.g. inside some CI system).

Since UTM macOS app supports app URLs for installing VMs & virtual machines are just macOS document bundles containing a config.plist + disk image files — means the can be “built” outside the app.  So, essentially, pkl-lang in my project is “orchestrating” the generation of one/more “UTM bundle” (i.e. a folder ending in .utm) from pkl "manifests”,  with a Makefile picking up the pieces pkl cannot do (i.e. copy/download large files and run shell commands).

The project is agnostic on OS/images, but since my immediate need was for RouterOS, that’s what works out-of-the-box (and wanted a ginny pig I knew).  But the CHR.pkl could be an example of extending "mikropkl" UTM package building to other OSes/images.  There is a README.md that explains more, and the pkl source is on GitHub.

The nuts-and-bolts are:

• make kicks off building UTM virtual machines from manifests by invoking pkl eval Manifests/* -m Machines …
• Pkl uses a -m for multiple output files, to construct the files contained in the UTM’s virtual machine bundle into a “Machines” directory, including:   
  • Converting a more “template friendly” pkl into the specific config.plist needed, and whose structure varies depending on the virtualization engine (QEMU or Apple) used.  Additionally, UTM is order-sensitive, which pkl seems to handle. Although, this makes extending/refactoring/etc more tricky - and, why there is a layer of abstraction in the "templates" directory and utmzip.pkl part of the code.
  • Since UTM bundles are large binary disk images of OSes/data, pkl cannot readily deal with a file system nor "shell commands"....  So for large file downloads, pkl generates a small file ending in .url, like filename.img.url, containing the download URL inside the file, that is later processed by make into filename.img after download. And uses additional file extensions for other operations, like a .localcp to move a "static file" from Files into a UTM bundle during build.  
  • UTM supports icons, so an SVG file is generated with pkl and part of the multiple file output.  For fun, you can specify icon color as a property in a template file,  then pkl will replace inside the SVG (*as string, using XmlRenderer was left for another day).
• make continues recursively after pkl to find any “placeholder” file like .url that needs download or other actions.  Inside the Makefile pattern matching files transform the pkl-generated placeholder files into something.  pkl appends an extra file extension which makes things easy on make to deal with “file operations” or "shell commands" that are beyond pkl.
• Ubuntu-based GitHub Actions can make and build the UTM document bundles usable in a utm:// URL from macOS. The project's Makefile can run locally on macOS with machines usable from the output directory as an alternative and for adding new machines or types.

Only a few minor “sour pickles”:

• UTM’s config.plist stores a <data> element when in Apple Virtualization mode which has a VZMacMachineIdentifier.  I posted about this in How to output a `.plist`'s <data> element from a pkl file? #981 which had a workaround to generate <data> in a Plist file.  And, apparently apple/pkl-evolution#14 appears to offer direct support for a Bytes type that map to <data> with PlistRenderer in future.
• “random() needs… “(see Custom Resource provider for shell commands? _or...how to get cheat Pkl and get a UUID for something_ #1006).  The config.plist also uses UUIDs for disk images, and one or more MacAddress are needed.  Now ideally UTM support would fill these on startup if missing, avoiding pkl for randomness… Instead, I used a hash as suggested in same discussion, which works well enough since pkl files have enough context for a unique hash. 
• Lack of some shell() nor simple pkl CLI way to map a generic/"MsgPack unaware" command-line process into a valid external reader - that would have been useful. While having compiled code (i.e. Swift, Go) could fill in that gap... but, if you're using pkl outside being the config system of an app/service, there was no "shell-friendly" analog for "external resources" when using pkl as more classic UNIX tool.

Overall, my implementation could be improved*, but works well enough to share.

* side notes:

• no pkl:test scripts are used - as one might expect, the UUIDs in output does make testing more challenging
• additionally the strict ordering of config.plist limited some refactoring since the generated output need be accepted by an unmodified UTM app.