kvert: a powerful tool to generate YAML files

kvert lets you generate yaml files in a declarative way. You can for example use it to manage your Kubernetes manifests.

It supports including parts of definitions into other ones, variables, generating files with a different shape based on a profile, reading values from environment variables... All of this allows you to manage your YAML files in an effective way.

The tool leverages the EDN format and the Aero library.

Why kvert ? It's simple, powerful and extensible. I think neither templating or using YAML to generate more YAML are good solutions to manage Kubernetes resources and that's why I built this tool.

Install

For Linux (x86-64), download the kvert binary from the release page and put it in your PATH. This binary is built using GraalVM, more targets may be added soon (help welcome).

You can alternatively download the jar file and then run it with java -jar kvert.jar (Java 17 needed).

A docker image is also provided. Note that this image uses java as well so executing it is a bit slower than the static binary built with GraalVM.

All example described below can be done using the Docker image by executing in the directory containing your templates docker run -v $(pwd):/data mcorbin/kvert:v0.2.0 <command>. The files will be availables in /data. Example: docker run -v $(pwd):/data mcorbin/kvert:v0.2.0 yaml -t /data/example.edn.

Quick start

Simple EDN example

Once kvert installed, you are ready to use it. Let's for example generate a yaml file from a simple EDN definition. Put in pod.edn this content:

;; this is a comment
{:apiVersion "v1"
 :kind "Pod"
 :metadata {:name "dnsutils"
            :namespace "default"}
 :spec {:containers [{:name "dnsutils"
                      :image "k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.3"
                      :command ["sleep" "3600"]
                      :imagePullPolicy "ifNotPresent"}]
        :restartPolicy "Always"}}

Now run kvert yaml --template pod.edn:

---
apiVersion: v1
kind: Pod
metadata:
  name: dnsutils
  namespace: default
spec:
  containers:
  - name: dnsutils
    image: k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.3
    command:
    - sleep
    - '3600'
    imagePullPolicy: ifNotPresent
  restartPolicy: Always

You can pass the -o (or --output) flag to save the output into a file.

As you can see, we can easily translate EDN to YAML. You can define multiple YAML resources into the same file as well:

[{:name "yaml-file-1"}
 {:name "yaml-file-2"}]

kvert will output:

---
name: yaml-file-1
---
name: yaml-file-2

Customizations

EDN supports readers, which can be used to extend it. Let's now put in pod.edn this content:

{:apiVersion "v1"
 :kind "Pod"
 :metadata {:name "dnsutils"
            :namespace #kvert/var :namespace}
 :spec {:containers [{:name "dnsutils"
                      :image #join ["k8s.gcr.io/e2e-test-images/jessie-dnsutils:" #kvert/var :container-version]
                      :command ["sleep" #or [#env SLEEP_DURATION 3600]]
                      :imagePullPolicy #profile {:production "ifNotPresent"
                                                 :default "Always"}}]
        :restartPolicy "Always"}}

As you an see, we use a few readers (which start with #) in this file:

• #kvert/var which will replace the next keyword (:namespace for example here) with a variable value
• #join which will concatenate several values together
• #or which allows you to define default values
• #env to read values from environment variables
• #profile which create a switch based on the value on the profile you used to run kvert (more on that later). This reader also supports default values in :default

Readers can be combined together, like in ["sleep" #or [#env SLEEP_DURATION 3600]] in this example which will first read the SLEEP_DURATION environment variable and fallback to 3600 if it's not defined.

Let's create a new file named config.edn:

{:variables {:namespace "default"
             :container-version "1.3"}
 :profile :production}

This file defines variables (referenced by #kvert/var in the pod.edn file) and the profile (:production).

Launch kvert yaml --template pod.edn -c config.edn, the output is:

---
apiVersion: v1
kind: Pod
metadata:
  name: dnsutils
  namespace: default
spec:
  containers:
  - name: dnsutils
    image: k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.3
    command:
    - sleep
    - 3600
    imagePullPolicy: ifNotPresent
  restartPolicy: Always

Thank to the readers we are able to customize our manifest. We could for example use variables and the profile to have variation between environment, kubernetes clusters...

Profile

We configured in this example the profile into the config.edn file. You can also set it by configuring the PROFILE environment variable when running kvert.

More readers

Aero, the library used by kvert to parse EDN files supports tons of readers out of the box. You can find them in the library documentation. Here are some interesting ones:

#include

For example, the #include reader allows you to include an EDN file into another one. Let's create a file named example.edn containing:

{:apiVersion "v1"
 :kind "Pod"
 :metadata {:labels #include "labels.edn"}}

labels.edn (which can also contain readers if you need to) being:

{:foo "bar"
 :environment "prod"}

The output of kvert yaml --template example.edn will be:

---
apiVersion: v1
kind: Pod
metadata:
  labels:
    foo: bar
    environment: prod

#kvert/include

THe #kvert/include reader works exactly like #include but allows you to pass additional variables and configure another profile to the included file:

{:apiVersion "v1"
 :kind "Pod"
 :metadata {:labels #kvert/include {:path "labels.edn"
                                     :variables {:foo "bar"}
                                     :profile :prod}}}

In this example, the content of the labels.edn file will be included. Additional variables (:foo in this example) are also provided and the profile is also overrided.

#ref

Another cool one is #ref, let's modify our example.edn with:

{:apiVersion "v1"
 :kind "Pod"
 :metadata {:name "foo"
            :labels {:name #ref [:metadata :name]}}}

This file will produce using kvert yaml --template example.edn:

---
apiVersion: v1
kind: Pod
metadata:
  name: foo
  labels:
    name: foo

As you can see, ref allows you to reference another part of your edn file.

Don't hesitate to check the Aero documentation for more examples !

Generate EDN from YAML

You can use kvert to convert an existing YAML file to EDN. This can help you to get started with the tool by using existing YAML files. An example with this file named pod.yml:

---
apiVersion: v1
kind: Pod
metadata:
  name: dnsutils
  namespace: default
spec:
  containers:
  - name: dnsutils
    image: k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.3
    command:
    - sleep
    - '3600'
    imagePullPolicy: ifNotPresent
  restartPolicy: Always

kvert edn --template pod.yaml will produce:

[{:apiVersion "v1",
  :kind "Pod",
  :metadata {:name "dnsutils", :namespace "default"},
  :spec
  {:containers
   [{:name "dnsutils",
     :image "k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.3",
     :command ["sleep" "3600"],
     :imagePullPolicy "ifNotPresent"}],
   :restartPolicy "Always"}}]