Merge pull request kubevirt#641 from petrkotas/offlinevm-devel-doc

Offlinevm devel doc

docs/devel/offline-virtual-machine.md

@@ -0,0 +1,325 @@
+# Offline Virtual Machine developer documentation
+
+This document introduces the OfflineVirtualMachine kind and provides a
+guide how to use it and build upon it.
+
+## What is Offline Virtual Machine
+
+Almost all virtual machine (VM) management systems allow you to manage both running
+and stopped virtual machines. Such system allows you to edit configuration of
+both types of VMs and show its statuses.
+
+To allow building such VM management systems on top of the KubeVirt, the
+OfflineVirtualMachine is introduced to provide the access to the stopped
+virtual machines. When working with running virtual machines, please see
+the [Virtual Machine object documentation](). The Virtual Machine object is
+designed to work in tandem with the OfflineVirtualMachine and provides the
+configuration and status for running virtual machines.
+
+OfflineVirtualMachine is a Kubernetes [custom resource definition](https://kubernetes.io/docs/concepts/api-extension/custom-resources/), which
+allows for using the Kubernetes machanisms for storing the objects and
+exposing it through the API.
+
+## What it does and how to use it
+
+The OfflineVirtualMachine provides the functionality to:
+
+* Store OfflineVirtualMachine,
+* Manipulate the OfflineVirtualMachine through the kubectl,
+* Manipulate the OfflineVirtualMachine through the Kubernetes API,
+* Watch for changes in the OfflineVirtualMachine and react to them:
+  * Convert the OfflineVirtualMachine to VirtualMachine and thus launch it
+  * Stop VirtualMachine and update status of OfflineVirtualMachine accordingly
+
+### Kubectl interface
+
+The kubectl allows you to manipulate the Kubernetes objects imperatively.
+You can create, delete, update and query objects in the API. More details on
+how to use kubectl and what it can do are in the [Kubernetes documentation](https://kubernetes.io/docs/reference/kubectl/overview/).
+
+Following are the examples of working with OfflineVirtualMachine and kubectl:
+
+```bash
+# Define an OfflineVirtualMachine:
+kubectl create -f myofflinevm.yaml
+
+# Start an OfflineVirtualMachine:
+kubectl patch offlinevirtualmachine myvm -p \
+    '{"spec":{"running" :"true"}}'
+
+# Look at OfflineVirtualMachine status and associated events:
+kubectl describe offlinevirtualmachine myvm
+
+# Look at the now created VirtualMachine status and associated events:
+kubectl describe virtualmachine myvm
+
+# Stop an OfflineVirtualMachine:
+kubectl patch offlinevirtualmachine myvm -p \
+    '{"spec":{"running":"false"}}'
+
+# Implicit cascade delete (first deletes the vm and then the ovm)
+kubectl delete offlinevirtualmachine myvm
+
+# Explicit cascade delete (first deletes the vm and then the ovm)
+kubectl delete offlinevirtualmachine myvm --cascade=true
+
+# Orphan delete (The running vm is only detached, not deleted)
+# Recreating the ovm would lead to the adoption of the vm
+kubectl delete offlinevirtualmachine myvm --cascade=false
+```
+
+### The REST API
+
+The kubectl is a handy tool that provides handy access to cluster, when you sit
+at the console. But, when you are writting an external application that
+needs to access the cluster programaticaly, it is better to have a API endpoint.
+Thats where the Kubernetes REST API endpoint comes right in. Kubernetes provides
+for its users the native REST API, which is easily extendable and in one place.
+
+The OfflineVirtualMachine object is a CRD, which implies that Kubernetes
+provides the API automatically. The API is located at the path
+
+```text
+<your-api-server-adress>/apis/kubevirt.io/v1alpha/offlinevirtualmachine/
+```
+
+and you can do typical REST manipulation, you would expect. All CRUD is
+supported, as shown in following example.
+
+```text
+POST /apis/kubevirt.io/v1alpha1/namespaces/{namespace}/offlinevirtualmachine
+GET /apis/kubevirt.io/v1alpha1/namespaces/{namespace}/offlinevirtualmachine
+GET /apis/kubevirt.io/v1alpha1/namespaces/{namespace}/offlinevirtualmachine/{name}
+DELETE /apis/kubevirt.io/v1alpha1/namespaces/{namespace}/offlinevirtualmachine/{name}
+PUT /apis/kubevirt.io/v1alpha1/namespaces/{namespace}/offlinevirtualmachine/{name}
+PATCH /apis/kubevirt.io/v1alpha1/namespaces/{namespace}/offlinevirtualmachine/{name}
+```
+
+By **POST** you can store new object in the etcd. With **GET** you either
+get the list of all OfflineVirtualMachines or get the concrete one. **DELETE**
+removes the object from etcd and all its resources. If you want to update the
+existing OfflineVirtualMachine object use **PUT** and if you want to change
+an item inside the object use **PATCH**.
+More details on the API are in the [documentation](https://kubevirt-incubator.github.io/api-reference/content/operations.html).
+
+To data format used when communicating with the API is the JSON. The format is
+set up the usual way by setting the Content-Type header to 'application/json'.
+The 'application/yaml' can also be used.
+
+## OfflineVirtualMachine object content
+
+Now its time to discuss the content of the OfflineVirtualMachine object.
+The object is defined in the same way as any other Kubernetes object. You can
+use YAML or JSON format to specify the content. The example structure is bellow:
+
+```yaml
+apiVersion: kubevirt.io/v1alpha1
+kind: OfflineVirtualMachine
+metadata:
+  name: myvm
+spec:
+  running: false
+  template:
+    metadata:
+      labels:
+        my: label
+    spec:
+      domain:
+        resources:
+          requests:
+            memory: 8Mi
+        devices:
+          disks:
+          - name: disk0
+            volumeName: mypcv
+      volumes:
+        - name: mypvc
+          persistentVolumeClaim:
+            claimName: myclaim
+```
+
+The file specification follows the Kubernetes guide. The apiVersion is linked
+with the Kubevirt release cycle.
+
+In the metadata section, there is a *required* field, the **name**. Then
+following the spec section, there are two important parts. The **running**, which
+indicates the current state of the VirtualMachine attached to this VirtualMachine.
+Second is the **template**, which is the VirtualMachine template.
+
+Let us go over each of these fields.
+
+### Name
+
+The `metadata.name` is important for the OfflineVirtualMachine, it is used to find
+created VirtualMachine. The VirtualMachine name is directly derived from
+the OfflineVirtualMachine. This means, if OfflineVirtualMachine is names 'testvm',
+the VirtualMachine is named 'testvm' too.
+
+Moreover, the pair namespace and name: `namepace/metadata.name` creates the
+unique identifier for the OfflineVirtualMachine. This fact implies that two
+identical names in the same namespace are considered as an error.
+
+### Template
+
+The spec.template is a VirtualMachine specification used to create an actual
+VirtualMachine. Below is an example of such VirtualMachine specification.
+
+```yaml
+metadata:
+spec:
+  domain:
+    resources:
+      requests:
+        memory: 8Mi
+    devices:
+      disks:
+      - name: disk0
+        volumeName: mypcv
+  volumes:
+    - name: mypvc
+      persistentVolumeClaim:
+        claimName: myclaim
+```
+
+It is easy to see that it is exactly the same as [VirtualMachine](https://kubevirt-incubator.github.io/api-reference/content/index.html),
+but it does not have a `kind` and `apiVersion`. These are implicitely added.
+
+Thus for the complete list of supported fields in the spec.template please refer
+to the [VirtualMachine](https://kubevirt-incubator.github.io/api-reference/content/index.html)
+documentation.
+
+## OfflineVirtualMachine behaviour
+
+The OfflineVirtualMachine has to be in sync with its VirtualMachine. This means
+that the OfflineVirtualMachine controller has to observe both, the OfflineVirtualMachine
+and created VirtualMachine. When the [link](#ownerreference) is established the config changes
+are translated to the VirtualMachine, and coresponding status changes are
+translated back to OfflineVirtualMachine.
+
+TBD this needs to be more specific
+
+### Status
+
+Shows the information about current OfflineVirtualMachine. The example status
+is shown below.
+
+```yaml
+status:
+  observedGeneration: 124 # current observed revision
+  virtualMachine: my-vm
+  running: true # is the attached VirtualMachine running
+  ready: true # based on http readiness check libvirt info
+  conditions: [] # additional possible states
+```
+
+The status of the VirtualMachine is watched and is reflected in the
+OfflineVirtualMachine status. The info propagated from the VirtualMachine is:
+
+* running state
+* readiness of the VM
+* name of the VirtualMachine
+
+For more information, user have to check the VirtualMachine object itself.
+
+The conditions can show additional information about state of VirtualMachine.
+One possible case would be to show how VirtualMachine was stopped, e.g.:
+
+```yaml
+status:
+  conditions:
+    - lastShutdown: 12.12.2018 00:00:00
+      Reason: error
+```
+
+### OwnerReference
+
+Linking the created VirtualMachine to its parent OfflineVirtualMachine pose
+a challenge. Using the same name is only part of the solution. To find the
+parent OfflineVirtualMachine programatically in the Kubernetes, the OwnerReference
+is used. As described in the
+[design document](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/controller-ref.md),
+the OwnerReference lives in the metadata section of the object and is created
+automaticaly. Example:
+
+```YAML
+apiVersion: kubevirt.io/v1alpha1
+kind: VirtualMachine
+metadata:
+  name: myvm
+  ownerReferences:
+    - controller: true
+      uid: 1234-1234-1234-1234
+      kind: OfflineVirtualMachine
+      version: kubevirt.io/v1alpha1
+```
+
+### Update strategy
+
+For now implicit: OnDelete. Can later be extended to RollingUpdate if needed.
+Spec changes have no direct effect on already running VMs, and they will not
+directly be propagated to the VM. If a VM should be running (spec.running=true)
+and it is powered down (VM object delete, OS shutdown, ...),
+the VM will be re-created by the controller with the new spec.
+
+### Delete strategy
+
+The delete has a cascade that deletes the created VirtualMachine. If a cascade
+is turned off the VirtualMachine is orphaned and leaved running.
+When the OfflineVirtualMachine with the same name as orphaned VirtualMachine
+is created, the VirtualMachine gets adopted and OwnerReference
+is updated accordingly.
+
+### Reset and Reboot
+
+This is not covered by the OfflineVirtualMachine. This functionality shall
+be achieved by subresources for the VirtualMachine (imperative),
+and will not result in a recreation of the VirtualMachine object or its Pod.
+From the KubeVirt perspective, the VirtualMachine is running all the time.
+Thus spec changes on OfflineVirtualMachine will not be propagated to
+the VirtualMachine in that case.
+
+## How it is implemented
+
+The first implementation of the OfflineVirtualMachine kind has two parts:
+
+1) The OfflineVirtualMachine custom resource definition,
+2) The controller watching for the changes and updating the state.
+
+### Custom resource definition
+
+The OfflineVirtualMachine custom resource is straightforward and is shown bellow:
+
+```yaml
+apiVersion: apiextensions.k8s.io/v1beta1
+kind: CustomResourceDefinition
+metadata:
+  name: offlinevirtualmachines.kubevirt.io
+spec:
+  scope: Namespaced
+  group: kubevirt.io
+  version: v1alpha1
+  names:
+    kind: OfflineVirtualMachine
+    plural: offlinevirtualmachines
+    singular: offlinevirtualmachine
+    shortNames:
+    - ovm
+    - ovms
+```
+
+Part of the definition of custom resource is a API specification used for
+autogenerating the Kubernetes resources: client, lister and watcher.
+
+### Controller
+
+The controller is responsible for watching the change in the registered
+offline virtual machines and update the state of the system. It is also
+responsible for creating new VirtualMachine when the `running` is set to `true`.
+Moreover the controller attaches the `metadata.OwnerReference` to the created
+VirtualMachine. With this mechanism it can link the OfflineVirtualMachine to the
+VirtualMachine and show combined status.
+
+The controller is designed to be a standalone service running in its own pod.
+Since the whole KubeVirt is designed to be modular, this approach allows for
+a more flexbility and less codebase in the core. Moreover it can be scaled
+up separately if the need arise.