windows.md

Microsoft Azure Container Service Engine - Kubernetes Windows Walkthrough

Deployment

Here are the steps to deploy a simple Kubernetes cluster with Windows:

1. install acs-engine
2. generate your ssh key
3. generate your service principal
4. edit the Kubernetes windows example and fill in the blank strings
5. generate the template
6. deploy the output azuredeploy.json and azuredeploy.parameters.json
7. Temporary workaround when deploying a cluster in a custom VNET with Kubernetes 1.6.0:

   1. After a cluster has been created in step 6 get id of the route table resource from Microsoft.Network provider in your resource group. The route table resource id is of the format: /subscriptions/SUBSCRIPTIONID/resourceGroups/RESOURCEGROUPNAME/providers/Microsoft.Network/routeTables/ROUTETABLENAME

   2. Update properties of all subnets in the newly created VNET that are used by Kubernetes cluster to refer to the route table resource by appending the following to subnet properties:

      "routeTable": {
              "id": "/subscriptions/<SubscriptionId>/resourceGroups/<ResourceGroupName>/providers/Microsoft.Network/routeTables/<RouteTableResourceName>"
            }

      E.g.:

      "subnets": [
          {
            "name": "subnetname",
            "id": "/subscriptions/<SubscriptionId>/resourceGroups/<ResourceGroupName>/providers/Microsoft.Network/virtualNetworks/<VirtualNetworkName>/subnets/<SubnetName>",
            "properties": {
              "provisioningState": "Succeeded",
              "addressPrefix": "10.240.0.0/16",
              "routeTable": {
                "id": "/subscriptions/<SubscriptionId>/resourceGroups/<ResourceGroupName>/providers/Microsoft.Network/routeTables/<RouteTableResourceName>"
              }
            ....
            }
            ....
          }
      ]

Walkthrough

Once your Kubernetes cluster has been created you will have a resource group containing:

1. 1 master accessible by SSH on port 22 or kubectl on port 443

2. a set of windows and linux nodes. The windows nodes can be accessed through an RDP SSH tunnel via the master node. To do this, follow these instructions, replacing port 80 with 3389. Since your windows machine is already using port 3389, it is recommended to use 3390 to Windows Node 0, 10.240.245.5, 3391 to Windows Node 1, 10.240.245.6, and so on as shown in the following image:

The following image shows the architecture of a container service cluster with 1 master, and 2 agents:

In the image above, you can see the following parts:

1. Master Components - The master runs the Kubernetes scheduler, api server, and controller manager. Port 443 is exposed for remote management with the kubectl cli.
2. Linux Nodes - the Kubernetes nodes run in an availability set. Azure load balancers are dynamically added to the cluster depending on exposed services.
3. Windows Nodes - the Kubernetes windows nodes run in an availability set.
4. Common Components - All VMs run a kubelet, Docker, and a Proxy.
5. Networking - All VMs are assigned an ip address in the 10.240.0.0/16 network. Each VM is assigned a /24 subnet for their pod CIDR enabling IP per pod. The proxy running on each VM implements the service network 10.0.0.0/16.

All VMs are in the same private VNET and are fully accessible to each other.

Create your First Kubernetes Service

After completing this walkthrough you will know how to:

• access Kubernetes cluster via SSH,
• deploy a simple Windows Docker application and expose to the world,
• and deploy a hybrid Windows / Linux Docker application.

1. After successfully deploying the template write down the master FQDN (Fully Qualified Domain Name).

   1. If using Powershell or CLI, the output parameter is in the OutputsString section named 'masterFQDN'
   2. If using Portal, to get the output you need to:
   3. navigate to "resource group"
   4. click on the resource group you just created
   5. then click on "Succeeded" under last deployment
   6. then click on the "Microsoft.Template"
   7. now you can copy the output FQDNs and sample SSH commands

   

2. SSH to the master FQDN obtained in step 1.

3. Explore your nodes and running pods:

4. to see a list of your nodes type kubectl get nodes. If you want full detail of the nodes, add -o yaml to become kubectl get nodes -o yaml.

5. to see a list of running pods type kubectl get pods --all-namespaces. By default DNS, heapster, and the dashboard pods will be assigned to the Linux nodes.

6. Start your first Docker image by editing a file named simpleweb.yaml filling in the contents below, and then apply by typing kubectl apply -f simpleweb.yaml. This will start a windows simple web application and expose to the world.

apiVersion: v1
kind: Service
metadata:
  name: win-webserver
  labels:
    app: win-webserver
spec:
  ports:
    # the port that this service should serve on
  - port: 80
    targetPort: 80
  selector:
    app: win-webserver
  type: LoadBalancer
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: win-webserver
  name: win-webserver
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: win-webserver
      name: win-webserver
    spec:
      containers:
      - name: windowswebserver
        image: microsoft/windowsservercore
        command:
        - powershell.exe
        - -command
        - "<#code used from https://gist.github.com/wagnerandrade/5424431#> ; $$listener = New-Object System.Net.HttpListener ; $$listener.Prefixes.Add('http://*:80/') ; $$listener.Start() ; $$callerCounts = @{} ; Write-Host('Listening at http://*:80/') ; while ($$listener.IsListening) { ;$$context = $$listener.GetContext() ;$$requestUrl = $$context.Request.Url ;$$clientIP = $$context.Request.RemoteEndPoint.Address ;$$response = $$context.Response ;Write-Host '' ;Write-Host('> {0}' -f $$requestUrl) ;  ;$$count = 1 ;$$k=$$callerCounts.Get_Item($$clientIP) ;if ($$k -ne $$null) { $$count += $$k } ;$$callerCounts.Set_Item($$clientIP, $$count) ;$$header='<html><body><H1>Windows Container Web Server</H1>' ;$$callerCountsString='' ;$$callerCounts.Keys | % { $$callerCountsString+='<p>IP {0} callerCount {1} ' -f $$_,$$callerCounts.Item($$_) } ;$$footer='</body></html>' ;$$content='{0}{1}{2}' -f $$header,$$callerCountsString,$$footer ;Write-Output $$content ;$$buffer = [System.Text.Encoding]::UTF8.GetBytes($$content) ;$$response.ContentLength64 = $$buffer.Length ;$$response.OutputStream.Write($$buffer, 0, $$buffer.Length) ;$$response.Close() ;$$responseStatus = $$response.StatusCode ;Write-Host('< {0}' -f $$responseStatus)  } ; "
      nodeSelector:
        beta.kubernetes.io/os: windows

5. Type watch kubectl get pods to watch the deployment of the service that takes about 30 seconds. Once running, type kubectl get svc and curl the 10.x address to see the output, eg. curl 10.244.1.4

6. Type watch kubectl get svc to watch the addition of the external IP address that will take about 2-5 minutes. Once there, you can take the external IP and view in your web browser.

7. The next step in this walkthrough is to deploy a hybrid Linux / Windows app. This application uses a Windows ASP.Net WebAPI front end, and a linux redis database as the backend. The ASP.Net WebAPI looks up the redis database via the fqdn redis-master.default.svc.cluster.local. To run this app, paste the contents below into a file named hybrid.yaml and type kubectl apply -f hybrid.yaml. This will take about 10 minutes to pull down the images.

apiVersion: v1
kind: Service
metadata:
  name: redis-master
  labels:
    app: redis
    tier: backend
    role: master
spec:
  ports:
    # the port that this service should serve on
  - port: 6379
    targetPort: 6379
  selector:
    app: redis
    tier: backend
    role: master
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: redis-master
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: redis
        role: master
        tier: backend
    spec:
      containers:
      - name: master
        image: redis
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        ports:
        - containerPort: 6379
      nodeSelector:
        beta.kubernetes.io/os: linux
---
apiVersion: v1
kind: Service
metadata:
  name: aspnet-webapi-todo
  labels:
    app: aspnet-webapi-todo
    tier: frontend
spec:
  ports:
    # the port that this service should serve on
  - port: 80
    targetPort: 80
  selector:
    app: aspnet-webapi-todo
    tier: frontend
  type: LoadBalancer
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: aspnet-webapi-todo
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: aspnet-webapi-todo
        tier: frontend
    spec:
      containers:
      - name: aspnet-webapi-todo
        image: anhowe/aspnet-web-api-todo2:latest
      nodeSelector:
        beta.kubernetes.io/os: windows

8. Type watch kubectl get pods to watch the deployment of the service that takes about 10 minutes. Once running, type kubectl get svc and for the app names aspnet-webapi-todo copy the external address and open in your webbrowser. As shown in the following image, the traffic flows from your webbrowser to the ASP.Net WebAPI frontend and then to the hybrid container.

   

Troubleshooting

If your cluster is not reachable, you can run the following command to check for common failures.

Misconfigured Service Principal

If your Service Principal is misconfigured, none of the Kubernetes components will come up in a healthy manner. You can check to see if this the problem:

ssh -i ~/.ssh/id_rsa USER@MASTERFQDN sudo journalctl -u kubelet | grep --text autorest

If you see output that looks like the following, then you have not configured the Service Principal correctly. You may need to check to ensure the credentials were provided accurately, and that the configured Service Principal has read and write permissions to the target Subscription.

Nov 10 16:35:22 k8s-master-43D6F832-0 docker[3177]: E1110 16:35:22.840688 3201 kubelet_node_status.go:69] Unable to construct api.Node object for kubelet: failed to get external ID from cloud provider: autorest#WithErrorUnlessStatusCode: POST https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/token?api-version=1.0 failed with 400 Bad Request: StatusCode=400

Learning More

Here are recommended links to learn more about Kubernetes:

1. Kubernetes Bootcamp - shows you how to deploy, scale, update, and debug containerized applications.
2. Kubernetes Userguide - provides information on running programs in an existing Kubernetes cluster.
3. Kubernetes Examples - provides a number of examples on how to run real applications with Kubernetes.