kubernetes

Installation of kubernetes cluster in private cloud using kubeadm.

Configuration

The following table lists the configurable parameters of the kubernetes cluster install and their default values.

Parameter	Description	Default
CLUSTER_NAME	Kubernetes cluster name	cloud.com
MOUNT_PATH	The mount path where kubernetes scripts and certificates are available.	/root
DNS_DOMAIN	The domains where vms are hosted.	cloud.uat
CERTIFICATE_PATH	The path where certificates are present.	/etc/kubernetes/pki
SERVER_DNS	The comma separated dns names where kubernetes master would be running, this also includes name of the kubernetes api service dns names, the dns name where ha proxy is running	master.cloud.com..
SERVER_IP	The comma separated list of all the ip addresses where master and ha proxy would be running (actual & virtual). This should also include the ip addess of kubernetes cluster api service ip.	11.0.0.1,..
HA_PROXY_PORT	Port of HA Proxy.	6443
HA_PROXY_HOSTNAME	IP/Hostname where HA Proxy running.	11.0.0.1
LOAD_BALANCER_URL	Endpoint of HA Proxy.	11.0.0.1:6443
APP_HOST	Hostname that would be put in ingress.	master.cloud.com
API_SERVERS	List of api servers which are used in creating certificates for ha proxy.	11.0.0.1:master.clud.com,..
OIDC_ISSUE_URL	OpenID Connect issuer url.	https://skmaji.auth0.com/
OIDC_CLIENT_ID	OpenID Connect Application ID.	Client ID
OIDC_USERNAME_CLAIM	Field name in the ID Token for username claim.	sub
OIDC_GROUPS_CLAIM	Field name in the ID Token for group claim.	http://localhost:8080/claims/groups

Installation steps:

• Install Cluster

cd /root/kubernetes/install_cluster
./install_master_node.sh

• Install kubernetes:

  • Master:

  cd /root/kubernetes/install_k8s
  ./install-k8s.sh

  • Worker:

  cd /root/kubernetes/install_k8s
  ./install-k8s-worker.sh

  • To join a worker node:

  sodo kubeadm join master_ip:master_port --token token_id --discovery-token-ca-cert-hash hash_cert

  • To remoave a kubernetes setup:

  kubeadm reset

  • To create a new join token

  kubeadm token create print-join-command

When installation of master is completed you should see below components in kube-system namespace

When you reboot the vms, the kubelet service may not run, you need to restart the kubelet.

sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
sudo swapoff -a

systemctl stop kubelet
syatemctl start kubelet

Or Execute below command

./start_cluster.sh

Other debugging steps

systemctl status kubelet
journalctl -u kubelet

Useful commands:

• To change namespace

alias kcd='kubectl config set-context $(kubectl config current-context) --namespace'
kcd name_space

• Inorder to login via OpenId Connect username/password

alias kctl='kubectl --kubeconfig=/root/oauth.conf --token=$(python3 /root/kubernetes/install_k8s/kube-login/cli-auth.py)'

In order to use the above approach, you must install and run

1. Ingress Instress ReadME
2. Kubeauthentication service KubeAuth ReadME

• To enable verbose(logging) of kubectl command

kubectl get pods --v 6

Useful commands

kubectl top nodes --v 6

kubectl get pod, svc, hpa -owide
watch -n1 !!

kubectl get componentstatus

Automation of build and deployment

Note: Ingress should be installed before this can begin.

• Execute below command to install ci/cd pipeline

cd /root/kubernetes/install_k8s/
./setuUp-devops.sh 

This would install local docker registry, Git webhook, Docker webhook and Helm webhook

Architecture of the pipeline

In order to test the pipeline use hlw repository, make a sample change and commit.

Application would be accessible in https://master.cloud.com:32028/hlw

Installing components

1. Ingress Controller: Link
2. Kube Login Webhook
3. Devops: Githook, Dockerhook,Reghook
4. Kubernetes Dashboard: Link
5. Kube Authentication
6. Ldap: Link
7. Kerberos: Link
8. Ldap Client: Link
9. Ldap Client2: Link

• Add ca.crt and server.crt file in chrome browser, please refer link on how to add certificate. Add server.crt in Other People tab and ca.crt in Trusted Root Certificate Authority tab.
• Add ip_address master.cloud.com to windows host file located in C:\Windows\System32\drivers\etc. e.g. 192.168.1.5 master.cloud.com >> host file.

Setup Https for Kubernetes Cluster

Work in progress

Refer to HTTPS Setup with Kubernetes