Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Helm templates to account for lack of external load balancer support #623

Open
scottslowe opened this issue Jul 28, 2023 · 1 comment
Labels
impact/reliability Something that feels unreliable or flaky impact/usability Something that impacts users' ability to use the product easily and intuitively kind/enhancement Improvements or new features needs-triage Needs attention from the triage team size/M Estimated effort to complete (up to 5 days).

Comments

@scottslowe
Copy link
Contributor

The current Helm templates deploy the Nginx ingress controller and expose it with a Service of type LoadBalancer. This is fine for production-grade environments where the Kubernetes clusters have external load balancer support, but this can cause issues when testing the programs on platforms like KinD or Minikube. These platforms typically do not have external load balancer support.

Without external load balancer support, the Service never comes out of a pending state, and this causes Pulumi to report messages like this:

 +   pulumi:pulumi:Stack               helm-kubernetes-go-dev  **creating failed**     1 error
 +   ├─ kubernetes:core/v1:Namespace   ingressns               created (0.18s)
 +   └─ kubernetes:helm.sh/v3:Release  ingresscontroller       **creating failed**     1 error; 1 warning


Diagnostics:
  kubernetes:helm.sh/v3:Release (ingresscontroller):
    warning: Helm release "ingresscontroller-ed059511" was created but has a failed status. Use the `helm` command to investigate the error, correct it, then retry. Reason: timed out waiting for the condition
    error: 1 error occurred:
    	* Helm release "nginx-ingress/ingresscontroller-ed059511" was created, but failed to initialize completely. Use Helm CLI to investigate.: failed to become available within allocated timeout. Error: Helm Release nginx-ingress/ingresscontroller-ed059511: timed out waiting for the condition

  pulumi:pulumi:Stack (helm-kubernetes-go-dev):
    error: update failed

To fix this, we need to update the Helm templates in three ways:

  1. Add a configuration value that specifies the Service type (LoadBalancer, ClusterIP, or NodePort).
  2. Provide this configuration value when creating the Helm Release object.
  3. Add comments to the code to explain what's happening

Defaulting to a value of ClusterIP will help tests pass successfully, but are not conducive to actual production deployments. Defaulting to LoadBalancer is more in line with actual production deployments, but tests will fail if external load balancer support isn't present. The recommendation is to default to ClusterIP with comments in the code on how to change that for production deployments. Changes to the associated landing pages on the website may also be needed.

@scottslowe scottslowe added impact/reliability Something that feels unreliable or flaky kind/enhancement Improvements or new features impact/usability Something that impacts users' ability to use the product easily and intuitively size/M Estimated effort to complete (up to 5 days). labels Jul 28, 2023
@Rajakavitha1
Copy link

Rajakavitha1 commented Oct 24, 2023

Hi @scottslowe Minikube does support LoadBalancer. However, you may have to run the command minikube tunnel in a separate terminal.

To perform an initial deployment, run `pulumi up`

warning: A new version of Pulumi is available. To upgrade from version '3.88.0' to '3.90.0', run 
   $ brew update && brew upgrade pulumi
or visit https://pulumi.com/docs/install/ for manual instructions and release notes.
rkodhandapani@blr-mpqob python % ls
Pulumi.yaml		requirements.txt
__main__.py		venv
rkodhandapani@blr-mpqob python % nano __main__.py
rkodhandapani@blr-mpqob python % pulumi up
Previewing update (dev)

View in Browser (Ctrl+O): https://app.pulumi.com/Rajie/helm/dev/previews/88bd4741-9757-4f40-98bd-0a0665f162bf

Downloading plugin: 37.45 MiB / 37.45 MiB [=========================] 100.00% 6s
                                                                               [resource plugin kubernetes-4.5.0] installing
     Type                              Name               Plan       
 +   pulumi:pulumi:Stack               helm-dev           create     
 +   ├─ kubernetes:core/v1:Namespace   ingressns          create     
 +   └─ kubernetes:helm.sh/v3:Release  ingresscontroller  create     

Outputs:
    name: "ingresscontroller-3b193d65"

Resources:
    + 3 to create

Do you want to perform this update? yes
Updating (dev)

View in Browser (Ctrl+O): https://app.pulumi.com/Rajie/helm/dev/updates/1

     Type                              Name               Status              
 +   pulumi:pulumi:Stack               helm-dev           created (34s)       
 +   ├─ kubernetes:core/v1:Namespace   ingressns          created (0.41s)     
 +   └─ kubernetes:helm.sh/v3:Release  ingresscontroller  created (30s)       

Outputs:
    name: "ingresscontroller-51639b48"

Resources:
    + 3 created

Duration: 36s

rkodhandapani@blr-mpqob python % kubectl get services
NAME                                       TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
ingresscontroller-51639b48-nginx-ingress   LoadBalancer   10.100.141.181   127.0.0.1     80:31089/TCP,443:31641/TCP   50s
kubernetes                                 ClusterIP      10.96.0.1        <none>        443/TCP                      5m8s
rkodhandapani@blr-mpqob python % curl  127.0.0.1
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.23.1</center>
</body>
</html>

However, if the port number is set to 8080. You don't have to provide the password for sudo user else, in the minikube tunnel terminal you have to provide the password.

🔗  Configuring bridge CNI (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
rkodhandapani@blr-mpqob ~ % kubectl get nodes
NAME       STATUS   ROLES           AGE   VERSION
minikube   Ready    control-plane   31s   v1.27.4
rkodhandapani@blr-mpqob ~ % minikube tunnel
✅  Tunnel successfully started

📌  NOTE: Please do not close this terminal as this process must stay alive for the tunnel to be accessible ...

❗  The service/ingress ingresscontroller-51639b48-nginx-ingress requires privileged ports to be exposed: [80 443]
🔑  sudo permission will be asked for it.
🏃  Starting tunnel for service ingresscontroller-51639b48-nginx-ingress.
Password:
Sorry, try again.
Password:

@pulumi-bot pulumi-bot added the needs-triage Needs attention from the triage team label Oct 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
impact/reliability Something that feels unreliable or flaky impact/usability Something that impacts users' ability to use the product easily and intuitively kind/enhancement Improvements or new features needs-triage Needs attention from the triage team size/M Estimated effort to complete (up to 5 days).
Projects
None yet
Development

No branches or pull requests

3 participants