Integrate istio ingress

[abaguas]

Fixes #552 The K8GB controller can now speak istio 🎉

User config

In #1557 the groundwork was done to be able to reference any ingress type.
The only implementation so far was Kubernetes Ingress resources. With this PR we bring to life the second implementation without touching any of the core logic: istio virtual services. The configuration looks as follows:

spec:
  resourceRef:
    apiVersion: networking.istio.io/v1
    kind: VirtualService
    matchLabels:
      app: istio

CRD changes

The previous API required CRD changes on every new integration because it was depending on a key with its name:

spec:
  resourceRef:
     istio:
        ...
     ingress:
        ...

To keep the CRD stable the structure is changed to:

spec:
  resourceRef:
    apiVersion: foo
    kind: bar

How istio ingress works

Istio uses istio ingress gateways for ingress. The users send traffic to the gateway, and then the gateway forward traffic to the application.
The ingress gateway is configured as follows:

• Gateway resources configure the ports open on the gateway. This resource contains also a list of hosts (Host header or SNI) that it expects on that port.
• Virtual Service resources configure to which kubernetes service the traffic should be routed, after it has been accepted by the ingress gateway. It contains a list of hosts (Host header or SNI), the service it should route the traffic to, and a reference to a Gateway resource.

How the k8gb controller integrates with istio

The k8gb controller is interested in three pieces of information: the domain of the endpoint and the IP address of the cluster ingress to create DNS entries; the kubernetes service in front of the application to monitor the health of the endpoints.
From the VirtualService resource all this information can be fetched, that is why this is the resource referenced by the GSLB. The domain and service are immediately available on the VirtualService resource. The IP address needs to be fetched from the istio ingress controller's LoadBalancer service which can reached by following -> VirtualService.spec.gateways -> Gateway.spec.selector.

Playground setup

To showcase this new integration the playground was extended with an istio ingress controller running side by side with the nginx ingress controller. The nginx ingress controller is still exposed on ports 80 and 81 (for clusters k3d-test-gslb1 and k3d-test-gslb2 respectively), and the istio ingress controller is exposed on ports 8080 and 8081.
In addition, since the podinfo pod behind the istio ingress requires a sidecar, a new namespace test-gslb-ingress was created to host this application and the respective GSLB resources.

Helm chart

The helm chart is adapted with a feature flag to enable the necessary RBAC read permissions.

Testing

These changes are backed by unit and e2e tests. The framework needed to be slightly adapted to support applying also VirtualService and Gateway resources but the core logic of both the frameworks and the tests remains.
In addition, a workflow was created to run the terratest suit for the v1beta1 apiVersion of istio, since we plan to support both v1beta1 and v1.
The PR looks big, but most of the changes are new manifests for the e2e testing setup.

[netlify]

✅ Deploy Preview for k8gb-preview ready!

Name	Link
🔨 Latest commit	c765b3c
🔍 Latest deploy log	https://app.netlify.com/sites/k8gb-preview/deploys/66dc1ed9f5f8e60008a9592c
😎 Deploy Preview	https://deploy-preview-1672--k8gb-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[ytsarev]

I've updated the default-full-local-setup exaples to use new resourceRef style in 08623be

Also resolved the conflict in GHA workflow to be able to merge with master

k -n test-gslb get gslb failover-ingress -o yaml
apiVersion: k8gb.absa.oss/v1beta1
kind: Gslb
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"k8gb.absa.oss/v1beta1","kind":"Gslb","metadata":{"annotations":{},"name":"failover-ingress","namespace":"test-gslb"},"spec":{"resourceRef":{"apiVersion":"networking.k8s.io/v1","kind":"Ingress","matchLabels":{"app":"test-gslb-failover"}},"strategy":{"primaryGeoTag":"eu","type":"failover"}}}
  creationTimestamp: "2024-09-15T21:48:05Z"
  finalizers:
  - k8gb.absa.oss/finalizer
  generation: 2
  name: failover-ingress
  namespace: test-gslb
  resourceVersion: "1038"
  uid: abf12c92-0490-453d-bf1f-75194962a0aa
spec:
  ingress: {}
  resourceRef:
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    matchLabels:
      app: test-gslb-failover
  strategy:
    dnsTtlSeconds: 30
    primaryGeoTag: eu
    splitBrainThresholdSeconds: 300
    type: failover
status:
  geoTag: us
  healthyRecords:
    failover.cloud.example.com:
    - 172.20.0.6
    - 172.20.0.7
  hosts: failover.cloud.example.com
  loadBalancer:
    exposedIps:
    - 172.20.0.10
    - 172.20.0.11
  servers:
  - host: failover.cloud.example.com
    services:
    - name: frontend-podinfo
      namespace: test-gslb
  serviceHealth:
    failover.cloud.example.com: Healthy

Local test with new style resourceRef looks great.

I've noticed that if we reference non-existing apiGroup or Kind , the Gslb will be silent and will not give the user any hints that it can't resolve resource and how to fix the situation

cat test-broken.yaml
apiVersion: k8gb.absa.oss/v1beta1
kind: Gslb
metadata:
  name: roundrobin-ingress-broken
  namespace: test-gslb
spec:
  resourceRef:
    apiVersion: networking.k8s.io/v1
    kind: Degress
    matchLabels:
      app: roundrobin-test-gslb
  strategy:
    type: roundRobin # Use a round robin load balancing strategy, when deciding which downstream clusters to route clients to
    splitBrainThresholdSeconds: 300 # Threshold after which external cluster is filtered out from delegated zone when it doesn't look alive
    dnsTtlSeconds: 30 # TTL value for automatically created DNS records

The Event stream will be empty

k describe -f test-broken.yaml
Name:         roundrobin-ingress-broken
Namespace:    test-gslb
Labels:       <none>
Annotations:  <none>
API Version:  k8gb.absa.oss/v1beta1
Kind:         Gslb
Metadata:
  Creation Timestamp:  2024-09-15T22:04:22Z
  Finalizers:
    k8gb.absa.oss/finalizer
  Generation:  2
  Managed Fields:
    API Version:  k8gb.absa.oss/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:finalizers:
          .:
          v:"k8gb.absa.oss/finalizer":
      f:spec:
        f:ingress:
    Manager:      k8gb
    Operation:    Update
    Time:         2024-09-15T22:04:22Z
    API Version:  k8gb.absa.oss/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
      f:spec:
        .:
        f:resourceRef:
        f:strategy:
          .:
          f:dnsTtlSeconds:
          f:splitBrainThresholdSeconds:
          f:type:
    Manager:         kubectl-client-side-apply
    Operation:       Update
    Time:            2024-09-15T22:04:22Z
  Resource Version:  2147
  UID:               e14fd3fb-0374-4f92-9fcf-73b1641e6d1a
Spec:
  Ingress:
  Resource Ref:
    API Version:  networking.k8s.io/v1
    Kind:         Degress
    Match Labels:
      App:  roundrobin-test-gslb
  Strategy:
    Dns Ttl Seconds:                30
    Split Brain Threshold Seconds:  300
    Type:                           roundRobin
Events:                             <none>

I suggest to create separate issue to track the verbosity enhancement and not bloat this PR

Approved, @abaguas thanks for the incredible work 🚀