feat: extend hsacle api to support Add or Delete replicas

[wangyelei]

Background

Some users have reported that their Redis cluster can become very large, consisting of numerous heterogeneous instances. They frequently need to scale out or in the cluster based on business scenarios, which may involve offline specified instances.

Currently, the HorizontalScaling opsRequest only supports declarative APIs. Each scaling operation requires users to manually calculate the number of replicas in the final cluster and manage the offline instances, which can be quite cumbersome.

Therefore, there is a need to support Add and Delete operations to simplify the complexity of computing and managing offline instances.

API

• replicasToAdd
  Adds replicas for the specified component and instance template.
• replicasToDelete
  Deletes replicas for the specified component and instance template.
• offlineInstancesToOnline
  Bring the specified offline instances back online.
  Replicas will be automatically adjust if the component and instance templates
  have no specified replicas and the component is not a sharding component.
• onlineInstancesToOffline
  Specifies the instance name that needs to be taken offline.
  Replicas will be automatically adjust if the component and instance templates
  have no specified replicas and the component is not a sharding component.
• instances.add
  Defines the configuration for new instances added during scaling, including resource requirements, labels, annotations, etc.
  New instances are created based on the provided InstanceTemplate.
• instances.change
  Modifies the desired replicas count for existing InstanceTemplate.

Limitations

1. If the submitted opsRequest attempts to delete instances created by a running opsRequest, it will be disallowed and will fail immediately.
2. If there is an ongoing opsRequest targeting the final state (i.e., an opsRequest specifying replicas for component and instances), the ongoing final state opsRequest will be terminated.

Examples

The original cluster is a MySQL cluster with 3 replicas and cpu/memory(1c2Gi).

1. scale in 2 replicas.

 horizontalScaling:
    - name: mysql
      replicasToDelete: 2
      # replicas: 1

2. scale out 2 replicas.

horizontalScaling:
 - name: mysql
   # replicas: 5
   replicasToAdd: 2

4. scale in 1 replicas with specified instance.

horizontalScaling:
   - name: mysql
     onlineInstancesToOffline: [pod-1]

5. then scale out the specified instance which is offline.

 horizontalScaling:
    - name: mysql
      offlineInstancesToOnline: [pod-1]

6. you can also scale in pod-1 but keep replicas is 3. and it will create a pod-3.

 horizontalScaling:
    - name: mysql
      replicasToDelete: 0
      onlineInstancesToOffline: [pod-1]

7. add an heterogeneous instance with diffrentent cpu/memory

 horizontalScaling:
    - name: mysql
      replicasToAdd: 1 # must add 1 total replicas, otherwise it will delete an existing pod and create the 2c4g pod.
      instances:
        add:
        - name: 2c4g
           replicasToAdd: 1 # add 1 replicas with 2c4Gi resources.
           resources:
              limits:
                 cpu: 2
                 memory: 4Gi            

8. delete the pod with cpu/memory 2c4g

 horizontalScaling:
    - name: mysql
      replicasToDelete: 1 # must delete 1 total replicas, otherwise it will create a new pod to keep replicas unchanged..
      instances:
        change:
        - name: 2c4g
           replicasToDelete: 1 # delete 1 replicas with 2c4Gi resources.         

[codecov]

Codecov Report

Attention: Patch coverage is 79.85948% with 86 lines in your changes are missing coverage. Please review.

Project coverage is 65.68%. Comparing base (07ea008) to head (da8cc4f).
Report is 14 commits behind head on main.

Files	Patch %	Lines
controllers/apps/operations/horizontal_scaling.go	87.71%	21 Missing and 8 partials ⚠️
apis/apps/v1alpha1/opsrequest_webhook.go	69.01%	15 Missing and 7 partials ⚠️
apis/apps/v1alpha1/opsrequest_types.go	0.00%	8 Missing ⚠️
controllers/apps/operations/upgrade.go	30.00%	6 Missing and 1 partial ⚠️
controllers/apps/operations/ops_progress_util.go	88.46%	4 Missing and 2 partials ⚠️
controllers/apps/operations/vertical_scaling.go	55.55%	3 Missing and 1 partial ⚠️
controllers/apps/operations/ops_comp_helper.go	81.25%	3 Missing ⚠️
controllers/apps/operations/volume_expansion.go	40.00%	2 Missing and 1 partial ⚠️
controllers/apps/operations/restart.go	33.33%	1 Missing and 1 partial ⚠️
controllers/apps/operations/stop.go	71.42%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7393      +/-   ##
==========================================
+ Coverage   65.54%   65.68%   +0.13%     
==========================================
  Files         342      342              
  Lines       41658    41918     +260     
==========================================
+ Hits        27306    27534     +228     
- Misses      11957    11982      +25     
- Partials     2395     2402       +7     

Flag	Coverage Δ
unittests	65.68% <79.85%> (+0.13%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.