naming: rename setup and cleanup tasks

Refs: #2181

Rename cnf_setup, cnf_cleanup and cluster_api_setup, cluster_api_cleanup tasks to
cnf_install, cnf_uninstall and cluster_api_install, cluster_api_uninstall.
Change documentation, specs and code accordingly.

Signed-off-by: Konstantin Yarovoy <[email protected]>

.github/workflows/actions.yml

@@ -394,7 +394,7 @@ jobs:
         cd /shared/$DIR
         ./cnf-testsuite setup
         wget -O cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/testsuite/${GITHUB_SHA}/example-cnfs/coredns/cnf-testsuite.yml
-        ./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
+        ./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
         LOG_LEVEL=info ./cnf-testsuite all ~compatibility ~resilience ~reasonable_startup_time ~reasonable_image_size ~platform ~increase_capacity ~decrease_capacity ~install_script_helm ~helm_chart_valid ~helm_chart_published verbose
     - name: Delete Cluster
       if: ${{ always() }}
@@ -474,7 +474,7 @@ jobs:
         cd /shared/$DIR
         ./cnf-testsuite setup
         wget -O cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/testsuite/${GITHUB_SHA}/example-cnfs/coredns/cnf-testsuite.yml
-        ./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
+        ./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
         LOG_LEVEL=info ./cnf-testsuite all ~resilience ~compatibility ~pod_network_latency ~platform ~increase_capacity ~decrease_capacity ~liveness ~readiness ~rolling_update ~rolling_downgrade ~rolling_version_change ~nodeport_not_used ~hostport_not_used ~hardcoded_ip_addresses_in_k8s_runtime_configuration ~install_script_helm ~helm_chart_valid ~helm_chart_published ~rollback ~secrets_used ~immutable_configmap verbose
     - name: Delete Cluster
       if: ${{ always() }}
@@ -554,7 +554,7 @@ jobs:
         cd /shared/$DIR
         ./cnf-testsuite setup
         wget -O cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/testsuite/${GITHUB_SHA}/example-cnfs/coredns/cnf-testsuite.yml
-        ./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
+        ./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
         LOG_LEVEL=info ./cnf-testsuite all ~resilience ~platform ~liveness ~readiness ~rolling_update ~rolling_downgrade ~rolling_version_change ~nodeport_not_used ~hostport_not_used ~hardcoded_ip_addresses_in_k8s_runtime_configuration ~rollback ~secrets_used ~immutable_configmap ~reasonable_startup_time ~reasonable_image_size verbose
     - name: Delete Cluster
       if: ${{ always() }}

CNF_TESTSUITE_YML_USAGE.md

@@ -1,7 +1,7 @@
 # Test Suite Configuration Usage: cnf-testsuite.yml
 ### What is the cnf-testsuite.yml and why is it required?:
 
-The cnf-testsuite.yml is used by `cnf_setup` in order to install the CNF to be tested onto an existing K8s cluster. 
+The cnf-testsuite.yml is used by `cnf_install` in order to install the CNF to be tested onto an existing K8s cluster. 
 
 
 The information in the cnf-testsuite.yml is also used for additional configuration of some tests e.g. `white_list_container_names` is used for exculding containers from the [privileged_containers](https://github.com/cnti-testcatalog/testsuite/blob/main/src/tasks/workload/security.cr#L138) container test.
@@ -122,7 +122,7 @@ Described below: [link](#5G-parameters)
 
 Deployments are defined as three arrays, each for different installation method. Each array element represents one deployment, and they are meant to represent a single CNF together.
 At least one deployment should exist in CNF config for it to be a proper config.
-All info for deployments is used in cnf_setup and cnf_cleanup tasks.
+All info for deployments is used in cnf_install and cnf_uninstall tasks.
 
 ```yaml
 deployments:

INSTALL.md

@@ -177,7 +177,7 @@ source test.sh
 </details>
 
 ### Configuration
-Now cnf-testsuite is setup, we're ready to configure it to point at a CNF to test.
+Now cnf-testsuite is prepared, we're ready to configure it to point at a CNF to test.
 
 #### Using an Example CNF
 
@@ -188,7 +188,7 @@ wget -O cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/tes
 ```
 - The wget gets a working config file, now tell cnf-testsuite to use it by doing the following:
 ```
-cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
+cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
 ```
 
 - There are other examples in the [examples-cnfs](https://github.com/cnti-testcatalog/testsuite/tree/master/example-cnfs) directory that can be used for testing as well.
@@ -200,14 +200,14 @@ If you've brought your own CNF to test, review the [CNF_TESTSUITE_YML_USAGE.md](
 If you've followed the [CNF_TESTSUITE_YML_USAGE.md](CNF_TESTSUITE_YML_USAGE.md) guide and have your cnf-testsuite.yml ready, you can run the same command we ran for the example CNF to set it up:
 
 ```
-cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
+cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
 ```
 
 ### Running cnf-testsuite for the first time
 
 #### Running Tests
 
-If you want to run all tests, do the following (this is assuming your `cnf_setup` ran without errors in the [configuration](#Configuration) steps:)
+If you want to run all tests, do the following (this is assuming your `cnf_install` ran without errors in the [configuration](#Configuration) steps:)
 _For complete usage, see the [USAGE.md](USAGE.md) doc._
 
 ```
@@ -233,12 +233,12 @@ A test log file, eg. `cnf-testsuite-results-20201216.txt`, will be created which
 
 For more details on points, see our [POINTS.md](./POINTS.md) documentation.
 
-#### Cleaning Up
+#### Uninstallation
 
 Run the following to uninstall the CNF (this is assuming you installed the cnf-testsuite.yml in your present working directory):
 ```
-cnf-testsuite cnf_cleanup
+cnf-testsuite cnf_uninstall
 ```
-You can also run `cleanall` and cnf-testsuite will attempt to cleanup everything.
+You can also run `cleanup` and cnf-testsuite will attempt to cleanup everything.
 
-_NOTE: Cleanup does not handle manually deployed CNFs_
+_NOTE: CNF uninstallation does not handle manually deployed CNFs_

README.md

@@ -19,7 +19,7 @@ Prereqs: Kubernetes cluster, wget, curl, helm 3.1.1 or greater on your system al
 1. Install the latest test suite binary: `source <(curl -s https://raw.githubusercontent.com/cnti-testcatalog/testsuite/main/curl_install.sh)`
 2. Run `setup` to prepare the cnf-testsuite: `cnf-testsuite setup`
 3. Pull down an example CNF configuration to try: `curl -o cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/testsuite/main/example-cnfs/coredns/cnf-testsuite.yml`
-4. Initialize the test suite for using the CNF: `cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml`
+4. Initialize the test suite for using the CNF: `cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml`
 5. Run all of application/workload tests: `cnf-testsuite workload`
 
 #### More Usage docs

SOURCE_INSTALL.md

@@ -8,7 +8,7 @@ This INSTALL guide will detail the minimum requirements needed for cnf-testsuite
 
 - [**Pre-Requisites**](#Pre-Requisites)
 - [**Installation**](#Installation)
-- [**Setup**](#Setup)
+- [**Preparation**](#Preparation)
 - [**Configuration**](#Configuration)
 - [**Running cnf-testsuite for the first time**](#Running-cnf-testsuite-for-the-first-time)
 
@@ -149,11 +149,11 @@ crystal spec
 
 </p></details>
 
-### Setup
+### Preparation
 
-Now that we have a `cnf-testsuite` binary, we can run `setup` to ensure it has all the pre-requisites needed in order to successfully run tests and setup required installed_cnf_files/ directory and other files required for cnf-testsuite.
+Now that we have a `cnf-testsuite` binary, we can run `setup` to ensure it has all the pre-requisites needed in order to successfully run tests and prepare required installed_cnf_files/ directory and other files required for cnf-testsuite.
 
-- Run the following to setup cnf-testsuite:
+- Run the following to prepare cnf-testsuite:
   ```
   ./cnf-testsuite setup
   ```
@@ -165,7 +165,7 @@ Now that we have a `cnf-testsuite` binary, we can run `setup` to ensure it has a
 
 ### Configuration
 
-Now that cnf-testsuite is installed and setup, we can now run CNF workloads and tests. We recommend installing and running a sample CNF to ensure cnf-testsuite is operational and set expectations of the output.
+Now that cnf-testsuite is installed and prepared, we can now run CNF workloads and tests. We recommend installing and running a sample CNF to ensure cnf-testsuite is operational and set expectations of the output.
 
 #### Configuring an example CNF
 
@@ -178,12 +178,12 @@ To use CoreDNS as an example CNF. Download the testsuite configuration to test C
 - Prepare the test suite to use the CNF by running:
   ```
   # via built binary
-  ./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
+  ./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
   ```
   Or
   ```
   # via crystal
-  crystal src/cnf-testsuite.cr cnf_setup cnf-config=./cnf-testsuite.yml
+  crystal src/cnf-testsuite.cr cnf_install cnf-config=./cnf-testsuite.yml
   ```
 
 There are other examples in the [example cnfs](example-cnfs) folder if you would like to test others.
@@ -197,7 +197,7 @@ There are other examples in the [example cnfs](example-cnfs) folder if you would
 
 #### Running Tests
 
-If you want to run all tests for CoreDNS Example CNF, do the following (this is assuming your `cnf_setup` ran without errors in the [configuration](#Configuring-an-example-CNF) steps:)
+If you want to run all tests for CoreDNS Example CNF, do the following (this is assuming your `cnf_install` ran without errors in the [configuration](#Configuring-an-example-CNF) steps:)
 _For complete usage, see the [USAGE.md](USAGE.md) doc._
 
 ```
@@ -221,7 +221,7 @@ You can also run via `crystal` by replacing the `./cnf-testsuite` with `crystal
 #### More Example Usage (also see the [complete usage documentation](https://github.com/cnti-testcatalog/testsuite/blob/main/USAGE.md))
 
 ```
-# These assume you've already run the cnf_setup pointing at a cnf-testsuite.yml config above. You can always specify your config at the end of each command as well, eg:
+# These assume you've already run the cnf_install pointing at a cnf-testsuite.yml config above. You can always specify your config at the end of each command as well, eg:
 ./cnf-testsuite all cnf-config=<path to your config yml>/cnf-testsuite.yml
 
 # Runs all ga tests (generally available workload and platform tests)
@@ -253,17 +253,18 @@ A test log file, eg. `cnf-testsuite-results-20201216.txt`, will be created which
 
 For more details on points, see our [POINTS.md](./POINTS.md) documentation.
 
-#### Cleaning Up
+#### Uninstallation
 
 Run the following to uninstall the CNF:
 
 ```
-./cnf-testsuite cnf_cleanup
+./cnf-testsuite cnf_uninstall
 ```
 
-You can also run `cleanall` and cnf-testsuite will attempt to cleanup everything.
+You can also run `cleanup` and cnf-testsuite will attempt to cleanup everything.
+
+_NOTE: CNF uninstallation does not handle manually deployed CNFs_
 
-_NOTE: Cleanup does not handle manually deployed CNFs_
 #### NOTE: If the OpenMetrics version changes, the protobuf file will need to be regenerated
 ```
 git clone https://github.com/jeromegn/protobuf.cr

USAGE.md

@@ -62,12 +62,12 @@ crystal build src/cnf-testsuite.cr
 #### Installing a cnf:
 
 ```
-./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
+./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
 ```
 
 ##### Skip waiting for resource readiness during installation:
 ```
-./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml skip_wait_for_install
+./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml skip_wait_for_install
 ```
 
 #### Running all of the platform and workload tests:

example-cnfs/coredns/README.md

@@ -24,7 +24,7 @@ Initialize the test suite
 Configure and deploy CoreDNS as the target CNF
 
 ```
-./cnf-testsuite cnf_setup cnf-path=example-cnfs/coredns/cnf-testsuite.yml
+./cnf-testsuite cnf_install cnf-path=example-cnfs/coredns/cnf-testsuite.yml
 ```
 
 Run the all the tests
@@ -38,5 +38,5 @@ Check the results file
 Uninstall the CNF (including undeployment of CoreDNS)
 
 ```
-./cnf-testsuite cnf_cleanup
+./cnf-testsuite cnf_uninstall
 ```

example-cnfs/envoy/README.md

@@ -21,7 +21,7 @@ Run cnf-testsuite setup
 Install Envoy
 
 ```
-./cnf-testsuite cnf_setup cnf-config=example-cnfs/envoy/cnf-testsuite.yml
+./cnf-testsuite cnf_install cnf-config=example-cnfs/envoy/cnf-testsuite.yml
 ```
 
 Run the test suite:
@@ -33,5 +33,5 @@ Run the test suite:
 Envoy uninstallation
 
 ```
-./cnf-testsuite cnf_cleanup
+./cnf-testsuite cnf_uninstall
 ```

example-cnfs/ip-forwarder/README.md

@@ -25,15 +25,15 @@ export KUBECONFIG=$(pwd)/<YourKubeConf> ; ./cnf-testsuite setup
 
 Setup and deploy  service chain
 ```
-export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite cnf_setup cnf-path=example-cnfs/ip-forwarder/cnf-testsuite.yml deploy_with_chart=false
+export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite cnf_install cnf-path=example-cnfs/ip-forwarder/cnf-testsuite.yml deploy_with_chart=false
 ```
 
 ### Testing
 Run the test suite: `export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite all`
 
 ### Automated uninstallation
 ```
-export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite cnf_cleanup
+export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite cnf_uninstall
 ```
 
 

example-cnfs/linkerd2/README.md

@@ -24,7 +24,7 @@ helm repo add linkerd https://helm.linkerd.io/stable
 
 helm install linkerd-crds linkerd/linkerd-crds -n linkerd --create-namespace 
 
-.cnf-testsuite cnf_setup cnf-path=example-cnfs/linkerd2/cnf-testsuite.yml
+.cnf-testsuite cnf_install cnf-path=example-cnfs/linkerd2/cnf-testsuite.yml
 ```
 
 Run the test suite:
@@ -36,5 +36,5 @@ Run the test suite:
 linkerd uninstallation
 
 ```
-./cnf-testsuite cnf_cleanup
+./cnf-testsuite cnf_uninstall
 ```

example-cnfs/nsm/README.md

@@ -20,7 +20,7 @@ Initialize the test suite
 Configure and deploy NSM as the target CNF
 
 ```
-./cnf-testsuite cnf_setup cnf-config=./example-cnfs/nsm/cnf-testsuite.yml deploy_with_chart=false
+./cnf-testsuite cnf_install cnf-config=./example-cnfs/nsm/cnf-testsuite.yml deploy_with_chart=false
 ```
 
 Run the all the tests
@@ -34,5 +34,5 @@ Check the results file
 Uninstall the CNF (including undeployment of NSM)
 
 ```
-./cnf-testsuite cnf_cleanup
+./cnf-testsuite cnf_uninstall
 ```

example-cnfs/pantheon-nsm-nat/README.md

@@ -28,9 +28,9 @@ Initialize the test suite
 Configure and deploy nsm and nsm-nat as the target CNF
 
 ```
-./cnf-testsuite cnf_setup cnf-config=./example-cnfs/nsm/cnf-testsuite.yml deploy_with_chart=false
+./cnf-testsuite cnf_install cnf-config=./example-cnfs/nsm/cnf-testsuite.yml deploy_with_chart=false
 
-./cnf-testsuite cnf_setup cnf-config=./example-cnfs/pantheon-nsm-nat/cnf-testsuite.yml deploy_with_chart=false
+./cnf-testsuite cnf_install cnf-config=./example-cnfs/pantheon-nsm-nat/cnf-testsuite.yml deploy_with_chart=false
 ```
 
 Run the all the tests
@@ -44,5 +44,5 @@ Check the results file
 Uninstall the CNF (including undeployment of nsm-nat)
 
 ```
-./cnf-testsuite cnf_cleanup
+./cnf-testsuite cnf_uninstall
 ```

example-cnfs/pantheon-nsm-nat/nat-cnf/README.md

@@ -20,7 +20,7 @@ crystal src/cnf-testsuite.cr setup
 Configure and deploy nsm-nat as the target CNF
 
 ```
-crystal src/cnf-testsuite.cr cnf_setup cnf-config=./example-cnfs/pantheon-nsm-nat/cnf-testsuite.yml deploy_with_chart=false
+crystal src/cnf-testsuite.cr cnf_install cnf-config=./example-cnfs/pantheon-nsm-nat/cnf-testsuite.yml deploy_with_chart=false
 ```
 
 Run the all the tests
@@ -34,5 +34,5 @@ Check the results file
 Uninstall the CNF (including undeployment of nsm-nat)
 
 ```
-crystal src/cnf-testsuite.cr cnf_cleanup
+crystal src/cnf-testsuite.cr cnf_uninstall
 ```

example-cnfs/vpp-3c2n-csp-use-case/README.md

@@ -23,14 +23,14 @@ export KUBECONFIG=$(pwd)/<YourKubeConf> ; ./cnf-testsuite setup
 
 Setup and deploy  service chain
 ```
-export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite cnf_setup example-cnf-path=example-cnfs/vpp-3c2n-csp-use-case/cnf-testsuite.yml
+export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite cnf_install example-cnf-path=example-cnfs/vpp-3c2n-csp-use-case/cnf-testsuite.yml
 ```
 
 ### Testing
 Run the test suite: `export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite all`
 
 ### Automated uninstallation
 ```
-export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite cnf_cleanup
+export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite cnf_uninstall
 ```
 

spec/5g/core_spec.cr

@@ -14,44 +14,44 @@ describe "Core" do
 
   it "'smf_upf_heartbeat' should pass if the smf_upf core is resilient to network latency", tags: ["core"]  do
     begin
-      ShellCmd.cnf_setup("cnf-config=sample-cnfs/sample_open5gs/cnf-testsuite.yml")
+      ShellCmd.cnf_install("cnf-config=sample-cnfs/sample_open5gs/cnf-testsuite.yml")
       result = ShellCmd.run_testsuite("smf_upf_heartbeat verbose")
       (/(PASSED).*(Chaos service degradation is less than 50%)/ =~ result[:output]).should_not be_nil
     ensure
-      result = ShellCmd.cnf_cleanup()
+      result = ShellCmd.cnf_uninstall()
       result[:status].success?.should be_true
     end
   end
 
   it "'smf_upf_heartbeat' should fail if the smf_upf core is not resilient to network latency", tags: ["core"]  do
     begin
-      ShellCmd.cnf_setup("cnf-config=sample-cnfs/sample_open5gs/cnf-testsuite.yml")
+      ShellCmd.cnf_install("cnf-config=sample-cnfs/sample_open5gs/cnf-testsuite.yml")
       result = ShellCmd.run_testsuite("smf_upf_heartbeat verbose baseline_count=300")
       (/(FAILED).*(Chaos service degradation is more than 50%)/ =~ result[:output]).should_not be_nil
     ensure
-      result = ShellCmd.cnf_cleanup()
+      result = ShellCmd.cnf_uninstall()
       result[:status].success?.should be_true
     end
   end
 
   it "'suci_enabled' should pass if the 5G core has suci enabled", tags: ["5g"]  do
     begin
-      ShellCmd.cnf_setup("cnf-config=sample-cnfs/sample_open5gs/cnf-testsuite.yml")
+      ShellCmd.cnf_install("cnf-config=sample-cnfs/sample_open5gs/cnf-testsuite.yml")
       result = ShellCmd.run_testsuite("suci_enabled verbose")
       (/(PASSED).*(Core uses SUCI 5g authentication)/ =~ result[:output]).should_not be_nil
     ensure
-      result = ShellCmd.cnf_cleanup()
+      result = ShellCmd.cnf_uninstall()
       result[:status].success?.should be_true
     end
   end
 
   it "'suci_enabled' should fail if the 5G core does not have suci enabled", tags: ["5g"]  do
     begin
-      ShellCmd.cnf_setup("cnf-config=sample-cnfs/sample_open5gs_no_auth/cnf-testsuite.yml")
+      ShellCmd.cnf_install("cnf-config=sample-cnfs/sample_open5gs_no_auth/cnf-testsuite.yml")
       result = ShellCmd.run_testsuite("suci_enabled verbose")
       (/(FAILED).*(Core does not use SUCI 5g authentication)/ =~ result[:output]).should_not be_nil
     ensure
-      result = ShellCmd.cnf_cleanup()
+      result = ShellCmd.cnf_uninstall()
       result[:status].success?.should be_true
     end
   end