Merge pull request #256 from shreyas-s-rao/doc/tests

Add test documentation

.ci/performance_regression_test

@@ -41,7 +41,7 @@ if [ "$PERF_TEST_KUBECONFIG" == "" ]; then
   PERF_TEST_KUBECONFIG=$TM_KUBECONFIG_PATH/shoot.config
 fi
 
-ETCD_VERSION=${ETCD_VERSION:-"v3.3.12"}
+ETCD_VERSION=${ETCD_VERSION:-"v3.3.17"}
 if [ "$ETCD_VERSION" != "" ]; then
   ETCD_IMAGE=${ETCD_IMAGE:-"quay.io/coreos/etcd:$ETCD_VERSION"}
 fi

README.md

@@ -26,6 +26,7 @@ Etcd-backup-restore is collection of components to backup and restore the [etcd]
 
 * [Setting up a local development environment](doc/development/local_setup.md)
 * [Testing and Dependency Management](doc/development/testing_and_dependencies.md)
+* [Tests](doc/development/tests.md)
 * [Adding support for a new object store provider](doc/development/new_cp_support.md)
 
 [etcd]: https://github.com/coreos/etcd

doc/development/testing_and_dependencies.md

@@ -19,10 +19,48 @@ The dependencies are installed into the `vendor` folder which **should be added*
 
 # Testing
 
+This section describes the process to execute tests. For more details about kind of tests that are executed, please refer [test documentation](./tests.md)
+
+### Unit tests
+
 We have created `make` target `verify` which will internally run different rules like `fmt` for formatting, `lint` for linting check and most importantly `test` which will check the code against predefined unit tests. As currently there aren't enough test cases written to cover the entire code, you must check for failure cases manually and include test cases before raising pull request. We will eventually add more test cases for complete code coverage.
 
 ```sh
 make verify
 ```
 
 By default, we run tests without computing code coverage. To get the code coverage, you can set the environment variable `COVER` to `true`. This will log the code coverage percentage at the end of test logs. Also, all cover profile files will be accumulated under `test/output/coverprofile.out` directory. You can visualize the exact code coverage by running `make show-coverage` after running `make verify` with code coverage enabled.
+
+### Integration tests
+
+You can also run integration tests for etcd-backup-restore on any given Kubernetes cluster. The test creates namespace `integration-test` on the cluster and deploys the [etcd-backup-restore helm chart](../../chart/etcd-backup-restore) which in turn deploys the required secrets, configmap, services and finally the statefulset which contains the pod that runs etcd and backup-restore as a sidecar.
+
+```sh
+make integration-test-cluster
+```
+
+:warning: Prerequisite for this command is to set the following environment variables:
+
+- INTEGRATION_TEST_KUBECONFIG: kubeconfig to the cluster on which you wish to run the test
+- ETCD_VERSION: optional, defaults to `v3.3.17`
+- ETCDBR_VERSION: optional, defaults to `v0.9.1`
+- ACCESS_KEY_ID: S3 credentials
+- SECRET_ACCESS_KEY: S3 credentials
+- REGION: S3 credentials
+- STORAGE_CONTAINER: S3 bucket name
+
+If you have a working setup of [TestMachinery](https://github.com/gardener/test-infra), you can run the integration tests on a TM-generated cluster as well.
+
+### Performance regression tests
+
+Furthermore, you can check any regression in performance in terms of memory consumption and CPU utilization, bby running the provided performance regression tests.
+
+```sh
+make perf-regression-test
+```
+
+:warning: Prerequisite for this command is to set the following environment variables:
+
+- PERF_TEST_KUBECONFIG: kubeconfig to the cluster on which you wish to run the test
+- ETCD_VERSION: optional, defaults to `v3.3.17`
+- ETCDBR_VERSION: etcd-backup-restore version to test against

doc/development/tests.md

@@ -0,0 +1,27 @@
+# Tests
+
+`etcd-backup-restore` makes use of three sets of tests - unit tests in each package, integration tests to ensure the working of the overall tool and performance regression tests to check changes in resource consumption between different versions of etcd-backup-restore.
+
+### Integration tests
+
+Integration tests include the basic working of:
+
+- **snapshotting**: successfully upload full and delta snapshots to the configured snapstore according to the specified schedule
+- **garbage collection**: garbage-collect old snapshots on the snapstore according to the specified policy
+- **defragmentation**: etcd data should be defragmented periodically to reduce db size
+- **http server**: http endpoints should work as expected:
+
+  - `/snapshot/full`: should take an on-demand full snapshot
+  - `/snapshot/delta`: should take an on-demand delta snapshot
+  - `/snapshot/latest`: should list the latest set of snapshots (full + deltas)
+
+- **data validation**: corrupted etcd data should be marked for deletion and restoration should be triggered
+- **restoration**: etcd data should be restored correctly from latest set of snapshots (full + deltas)
+
+### Unit tests
+
+Each package within this repo contains its own set of unit tests to test the functionality of the methods contained within the packages.
+
+### Performance regression tests
+
+These tests help check any regression in performance in terms of memory consumption and CPU utilization.