Functional testing for Operator Charms

[balbirthomas]

Introduction

Zaza is an excellent platform for integration testing of Operator Charms. So for instance it allows deploying a group of charms, relating them and checking that the reach a desired goal state. It seems possible to extend Zaza to support functional testing of charms in isolation (as discussed below). The operator framework's unit testing harness also supports testing charms in isolation. However this unit testing does not actually deploy the charm and hence the unit tests need to mock/patch the underlying application interface. For example a unit tests of a MongoDB charm do not test the charm using a real deployed MongoDB instance. Further even though the Operator Framework provides a rich feature set for unit testing relation handling by charms they do so in a very controlled environment in isolation from the underlying application.

Proposal

Zaza can support functional testing of all functionality of charms (in isolation from other charms), using an Autogenerated Proxy Charm. For example suppose one would like to test a MongoDB charm. The metadata.yaml of the MongoDB charm lists all the relations that the MongoDB charm supports. This should be sufficient information to generate a Proxy charm that allows a Zaza functional test to interact with the MongoDB charm.