Open
Description
Prerequisites
- This issue has an informative and human-readable title.
💡 Summary
Manually test ScubaGoggles in a variety of different ways (more details below).
Motivation and context
To minimize issues in the upcoming release.
Implementation notes
Notes for installation tests:
- Follow the installation instructions in README
- Don't just
pip install ...and call it good. Also test thescubagoggles setupcommand. Then run a quick sanity tests likescubagoggles gws, nothing thorough, just see if it can run to completion without errors
Notes for baseline tests:
- Run scubagoggles against that baseline
- For each control, verify that the tenant's current state is accurately reflected in the report
- For each control, if feasible, change the setting in the tenant, verify scubagoggles accurately reflects the change, then restore the original tenant state.
- Ensure that at least one setting is applied to a custom OU and is accurately reflected in the report
- Ensure that at least one setting is applied to a group and is accurately reflected in the report.
Notes for sanity checks:
- Don't need to be thorough
- Just basic, "does this seem to be working?" tests
Acceptance criteria
Test the installation process:
- Test fresh install on Mac @amart241
- Test fresh install on Windows @isabella-mont
- Test fresh install on Linux @snarve
- Test upgrading from existing installation (with ~/.scubagoggles existing as a text file) to 1.0
Test each baseline:
- Calendar functional tests @adhilto
- Chat functional tests @adhilto
- Classroom functional tests @adhilto
- CommonControls functional tests @adhilto
- Drive functional tests @adhilto
- Gmail functional tests
- Groups functional tests @snarve
- Meet functional tests @snarve
- Sites functional tests @adhilto
Final sanity checks:
- Run ScubaGoggles in our secondary test tenants @adhilto
- Run ScubaGoggles both with OAUTH credentials and with a service account @mitchelbaker-cisa
- Test on Python 3.13 @isabella-mont
- Test on Python 3.9 @isabella-mont