-
-
Notifications
You must be signed in to change notification settings - Fork 46
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Model regression test setup #435
Comments
There has been a superhuman release of https://github.com/EnviroDIY/ModularSensors/releases/tag/v0.35.0 My question is, how should it be regression tested for all previous functionality ? For open source software, regression testing is often something that the wider community can contribute. However this is only possible if its set up to be testable. Practically speaking for my fork, "release1" stream based on the core (master), its going to take time to merge https://github.com/neilh10/ModularSensors/releases/tag/v0.34.1-abe and then I will be able to feedback any issues I find. |
For testing community PRs on core code, it would be useful to have a model test setup - a Maylfy 1.1 with battery and optionally a variety of communication devices.
For EASY regression testing, it must be possible to generate a new build, put it on the test system and then let it run for some time period for easy proof it works. Weekends are ideal times to leave a system running for an extended soak.
The mayfly 1.1 has four basic sensors that can prove the reliability of the data flow.
Stability of a systems core software functions on an embedded uP is one of the most challenging aspects of an embedded system. Typically this is performed on a private sandbox, and consists of adding functionality and refactoring.
The stability/function testing are called "regression testing", and can be both time consuming and boring. The opposite of the systems objective to be able easily extend a function, compile and commit.
There are many academic papers on the subject of adding functions and verifying "software correctness", promoting functionality through stages of stability - and there are many examples of embbeded at its essence there are three function
The git workflow supports branches and also a "develop" stream, for gaining confidence with new code.
Currently, it seems to me that SWRC, @SRGDamia1 critically performs item 2), It would be nice for that process to be supported with good data by users wanting 1) and prepared to do 3).
Its critical that one person isn't response for system stability, and there is an easy way of distributing the ability to do regression testing, reporting, and collectively analyzing the results.
I currently have a version of this method on my own branch and would be happy to create a PR.
This would support
a) say extras\ref_platform or examples\ref_platform -
b) configuration (through local .ini) of the test setup, including defined all UUIDs in the .ini
This could be modified later for git hub actions to automatically do a build, producing a .hex - that can then downloaded to a real system for core regression testing.
The text was updated successfully, but these errors were encountered: