Support for finding steps which do not match without .yml file

[davidghiurco]

I am aware of matching testing functionality under the Testing Step Patterns heading in the radish documentation.

Right now, one needs to write a .yml file which would describe what functions a step sentence should match. And that's greatly useful for thorough testing. Especially useful when there exist very similar step scenarios with slightly different wording and such.

However, this testing strategy is a bit bulky and cumbersome in same cases. What I if I just want to execute a command to find steps which do not have a matching implementation yet?

Right now the only way I'm aware to do that (and correct me if I'm wrong) is to outright run radish on a feature file with the corresponding step file(s) and if it does not match all step sentences it will throw an error. However if it does match everything, it will start executing the steps, which you might not want if you're just trying to figure out if a step has a matching implementation.

Just a feature suggestion for future releases (unless it already exists and I just blatantly missed it)
🥇

[timofurrer]

The only thing which is in place right now and comes close to what you want is the Dry run feature.

However, I think that does not fit your use-case perfectly.

The dry-run feature is somehow weird that's why this issue exists: #35
Thus, there might change some stuff in the future. Maybe something towards you are looking for?

How would you expect such a feature to behave?

[davidghiurco]

If you don't mind, could you explain what the dry-run does/ is supposed to do? I'm not quite understanding it from the documentation.

Specifically, I don't understand this part of the dry-run documentation, It seems that --dry-run does the same thing as --profile and --user-data ??? Which doesn't make any sense to me:

Run - Profile¶

Radish allows you to pass custom data to a Terrain hook code or to the Step implementations using the -p or --profile command line option. This can be used to customize your test runs as needed.

The value specified to the -p / --profile command line option is made available in world.config.profile. Please see World for for an example.

A common usage of profile s setting it to some environment value such as stage or production.

radish SomeFeature.feature -p stage
radish SomeFeature.feature --profile stage

Note: -p / --profile is being deprecated and will be removed in a future version of Radish. Please use -u / --user-data instead. See Arbitrary User Data for details.
Run - Dry run

Radish allows you to pass custom flags to a Terrain hook code or to Step implementations using the -d or --dry-run command line option. This can be used to customize your test runs as needed.

The -d / --dry-run command line switch is made available in world.config.dry_run which is set to True. Please see World for an example.

radish SomeFeature.feature -d
radish SomeFeature.feature --dry-run

Also, the way I would imagine this feature looking like is something along the lines of

$ radish --show-unmatched-steps myFeature.feature

And printed would be a list of steps which are not implemented in the specified feature file(s).

Right now, radish throws an error when you give it a feature file which has an unmatched step. It will throw an error and show you the first step which is unmatched. If you have multiple unmatched steps, after fixing the previous unmatched step, it will throw an error on the next unmatched step and show you it, until finally it matches all steps and then begins test executions.

So I think bundling that up into a CLI argument that does just verifying matching for all steps and printing them out to console (with an optional report format file, maybe) would be a nice complement to the step testing suite.

Also if you wanted to get really fancy, you could suggest fixing closely matched steps based on a cosine similarity between the test step and the step implementation regex.

[github-actions]

This Issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days

[dduleba]

Hi @davidghiurco,
I have developed something that may be useful for you.
I developed some extensions to simplify my everyday work with radish.

I added there steps generator also:
https://github.com/nokia/radish-bdd-extensions/blob/master/pkg_radish_ext/radish_ext/radish/stub_generator.py

Running the runner https://github.com/nokia/radish-bdd-extensions/blob/master/pkg_radish_ext/radish_ext/tools/main_radish_ext.py
with additional option --generate-steps-file generated_steps.py generate steps py file.

Best Regards,
Darek

[github-actions]

This Issue is stale because it has been open 180 days with no activity. Remove stale label or comment or this will be closed in 7 days