glob step and special tag merge

• pypyr.steps.glob lets you get a list of paths from a glob, and combine different globs into a combined output result list. Issue #145
• with much thanks to @Reskov 🎉 , resolves issue #143 - merge step now supports special tag directives

Retry Decorator and until-style While Loops

• new Retry decorator allows steps to retry automatically when step encounters an error. ref #130
• fix run_step AttributeError might have caught or hidden AttributeErrors in the step code itself, rather than just when the step module didn't contain a run_step function. ref #129
• while loop checks stop condition only at end of each iteration. previously if a stop condition evaluated True even before the loop started the loop wouldn't run at all. now the loop will run once. If you don't want the step to run, use the skip: True decorator instead.
• it's now easier to write infinite While loops with an explicit stop: False condition

pypyr.steps.pype inputs now support string interpolation

• add string interpolation to pypyr.steps.pype input.

pypyr.steps.now and pypyr.steps.nowutc

• get the current datetime stamp into the pypyr context as a formattable string, where you can specify your own datetime formats using the standard python date time replacement tokens like %Y-%m-%d for YYYY-MM-DD.

change the current working directory for cmd and shell

use the new cwd input on pypyr.steps.cmd and pypyr.steps.shell to set the current working directory for the program/shell that you want to execute. Supports relative paths.

pypyr.steps.pathcheck & deprecation of old style fetchjson/fetchyaml args

• The new pypyr.steps.pathcheck step allows you see if a path exists on the filesystem. It supports literal paths and glob expansions. It writes handy values into pathCheckOutwith bool for existence and count of files found for the given path. #114
• deprecate old style multi context inputs for fetchjson and fetchyaml, #118. this is not just arbitrary: reason is when step is used multiple times in same pipeline, it becomes easy to have left-over values from previous step run left in context that then cause surprising behaviour with the downstream step. the old style is still supported by virtue of pypyr creating the new style inputs under the hood if it finds the old style arguments - but you can expect this to go away by the next major release.
• add info output logging to envget

pypyr.steps.getenv

• add new step pypyr.steps.getenv Whereas pypyr.steps.env raises an error if you are getting an environment variable that doesn't exist, the new envget allows you to specify a default value to use instead. ref #111
• some documentation updates to demonstrate how to use py strings for ternary assignments
• built-in pypes use comment rather than description where output is not meant for operator consumption ref #109
• alias the --loglevel switch with --log, so you can now do pypyr mypipe --log 10 think of the savings just on keyboard wear and tear!

Dynamic pype loading and KeyNotInContextError == KeyError

This release is all @Reskov! 🎉 Much thanks for your excellent contributions to some tricky bits of the pypyr core!

• Context KeyNotInContextError now also derives from KeyError. This makes it easier to catch missing key exceptions in scenarios where Context keys contain other dicts.
• Allow dynamic loading of pipeline loaders. This allows core extensibility to load pipelines differently and from other places than the default file loader. Plugins can now do things like myplugin.pypeloaders.getpypefromsomewhere closes #106

Write step description to logger INFO on step execution

• write step description to INFO output on step execution, if it exists. this should help with debugging.
• minor bug fix: legacy support for envs, recreate env key on legacy key found

globs, save cmd out, backwards compatible context change for multi-key steps

• ATTENTION, FUTURE WARNING: all built-in steps that used to take multiple input keys now instead take a dict input. This makes life significantly easier if you re-run the step in the same pipeline in that you don't have to clear down context before the subsequent step runs anymore.
  • The old-style is now deprecated - for the moment pypyr will make the old-style work seamlessly for you by creating the new keys under the hood for you and raising a warning without stopping, but be aware legacy support will be removed on the next major release.
  • Affected steps are:
    • assert
    • env
    • fileformat
    • fileformatjson
    • fileformatyaml
    • filereplace
    • tar
• enable globbing on file format and file replace style steps. ref #58
• pypyr.steps.debug for easy pretty print debugging to the console ref #95
• contextsetf allows formatting on key now too, not just value, ref #96
• pypyr.parser.dict to parse input key value pairs into a specified context key. ref #97
• pypyr.steps.shell/safeshell can save return code, stdout and stderr. ref #71
• rename pypyr.steps.safeshell to pypyr.steps.cmd
  • the old name (safeshell) will keep on working as an alias for the new name.
  • the functionality is exactly the same, so this is not a breaking change.
  • hopefully "cmd" is less confusing than "safeshell" as to what it does, especially with contrast to pypyr.steps.shell
• make built-in context parsers input arguments optional to allow pipelines with context_parsers to be executed with or without the 2nd argument from the cmd line. ref #69