Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardize logging - final places #4665

Merged

Conversation

GarethCabournDavies
Copy link
Contributor

@GarethCabournDavies GarethCabournDavies commented Mar 19, 2024

Final places where we need to standardise including the logging

This looks like more places than it is because I've added two things in order to simplify logging setup:

  • a default_level kwarg to init_logging, which means we can easily state that logging should happen, but --verbose make it a higher level
  • a to_file kwarg, which sets up logging to a file instead of to the stderr

@GarethCabournDavies
Copy link
Contributor Author

To check that all executables have the standardized logging, when we check the repo for files which don't have init_logging, I get no output, i.e. (grep -Le means list of files which don't contain the string):

$ grep -Le init_logging bin/**/pycbc_*

returns nothing

The common pycbc options added to the parser is slightly more complicated, as some of the pygrb codes use a common parser setup, so we see:

$ grep -Le common_pycbc bin/**/pycbc_*
bin/pygrb/pycbc_pygrb_efficiency
bin/pygrb/pycbc_pygrb_page_tables
bin/pygrb/pycbc_pygrb_plot_chisq_veto
bin/pygrb/pycbc_pygrb_plot_coh_ifosnr
bin/pygrb/pycbc_pygrb_plot_injs_results
bin/pygrb/pycbc_pygrb_plot_null_stats
bin/pygrb/pycbc_pygrb_plot_skygrid
bin/pygrb/pycbc_pygrb_plot_snr_timeseries
bin/pygrb/pycbc_pygrb_plot_stats_distribution

but the common options are called within pygrb_initialize_plot_parser, so we can see the same group of files which have that option (grep -le means list of files which do contain the string):

$ grep -le pygrb_initialize_plot_parser bin/**/pycbc_*
bin/pygrb/pycbc_pygrb_efficiency
bin/pygrb/pycbc_pygrb_page_tables
bin/pygrb/pycbc_pygrb_plot_chisq_veto
bin/pygrb/pycbc_pygrb_plot_coh_ifosnr
bin/pygrb/pycbc_pygrb_plot_injs_results
bin/pygrb/pycbc_pygrb_plot_null_stats
bin/pygrb/pycbc_pygrb_plot_skygrid
bin/pygrb/pycbc_pygrb_plot_snr_timeseries
bin/pygrb/pycbc_pygrb_plot_stats_distribution

@GarethCabournDavies
Copy link
Contributor Author

Codeclimate failures are unrelated to these changes

@GarethCabournDavies
Copy link
Contributor Author

GarethCabournDavies commented Jun 11, 2024

Note i realised i skipped a couple of places for the init_logging check, and saw

grep -Le init_logging bin/pycbc_*
bin/pycbc_copy_output_map
bin/pycbc_live_nagios_monitor
bin/pycbc_splitbank
bin/pycbc_split_inspinj
bin/pycbc_stageout_failed_workflow
bin/pycbc_submit_dax

of these, pycbc_copy_output_map, pycbc_stageout_failed_workflow and pycbc_submit_dax are bash scripts, the others do not have any logging

@@ -0,0 +1,3 @@
[BASIC]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This silences the codeclimate "logger is not a valid name" complaints

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other variables are the pyint defaults

@@ -86,7 +86,8 @@ def add_common_pycbc_options(parser):
'logging at the info level, but -vv or '
'--verbose --verbose provides debug logging.')

def init_logging(verbose=False,

def init_logging(verbose=False, default_level=0, to_file=None,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably the best place to start the review - the more significant change is here:

  • We add a default_level kwarg so that the code can define the default level of logging to be higher, but --verbose still adds incremental logging levels
  • We add a to_file kwarg, so that we can define logging to a file through the init_logging interface. If wanting logging to more than one file, or to both stderr and the file, we can call init_logging twice with different to_file values

@@ -57,9 +61,8 @@ def pygrb_initialize_plot_parser(description=None, version=None):
formatter_class = argparse.ArgumentDefaultsHelpFormatter
parser = argparse.ArgumentParser(description=description,
formatter_class=formatter_class)
add_common_pycbc_options(parser)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outwith this change, all the changes in pycbc directory are:

  • adding the named logger to the module
  • updating logging --> logger to use the named logger
  • some reordering/reformatting of imports

Copy link
Contributor

@ArthurTolley ArthurTolley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sure that 99% of my changes (the most I've ever done by far) are related to simple changes or wanting to understand a bit more the 'standardised' part of things. I imagine I'm just misinformed on a few pythonic things, such as my import pycbc comment so if you answer it once then feel free to ignore the rest.

bin/hwinj/pycbc_insert_frame_hwinj Outdated Show resolved Hide resolved
bin/inference/pycbc_inference_create_fits Show resolved Hide resolved
bin/plotting/pycbc_page_injtable Outdated Show resolved Hide resolved
bin/inference/pycbc_validate_test_posterior Show resolved Hide resolved
bin/pygrb/pycbc_pygrb_plot_coh_ifosnr Outdated Show resolved Hide resolved
bin/pygrb/pycbc_pygrb_pp_workflow Show resolved Hide resolved
examples/live/check_results.py Outdated Show resolved Hide resolved
pycbc/io/hdf.py Outdated Show resolved Hide resolved
Copy link
Contributor

@ArthurTolley ArthurTolley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lfg

@GarethCabournDavies GarethCabournDavies merged commit 7786c1a into gwastro:master Jul 2, 2024
32 of 33 checks passed
@GarethCabournDavies GarethCabournDavies deleted the standard_logging branch July 2, 2024 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants