Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements to yuma #223

Open
10 of 14 tasks
nikitapond opened this issue Nov 14, 2023 · 3 comments
Open
10 of 14 tasks

Improvements to yuma #223

nikitapond opened this issue Nov 14, 2023 · 3 comments

Comments

@nikitapond
Copy link
Contributor

nikitapond commented Nov 14, 2023

Yuma, introduced in #204 , allows for plotting common flavour tagging results via a yaml config, reducing requirements for boilerplate.
Some areas that could probably still be implemented

  • Finer configurable control over backgrounds in plots, to allow for tau-rejection.
  • Add yaml config plotting of jet and track variables
  • Fix bug where yuma crashes with one configured tagger
  • Default the tagger name to the config key name in the taggers config
  • Better entry point - make yuma an installed script and/or let the user import the module (in case of pip install)
  • Use the same yaml including we use in UPP to allow for contained single file configs
  • Fix warning about no reference tagger
  • Remove trailing underscores e.g. ttbar_bjets_probs_pb__.png and ttbar_bjets_roc_.png
  • Streamline yuma config by moving logic to results/tagger API
  • Make plotting generic - instead of definining a yuma plotting function for each plot type, we should put any additional logic into the results or plot classes and call plot from yuma in a consistent way
  • Clean up tagger loading. The logic here should be handled by the Results/Tagger classes, and we should have single way to load taggers into the results object, whereas at the moment we have load, add_taggers_from_file and load_taggers_from_file
  • Avoid stateful effects and instead pass things as arguments to functions. This happens sometimes when we modify the Results atlas tags, and also when we select the taggers to plot
  • Passing a directory should loop over all configs in the dir and plot them all
  • ensure that no unexpected/unused arguments are provided to the plot configuration (e.g. beyond signal and plot type)
@samvanstroud
Copy link
Contributor

With

taggers: 
  - GN2v01
  - DL1dv01
reference_tagger: GN2v01

I get a warning

WARNING:puma: No reference set for plot, using DL1dv01 (DL1dv01) as reference

Adding this to the above list

@nikitapond
Copy link
Contributor Author

nikitapond commented Feb 14, 2024

With

taggers: 
  - GN2v01
  - DL1dv01
reference_tagger: GN2v01

I get a warning

WARNING:puma: No reference set for plot, using DL1dv01 (DL1dv01) as reference

Adding this to the above list

I think this is as expected. In the default example config, we have a series of ROC plots. We have defined

  -  args:
      signal: bjets
      x_range: [0.5, 1.0]
      include_taggers: ['GN2v01']
  -  args:
      signal: bjets
      x_range: [0.5, 1.0]
      exclude_taggers: ['GN2v01']

What is happening is for the first plot, we have include_taggers: 'GN2v01'. This is fine, the code is happy, as GN2v01 is the default reference tagger. For the second plot, we have exclude_taggers: 'GN2v01', so it will show all other taggers (in this case, only DL1dv01). When it is showing this plot, it doesn't have the reference tagger, as we've excluded it, so it automatically sets DL1dv01 as the reference tagger in this plot. If you wanted to remove the warning you would add `reference: DL1dv01' to the plot config above.

I think this is the same error for the point "Fix bug where yuma crashes with one configured tagger", if you have only 1 tagger, and you exclude that tagger, then it'll have nothing to plot. I'll add in an exception here to make it clearer.

@WeiShengL
Copy link
Contributor

Hey, I noticed that when using yuma it ignores the taggers selection in the plot config and plots every tagger that is defined in taggers.yaml
I've create a PR for a quick fix here #247

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants