Special diagrams
A number of the diagrams specified by -m are diagrams that combine several regular metrics. In many cases (but not all), options such as specifying dimension (-x) and aggregation method (-agg) do not apply to these diagram. A warning will be given if an option has no effect.
The scatter plot (-m scatter) shows observations on the x-axis and corresponding forecasts on the y-axis. The figure is clearest if only one input is used.
verif ECMWF.nc -m scatter
The black lines show the distribution (in terms of percentiles) of the observations for different forecast values. These can be hidden by using -simple.
By default, all observation/forecast points are shown in the plot. Use -x to aggregate the pairs along a dimension. On this case, one point is shown for each value along the aggregation dimension. Use -agg to specify how to aggregate the points.
The quantile-quantile plot (-m qq) is similar to the scatter plot, except the observations and forecasts independently. This allows you to see if the observations and forecasts are distributed equally.
verif ECMWF.nc MEPS.nc -m qq -sp
The -sp (show perfect) flag adds the gray diagonal line showing where lines would end up if the observations and forecasts have the same distribution. In this example, the raw forecasts have fewer high precipitation values than observed.
The conditional plot (-m cond) shows the average forecast value issued for different observations and vice versa. To compute the F|O line, observations are first placed into bins (whose edges are determined automatically, or using -r). Then for each bin, the average forecast value corresponding to the observations in the bin is computed. A different aggregator than the average can be specified using -agg. The same procedure is used for the O|F line, except that forecasts are binned.
verif ECMWF.nc MEPS.nc -m cond -sp -r 0:10,12:2:20,25,30
Verif tries to use appropriate thresholds, however in some cases it might be necessary to override these with -r, as is done above.
The Taylor diagram combines information about correlation, standard error, and RMSE into one diagram:
verif ECMWF.nc MEPS.nc -m taylor
By default, one point is shown for each input, which is calculated by aggregating all data. To show points for different leadtimes use -x leadtime (any other -x can also be used). When multiple points are plotted, then the variances are normalized to 1, so that only one observation point needs to be drawn.
The change diagram (-m change) shows how the forecast error (MAE) varies as a function of how different the weather has changed since the last forecast initialization time. Some post-processing methods (like the Kalman Filter) has a lag in correcting forecasts, thereby performing poorly after weather regime changes.
The fractions skill score (-m fss) shows the forecasts ability to predict the correct frequency of events at different spatial or temporal scales. Use -r to set the event threshold, and -b to set the interval type. By default, -m fss shows FSS in space, but you can use -x leadtime to compute the score across different temporal scales.
Since Verif does not support gridded fields, the spatial FSS values produced by Verif are only an approximation to the real FSS values you would get using a full gridded field. For a given spatial scale S, Verif computes the Brier score for each location, using all observations within a radius of S. This is then averaged and a Brier Skill Score is computed for that scale. As observational networks are irregularly spaced, the FSS is sampled unevenly in space, which can affect the results.
Since Verif version 1.3, FSS can also be computed across leadtimes (use -x leadtime). Verif computes the frequency of the event across different time spans.