So far, I convert the test to test_func format, but only test_sampling is working. test_sampling_with_seasonality would not be able to actually sample and would throw pymc3.exceptions.SamplingError: Bad initial energy. Additionally, since we changed the simulate_poiszero_hmm to the SwitchingProcess, additional adjustment need to be made. Also, I will still need to reformat so that it would fit format requirement.

Don't forget to resolve the review comments when you commit/rebase relevant changes!

To fix the ValueErrors, try uncommenting this line and changing the string value to "warn". That should create the missing mu[0].tag.test_value.

With that ValueError out of the way, I can see the SamplingError. This occurs within the HMC sampler when the sampled point value is

{'p_0_stickbreaking__': array([0., 0.]),
 'p_1_stickbreaking__': array([0., 0.]),
 'p_2_stickbreaking__': array([0., 0.]),
 'S_t': array([1, 2, 2, 2, 2, 0, 0, 2, 2, 2, 0, 2, 2, 2, 2, 2, 0, 0, 2, 2, 1, 1,
        1, 1, 1, 1, 2, 1, 2, 2, 1, 1, 0, 2, 2, 2, 2, 2, 2, 2, 2, 2, 0, 2,
        2, 0, 2, 2, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 2, 2, 2, 1, 1, 1,
        1, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 2,
        2, 0, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 0, 0, 2, 2, 2, 2, 2,
        2, 2, 2, 2, 2, 2, 2, 0, 0, 0, 2, 2, 2, 2, 2, 2, 0, 0, 2, 2, 2, 2,
        2, 2, 2, 2, 0, 2, 2, 0, 2, 2, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 0, 2, 2, 2, 2, 2, 2, 2, 2, 2, 0, 0,
        0, 0, 0, 2, 0, 2, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 2, 2, 2, 2, 2, 0,
        0, 1]),
 'mu_1_log__': array(8.00636757),
 'mu_2_log__': array(nan),
 'beta_s_log__': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}

Of course, the mu_2_log__ value (i.e. nan) is a problem. Inspecting mu_2_rv.tag.test_value in the parent frames show that the nan value was present from the start, and it's likely cause is the undefined pm.Gamma prior with negative E_2_mu and Var_2_mu parameters (i.e. -2000 and -1800, respectively).

In summary, it looks like there might be a sign error such that (mu_2 - mu_1) at line 31 should be reversed or the absolute value should be used.

Adjusted according to recommendations from @brandonwillard, solved both ValueError and SamplingError. However, with current fixed effect setting for seasonality, posterior prediction is a bit far away from simulation. Thus the failure in Assertion

Sampling for seasonality with fixed effect would generate positive Poisson prediction with MAPE around 30%. Percentage of true value falls into predicted 95% CI is also about 30%. Also moved metric checking function into utility.

changed up some formatting issue and added in descriptions for metrics checking function. A not working version of ZeroTruncatedPoission. It is not working at this point and it would throw error with bad initial energy.

Just adjust logp as requested. and both seasonality and seasonality_ztp is able to pass individual test. but not when call make test. It seems like i was failing at a function perform(self, node, inputs, output_storage): and the only place I can find it across the project is testing-report.html

I switched the metrics from posterior estimation to measuring the distance from the real parameters of the distributions.

In addition I added in [ Conway-Maxwell-Poission distribution]
(https://gist.github.com/dadaromeo/33e581d9e3bcbad83531b4a91a87509f). However, this method would fail on both states estimation and the parameter estimation.

Additionally, we are able to reduce the time to run the script by removing sampling posterior only to an extent. For tests with seasonality we would still need reasonable amount of time just to train the model.

In addition I added in [ Conway-Maxwell-Poission distribution]
(https://gist.github.com/dadaromeo/33e581d9e3bcbad83531b4a91a87509f). However, this method would fail on both states estimation and the parameter estimation.

This should be in a separate PR.

Looks like we'll have to resubmit this to the main branch instead.