Gibbs test | Fix dynamic model test in Gibbs sampler suite #2579

AoifeHughes · 2025-06-04T09:23:57Z

The existing dynamic model test was problematic:

Used a complex Dirichlet Process mixture model that was slow (10,000 samples)
Only performed regression testing against "old" values with a TODO indicating it wasn't ideal
Failed to properly test the core dynamic model functionality that Gibbs sampling should support

Here I've changed this with:

a new dynamic_bernoulli_normal model that:

Has varying dimensions: A Bernoulli variable b controls whether the model has 1 or 2 parameters
Has analytical ground truth: Marginal likelihoods and posterior for b can be computed exactly
Is fast to sample: Only 1,000 samples vs 10,000, runs in ~7-12 seconds
Tests core functionality: Validates that Gibbs sampling correctly handles dimension changes without errors

…ce sampling validation

AoifeHughes · 2025-06-04T09:24:41Z

Leave this as a draft for a teeny bit, I need to properly test and fully understand how julia tests work so I'm not breaking what I think I understand - no need for any reviews yet.

AoifeHughes · 2025-06-04T09:28:08Z

This should address: #2402 when completed.

mhauru · 2025-06-04T10:00:39Z

As discussed on Slack, feel free to change the base branch of this to main. Since it's not a user-interface breaking change no need to make a breaking release for it and make it go through the breaking branch.

AoifeHughes · 2025-06-04T11:57:09Z

Results from my local testing:

Test Summary:                                          | Pass  Total      Time
Turing                                                 | 2613   2613  19m50.9s
  Aqua                                                 |           0      0.1s
  AD                                                   |           0      0.0s
  essential                                            |           0      0.0s
  samplers (without AD)                                |           0      0.0s
  inference with samplers                              | 2613   2613  19m50.8s
    GibbsContext                                       |  868    868     27.7s
      type stability                                   |  868    868     27.7s
    Invalid Gibbs constructor                          |    8      8      0.1s
    Sampler call order                                 |    1      1     53.9s
    Equivalence of RepeatSampler and repeating Sampler |    1      1      4.5s
    Gibbs warmup                                       |    8      8      2.3s
    Testing gibbs.jl                                   | 1727   1727  18m19.3s
      Gibbs constructors                               |   13     13     19.7s
      Gibbs inference                                  |   22     22   6m06.7s
      transitions                                      |           0      2.1s
      dynamic model with analytical posterior          |    5      5      8.4s
      dynamic model with ESS                           |    6      6     33.4s
      dynamic model with dot tilde                     |           0     11.3s
      Demo model                                       | 1564   1564   8m32.2s
      multiple varnames                                |   20     20      1.5s
      non-identity varnames                            |    2      2      9.2s
      submodels                                        |    2      2     11.3s
      CSMC + ESS                                       |   42     42     38.9s
      CSMC + ESS (usage of implicit varname)           |   42     42     51.8s
      externalsampler                                  |    8      8     28.2s
      linking changes dimension                        |    1      1      4.5s
  variational algorithms                               |           0      0.0s
  mode estimation                                      |           0      0.0s
  variational optimisers                               |           0      0.0s
  stdlib                                               |           0      0.0s
  utilities                                            |           0      0.0s
───────────────────────────────────────────────────────────
                                Time          Allocations
                          ───────────────   ───────────────
     Total measured:           1191s             534GiB

Section           ncalls     time    %tot     alloc    %tot
───────────────────────────────────────────────────────────
inference              1    1191s  100.0%    534GiB  100.0%
  mcmc/gibbs.jl        1    1191s  100.0%    534GiB  100.0%
───────────────────────────────────────────────────────────     Testing Turing tests passed

github-actions · 2025-06-05T10:06:06Z

test/mcmc/gibbs.jl

+        # When b=1: two parameters θ₁, θ₂ where we observe their sum
+        @model function dynamic_bernoulli_normal(y_obs=2.0)
+            b ~ Bernoulli(0.3)
+


[JuliaFormatter v1.0.62] _{reported by reviewdog 🐶}

Suggested change

github-actions · 2025-06-05T10:06:06Z

test/mcmc/gibbs.jl

+            b ~ Bernoulli(0.3)
+
+            if b == 0
+                θ = Vector{Float64}(undef, 1) 


[JuliaFormatter v1.0.62] _{reported by reviewdog 🐶}

Suggested change

θ = Vector{Float64}(undef, 1)

θ = Vector{Float64}(undef, 1)

github-actions · 2025-06-05T10:06:06Z

test/mcmc/gibbs.jl

+            StableRNG(42), model, Gibbs(:b => MH(), :θ => HMC(0.1, 10)), 1000;
+            discard_initial=500


[JuliaFormatter v1.0.62] _{reported by reviewdog 🐶}

Suggested change

StableRNG(42), model, Gibbs(:b => MH(), :θ => HMC(0.1, 10)), 1000;

discard_initial=500

StableRNG(42),

model,

Gibbs(:b => MH(), :θ => HMC(0.1, 10)),

1000;

discard_initial=500,

github-actions · 2025-06-05T10:06:06Z

test/mcmc/gibbs.jl

-        # Issue ref: https://github.com/TuringLang/Turing.jl/issues/2402
-        @test isapprox(mean(num_ms), 8.6087; atol=0.8)
-        @test isapprox(std(num_ms), 1.8865; atol=0.03)
+


[JuliaFormatter v1.0.62] _{reported by reviewdog 🐶}

Suggested change

github-actions · 2025-06-05T10:06:06Z

test/mcmc/gibbs.jl

+
+        # Test that sampling completes without error
+        @test size(chn, 1) == 1000
+


[JuliaFormatter v1.0.62] _{reported by reviewdog 🐶}

Suggested change

github-actions · 2025-06-05T10:06:06Z

test/mcmc/gibbs.jl

+        b_samples = chn[:b]
+        unique_b_values = unique(skipmissing(b_samples))
+        @test length(unique_b_values) >= 1  # At least one value should be sampled
+


[JuliaFormatter v1.0.62] _{reported by reviewdog 🐶}

Suggested change

github-actions · 2025-06-05T10:06:07Z

test/mcmc/gibbs.jl

+            @test all(isfinite, theta1_samples)  # All samples should be finite
+            @test std(theta1_samples) > 0.1     # Should show some variation
+        end
+


[JuliaFormatter v1.0.62] _{reported by reviewdog 🐶}

Suggested change

github-actions · 2025-06-05T10:06:07Z

test/mcmc/gibbs.jl

+            # θ[2] should have some missing values (when b=0) and some non-missing (when b=1)
+            n_missing_theta2 = sum(ismissing.(theta2_samples))
+            n_present_theta2 = sum(.!ismissing.(theta2_samples))
+


[JuliaFormatter v1.0.62] _{reported by reviewdog 🐶}

Suggested change

github-actions · 2025-06-05T10:06:15Z

Turing.jl documentation for PR #2579 is available at:
https://TuringLang.github.io/Turing.jl/previews/PR2579/

codecov · 2025-06-05T10:11:23Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.86%. Comparing base (58232e4) to head (6696fc4).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2579   +/-   ##
=======================================
  Coverage   85.86%   85.86%           
=======================================
  Files          24       24           
  Lines        1429     1429           
=======================================
  Hits         1227     1227           
  Misses        202      202

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

coveralls · 2025-06-05T10:48:23Z

Pull Request Test Coverage Report for Build 15464256226

Details

0 of 0 changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 85.924%

Totals
Change from base Build 15463374492:	0.0%
Covered Lines:	1227
Relevant Lines:	1428

💛 - Coveralls

Copilot

Pull Request Overview

This PR improves the dynamic model test in the Gibbs sampler suite by replacing slow, less informative tests with a fast, analytically tractable dynamic_bernoulli_normal model and updating model signature syntax to the new standards.

Updates model function parameter syntax across several test files
Refactors multi-line expressions for clarity and consistency in optimisation and MCMC files
Introduces a new dynamic model test that validates dimension changes and ensures sampler robustness

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
test/skipped/vec_assume_mat.jl	Updated model function signature to new parameter syntax
test/optimisation/Optimisation.jl	Reformatted multi-line expression for setting child context
test/mcmc/hmc.jl	Adjusted formatting and signature updates in several models
test/mcmc/gibbs.jl	Introduced dynamic_bernoulli_normal model and updated test logic
test/mcmc/Inference.jl	Updated model function signatures to new syntax
src/mcmc/external_sampler.jl	Adjusted function parameter handling in sampler construction

Refactor dynamic model test to include analytical posterior and enhan…

e4df2ae

…ce sampling validation

AoifeHughes requested a review from Copilot June 4, 2025 09:23

AoifeHughes self-assigned this Jun 4, 2025

This comment was marked as outdated.

Sign in to view

AoifeHughes added the enhancement: tests label Jun 4, 2025

AoifeHughes changed the base branch from breaking to main June 4, 2025 10:01

AoifeHughes marked this pull request as ready for review June 5, 2025 10:02

Merge branch 'main' into gibbs-test

6696fc4

github-actions bot reviewed Jun 5, 2025

View reviewed changes

Appeasing the formatter gods

9389e59

AoifeHughes requested a review from Copilot June 9, 2025 07:50

Copilot AI reviewed Jun 9, 2025

View reviewed changes

	θ = Vector{Float64}(undef, 1)
	θ = Vector{Float64}(undef, 1)

		StableRNG(42), model, Gibbs(:b => MH(), :θ => HMC(0.1, 10)), 1000;
		discard_initial=500


		# Test that sampling completes without error
		@test size(chn, 1) == 1000

Gibbs test | Fix dynamic model test in Gibbs sampler suite #2579

Are you sure you want to change the base?

Gibbs test | Fix dynamic model test in Gibbs sampler suite #2579

Uh oh!

Conversation

AoifeHughes commented Jun 4, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

AoifeHughes commented Jun 4, 2025

Uh oh!

AoifeHughes commented Jun 4, 2025

Uh oh!

mhauru commented Jun 4, 2025

Uh oh!

AoifeHughes commented Jun 4, 2025

Uh oh!

github-actions bot Jun 5, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Jun 5, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Jun 5, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Jun 5, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Jun 5, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Jun 5, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Jun 5, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Jun 5, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jun 5, 2025

Uh oh!

codecov bot commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

coveralls commented Jun 5, 2025

Pull Request Test Coverage Report for Build 15464256226

Details

💛 - Coveralls

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

codecov bot commented Jun 5, 2025 •

edited

Loading