ENH: Local sensitivity checks for PyMC posterior predictions . #659
Description
This is a small enhancement idea rather than a concrete feature request.
pymc models are often used for interpretability and posterior analysis, but there is currently no simple way to assess how sensitive posterior predictions or model outputs are to small changes in the observed data. in practice, users sometimes want to know whether a result changes a lot if a single observation is removed, for example the most recent point in a rolling or time series setting.
given a fitted pymc model with posterior predictive function g(·):
full fit
X -> g(X) using the full dataset
leave one out fit
X_-1 -> g(X_-1) using the same data with one observation removed, for example the last point
a simple stability check compares g(X) and g(X_-1) by looking at how much posterior predictions or derived quantities change. if the change is small, the result looks locally stable. if the change is large, the result may be sensitive to that observation. this is related to leave one out or case deletion ideas, but it is not meant as a formal uncertainty or variance estimate.
one possible lightweight api in pymc-extras could look something like:
pmx.sensitivity(
model,
idata,
X,
index=-1,
target="posterior_predictive",
norm="l2",
eps=1e-8
)
conceptually, the quantity being computed could be something like:
sensitivity = || g(X) - g(X_-1) || / (|| g(X) || + eps)
where eps is a small stabilizer and the norm is chosen to match the output (scalar or vector, predictions or summaries).
This is intended as a quick practical stability check rather than a statistically principled diagnostic. if there is interest and this seems reasonable for pymc-extras i would be happy to explore an initial implementation and raise a pr to see how it fits..