Skip to content

Stabilize Gaussian mixture backward passes#3452

Open
jburroni wants to merge 3 commits into
pyro-ppl:devfrom
jburroni:dev
Open

Stabilize Gaussian mixture backward passes#3452
jburroni wants to merge 3 commits into
pyro-ppl:devfrom
jburroni:dev

Conversation

@jburroni
Copy link
Copy Markdown
Contributor

Summary

This PR fixes numerical instabilities in the backward passes for Pyro's Gaussian mixture distributions:

  • MixtureOfDiagNormals

  • GaussianScaleMixture

  • MixtureOfDiagNormalsSharedCovariance

The affected backward paths can materialize very small density/product terms in probability space. In float32, these terms can underflow to zero, which can then produce nonfinite gradients during backpropagation.

Changes

  • Adds regression tests that assert finite gradients in numerically sensitive high-dimensional / small-scale cases.

  • Updates the affected backward computations to avoid unstable probability-space divisions/products where possible.

  • Format code to pass lint.

Tests

Added regression coverage for:

  • small-scale MixtureOfDiagNormals

  • small-scale GaussianScaleMixture

  • high-dimensional MixtureOfDiagNormalsSharedCovariance

Each test verifies that samples, losses, and parameter gradients remain finite.

@martinjankowiak
Copy link
Copy Markdown
Collaborator

thanks @jburroni! your diff changed a bunch of other files so you'll need to fix that if you want this reviewed and merged

@jburroni
Copy link
Copy Markdown
Contributor Author

Hi @martinjankowiak!
The commits for this PR are ebd8d57 and 8384f6f.
The third one, which modified many files, was made by running make lint.

The GH actions seem to be broken, and I don't know how to proceed. What do you suggest?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants