Skip to content

Releases: tfjgeorge/nngeometry

Internal changes in Jacobian generator, new features, new layers

23 May 20:03
93897d9
Compare
Choose a tag to compare

Internal changes:

  • The DataLoader is not part of the generator anymore
  • Full backward hooks in jacobian generator
  • Migrates gradient computation outside of Jacobian class to JacobianFactory
  • Adds efficient grads for convolution implementation, with switch to choose between 2 implementations
  • Changes solve implementation

New features

  • PVector add_to_model method
  • Matrix-matrix products for KFAC, Dense, BlockDiag; Diag representations
  • Solve for PMatLowRank
  • Dot between 2 PVector objects
  • Pickle for PMat and PVector objects
  • Grad method for PVector objects
  • MC Fisher helper for segmentation
  • Adds get_kfe method to PMatEKFAC

New layers

  • Custom layers: Cosine, Weightnorm, Affine
  • Support for additional layers: ConvTranspose2d

Minor release: Jacobian generator now raises an Exception when using batch norm in training mode instead of silently failing

10 Feb 21:44
Compare
Choose a tag to compare

This minor releases makes sure that users are aware of the fact that we currently do not support batch norm in training mode. Prior releases would instead silently fail, without telling the user that the returned objects are likely wrong.

Layer Collections, support for GroupNorm, QuasiDiag representation

09 Oct 14:57
67043d5
Compare
Choose a tag to compare

This release revamps NNGeometry in order to have LayerCollection at its heart, which allows:

  • flexibility in parameter space definition (e.g. if you want to compute FIM or NTK for a subset of your layers)
  • flexbility in some operations, e.g. in a Fv product F.mv(v) F and v do not need to be created from the same model instance, but as long as they share the same LayerCollection it is enough

We also added support for GroupNorm layers, and included the QuasiDiag parameter space representation.

This release includes some documentation (not perfect !), and supports continuous integration tools.