Skip to content

PETSc XZHermiteSpline #2917

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: next
Choose a base branch
from
Open

PETSc XZHermiteSpline #2917

wants to merge 14 commits into from

Conversation

ZedThree
Copy link
Member

Continues #2858 pulling out a separate class for PetscXZHermiteSpline that was started in #2651

This does have a working implementation, but I've tried to switch it the existing PetscMatrix wrapper, and unfortunately I've gone wrong somewhere -- probably at the boundaries.

Double unfortunately, I really don't have much time to spend digging into this. @dschwoerer Maybe you want to take a look?

bendudson and others added 10 commits February 3, 2024 11:14
Reduce duplication by moving some overloads to the base class.
Optimised build was omitting XZInterpolation factory registrations.
* next: (194 commits)
  invert_laplace: Fix outer boundary metrics
  Apply clang-format changes
  Apply clang-format changes
  Apply suggestions from code review
  Fix missed adios
  Prefer ADIOS2 over ADIOS in more cases
  Update tests/unit/test_extras.hxx
  Apply clang-format changes
  Silence warning about assignment in `if` statement
  Fix some short identifiers
  Make a couple of implicit casts explicit
  CI: Suppress some warnings about short identifiers
  Move some static functions to anonymous namespace
  CI: Remove deprecated clang-tidy option
  Include header for `BoutReal`; remove unused header
  Apply clang-format
  Add options to set Butcher tables by name
  Simplify SUNDIALS cmake
  Switch to [[maybe_unused]]
  Apply suggestions from code review
  ...
Currently gives pretty bad results, not sure why
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@boutproject boutproject deleted a comment from github-actions bot May 30, 2024
@ZedThree ZedThree marked this pull request as ready for review May 30, 2024 16:57
@ZedThree
Copy link
Member Author

Ok, I have managed to convert this to use the PetscMatrix interface. I need to spend a lot of time faffing about with GlobalIndexer and realised the source of my pain: it can only convert indices that are both local to and owned by the current process. For the interpolation here, we want to deal with indices that are on other processes. And because PetscMatrix wants to convert local indices to global ones, we need to first convert the global index of the (x, z) offsets into local ones, and then convert those to global indices.

So I've implemented a new GlobalIndexer subclass that will probably only work for very simple parallel decompositions, but can convert e.g. negative local indices into global ones. That gets an index converter that is actually pretty straightforward and can be used with PetscMatrix, which then allows a pretty clean implementation in our new interpolation!

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

Although `getGlobal` is only valid for local indices, unfortunately
`isLocal` calls `getGlobal`, so we can't actually check!
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@ZedThree ZedThree changed the title WIP: PETSc XZHermiteSpline PETSc XZHermiteSpline May 31, 2024
@ZedThree ZedThree requested a review from dschwoerer October 23, 2024 16:25
Copy link
Contributor

@dschwoerer dschwoerer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few things are indeed much nicer then with the abstraction.

I am not sure whether z-splitting will be tricky. Right now it works because only x is split, and x is the slowest index.

One advantage with the alternative implementation is, that the monotonic version is implementation for all hermite spline interpolation, although we probably want to keep at most the non-petsc version, although I am not sure of that, we might just say: Get PETSc if you want to do FCI ...

Have you by chance did a performance measurement? This implementation should copy slightly less data ...

Comment on lines +257 to +258
// Actually, I'm not 100% sure what this does, but this appears to the minimum
// necessary to get things to work
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not to sure either.
I have to say I am worried, if no-one understands how this works ...

Why do we not need .xp(2).zp(2) and similar? Does that include the corners, or just the boundaries?

Comment on lines +333 to +353
using RegisterUnavailableXZInterpolation =
XZInterpolationFactory::RegisterUnavailableInFactory;

namespace {
const inline RegisterXZInterpolation<XZHermiteSpline> registerinterphermitespline{
"hermitespline"};
const inline RegisterXZInterpolation<XZMonotonicHermiteSpline>
registerinterpmonotonichermitespline{"monotonichermitespline"};
const inline RegisterXZInterpolation<XZLagrange4pt> registerinterplagrange4pt{
"lagrange4pt"};
const inline RegisterXZInterpolation<XZBilinear> registerinterpbilinear{"bilinear"};

#if BOUT_HAS_PETSC
const inline RegisterXZInterpolation<PetscXZHermiteSpline> registerpetschermitespline{
"petschermitespline"};
#else
const inline RegisterUnavailableXZInterpolation register_unavailable_petsc_hermite_spline{
"petschermitespline", "BOUT++ was not configured with PETSc"};
#endif
} // namespace

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should they really be in the header?

const int xend = (localmesh->xend - localmesh->xstart + 1) * localmesh->getNXPE()
+ localmesh->xstart - 1;

// TODO: work out why using `getRegion(region)` directly causes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this still an issue?

Comment on lines +268 to +280
// Basis functions for cubic Hermite spline interpolation
// see http://en.wikipedia.org/wiki/Cubic_Hermite_spline
// The h00 and h01 basis functions are applied to the function itself
// and the h10 and h11 basis functions are applied to its derivative
// along the interpolation direction.
Field3D h00_x;
Field3D h01_x;
Field3D h10_x;
Field3D h11_x;
Field3D h00_z;
Field3D h01_z;
Field3D h10_z;
Field3D h11_z;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They should only be used in the constructor, so no need to keep them around

// Need to convert from global indices to local
const auto i_c = Field3D::ind_type(
(((localmesh->getLocalXIndex(i_corn) * ny) + (y + y_offset)) * nz
+ localmesh->getLocalZIndex(k_corner(x, y, z))),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this seems like it would work with Z-splitting, but it does not!

This is inherently incompatible with z-splitting. It only works if ny and nz are the same as the local ...

const PetscVector<Field3D> f_result = weights * f_petsc;
// TODO: we should only consider the passed-in region
// const auto region2 = y_offset == 0 ? region : fmt::format("RGN_YPAR_{:+d}", y_offset);
return f_result.toField();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not set the region - is this set in fci.cxx/fci.hxx?
Otherwise this will not work with high check level and 3d metrics, afaik.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants