templating integrator functions on NumSpec, actual_rhs

[BenWibking]

There is somewhat of a design "impedance mismatch" between Quokka and Microphysics.

I think this can be solved with two straightforward API changes:

• making actual_rhs a functor object, and
• templating the integrator functions on:
  • the number of species/equations NumSpec
  • the type of the actual_rhs functor object

Then it should be possible to call any integrator (either BackwardEuler or VODE) with something like:
integrate<NumSpec, F>(F actual_rhs, ...) from the application code, or from the existing burner. Note that since the type of F is known at compile time, the compiler can inline the function call to actual_rhs() and there is no performance loss (it is not equivalent to a function pointer).

The functor could look like this:

struct QuokkaActualRhsFunctor {
        constexpr int num_groups = 1;
        amrex::Real my_runtime_parameter = 1.5;
	AMREX_GPU_DEVICE void operator()(amrex::GpuArray<num_groups> &rhs) const
	{
		// compute rhs here
		rhs(0) = M_PI * my_runtime_parameter;
	}
};

Calling the integrator could look like this:

QuokkaActualRhsFunctor my_actual_rhs;
my_actual_rhs.my_runtime_parameter = 42.0;
integrate<NumGroups>(my_actual_rhs, dt);

where the template parameter F is deduced by the compiler from the type of my_actual_rhs.

For existing Microphysics networks, I think this could be accomplished with code changes only trivial syntactic changes to the generic burner code (although potentially changing a significant number of lines of code). But it would be good to hear from other Microphysics developers on this point.

Background information

• "Function Objects in the C++ Standard Library"
  • "A function object, or functor, is any type that implements operator(). This operator is referred to as the call operator or sometimes the application operator. The C++ Standard Library uses function objects primarily as sorting criteria for containers and in algorithms."
  • "Function objects provide two main advantages over a straight function call. The first is that a function object can contain state. The second is that a function object is a type and therefore can be used as a template parameter."
• "What is a functor?"
  • "So what is the purpose of using a functor? It helps us implement a common software engineering design pattern used in coding -- the strategy pattern. The strategy pattern is the idea of passing a function into a function -- i.e. a pluggable algorithm."

cc @chongchonghe @markkrumholz @psharda @zingale

[BenWibking]

Since everything is now templated on BurnT, we should be able to just add an operator() to the state object and call it inside rhs:

Microphysics/integration/integrator_rhs_strang.H

Line 25 in 533f712

void rhs (const amrex::Real time, BurnT& state, T& int_state, RArray1D& ydot, [[maybe_unused]] const bool in_jacobian=false)

[BenWibking]

For reference, the Kokkos ODE solver uses exactly this kind of API design: https://github.com/kokkos/kokkos-kernels/blob/develop/ode/unit_test/Test_ODE_BDF.hpp