FillPatchTwoLevels does not fill fine-level ghost cells – FPinfo shows no overlap detected

[Afraid31]

Title: FillPatchTwoLevels does not fill fine-level ghost cells – TheFPinfo shows no overlap detected
Hi all,
I’m working with a two-level AMReX setup and encountered an issue when using FillPatchTwoLevels(). Specifically, the valid region of the fine level is filled correctly, but the ghost cells remain unfilled. Here's a simplified version of my domain and geometry setup:

int nx_cell = 16;
int ny_cell = 8;
dom_lo.push_back({0, 0});
dom_lo.push_back({nx_cell / 4, ny_cell / 4});
dom_hi.push_back({nx_cell - 1, ny_cell - 1});
dom_hi.push_back({nx_cell * 3 / 4 - 1, ny_cell * 3 / 4 - 1});

amrex::Vector<RealBox> real_box;
real_box.push_back(RealBox({0., 0.}, {double(nx_cell), double(ny_cell)}));
real_box.push_back(RealBox({double(nx_cell) / 4, double(ny_cell) / 4},
                           {double(nx_cell) * 3 / 4, double(ny_cell) * 3 / 4}));

Array<int, AMREX_SPACEDIM> is_periodic{0, 0};

for (int lev = 0; lev < 2; ++lev) {
    domain[lev] = Box(dom_lo[lev], dom_hi[lev]).refine(lev + 1);
    ba[lev].define(domain[lev]);
    geom[lev].define(domain[lev], real_box[lev], CoordSys::cartesian, is_periodic);
    dm[lev] = DistributionMapping(ba[lev]);
}

I define my data like this:

int Nghost[2] = {1, 4};
amrex::Vector<MultiFab> fi(2), fi_p(2);
int ncomp = 9;
for (int lev = 0; lev < 2; ++lev) {
    fi[lev].define(ba[lev], dm[lev], ncomp, Nghost[lev]);
    fi_p[lev].define(ba[lev], dm[lev], ncomp, Nghost[lev]);
}

All boundary conditions are set to int_dir:

amrex::Vector<amrex::BCRec> bcs(ncomp);
for (int n = 0; n < ncomp; ++n) {
    for (int dir = 0; dir < AMREX_SPACEDIM; ++dir) {
        bcs[n].setLo(dir, amrex::BCType::int_dir);
        bcs[n].setHi(dir, amrex::BCType::int_dir);
    }
}

I then try to fill ghost cells at level 1 from level 0 using:

MultiFab mf(ba[1], dm[1], fi_p[1].nComp(), fi_p[1].nGrow());

Vector<MultiFab*> cmf, fmf;
Vector<Real> ctime, ftime;
cmf.push_back(&fi_p[0]);
ctime.push_back(time);
fmf.push_back(&fi_p[1]);
ftime.push_back(time);

Interpolater* mapper = &cell_cons_interp;
CpuBndryFuncFab bndry_func(nullptr);
PhysBCFunct<CpuBndryFuncFab> cphysbc(geom[0], bcs, bndry_func);
PhysBCFunct<CpuBndryFuncFab> fphysbc(geom[1], bcs, bndry_func);

amrex::FillPatchTwoLevels(mf, time, cmf, ctime, fmf, ftime,
    0, 0, ncomp, geom[0], geom[1],
    cphysbc, 0, fphysbc, 0, amrex::IntVect(2),
    mapper, bcs, 0);

Although the valid region of the fine grid ‘mf’ is filled, the ghost cells remain zero or unfilled.
I dug into the source and added debug output in ‘FillPatchTwoLevels’. I noticed that:

const FabArrayBase::FPinfo& fpc = FabArrayBase::TheFPinfo(*fmf[0], mf,
                                                          nghost,
                                                          coarsener,
                                                          fgeom,
                                                          cgeom,
                                                          index_space);

returns a structure where !fpc.ba_crse_patch.empty() is false – which indicates that no coarse-fine overlap was detectedNow here’s the issue:
I’m still new to AMReX and multilevel mesh handling, so I may be misunderstanding some details. However, based on visualization in ParaView, the grids appear to overlap reasonably well. I’ve confirmed that the ghost cells on the fine level extend beyond the valid region and into areas covered by coarse-level data.

So my questions are:

1. Is there anything wrong with how I define the domains or BoxArrays that could prevent FillPatchTwoLevels() from detecting a coarse-fine patch?
2. Is it valid to construct the fine BoxArray as a refined subregion like I did here, or should it align differently with the coarse level?
   Thanks in advance for any help. I’m happy to provide the ParaView image or more debug output if needed.

[WeiqunZhang]

The way you are building Geometry is incorrect. The fine level's Geometry should have the same RealBox as the coarse level. Its Geometry's domain box should be refined version of the coarse level's domain box. In your code the fine level's domain box only covers the middle of the domain. You should think Geometry's domain box as if we have refinement everywhere.

You could build your fine level's geometry by calling Geometry refine (Geometry const& crse, int rr).

[WeiqunZhang]

Here is an example of building Geometry objects.

amrex/Src/AmrCore/AMReX_AmrMesh.cpp

Line 64 in 335f529

geom.push_back(level_0_geom);

[WeiqunZhang]

Why are you not using AmrCore or Amr classes?