Summary
Castro::add_sdc_source_to_states() applies SDC reaction source terms to face states on nodal boxes and, for the left state, explicitly reads from the adjacent cell (i-1, j-1, or k-1 depending on direction). This is an indexing pattern that relies on ghost values of sdc_src being valid at every grid/tile boundary.
In construct_ctu_hydro_source(), this neighbor-read path is used, but there is no local FillBoundary/FillPatch immediately before these accesses. The correctness therefore depends on upstream code having already filled and synchronized Simplified_SDC_React_Type ghost zones.
Evidence
In Source/hydro/Castro_ctu.cpp:
if (idir == 0) {
qleft(...,QPRES) += 0.5 * dt * sdc_src(i-1,j,k,QPRES);
qleft(...,QREINT) += 0.5 * dt * sdc_src(i-1,j,k,QREINT);
} else if (idir == 1) {
qleft(...,QPRES) += 0.5 * dt * sdc_src(i,j-1,k,QPRES);
...
} else {
qleft(...,QPRES) += 0.5 * dt * sdc_src(i,j,k-1,QPRES);
...
}
In Source/hydro/Castro_ctu_hydro.cpp, this function is called on nodal face boxes (xbx/ybx/zbx) with sdc_src_arr = SDC_react_source.array(mfi), but there is no immediate local ghost fill of SDC_react_source in this routine.
Simplified_SDC_React_Type is defined with only one ghost cell in setup (addDescriptor(..., 1, NQ, ...)), so correctness is especially sensitive to whether that one layer is current and valid before hydro uses it.
Why this is risky
This can create boundary/grid-layout-dependent behavior:
- On interior faces near a Fab boundary,
i-1/j-1/k-1 may land in ghost cells.
- If ghosts are stale/not filled for the current time, advection states get inconsistent source contributions.
- Problems may appear only with certain decomposition/tiling layouts, making debugging difficult.
Suggested fix
Before calling add_sdc_source_to_states(), enforce a local ghost synchronization for SDC_react_source at the current time (e.g., FillBoundary/FillPatch as appropriate), or explicitly document/invariant-check that this has already been done.
Optionally, add debug assertions (in non-optimized builds) that the required neighbor region is contained in the valid+ghost box for sdc_src.
Summary
Castro::add_sdc_source_to_states()applies SDC reaction source terms to face states on nodal boxes and, for the left state, explicitly reads from the adjacent cell (i-1,j-1, ork-1depending on direction). This is an indexing pattern that relies on ghost values ofsdc_srcbeing valid at every grid/tile boundary.In
construct_ctu_hydro_source(), this neighbor-read path is used, but there is no localFillBoundary/FillPatchimmediately before these accesses. The correctness therefore depends on upstream code having already filled and synchronizedSimplified_SDC_React_Typeghost zones.Evidence
In
Source/hydro/Castro_ctu.cpp:In
Source/hydro/Castro_ctu_hydro.cpp, this function is called on nodal face boxes (xbx/ybx/zbx) withsdc_src_arr = SDC_react_source.array(mfi), but there is no immediate local ghost fill ofSDC_react_sourcein this routine.Simplified_SDC_React_Typeis defined with only one ghost cell in setup (addDescriptor(..., 1, NQ, ...)), so correctness is especially sensitive to whether that one layer is current and valid before hydro uses it.Why this is risky
This can create boundary/grid-layout-dependent behavior:
i-1/j-1/k-1may land in ghost cells.Suggested fix
Before calling
add_sdc_source_to_states(), enforce a local ghost synchronization forSDC_react_sourceat the current time (e.g.,FillBoundary/FillPatchas appropriate), or explicitly document/invariant-check that this has already been done.Optionally, add debug assertions (in non-optimized builds) that the required neighbor region is contained in the valid+ghost box for
sdc_src.