GMRES for Poisson Equation#6367
Conversation
The non-EB case gradients are handled in EffectivePotentialES.cpp::ComputeSpaceChargeField() The MLMG + EB case gradients are haneled in post_phi_calculation.value()
…s" in inputs file
… gmres_for_warpx
|
I've tried this out and it works quite nicely. It's fully functional for my needs, in 2D, with Dirichlet and Neumann boundaries, and embedded boundaries. Comparing it to multigrid, it is consistently faster. On a Mac with the small grid (80x200), the multigrid takes ~0.047 s per solve, the gmres 0.028 s per solve. (Though by comparison, the multigrid solver in Warp takes 0.0024 s per solve.) On AMD GPU for a larger grid (360x800), the multigrid takes 0.058 s and gmres 0.048 s per solve, not a big difference. On parallel CPU (256 processors) with the larger grid, multigrid taks 0.032 s, and gmres 0.029 s, also not a big difference. (For reference, the multigrid solver in Warp takes 0.01 s per solve.) |
|
What is (if any) the precondition matrix solver? |
It says that it does 1 MLMG cycle. |
RevathiJambunathan
requested review from
JustinRayAngus,
RemiLehe,
RevathiJambunathan and
dpgrote
|
@ajnonaka can you merge |
| #if defined(WARPX_DIM_1D_Z) | ||
| amrex::Array<amrex::MultiFab*, 1>{ | ||
| warpx.m_fields.get(warpx::fields::FieldType::Efield_fp, Direction{2}, lev) | ||
| } | ||
| #elif defined(WARPX_DIM_XZ) || defined(WARPX_DIM_RZ) | ||
| amrex::Array<amrex::MultiFab*, 2>{ | ||
| warpx.m_fields.get(warpx::fields::FieldType::Efield_fp, Direction{0}, lev), | ||
| warpx.m_fields.get(warpx::fields::FieldType::Efield_fp, Direction{2}, lev) | ||
| } | ||
| #elif defined(WARPX_DIM_3D) | ||
| amrex::Array<amrex::MultiFab *, 3>{ | ||
| warpx.m_fields.get(warpx::fields::FieldType::Efield_fp, Direction{0}, lev), | ||
| warpx.m_fields.get(warpx::fields::FieldType::Efield_fp, Direction{1}, lev), | ||
| warpx.m_fields.get(warpx::fields::FieldType::Efield_fp, Direction{2}, lev) | ||
| } | ||
| #endif |
There was a problem hiding this comment.
@dpgrote Is electrostatic mode in WarpX not setup for RCYLINDER and RSPHERE geometries?
There was a problem hiding this comment.
@JustinRayAngus Those geometries are not supported in electrostatic. Also, this block of code is for EB which is also not supported.
| } else { | ||
| amrex::GMRESMLMG gmsolve(mlmg); | ||
| gmsolve.solve( *phi[lev], *rho[lev], relative_tolerance, absolute_tolerance ); | ||
| linop->postSolve({phi[lev]}); |
There was a problem hiding this comment.
What is the need for linop->postSolve() here ? Why is it not needed when using mlmg?
There was a problem hiding this comment.
MLMG calls postSolve within, which sets the value of phi in covered cells to the Dirichlet value. GMRES initially left phi in covered cells at zero; I added this call so the plotfiles would match up in the covered cells.
| amrex::MLMG mlmg(*linop); // actual solver defined here | ||
| mlmg.setVerbose(verbosity); | ||
| mlmg.setMaxIter(max_iters); | ||
| mlmg.setConvergenceNormType(amrex::MLMGNormType::greater); | ||
| mlmg.setNoGpuSync(true); |
There was a problem hiding this comment.
Right now it doesn't look like this PR is setup to change additional parameters used by GMRES. I think it would be a good ideal to make the linear solver a member variable defined once on initialization, then you could have control over more options. For example, you may want to manually set the restart length:
m_linear_solver->setRestartLength( m_linsol_restart_length );
See Source/NonlinearSolvers/NewtonSolver.H for an example of what I'm referring to.
There was a problem hiding this comment.
Reva wants to discuss at the next dev meeting; the MLMG option has no exposure either.
| amrex::GMRESMLMG gmsolve(mlmg); | ||
| gmsolve.solve( *phi[lev], *rho[lev], relative_tolerance, absolute_tolerance ); |
There was a problem hiding this comment.
@RemiLehe commented: can we also handle max_iters here? Otherwise it is silently ignored if user-set
set with warpx.magnetostatic_solver_max_iters
5 participants
GMRES implementation of poisson solver, enabled with
warpx.poisson_solver = gmres.Instead of pure multigrid via the AMReX MLMG class, this uses an MLMG-preconditioned GMRES algorithm to solve the Poisson equation. More documentation forthcoming.