Merge branch 'development' into use_modulusshuffle_instead_of_yates

Author: dpgrote

Docs/source/install/dependencies.rst

@@ -6,7 +6,7 @@ List of Dependencies
 WarpX depends on the following popular third party software.
 Please see installation instructions below.
 
-- a mature `C++20 <https://en.wikipedia.org/wiki/C%2B%2B17>`__ compiler, e.g., GCC 12+, Clang 14, NVCC 12.4, MSVC 19.39 or newer
+- a mature `C++20 <https://en.wikipedia.org/wiki/C%2B%2B20>`__ compiler, e.g., GCC 12+, Clang 14, NVCC 12.4, MSVC 19.39 or newer
 - `CMake 3.24.0+ <https://cmake.org>`__
 - `Git 2.18+ <https://git-scm.com>`__
 - `AMReX <https://amrex-codes.github.io>`__: we automatically download and compile a copy of AMReX
@@ -23,8 +23,8 @@ Optional dependencies include:
 - for on-node accelerated compute *one of either*:
 
   - `OpenMP 3.1+ <https://www.openmp.org>`__: for threaded CPU execution or
-  - `CUDA Toolkit 11.7+ <https://developer.nvidia.com/cuda-downloads>`__: for Nvidia GPU support (see `matching host-compilers <https://gist.github.com/ax3l/9489132>`__) or
-  - `ROCm 5.2+ (5.5+ recommended) <https://gpuopen.com/learn/amd-lab-notes/amd-lab-notes-rocm-installation-readme/>`__: for AMD GPU support
+  - `CUDA Toolkit 12.2+ <https://developer.nvidia.com/cuda-downloads>`__: for Nvidia GPU support (see `matching host-compilers <https://gist.github.com/ax3l/9489132>`__) or
+  - `ROCm 5.5+ <https://gpuopen.com/learn/amd-lab-notes/amd-lab-notes-rocm-installation-readme/>`__: for AMD GPU support
   - `oneAPI <https://www.intel.com/content/www/us/en/developer/tools/oneapi/overview.html>`__: for Intel GPU support
 - `FFTW3 <http://www.fftw.org>`__: for spectral solver (PSATD or IGF) support when running on CPU or SYCL
 
@@ -35,10 +35,9 @@ Optional dependencies include:
 
   - see `optional I/O backends <https://github.com/openPMD/openPMD-api#dependencies>`__, i.e., ADIOS2 and/or HDF5
 - `Ascent 0.8.0+ <https://ascent.readthedocs.io>`__: for in situ 3D visualization
-- `SENSEI 4.0.0+ <https://sensei-insitu.org>`__: for in situ analysis and visualization
 - `CCache <https://ccache.dev>`__: to speed up rebuilds (For CUDA support, needs version 3.7.9+ and 4.2+ is recommended)
 - `Ninja <https://ninja-build.org>`__: for faster parallel compiles
-- `Python 3.9+ <https://www.python.org>`__
+- `Python 3.11+ <https://www.python.org>`__
 
   - `mpi4py <https://mpi4py.readthedocs.io>`__
   - `numpy <https://numpy.org>`__

Docs/source/usage/parameters.rst

@@ -3100,7 +3100,8 @@ Details about the collision models can be found in the :ref:`theory section <mul
     :default: 0
     :optional:
 
-    For pairwisecoulomb collisions, whether to correct the energy and momentum after the collisions so that they are conserved.
+    Only for ``pairwisecoulomb`` collisions, whether to correct the energy and momentum after the collisions so that they are conserved.
+    This can be set for each collision using :pp:param:`<collision_name>.correct_energy_momentum`.
     In binary collisions, if the weights of the colliding particles are not the same, the collision does not
     exactly conserve energy and momentum. When this option is on, after the collisions, small modifications are made to the
     particle momentum so that the energy and momentum are exactly conserved in each cell.
@@ -3111,25 +3112,39 @@ Details about the collision models can be found in the :ref:`theory section <mul
     :default: 0.05
     :optional:
 
-    For pairwisecoulomb collisions, when correcting the energy and momentum conservation, the energy correction is applied to pairs of particles in their center of momentum frame.
+    Only for ``pairwisecoulomb`` collisions, with :pp:param:`collisions.correct_energy_momentum` set, the energy correction is applied to pairs of particles in their center of momentum frame.
+    This can be set for each collision using :pp:param:`<collision_name>.energy_fraction`.
     This parameter is the fraction of the relative energy in the COM frame that is used in the correction.
 
 .. pp:param:: collisions.energy_fraction_max
     :type: ``float``
     :default: 0.5
     :optional:
 
-    For pairwisecoulomb collisions, when correcting the energy and momentum conservation, the energy correction is applied to pairs of particles in their center of momentum frame.
-    This parameter is the fraction of the total relative energy in the COM frame of all pairs that is used in the correction.
+    Only for ``pairwisecoulomb`` collisions, with :pp:param:`collisions.correct_energy_momentum` set, the energy correction is applied to pairs of particles in their center of momentum frame.
+    This can be set for each collision using :pp:param:`<collision_name>.energy_fraction_max`.
+    This parameter sets the maximum allowed value for :pp:param:`collisions.energy_fraction`.
+    If residual energy error remains after a pass over all particle pairs in a cell, the algorithm increases the effective correction fraction for the next pass based on an estimate of the remaining error.
+    If this computed value exceeds the limit imposed by this parameter, the correction is deemed to have failed and particle velocities in the cell are restored to their pre-collision values.
 
 .. pp:param:: collisions.beta_weight_exponent
     :type: ``float``
     :default: 1.
     :optional:
 
-    For pairwisecoulomb collisions, when correcting the energy and momentum conservation, this parameter controls the exponent used on the particle weight when distributing the momentum correction.
+    Only for ``pairwisecoulomb`` collisions, with :pp:param:`collisions.correct_energy_momentum` set, this parameter controls the exponent used on the particle weight when distributing the momentum correction.
+    This can be set for each collision using :pp:param:`<collision_name>.beta_weight_exponent`.
     With a value greater than 1, it will distribute more of the correction to particles with higher weights.
 
+.. pp:param:: collisions.energy_correction_sort_by_weight
+    :type: ``bool``
+    :default: 0
+    :optional:
+
+    Only for ``pairwisecoulomb`` collisions, with :pp:param:`collisions.correct_energy_momentum` set, specifies whether the particles are sorted by weight when the energy correction is applied.
+    This can be set for each collision using :pp:param:`<collision_name>.energy_correction_sort_by_weight`.
+    When the particles have a range of weights, sorting improves the correction by applying more of it to the heavier weighted particles, which has a proportionately smaller effect on their momenta, and typically reduces the number of particles that the correction is applied to.
+
 .. pp:param:: collisions.split_momentum_push
     :type: ``bool``
     :default: 1
@@ -3140,30 +3155,6 @@ Details about the collision models can be found in the :ref:`theory section <mul
     This is only implemented for the explicit evolve scheme and is not available for the implicit evolve schemes, because the implicit
     formulation is intrinsically energy-conserving when combined with MCC collisions, as shown in `Angus et al., J. Comput. Phys. 456, 2022 <https://doi.org/10.1016/j.jcp.2022.111030>`__.
 
-.. pp:param:: <collision_name>.correct_energy_momentum
-    :type: ``bool``
-    :optional:
-
-    For pairwisecoulomb collisions, override the parameter :pp:param:`collisions.correct_energy_momentum` for the specific collision.
-
-.. pp:param:: <collision_name>.energy_fraction
-    :type: ``float``
-    :optional:
-
-    For pairwisecoulomb collisions, override the parameter :pp:param:`collisions.energy_fraction` for the specific collision.
-
-.. pp:param:: <collision_name>.energy_fraction_max
-    :type: ``float``
-    :optional:
-
-    For pairwisecoulomb collisions, override the parameter :pp:param:`collisions.energy_fraction_max` for the specific collision.
-
-.. pp:param:: <collision_name>.beta_weight_exponent
-    :type: ``float``
-    :optional:
-
-    For pairwisecoulomb collisions, override the parameter :pp:param:`collisions.beta_weight_exponent` for the specific collision.
-
 .. pp:param:: collisions.shuffling_method
     :type: ``bool``
     :default: 0

Examples/Tests/collision/CMakeLists.txt

@@ -131,6 +131,16 @@ add_warpx_test(
     OFF  # dependency
 )
 
+add_warpx_test(
+    test_2d_charge_exchange_dsmc  # name
+    2  # dims
+    1  # nprocs
+    inputs_test_2d_charge_exchange_dsmc  # inputs
+    "analysis_test_2d_collisions_split_momentum_push.py --path diags/reducedfiles/"  # analysis
+    "analysis_default_regression.py --path diags/diag1/"  # checksum
+    OFF  # dependency
+)
+
 add_warpx_test(
     test_2d_collisions_split_momentum_push_electrostatic  # name
     2  # dims

Examples/Tests/collision/analysis_test_2d_collisions_split_momentum_push.py

@@ -42,6 +42,7 @@ def analyze(args: argparse.Namespace) -> None:
     num_particles_per_cell_array = np.array(
         [int(nppc) for nppc in num_particles_per_cell_list]
     )
+    num_species = len(input_dict["particles.species_names"])
     num_particles_per_cell = np.prod(num_particles_per_cell_array)
     electrostatic = (
         True
@@ -52,9 +53,9 @@ def analyze(args: argparse.Namespace) -> None:
         else False
     )
     if electrostatic:
-        equipartition_value = 1 / (6 * num_particles_per_cell + 1)
+        equipartition_value = 1 / (3 * num_species * num_particles_per_cell + 1)
     else:
-        equipartition_value = 5 / (6 * num_particles_per_cell + 5)
+        equipartition_value = 5 / (3 * num_species * num_particles_per_cell + 5)
 
     # normalize the field energy variation
     electron_temperature = float(input_dict["my_constants.Te"][0])
@@ -117,7 +118,7 @@ def analyze(args: argparse.Namespace) -> None:
         )
 
     # compare the final value of the field energy variation to the reference equipartition value
-    relative_tolerance = 1e-1 if electrostatic else 5e-1
+    relative_tolerance = 1.1e-1 if electrostatic else 5e-1
     # average over the last 100 time steps to reduce noise
     field_energy_error_avg = np.mean(field_energy_error[-100:])
     if not np.isclose(

Examples/Tests/collision/inputs_test_2d_charge_exchange_dsmc

@@ -0,0 +1,125 @@
+# 2D charge exchange collision test (DSMC), with electrostatic solver.
+# Electrons (neutralizing background) + He+ ions + He neutrals.
+# He+ and He undergo DSMC charge exchange.
+# The electrostatic solver is active; total energy (field + particle) conservation
+# is verified using analysis_test_2d_collisions_split_momentum_push.py.
+
+# algo
+algo.field_gathering = energy-conserving
+algo.particle_shape = 2
+
+# amr
+amr.max_level = 0
+amr.n_cell = 16 16
+
+# boundary
+boundary.field_hi = periodic periodic
+boundary.field_lo = periodic periodic
+boundary.particle_hi = periodic periodic
+boundary.particle_lo = periodic periodic
+
+# collisions
+collisions.collision_names = cx_col
+cx_col.type = dsmc
+cx_col.scattering_processes = charge_exchange
+cx_col.species = Heplus He
+cx_col.product_species = He Heplus
+cx_col.charge_exchange_cross_section = ../../../../warpx-data/MCC_cross_sections/He/charge_exchange.dat
+
+# diagnostics
+diagnostics.diags_names = diag1
+diag1.diag_type = Full
+diag1.format = openpmd
+diag1.intervals = 2000
+diag1.fields_to_plot = Ex Ey Ez Bx By Bz jx jy jz rho
+diag1.write_species = 1
+diag1.species = electrons Heplus He
+diag1.electrons.variables = w x z ux uy uz
+diag1.Heplus.variables = w x z ux uy uz
+diag1.He.variables = w x z ux uy uz
+
+# electrons (neutralizing background for He+)
+electrons.charge = -q_e
+electrons.density = n0
+electrons.injection_style = "NUniformPerCell"
+electrons.mass = m_e
+electrons.momentum_distribution_type = gaussian
+electrons.num_particles_per_cell_each_dim = 8 8
+electrons.profile = constant
+electrons.species_type = electron
+electrons.ux_th = sqrt(Te*q_e/m_e)/clight
+electrons.uy_th = sqrt(Te*q_e/m_e)/clight
+electrons.uz_th = sqrt(Te*q_e/m_e)/clight
+electrons.xmax = 10*de0
+electrons.xmin = 0
+electrons.zmax = 10*de0
+electrons.zmin = 0
+
+# Heplus (He+ ions undergoing charge exchange)
+Heplus.charge = q_e
+Heplus.density = n0
+Heplus.injection_style = "NUniformPerCell"
+Heplus.mass = m_He
+Heplus.momentum_distribution_type = gaussian
+Heplus.num_particles_per_cell_each_dim = 8 8
+Heplus.profile = constant
+Heplus.ux_th = sqrt(Ti*q_e/m_He)/clight
+Heplus.uy_th = sqrt(Ti*q_e/m_He)/clight
+Heplus.uz_th = sqrt(Ti*q_e/m_He)/clight
+Heplus.xmax = 10*de0
+Heplus.xmin = 0
+Heplus.zmax = 10*de0
+Heplus.zmin = 0
+
+# He (neutral He atoms undergoing charge exchange)
+He.charge = 0
+He.density = n0
+He.injection_style = "NUniformPerCell"
+He.mass = m_He
+He.momentum_distribution_type = gaussian
+He.num_particles_per_cell_each_dim = 8 8
+He.profile = constant
+He.ux_th = sqrt(Ti*q_e/m_He)/clight
+He.uy_th = sqrt(Ti*q_e/m_He)/clight
+He.uz_th = sqrt(Ti*q_e/m_He)/clight
+He.xmax = 10*de0
+He.xmin = 0
+He.zmax = 10*de0
+He.zmin = 0
+
+# reduced diagnostics
+field_energy.intervals = 1
+field_energy.type = FieldEnergy
+particle_energy.intervals = 1
+particle_energy.type = ParticleEnergy
+
+# geometry
+geometry.dims = 2
+geometry.prob_hi = 10*de0 10*de0
+geometry.prob_lo = 0. 0.
+
+# interpolation
+interpolation.galerkin_scheme = 1
+
+# max_step
+max_step = 2000
+
+# my_constants
+my_constants.m_He = 4*m_p           # He mass, kg
+my_constants.n0 = 1e30              # number density, m^-3
+my_constants.Te = 100.              # electron temperature, eV
+my_constants.Ti = 100.              # He ion/neutral temperature, eV
+my_constants.wpe = q_e*sqrt(n0/(m_e*epsilon0))   # electron plasma frequency, rad/s
+my_constants.de0 = clight/wpe       # electron skin depth, m
+my_constants.dt = 0.1/wpe           # time step (electron timescale), s
+
+# particles
+particles.species_names = electrons Heplus He
+
+# warpx
+warpx.const_dt = dt
+warpx.do_electrostatic = labframe
+warpx.reduced_diags_names = field_energy particle_energy
+warpx.serialize_initial_conditions = 1
+warpx.use_filter = 0
+warpx.verbose = 1

Examples/Tests/secondary_ion_emission/analysis.py

@@ -90,7 +90,7 @@
 
 # The tolerance is expressed as a fraction of the sphere radius R, and covers
 # only the EB discretization error (the thermal kick is removed by back-propagation).
-tolerance = 0.05
+tolerance = 0.02
 
 for i in range(0, N_sec_e):
     # For each candidate parent ion j, back-propagate the electron from ts.t[-1]

Regression/Checksum/benchmarks_json/test_1d_charge_exchange_dsmc.json

@@ -1,11 +1,11 @@
 {
   "lev=0": {
-    "rho": 0.0007701705979872025
+    "rho": 0.0007712764213131624
   },
   "H": {
     "particle_position_x": 0.0,
     "particle_position_y": 0.0,
-    "particle_position_z": 2530.8479274154333,
+    "particle_position_z": 2530.062259029084,
     "particle_momentum_x": 0.0,
     "particle_momentum_y": 0.0,
     "particle_momentum_z": 5.948838493809116e-17,
@@ -23,7 +23,7 @@
   "H2plus": {
     "particle_position_x": 0.0,
     "particle_position_y": 0.0,
-    "particle_position_z": 1021.6452859116588,
+    "particle_position_z": 1023.4987864862421,
     "particle_momentum_x": 0.0,
     "particle_momentum_y": 0.0,
     "particle_momentum_z": 9.753884936303964e-28,
@@ -38,4 +38,4 @@
     "particle_momentum_z": 5.434778911325365e-18,
     "particle_weight": 1104206441820.484
   }
-}
+}

Regression/Checksum/benchmarks_json/test_2d_charge_exchange_dsmc.json

@@ -0,0 +1,41 @@
+{
+  "lev=0": {
+    "Bx": 0.0,
+    "By": 0.0,
+    "Bz": 0.0,
+    "Ex": 14448333080954.348,
+    "Ey": 0.0,
+    "Ez": 17169859243066.512,
+    "jx": 7.803855689663588e+18,
+    "jy": 7.215854549193545e+18,
+    "jz": 6.576572210896873e+18,
+    "rho": 53358037967.6566
+  },
+  "He": {
+    "particle_position_x": 0.000435321087901117,
+    "particle_position_y": 0.0,
+    "particle_position_z": 0.00043532495740775497,
+    "particle_momentum_x": 4.2993604934866694e-18,
+    "particle_momentum_y": 4.284989545488133e-18,
+    "particle_momentum_z": 4.295691097769843e-18,
+    "particle_weight": 2823958725036869.5
+  },
+  "Heplus": {
+    "particle_position_x": 0.00043531587816417113,
+    "particle_position_y": 0.0,
+    "particle_position_z": 0.0004353330259468823,
+    "particle_momentum_x": 4.306240128575128e-18,
+    "particle_momentum_y": 4.269568445141001e-18,
+    "particle_momentum_z": 4.2453148876079736e-18,
+    "particle_weight": 2823958725036869.5
+  },
+  "electrons": {
+    "particle_position_x": 0.00043485561435631804,
+    "particle_position_y": 0.0,
+    "particle_position_z": 0.0004366002075857368,
+    "particle_momentum_x": 4.9302740059601715e-20,
+    "particle_momentum_y": 5.076441659627337e-20,
+    "particle_momentum_z": 4.984100225110185e-20,
+    "particle_weight": 2823958725036869.5
+  }
+}