Homec4science
Diffusion lammps (master) Browse

lammps/doc/html/processors.html
04eadb634194master

Last Change
Raw File

processors.html
View Options

<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>processors command &mdash; LAMMPS documentation</title>
<link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="_static/sphinxcontrib-images/LightBox2/lightbox2/css/lightbox.css" type="text/css" />
<link rel="top" title="LAMMPS documentation" href="index.html"/>
<link rel="up" title="Commands" href="commands.html"/>
<link rel="next" title="python command" href="python.html"/>
<link rel="prev" title="print command" href="print.html"/>
<script src="_static/js/modernizr.min.js"></script>
</head>
<body class="wy-body-for-nav" role="document">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-nav-search">
<a href="Manual.html" class="icon icon-home"> LAMMPS
</a>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
<input type="text" name="q" placeholder="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
<p class="caption"><span class="caption-text">User Documentation</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="Section_intro.html">1. Introduction</a></li>
<li class="toctree-l1"><a class="reference internal" href="Section_start.html">2. Getting Started</a></li>
<li class="toctree-l1"><a class="reference internal" href="Section_commands.html">3. Commands</a></li>
<li class="toctree-l1"><a class="reference internal" href="Section_packages.html">4. Packages</a></li>
<li class="toctree-l1"><a class="reference internal" href="Section_accelerate.html">5. Accelerating LAMMPS performance</a></li>
<li class="toctree-l1"><a class="reference internal" href="Section_howto.html">6. How-to discussions</a></li>
<li class="toctree-l1"><a class="reference internal" href="Section_example.html">7. Example problems</a></li>
<li class="toctree-l1"><a class="reference internal" href="Section_perf.html">8. Performance &amp; scalability</a></li>
<li class="toctree-l1"><a class="reference internal" href="Section_tools.html">9. Additional tools</a></li>
<li class="toctree-l1"><a class="reference internal" href="Section_modify.html">10. Modifying &amp; extending LAMMPS</a></li>
<li class="toctree-l1"><a class="reference internal" href="Section_python.html">11. Python interface to LAMMPS</a></li>
<li class="toctree-l1"><a class="reference internal" href="Section_errors.html">12. Errors</a></li>
<li class="toctree-l1"><a class="reference internal" href="Section_history.html">13. Future and history</a></li>
</ul>
<p class="caption"><span class="caption-text">Index</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="tutorials.html">Tutorials</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="commands.html">Commands</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="angle_coeff.html">angle_coeff command</a></li>
<li class="toctree-l2"><a class="reference internal" href="angle_style.html">angle_style command</a></li>
<li class="toctree-l2"><a class="reference internal" href="atom_modify.html">atom_modify command</a></li>
<li class="toctree-l2"><a class="reference internal" href="atom_style.html">atom_style command</a></li>
<li class="toctree-l2"><a class="reference internal" href="balance.html">balance command</a></li>
<li class="toctree-l2"><a class="reference internal" href="bond_coeff.html">bond_coeff command</a></li>
<li class="toctree-l2"><a class="reference internal" href="bond_style.html">bond_style command</a></li>
<li class="toctree-l2"><a class="reference internal" href="bond_write.html">bond_write command</a></li>
<li class="toctree-l2"><a class="reference internal" href="boundary.html">boundary command</a></li>
<li class="toctree-l2"><a class="reference internal" href="box.html">box command</a></li>
<li class="toctree-l2"><a class="reference internal" href="change_box.html">change_box command</a></li>
<li class="toctree-l2"><a class="reference internal" href="clear.html">clear command</a></li>
<li class="toctree-l2"><a class="reference internal" href="comm_modify.html">comm_modify command</a></li>
<li class="toctree-l2"><a class="reference internal" href="comm_style.html">comm_style command</a></li>
<li class="toctree-l2"><a class="reference internal" href="compute.html">compute command</a></li>
<li class="toctree-l2"><a class="reference internal" href="compute_modify.html">compute_modify command</a></li>
<li class="toctree-l2"><a class="reference internal" href="create_atoms.html">create_atoms command</a></li>
<li class="toctree-l2"><a class="reference internal" href="create_bonds.html">create_bonds command</a></li>
<li class="toctree-l2"><a class="reference internal" href="create_box.html">create_box command</a></li>
<li class="toctree-l2"><a class="reference internal" href="delete_atoms.html">delete_atoms command</a></li>
<li class="toctree-l2"><a class="reference internal" href="delete_bonds.html">delete_bonds command</a></li>
<li class="toctree-l2"><a class="reference internal" href="dielectric.html">dielectric command</a></li>
<li class="toctree-l2"><a class="reference internal" href="dihedral_coeff.html">dihedral_coeff command</a></li>
<li class="toctree-l2"><a class="reference internal" href="dihedral_style.html">dihedral_style command</a></li>
<li class="toctree-l2"><a class="reference internal" href="dimension.html">dimension command</a></li>
<li class="toctree-l2"><a class="reference internal" href="displace_atoms.html">displace_atoms command</a></li>
<li class="toctree-l2"><a class="reference internal" href="dump.html">dump command</a></li>
<li class="toctree-l2"><a class="reference internal" href="dump.html#dump-custom-vtk-command"><code class="docutils literal"><span class="pre">dump</span> <span class="pre">custom/vtk</span></code> command</a></li>
<li class="toctree-l2"><a class="reference internal" href="dump.html#dump-h5md-command"><code class="docutils literal"><span class="pre">dump</span> <span class="pre">h5md</span></code> command</a></li>
<li class="toctree-l2"><a class="reference internal" href="dump.html#dump-image-command"><code class="docutils literal"><span class="pre">dump</span> <span class="pre">image</span></code> command</a></li>
<li class="toctree-l2"><a class="reference internal" href="dump.html#dump-movie-command"><code class="docutils literal"><span class="pre">dump</span> <span class="pre">movie</span></code> command</a></li>
<li class="toctree-l2"><a class="reference internal" href="dump.html#dump-molfile-command"><code class="docutils literal"><span class="pre">dump</span> <span class="pre">molfile</span></code> command</a></li>
<li class="toctree-l2"><a class="reference internal" href="dump_custom_vtk.html">dump custom/vtk command</a></li>
<li class="toctree-l2"><a class="reference internal" href="dump_h5md.html">dump h5md command</a></li>
<li class="toctree-l2"><a class="reference internal" href="dump_image.html">dump image command</a></li>
<li class="toctree-l2"><a class="reference internal" href="dump_image.html#dump-movie-command">dump movie command</a></li>
<li class="toctree-l2"><a class="reference internal" href="dump_modify.html">dump_modify command</a></li>
<li class="toctree-l2"><a class="reference internal" href="dump_molfile.html">dump molfile command</a></li>
<li class="toctree-l2"><a class="reference internal" href="echo.html">echo command</a></li>
<li class="toctree-l2"><a class="reference internal" href="fix.html">fix command</a></li>
<li class="toctree-l2"><a class="reference internal" href="fix_modify.html">fix_modify command</a></li>
<li class="toctree-l2"><a class="reference internal" href="group.html">group command</a></li>
<li class="toctree-l2"><a class="reference internal" href="group2ndx.html">group2ndx command</a></li>
<li class="toctree-l2"><a class="reference internal" href="group2ndx.html#ndx2group-command">ndx2group command</a></li>
<li class="toctree-l2"><a class="reference internal" href="if.html">if command</a></li>
<li class="toctree-l2"><a class="reference internal" href="improper_coeff.html">improper_coeff command</a></li>
<li class="toctree-l2"><a class="reference internal" href="improper_style.html">improper_style command</a></li>
<li class="toctree-l2"><a class="reference internal" href="include.html">include command</a></li>
<li class="toctree-l2"><a class="reference internal" href="info.html">info command</a></li>
<li class="toctree-l2"><a class="reference internal" href="jump.html">jump command</a></li>
<li class="toctree-l2"><a class="reference internal" href="kspace_modify.html">kspace_modify command</a></li>
<li class="toctree-l2"><a class="reference internal" href="kspace_style.html">kspace_style command</a></li>
<li class="toctree-l2"><a class="reference internal" href="label.html">label command</a></li>
<li class="toctree-l2"><a class="reference internal" href="lattice.html">lattice command</a></li>
<li class="toctree-l2"><a class="reference internal" href="log.html">log command</a></li>
<li class="toctree-l2"><a class="reference internal" href="mass.html">mass command</a></li>
<li class="toctree-l2"><a class="reference internal" href="min_modify.html">min_modify command</a></li>
<li class="toctree-l2"><a class="reference internal" href="min_style.html">min_style command</a></li>
<li class="toctree-l2"><a class="reference internal" href="minimize.html">minimize command</a></li>
<li class="toctree-l2"><a class="reference internal" href="molecule.html">molecule command</a></li>
<li class="toctree-l2"><a class="reference internal" href="neb.html">neb command</a></li>
<li class="toctree-l2"><a class="reference internal" href="neigh_modify.html">neigh_modify command</a></li>
<li class="toctree-l2"><a class="reference internal" href="neighbor.html">neighbor command</a></li>
<li class="toctree-l2"><a class="reference internal" href="newton.html">newton command</a></li>
<li class="toctree-l2"><a class="reference internal" href="next.html">next command</a></li>
<li class="toctree-l2"><a class="reference internal" href="package.html">package command</a></li>
<li class="toctree-l2"><a class="reference internal" href="pair_coeff.html">pair_coeff command</a></li>
<li class="toctree-l2"><a class="reference internal" href="pair_modify.html">pair_modify command</a></li>
<li class="toctree-l2"><a class="reference internal" href="pair_style.html">pair_style command</a></li>
<li class="toctree-l2"><a class="reference internal" href="pair_write.html">pair_write command</a></li>
<li class="toctree-l2"><a class="reference internal" href="partition.html">partition command</a></li>
<li class="toctree-l2"><a class="reference internal" href="prd.html">prd command</a></li>
<li class="toctree-l2"><a class="reference internal" href="print.html">print command</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">processors command</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#syntax">Syntax</a></li>
<li class="toctree-l3"><a class="reference internal" href="#examples">Examples</a></li>
<li class="toctree-l3"><a class="reference internal" href="#description">Description</a></li>
<li class="toctree-l3"><a class="reference internal" href="#restrictions">Restrictions</a></li>
<li class="toctree-l3"><a class="reference internal" href="#related-commands">Related commands</a></li>
<li class="toctree-l3"><a class="reference internal" href="#default">Default</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="python.html">python command</a></li>
<li class="toctree-l2"><a class="reference internal" href="quit.html">quit command</a></li>
<li class="toctree-l2"><a class="reference internal" href="read_data.html">read_data command</a></li>
<li class="toctree-l2"><a class="reference internal" href="read_dump.html">read_dump command</a></li>
<li class="toctree-l2"><a class="reference internal" href="read_restart.html">read_restart command</a></li>
<li class="toctree-l2"><a class="reference internal" href="region.html">region command</a></li>
<li class="toctree-l2"><a class="reference internal" href="replicate.html">replicate command</a></li>
<li class="toctree-l2"><a class="reference internal" href="rerun.html">rerun command</a></li>
<li class="toctree-l2"><a class="reference internal" href="reset_timestep.html">reset_timestep command</a></li>
<li class="toctree-l2"><a class="reference internal" href="restart.html">restart command</a></li>
<li class="toctree-l2"><a class="reference internal" href="run.html">run command</a></li>
<li class="toctree-l2"><a class="reference internal" href="run_style.html">run_style command</a></li>
<li class="toctree-l2"><a class="reference internal" href="set.html">set command</a></li>
<li class="toctree-l2"><a class="reference internal" href="shell.html">shell command</a></li>
<li class="toctree-l2"><a class="reference internal" href="special_bonds.html">special_bonds command</a></li>
<li class="toctree-l2"><a class="reference internal" href="suffix.html">suffix command</a></li>
<li class="toctree-l2"><a class="reference internal" href="tad.html">tad command</a></li>
<li class="toctree-l2"><a class="reference internal" href="temper.html">temper command</a></li>
<li class="toctree-l2"><a class="reference internal" href="thermo.html">thermo command</a></li>
<li class="toctree-l2"><a class="reference internal" href="thermo_modify.html">thermo_modify command</a></li>
<li class="toctree-l2"><a class="reference internal" href="thermo_style.html">thermo_style command</a></li>
<li class="toctree-l2"><a class="reference internal" href="timer.html">timer command</a></li>
<li class="toctree-l2"><a class="reference internal" href="timestep.html">timestep command</a></li>
<li class="toctree-l2"><a class="reference internal" href="uncompute.html">uncompute command</a></li>
<li class="toctree-l2"><a class="reference internal" href="undump.html">undump command</a></li>
<li class="toctree-l2"><a class="reference internal" href="unfix.html">unfix command</a></li>
<li class="toctree-l2"><a class="reference internal" href="units.html">units command</a></li>
<li class="toctree-l2"><a class="reference internal" href="variable.html">variable command</a></li>
<li class="toctree-l2"><a class="reference internal" href="velocity.html">velocity command</a></li>
<li class="toctree-l2"><a class="reference internal" href="write_coeff.html">write_coeff command</a></li>
<li class="toctree-l2"><a class="reference internal" href="write_data.html">write_data command</a></li>
<li class="toctree-l2"><a class="reference internal" href="write_dump.html">write_dump command</a></li>
<li class="toctree-l2"><a class="reference internal" href="write_restart.html">write_restart command</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="fixes.html">Fixes</a></li>
<li class="toctree-l1"><a class="reference internal" href="computes.html">Computes</a></li>
<li class="toctree-l1"><a class="reference internal" href="pairs.html">Pair Styles</a></li>
<li class="toctree-l1"><a class="reference internal" href="bonds.html">Bond Styles</a></li>
<li class="toctree-l1"><a class="reference internal" href="angles.html">Angle Styles</a></li>
<li class="toctree-l1"><a class="reference internal" href="dihedrals.html">Dihedral Styles</a></li>
<li class="toctree-l1"><a class="reference internal" href="impropers.html">Improper Styles</a></li>
</ul>
</div>
&nbsp;
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
<nav class="wy-nav-top" role="navigation" aria-label="top navigation">
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="Manual.html">LAMMPS</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="wy-breadcrumbs">
<li><a href="Manual.html">Docs</a> &raquo;</li>
<li><a href="commands.html">Commands</a> &raquo;</li>
<li>processors command</li>
<li class="wy-breadcrumbs-aside">
<a href="http://lammps.sandia.gov">Website</a>
<a href="Section_commands.html#comm">Commands</a>
</li>
</ul>
<hr/>
<div class="rst-footer-buttons" style="margin-bottom: 1em" role="navigation" aria-label="footer navigation">
<a href="python.html" class="btn btn-neutral float-right" title="python command" accesskey="n">Next <span class="fa fa-arrow-circle-right"></span></a>
<a href="print.html" class="btn btn-neutral" title="print command" accesskey="p"><span class="fa fa-arrow-circle-left"></span> Previous</a>
</div>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<div class="section" id="processors-command">
<span id="index-0"></span><h1>processors command</h1>
<div class="section" id="syntax">
<h2>Syntax</h2>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">processors</span> <span class="n">Px</span> <span class="n">Py</span> <span class="n">Pz</span> <span class="n">keyword</span> <span class="n">args</span> <span class="o">...</span>
</pre></div>
</div>
<ul>
<li><p class="first">Px,Py,Pz = # of processors in each dimension of 3d grid overlaying the simulation domain</p>
</li>
<li><p class="first">zero or more keyword/arg pairs may be appended</p>
</li>
<li><p class="first">keyword = <em>grid</em> or <em>map</em> or <em>part</em> or <em>file</em></p>
<pre class="literal-block">
<em>grid</em> arg = gstyle params ...
gstyle = <em>onelevel</em> or <em>twolevel</em> or <em>numa</em> or <em>custom</em>
onelevel params = none
twolevel params = Nc Cx Cy Cz
Nc = number of cores per node
Cx,Cy,Cz = # of cores in each dimension of 3d sub-grid assigned to each node
numa params = none
custom params = infile
infile = file containing grid layout
<em>map</em> arg = <em>cart</em> or <em>cart/reorder</em> or <em>xyz</em> or <em>xzy</em> or <em>yxz</em> or <em>yzx</em> or <em>zxy</em> or <em>zyx</em>
cart = use MPI_Cart() methods to map processors to 3d grid with reorder = 0
cart/reorder = use MPI_Cart() methods to map processors to 3d grid with reorder = 1
xyz,xzy,yxz,yzx,zxy,zyx = map procesors to 3d grid in IJK ordering
<em>numa</em> arg = none
<em>part</em> args = Psend Precv cstyle
Psend = partition # (1 to Np) which will send its processor layout
Precv = partition # (1 to Np) which will recv the processor layout
cstyle = <em>multiple</em>
<em>multiple</em> = Psend grid will be multiple of Precv grid in each dimension
<em>file</em> arg = outfile
outfile = name of file to write 3d grid of processors to
</pre>
</li>
</ul>
</div>
<div class="section" id="examples">
<h2>Examples</h2>
<pre class="literal-block">
processors * * 5
processors 2 4 4
processors * * 8 map xyz
processors * * * grid numa
processors * * * grid twolevel 4 * * 1
processors 4 8 16 grid custom myfile
processors * * * part 1 2 multiple
</pre>
</div>
<div class="section" id="description">
<h2>Description</h2>
<p>Specify how processors are mapped as a regular 3d grid to the global
simulation box. The mapping involves 2 steps. First if there are P
processors it means choosing a factorization P = Px by Py by Pz so
that there are Px processors in the x dimension, and similarly for the
y and z dimensions. Second, the P processors are mapped to the
regular 3d grid. The arguments to this command control each of these
2 steps.</p>
<p>The Px, Py, Pz parameters affect the factorization. Any of the 3
parameters can be specified with an asterisk &#8220;*&#8221;, which means LAMMPS
will choose the number of processors in that dimension of the grid.
It will do this based on the size and shape of the global simulation
box so as to minimize the surface-to-volume ratio of each processor&#8217;s
sub-domain.</p>
<p>Choosing explicit values for Px or Py or Pz can be used to override
the default manner in which LAMMPS will create the regular 3d grid of
processors, if it is known to be sub-optimal for a particular problem.
E.g. a problem where the extent of atoms will change dramatically in a
particular dimension over the course of the simulation.</p>
<p>The product of Px, Py, Pz must equal P, the total # of processors
LAMMPS is running on. For a <a class="reference internal" href="dimension.html"><span class="doc">2d simulation</span></a>, Pz must
equal 1.</p>
<p>Note that if you run on a prime number of processors P, then a grid
such as 1 x P x 1 will be required, which may incur extra
communication costs due to the high surface area of each processor&#8217;s
sub-domain.</p>
<p>Also note that if multiple partitions are being used then P is the
number of processors in this partition; see <a class="reference internal" href="Section_start.html#start-7"><span class="std std-ref">this section</span></a> for an explanation of the
-partition command-line switch. Also note that you can prefix the
processors command with the <a class="reference internal" href="partition.html"><span class="doc">partition</span></a> command to
easily specify different Px,Py,Pz values for different partitions.</p>
<p>You can use the <a class="reference internal" href="partition.html"><span class="doc">partition</span></a> command to specify
different processor grids for different partitions, e.g.</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">partition</span> <span class="n">yes</span> <span class="mi">1</span> <span class="n">processors</span> <span class="mi">4</span> <span class="mi">4</span> <span class="mi">4</span>
<span class="n">partition</span> <span class="n">yes</span> <span class="mi">2</span> <span class="n">processors</span> <span class="mi">2</span> <span class="mi">3</span> <span class="mi">2</span>
</pre></div>
</div>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">This command only affects the initial regular 3d grid created
when the simulation box is first specified via a
<a class="reference internal" href="create_box.html"><span class="doc">create_box</span></a> or <a class="reference internal" href="read_data.html"><span class="doc">read_data</span></a> or
<a class="reference internal" href="read_restart.html"><span class="doc">read_restart</span></a> command. Or if the simulation box is
re-created via the <a class="reference internal" href="replicate.html"><span class="doc">replicate</span></a> command. The same
regular grid is initially created, regardless of which
<a class="reference internal" href="comm_style.html"><span class="doc">comm_style</span></a> command is in effect.</p>
</div>
<p>If load-balancing is never invoked via the <a class="reference internal" href="balance.html"><span class="doc">balance</span></a> or
<a class="reference internal" href="fix_balance.html"><span class="doc">fix balance</span></a> commands, then the initial regular grid
will persist for all simulations. If balancing is performed, some of
the methods invoked by those commands retain the logical toplogy of
the initial 3d grid, and the mapping of processors to the grid
specified by the processors command. However the grid spacings in
different dimensions may change, so that processors own sub-domains of
different sizes. If the <a class="reference internal" href="comm_style.html"><span class="doc">comm_style tiled</span></a> command is
used, methods invoked by the balancing commands may discard the 3d
grid of processors and tile the simulation domain with sub-domains of
different sizes and shapes which no longer have a logical 3d
connectivity. If that occurs, all the information specified by the
processors command is ignored.</p>
<hr class="docutils" />
<p>The <em>grid</em> keyword affects the factorization of P into Px,Py,Pz and it
can also affect how the P processor IDs are mapped to the 3d grid of
processors.</p>
<p>The <em>onelevel</em> style creates a 3d grid that is compatible with the
Px,Py,Pz settings, and which minimizes the surface-to-volume ratio of
each processor&#8217;s sub-domain, as described above. The mapping of
processors to the grid is determined by the <em>map</em> keyword setting.</p>
<p>The <em>twolevel</em> style can be used on machines with multicore nodes to
minimize off-node communication. It insures that contiguous
sub-sections of the 3d grid are assigned to all the cores of a node.
For example if <em>Nc</em> is 4, then 2x2x1 or 2x1x2 or 1x2x2 sub-sections of
the 3d grid will correspond to the cores of each node. This affects
both the factorization and mapping steps.</p>
<p>The <em>Cx</em>, <em>Cy</em>, <em>Cz</em> settings are similar to the <em>Px</em>, <em>Py</em>, <em>Pz</em>
settings, only their product should equal <em>Nc</em>. Any of the 3
parameters can be specified with an asterisk &#8220;*&#8221;, which means LAMMPS
will choose the number of cores in that dimension of the node&#8217;s
sub-grid. As with Px,Py,Pz, it will do this based on the size and
shape of the global simulation box so as to minimize the
surface-to-volume ratio of each processor&#8217;s sub-domain.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">For the <em>twolevel</em> style to work correctly, it assumes the MPI
ranks of processors LAMMPS is running on are ordered by core and then
by node. E.g. if you are running on 2 quad-core nodes, for a total of
8 processors, then it assumes processors 0,1,2,3 are on node 1, and
processors 4,5,6,7 are on node 2. This is the default rank ordering
for most MPI implementations, but some MPIs provide options for this
ordering, e.g. via environment variable settings.</p>
</div>
<p>The <em>numa</em> style operates similar to the <em>twolevel</em> keyword except
that it auto-detects which cores are running on which nodes.
Currently, it does this in only 2 levels, but it may be extended in
the future to account for socket topology and other non-uniform memory
access (NUMA) costs. It also uses a different algorithm than the
<em>twolevel</em> keyword for doing the two-level factorization of the
simulation box into a 3d processor grid to minimize off-node
communication, and it does its own MPI-based mapping of nodes and
cores to the regular 3d grid. Thus it may produce a different layout
of the processors than the <em>twolevel</em> options.</p>
<p>The <em>numa</em> style will give an error if the number of MPI processes is
not divisible by the number of cores used per node, or any of the Px
or Py of Pz values is greater than 1.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Unlike the <em>twolevel</em> style, the <em>numa</em> style does not require
any particular ordering of MPI ranks i norder to work correctly. This
is because it auto-detects which processes are running on which nodes.</p>
</div>
<p>The <em>custom</em> style uses the file <em>infile</em> to define both the 3d
factorization and the mapping of processors to the grid.</p>
<p>The file should have the following format. Any number of initial
blank or comment lines (starting with a &#8220;#&#8221; character) can be present.
The first non-blank, non-comment line should have
3 values:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">Px</span> <span class="n">Py</span> <span class="n">Py</span>
</pre></div>
</div>
<p>These must be compatible with the total number of processors
and the Px, Py, Pz settings of the processors commmand.</p>
<p>This line should be immediately followed by
P = Px*Py*Pz lines of the form:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">ID</span> <span class="n">I</span> <span class="n">J</span> <span class="n">K</span>
</pre></div>
</div>
<p>where ID is a processor ID (from 0 to P-1) and I,J,K are the
processors location in the 3d grid. I must be a number from 1 to Px
(inclusive) and similarly for J and K. The P lines can be listed in
any order, but no processor ID should appear more than once.</p>
<hr class="docutils" />
<p>The <em>map</em> keyword affects how the P processor IDs (from 0 to P-1) are
mapped to the 3d grid of processors. It is only used by the
<em>onelevel</em> and <em>twolevel</em> grid settings.</p>
<p>The <em>cart</em> style uses the family of MPI Cartesian functions to perform
the mapping, namely MPI_Cart_create(), MPI_Cart_get(),
MPI_Cart_shift(), and MPI_Cart_rank(). It invokes the
MPI_Cart_create() function with its reorder flag = 0, so that MPI is
not free to reorder the processors.</p>
<p>The <em>cart/reorder</em> style does the same thing as the <em>cart</em> style
except it sets the reorder flag to 1, so that MPI can reorder
processors if it desires.</p>
<p>The <em>xyz</em>, <em>xzy</em>, <em>yxz</em>, <em>yzx</em>, <em>zxy</em>, and <em>zyx</em> styles are all
similar. If the style is IJK, then it maps the P processors to the
grid so that the processor ID in the I direction varies fastest, the
processor ID in the J direction varies next fastest, and the processor
ID in the K direction varies slowest. For example, if you select
style <em>xyz</em> and you have a 2x2x2 grid of 8 processors, the assignments
of the 8 octants of the simulation domain will be:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">proc</span> <span class="mi">0</span> <span class="o">=</span> <span class="n">lo</span> <span class="n">x</span><span class="p">,</span> <span class="n">lo</span> <span class="n">y</span><span class="p">,</span> <span class="n">lo</span> <span class="n">z</span> <span class="n">octant</span>
<span class="n">proc</span> <span class="mi">1</span> <span class="o">=</span> <span class="n">hi</span> <span class="n">x</span><span class="p">,</span> <span class="n">lo</span> <span class="n">y</span><span class="p">,</span> <span class="n">lo</span> <span class="n">z</span> <span class="n">octant</span>
<span class="n">proc</span> <span class="mi">2</span> <span class="o">=</span> <span class="n">lo</span> <span class="n">x</span><span class="p">,</span> <span class="n">hi</span> <span class="n">y</span><span class="p">,</span> <span class="n">lo</span> <span class="n">z</span> <span class="n">octant</span>
<span class="n">proc</span> <span class="mi">3</span> <span class="o">=</span> <span class="n">hi</span> <span class="n">x</span><span class="p">,</span> <span class="n">hi</span> <span class="n">y</span><span class="p">,</span> <span class="n">lo</span> <span class="n">z</span> <span class="n">octant</span>
<span class="n">proc</span> <span class="mi">4</span> <span class="o">=</span> <span class="n">lo</span> <span class="n">x</span><span class="p">,</span> <span class="n">lo</span> <span class="n">y</span><span class="p">,</span> <span class="n">hi</span> <span class="n">z</span> <span class="n">octant</span>
<span class="n">proc</span> <span class="mi">5</span> <span class="o">=</span> <span class="n">hi</span> <span class="n">x</span><span class="p">,</span> <span class="n">lo</span> <span class="n">y</span><span class="p">,</span> <span class="n">hi</span> <span class="n">z</span> <span class="n">octant</span>
<span class="n">proc</span> <span class="mi">6</span> <span class="o">=</span> <span class="n">lo</span> <span class="n">x</span><span class="p">,</span> <span class="n">hi</span> <span class="n">y</span><span class="p">,</span> <span class="n">hi</span> <span class="n">z</span> <span class="n">octant</span>
<span class="n">proc</span> <span class="mi">7</span> <span class="o">=</span> <span class="n">hi</span> <span class="n">x</span><span class="p">,</span> <span class="n">hi</span> <span class="n">y</span><span class="p">,</span> <span class="n">hi</span> <span class="n">z</span> <span class="n">octant</span>
</pre></div>
</div>
<p>Note that, in principle, an MPI implementation on a particular machine
should be aware of both the machine&#8217;s network topology and the
specific subset of processors and nodes that were assigned to your
simulation. Thus its MPI_Cart calls can optimize the assignment of
MPI processes to the 3d grid to minimize communication costs. In
practice, however, few if any MPI implementations actually do this.
So it is likely that the <em>cart</em> and <em>cart/reorder</em> styles simply give
the same result as one of the IJK styles.</p>
<p>Also note, that for the <em>twolevel</em> grid style, the <em>map</em> setting is
used to first map the nodes to the 3d grid, then again to the cores
within each node. For the latter step, the <em>cart</em> and <em>cart/reorder</em>
styles are not supported, so an <em>xyz</em> style is used in their place.</p>
<hr class="docutils" />
<p>The <em>part</em> keyword affects the factorization of P into Px,Py,Pz.</p>
<p>It can be useful when running in multi-partition mode, e.g. with the
<a class="reference internal" href="run_style.html"><span class="doc">run_style verlet/split</span></a> command. It specifies a
dependency bewteen a sending partition <em>Psend</em> and a receiving
partition <em>Precv</em> which is enforced when each is setting up their own
mapping of their processors to the simulation box. Each of <em>Psend</em>
and <em>Precv</em> must be integers from 1 to Np, where Np is the number of
partitions you have defined via the <a class="reference internal" href="Section_start.html#start-7"><span class="std std-ref">-partition command-line switch</span></a>.</p>
<p>A &#8220;dependency&#8221; means that the sending partition will create its
regular 3d grid as Px by Py by Pz and after it has done this, it will
send the Px,Py,Pz values to the receiving partition. The receiving
partition will wait to receive these values before creating its own
regular 3d grid and will use the sender&#8217;s Px,Py,Pz values as a
constraint. The nature of the constraint is determined by the
<em>cstyle</em> argument.</p>
<p>For a <em>cstyle</em> of <em>multiple</em>, each dimension of the sender&#8217;s processor
grid is required to be an integer multiple of the corresponding
dimension in the receiver&#8217;s processor grid. This is a requirement of
the <a class="reference internal" href="run_style.html"><span class="doc">run_style verlet/split</span></a> command.</p>
<p>For example, assume the sending partition creates a 4x6x10 grid = 240
processor grid. If the receiving partition is running on 80
processors, it could create a 4x2x10 grid, but it will not create a
2x4x10 grid, since in the y-dimension, 6 is not an integer multiple of
4.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">If you use the <a class="reference internal" href="partition.html"><span class="doc">partition</span></a> command to invoke
different &#8220;processsors&#8221; commands on different partitions, and you also
use the <em>part</em> keyword, then you must insure that both the sending and
receiving partitions invoke the &#8220;processors&#8221; command that connects the
2 partitions via the <em>part</em> keyword. LAMMPS cannot easily check for
this, but your simulation will likely hang in its setup phase if this
error has been made.</p>
</div>
<hr class="docutils" />
<p>The <em>file</em> keyword writes the mapping of the factorization of P
processors and their mapping to the 3d grid to the specified file
<em>outfile</em>. This is useful to check that you assigned physical
processors in the manner you desired, which can be tricky to figure
out, especially when running on multiple partitions or on, a multicore
machine or when the processor ranks were reordered by use of the
<a class="reference internal" href="Section_start.html#start-7"><span class="std std-ref">-reorder command-line switch</span></a> or due to
use of MPI-specific launch options such as a config file.</p>
<p>If you have multiple partitions you should insure that each one writes
to a different file, e.g. using a <a class="reference internal" href="variable.html"><span class="doc">world-style variable</span></a>
for the filename. The file has a self-explanatory header, followed by
one-line per processor in this format:</p>
<p>world-ID universe-ID original-ID: I J K: name</p>
<p>The IDs are the processor&#8217;s rank in this simulation (the world), the
universe (of multiple simulations), and the original MPI communicator
used to instantiate LAMMPS, respectively. The world and universe IDs
will only be different if you are running on more than one partition;
see the <a class="reference internal" href="Section_start.html#start-7"><span class="std std-ref">-partition command-line switch</span></a>.
The universe and original IDs will only be different if you used the
<a class="reference internal" href="Section_start.html#start-7"><span class="std std-ref">-reorder command-line switch</span></a> to reorder
the processors differently than their rank in the original
communicator LAMMPS was instantiated with.</p>
<p>I,J,K are the indices of the processor in the regular 3d grid, each
from 1 to Nd, where Nd is the number of processors in that dimension
of the grid.</p>
<p>The <em>name</em> is what is returned by a call to MPI_Get_processor_name()
and should represent an identifier relevant to the physical processors
in your machine. Note that depending on the MPI implementation,
multiple cores can have the same <em>name</em>.</p>
</div>
<hr class="docutils" />
<div class="section" id="restrictions">
<h2>Restrictions</h2>
<p>This command cannot be used after the simulation box is defined by a
<a class="reference internal" href="read_data.html"><span class="doc">read_data</span></a> or <a class="reference internal" href="create_box.html"><span class="doc">create_box</span></a> command.
It can be used before a restart file is read to change the 3d
processor grid from what is specified in the restart file.</p>
<p>The <em>grid numa</em> keyword only currently works with the <em>map cart</em>
option.</p>
<p>The <em>part</em> keyword (for the receiving partition) only works with the
<em>grid onelevel</em> or <em>grid twolevel</em> options.</p>
</div>
<div class="section" id="related-commands">
<h2>Related commands</h2>
<p><a class="reference internal" href="partition.html"><span class="doc">partition</span></a>, <a class="reference internal" href="Section_start.html#start-7"><span class="std std-ref">-reorder command-line switch</span></a></p>
</div>
<div class="section" id="default">
<h2>Default</h2>
<p>The option defaults are Px Py Pz = * * *, grid = onelevel, and map =
cart.</p>
</div>
</div>
</div>
</div>
<footer>
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
<a href="python.html" class="btn btn-neutral float-right" title="python command" accesskey="n">Next <span class="fa fa-arrow-circle-right"></span></a>
<a href="print.html" class="btn btn-neutral" title="print command" accesskey="p"><span class="fa fa-arrow-circle-left"></span> Previous</a>
</div>
<hr/>
<div role="contentinfo">
<p>
&copy; Copyright 2013 Sandia Corporation.
</p>
</div>
Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'./',
VERSION:'',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="_static/sphinxcontrib-images/LightBox2/lightbox2/js/jquery-1.11.0.min.js"></script>
<script type="text/javascript" src="_static/sphinxcontrib-images/LightBox2/lightbox2/js/lightbox.min.js"></script>
<script type="text/javascript" src="_static/sphinxcontrib-images/LightBox2/lightbox2-customize/jquery-noconflict.js"></script>
<script type="text/javascript" src="_static/js/theme.js"></script>
<script type="text/javascript">
jQuery(function () {
SphinxRtdTheme.StickyNav.enable();
});
</script>
</body>
</html>
c4science · Help