package.html
d38957c1c0b5runner

package.html
View Options

			<HTML>
			<CENTER><A HREF = "http://lammps.sandia.gov">LAMMPS WWW Site</A> - <A HREF = "Manual.html">LAMMPS Documentation</A> - <A HREF = "Section_commands.html#comm">LAMMPS Commands</A>
			</CENTER>






			<HR>

			<H3>package command
			</H3>
			<P><B>Syntax:</B>
			</P>
			<PRE>package style args
			</PRE>
			<UL><LI>style = <I>cuda</I> or <I>gpu</I> or <I>intel</I> or <I>kokkos</I> or <I>omp</I>

			<LI>args = arguments specific to the style

			<PRE> <I>cuda</I> args = Ngpu keyword value ...
			Ngpu = # of GPUs per node
			zero or more keyword/value pairs may be appended
			keywords = <I>newton</I> or <I>gpuID</I> or <I>timing</I> or <I>test</I> or <I>thread</I>
			<I>newton</I> = <I>off</I> or <I>on</I>
			off = set Newton pairwise and bonded flags off (default)
			on = set Newton pairwise and bonded flags on
			<I>gpuID</I> values = gpu1 .. gpuN
			gpu1 .. gpuN = IDs of the Ngpu GPUs to use
			<I>timing</I> values = none
			<I>test</I> values = id
			id = atom-ID of a test particle
			<I>thread</I> = auto or tpa or bpa
			auto = test whether tpa or bpa is faster
			tpa = one thread per atom
			bpa = one block per atom
			<I>gpu</I> args = Ngpu keyword value ...
			Ngpu = # of GPUs per node
			zero or more keyword/value pairs may be appended
			keywords = <I>neigh</I> or <I>newton</I> or <I>binsize</I> or <I>split</I> or <I>gpuID</I> or <I>tpa</I> or <I>device</I>
			<I>neigh</I> value = <I>yes</I> or <I>no</I>
			yes = neighbor list build on GPU (default)
			no = neighbor list build on CPU
			<I>newton</I> = <I>off</I> or <I>on</I>
			off = set Newton pairwise flag off (default and required)
			on = set Newton pairwise flag on (currently not allowed)
			<I>binsize</I> value = size
			size = bin size for neighbor list construction (distance units)
			<I>split</I> = fraction
			fraction = fraction of atoms assigned to GPU (default = 1.0)
			<I>gpuID</I> values = first last
			first = ID of first GPU to be used on each node
			last = ID of last GPU to be used on each node
			<I>tpa</I> value = Nthreads
			Nthreads = # of GPU threads used per atom
			<I>device</I> value = device_type
			device_type = <I>kepler</I> or <I>fermi</I> or <I>cypress</I> or <I>generic</I>
			<I>blocksize</I> value = size
			size = thread block size for pair force computation
			<I>intel</I> args = NPhi keyword value ...
			Nphi = # of coprocessors per node
			zero or more keyword/value pairs may be appended
			keywords = <I>omp</I> or <I>mode</I> or <I>balance</I> or <I>ghost</I> or <I>tpc</I> or <I>tptask</I> or <I>no_affinity</I>
			<I>omp</I> value = Nthreads
			Nthreads = number of OpenMP threads to use on CPU (default = 0)
			<I>mode</I> value = <I>single</I> or <I>mixed</I> or <I>double</I>
			single = perform force calculations in single precision
			mixed = perform force calculations in mixed precision
			double = perform force calculations in double precision
			<I>balance</I> value = split
			split = fraction of work to offload to coprocessor, -1 for dynamic
			<I>ghost</I> value = <I>yes</I> or <I>no</I>
			yes = include ghost atoms for offload
			no = do not include ghost atoms for offload
			<I>tpc</I> value = Ntpc
			Ntpc = max number of coprocessor threads per coprocessor core (default = 4)
			<I>tptask</I> value = Ntptask
			Ntptask = max number of coprocessor threads per MPI task (default = 240)
			<I>no_affinity</I> values = none
			<I>kokkos</I> args = keyword value ...
			zero or more keyword/value pairs may be appended
			keywords = <I>neigh</I> or <I>newton</I> or <I>binsize</I> or <I>comm</I> or <I>comm/exchange</I> or <I>comm/forward</I>
			<I>neigh</I> value = <I>full</I> or <I>half/thread</I> or <I>half</I> or <I>n2</I> or <I>full/cluster</I>
			full = full neighbor list
			half/thread = half neighbor list built in thread-safe manner
			half = half neighbor list, not thread-safe, only use when 1 thread/MPI task
			n2 = non-binning neighbor list build, O(N^2) algorithm
			full/cluster = full neighbor list with clustered groups of atoms
			<I>newton</I> = <I>off</I> or <I>on</I>
			off = set Newton pairwise and bonded flags off (default)
			on = set Newton pairwise and bonded flags on
			<I>binsize</I> value = size
			size = bin size for neighbor list construction (distance units)
			<I>comm</I> value = <I>no</I> or <I>host</I> or <I>device</I>
			use value for both comm/exchange and comm/forward
			<I>comm/exchange</I> value = <I>no</I> or <I>host</I> or <I>device</I>
			<I>comm/forward</I> value = <I>no</I> or <I>host</I> or <I>device</I>
			no = perform communication pack/unpack in non-KOKKOS mode
			host = perform pack/unpack on host (e.g. with OpenMP threading)
			device = perform pack/unpack on device (e.g. on GPU)
			<I>omp</I> args = Nthreads keyword value ...
			Nthread = # of OpenMP threads to associate with each MPI process
			zero or more keyword/value pairs may be appended
			keywords = <I>neigh</I>
			<I>neigh</I> value = <I>yes</I> or <I>no</I>
			yes = threaded neighbor list build (default)
			no = non-threaded neighbor list build
			</PRE>

			</UL>
			<P><B>Examples:</B>
			</P>
			<PRE>package gpu 1
			package gpu 1 split 0.75
			package gpu 2 split -1.0
			package cuda 2 gpuID 0 2
			package cuda 1 test 3948
			package kokkos neigh half/thread comm device
			package omp 0 neigh no
			package omp 4
			package intel 1
			package intel 2 omp 4 mode mixed balance 0.5
			</PRE>
			<P><B>Description:</B>
			</P>
			<P>This command invokes package-specific settings for the various
			accelerator packages available in LAMMPS. Currently the following
			packages use settings from this command: USER-CUDA, GPU, USER-INTEL,
			KOKKOS, and USER-OMP.
			</P>
			<P>If this command is specified in an input script, it must be near the
			top of the script, before the simulation box has been defined. This
			is because it specifies settings that the accelerator packages use in
			their intialization, before a simultion is defined.
			</P>
			<P>This command can also be specified from the command-line when
			launching LAMMPS, using the "-pk" <A HREF = "Section_start.html#start_7">command-line
			switch</A>. The syntax is exactly the same as
			when used in an input script.
			</P>
			<P>Note that all of the accelerator packages require the package command
			to be specified (except the OPT package), if the package is to be used
			in a simulation (LAMMPS can be built with an accelerator package
			without using it in a particular simulation). However, in all cases,
			a default version of the command is typically invoked by other
			accelerator settings.
			</P>
			<P>The USER-CUDA and KOKKOS packages require a "-c on" or "-k on"
			<A HREF = "Section_start.html#start_7">command-line switch</A> respectively, which
			invokes a "package cuda" or "package kokkos" command with default
			settings.
			</P>
			<P>For the GPU, USER-INTEL, and USER-OMP packages, if a "-sf gpu" or "-sf
			intel" or "-sf omp" <A HREF = "Section_start.html#start_7">command-line switch</A>
			is used to auto-append accelerator suffixes to various styles in the
			input script, then those switches also invoke a "package gpu",
			"package intel", or "package omp" command with default settings.
			</P>
			<P>IMPORTANT NOTE: A package command for a particular style can be
			invoked multiple times when a simulation is setup, e.g. by the "-c
			on", "-k on", "-sf", and "-pk" <A HREF = "Section_start.html#start_7">command-line
			switches</A>, and by using this command in an
			input script. Each time it is used all of the style options are set,
			either to default values or to specified settings. I.e. settings from
			previous invocations do not persist across multiple invocations.
			</P>
			<P>See the <A HREF = "Section_accelerate.html">Section Accelerate</A> section of the
			manual for more details about using the various accelerator packages
			for speeding up LAMMPS simulations.
			</P>
			<HR>

			<P>The <I>cuda</I> style invokes settings associated with the use of the
			USER-CUDA package.
			</P>
			<P>The <I>Ngpus</I> argument sets the number of GPUs per node. There must be
			exactly one MPI task per GPU, as set by the mpirun or mpiexec command.
			</P>
			<P>Optional keyword/value pairs can also be specified. Each has a
			default value as listed below.
			</P>
			<P>The <I>newton</I> keyword sets the Newton flags for pairwise and bonded
			interactions to <I>off</I> or <I>on</I>, the same as the <A HREF = "newton.html">newton</A>
			command allows. The default is <I>off</I> because this will almost always
			give better performance for the USER-CUDA package. This means
			more computation is done, but less communication.
			</P>
			<P>The <I>gpuID</I> keyword allows selection of which GPUs on each node will
			be used for a simulation. GPU IDs range from 0 to N-1 where N is the
			physical number of GPUs/node. An ID is specified for each of the
			Ngpus being used. For example if you have three GPUs on a machine,
			one of which is used for the X-Server (the GPU with the ID 1) while
			the others (with IDs 0 and 2) are used for computations you would
			specify:
			</P>
			<PRE>package cuda 2 gpuID 0 2
			</PRE>
			<P>The purpose of the <I>gpuID</I> keyword is to allow two (or more)
			simulations to be run on one workstation. In that case one could set
			the first simulation to use GPU 0 and the second to use GPU 1. This is
			not necessary however, if the GPUs are in what is called <I>compute
			exclusive</I> mode. Using that setting, every process will get its own
			GPU automatically. This <I>compute exclusive</I> mode can be set as root
			using the <I>nvidia-smi</I> tool which is part of the CUDA installation.
			</P>
			<P>Also note that if the <I>gpuID</I> keyword is not used, the USER-CUDA
			package sorts existing GPUs on each node according to their number of
			multiprocessors. This way, compute GPUs will be priorized over
			X-Server GPUs.
			</P>
			<P>If the <I>timing</I> keyword is specified, detailed timing information for
			various subroutines will be output.
			</P>
			<P>If the <I>test</I> keyword is specified, information for the specified atom
			with atom-ID will be output at several points during each timestep.
			This is mainly usefull for debugging purposes. Note that the
			simulation slow down dramatically if this option is used.
			</P>
			<P>The <I>thread</I> keyword can be used to specify how GPU threads are
			assigned work during pair style force evaluation. If the value =
			<I>tpa</I>, one thread per atom is used. If the value = <I>bpa</I>, one block
			per atom is used. If the value = <I>auto</I>, a short test is performed at
			the beginning of each run to determing where <I>tpa</I> or <I>bpa</I> mode is
			faster. The result of this test is output. Since <I>auto</I> is the
			default value, it is usually not necessary to use this keyword.
			</P>
			<HR>

			<P>The <I>gpu</I> style invokes settings associated with the use of the GPU
			package.
			</P>
			<P>The <I>Ngpu</I> argument sets the number of GPUs per node. There must be
			at least as many MPI tasks per node as GPUs, as set by the mpirun or
			mpiexec command. If there are more MPI tasks (per node)
			than GPUs, multiple MPI tasks will share each GPU.
			</P>
			<P>Optional keyword/value pairs can also be specified. Each has a
			default value as listed below.
			</P>
			<P>The <I>neigh</I> keyword specifies where neighbor lists for pair style
			computation will be built. If <I>neigh</I> is <I>yes</I>, which is the default,
			neighbor list building is performed on the GPU. If <I>neigh</I> is <I>no</I>,
			neighbor list building is performed on the CPU. GPU neighbor list
			building currently cannot be used with a triclinic box. GPU neighbor
			list calculation currently cannot be used with
			<A HREF = "pair_hybrid.html">hybrid</A> pair styles. GPU neighbor lists are not
			compatible with comannds that are not GPU-enabled. When a non-GPU
			enabled command requires a neighbor list, it will also be built on the
			CPU. In these cases, it will typically be more efficient to only use
			CPU neighbor list builds.
			</P>
			<P>The <I>newton</I> keyword sets the Newton flags for pairwise (not bonded)
			interactions to <I>off</I> or <I>on</I>, the same as the <A HREF = "newton.html">newton</A>
			command allows. Currently, only an <I>off</I> value is allowed, since all
			the GPU package pair styles require this setting. This means more
			computation is done, but less communication. In the future a value of
			<I>on</I> may be allowed, so the <I>newton</I> keyword is included as an option
			for compatibility with the package command for other accelerator
			styles. Note that the newton setting for bonded interactions is not
			affected by this keyword.
			</P>
			<P>The <I>binsize</I> keyword sets the size of bins used to bin atoms in
			neighbor list builds performed on the GPU, if <I>neigh</I> = <I>yes</I> is set.
			If <I>binsize</I> is set to 0.0 (the default), then bins = the size of the
			pairwise cutoff + neighbor skin distance. This is 2x larger than the
			LAMMPS default used for neighbor list building on the CPU. This will
			be close to optimal for the GPU, so you do not normally need to use
			this keyword. Note that if you use a longer-than-usual pairwise
			cutoff, e.g. to allow for a smaller fraction of KSpace work with a
			<A HREF = "kspace_style.html">long-range Coulombic solver</A> because the GPU is
			faster at performing pairwise interactions, then it may be optimal to
			make the <I>binsize</I> smaller than the default. For example, with a
			cutoff of 20*sigma in LJ <A HREF = "units.html">units</A> and a neighbor skin
			distance of sigma, a <I>binsize</I> = 5.25*sigma can be more efficient than
			the default.
			</P>
			<P>The <I>split</I> keyword can be used for load balancing force calculations
			between CPU and GPU cores in GPU-enabled pair styles. If 0 < <I>split</I> <
			1.0, a fixed fraction of particles is offloaded to the GPU while force
			calculation for the other particles occurs simulataneously on the CPU.
			If <I>split</I> < 0.0, the optimal fraction (based on CPU and GPU timings)
			is calculated every 25 timesteps, i.e. dynamic load-balancing across
			the CPU and GPU is performed. If <I>split</I> = 1.0, all force
			calculations for GPU accelerated pair styles are performed on the GPU.
			In this case, other <A HREF = "pair_hybrid.html">hybrid</A> pair interactions,
			<A HREF = "bond_style.html">bond</A>, <A HREF = "angle_style.html">angle</A>,
			<A HREF = "dihedral_style.html">dihedral</A>, <A HREF = "improper_style.html">improper</A>, and
			<A HREF = "kspace_style.html">long-range</A> calculations can be performed on the
			CPU while the GPU is performing force calculations for the GPU-enabled
			pair style. If all CPU force computations complete before the GPU
			completes, LAMMPS will block until the GPU has finished before
			continuing the timestep.
			</P>
			<P>As an example, if you have two GPUs per node and 8 CPU cores per node,
			and would like to run on 4 nodes (32 cores) with dynamic balancing of
			force calculation across CPU and GPU cores, you could specify
			</P>
			<PRE>mpirun -np 32 -sf gpu -in in.script # launch command
			package gpu 2 split -1 # input script command
			</PRE>
			<P>In this case, all CPU cores and GPU devices on the nodes would be
			utilized. Each GPU device would be shared by 4 CPU cores. The CPU
			cores would perform force calculations for some fraction of the
			particles at the same time the GPUs performed force calculation for
			the other particles.
			</P>
			<P>The <I>gpuID</I> keyword allows selection of which GPUs on each node will
			be used for a simulation. The <I>first</I> and <I>last</I> values specify the
			GPU IDs to use (from 0 to Ngpu-1). By default, first = 0 and last =
			Ngpu-1, so that all GPUs are used, assuming Ngpu is set to the number
			of physical GPUs. If you only wish to use a subset, set Ngpu to a
			smaller number and first/last to a sub-range of the available GPUs.
			</P>
			<P>The <I>tpa</I> keyword sets the number of GPU thread per atom used to
			perform force calculations. With a default value of 1, the number of
			threads will be chosen based on the pair style, however, the value can
			be set explicitly with this keyword to fine-tune performance. For
			large cutoffs or with a small number of particles per GPU, increasing
			the value can improve performance. The number of threads per atom must
			be a power of 2 and currently cannot be greater than 32.
			</P>
			<P>The <I>blocksize</I> keyword allows you to tweak the number of threads used
			per thread block. This number should be a multiple of 32 (for GPUs)
			and its maximum depends on the specific GPU hardware. Typical choices
			are 64, 128, or 256. A larger blocksize increases occupancy of
			individual GPU cores, but reduces the total number of thread blocks,
			thus may lead to load imbalance.
			</P>
			<P>The <I>device</I> keyword can be used to tune parameters optimized for a
			specific accelerator, when using OpenCL. For CUDA, the <I>device</I>
			keyword is ignored. Currently, the device type is limited to NVIDIA
			Kepler, NVIDIA Fermi, AMD Cypress, or a generic device. More devices
			may be added later. The default device type can be specified when
			building LAMMPS with the GPU library, via settings in the
			lib/gpu/Makefile that is used.
			</P>
			<HR>

			<P>The <I>intel</I> style invokes settings associated with the use of the
			USER-INTEL package. All of its settings, except the <I>omp</I> and <I>mode</I>
			keywords, are ignored if LAMMPS was not built with Xeon Phi
			coprocessor support. All of its settings, including the <I>omp</I> and
			<I>mode</I> keyword are applicable if LAMMPS was built with coprocessor
			support.
			</P>
			<P>The <I>Nphi</I> argument sets the number of coprocessors per node.
			This can be set to any value, including 0, if LAMMPS was not
			built with coprocessor support.
			</P>
			<P>Optional keyword/value pairs can also be specified. Each has a
			default value as listed below.
			</P>
			<P>The <I>omp</I> keyword determines the number of OpenMP threads allocated
			for each MPI task when any portion of the interactions computed by a
			USER-INTEL pair style are run on the CPU. This can be the case even
			if LAMMPS was built with coprocessor support; see the <I>balance</I>
			keyword discussion below. If you are running with less MPI tasks/node
			than there are CPUs, it can be advantageous to use OpenMP threading on
			the CPUs.
			</P>
			<P>IMPORTANT NOTE: The <I>omp</I> keyword has nothing to do with coprocessor
			threads on the Xeon Phi; see the <I>tpc</I> and <I>tptask</I> keywords below for
			a discussion of coprocessor threads.
			</P>
			<P>The <I>Nthread</I> value for the <I>omp</I> keyword sets the number of OpenMP
			threads allocated for each MPI task. Setting <I>Nthread</I> = 0 (the
			default) instructs LAMMPS to use whatever value is the default for the
			given OpenMP environment. This is usually determined via the
			<I>OMP_NUM_THREADS</I> environment variable or the compiler runtime, which
			is usually a value of 1.
			</P>
			<P>For more details, including examples of how to set the OMP_NUM_THREADS
			environment variable, see the discussion of the <I>Nthreads</I> setting on
			this doc page for the "package omp" command. Nthreads is a required
			argument for the USER-OMP package. Its meaning is exactly the same
			for the USER-INTEL pacakge.
			</P>
			<P>IMPORTANT NOTE: If you build LAMMPS with both the USER-INTEL and
			USER-OMP packages, be aware that both packages allow setting of the
			<I>Nthreads</I> value via their package commands, but there is only a
			single global <I>Nthreads</I> value used by OpenMP. Thus if both package
			commands are invoked, you should insure the two values are consistent.
			If they are not, the last one invoked will take precedence, for both
			packages. Also note that if the "-sf intel" <A HREF = <A HREF = "Section_start.html#start_7">command-line"></A>
			switch</A> is used, it invokes a "package
			intel" command, followed by a "package omp" command, both with a
			setting of <I>Nthreads</I> = 0.
			</P>
			<P>The <I>mode</I> keyword determines the precision mode to use for
			computing pair style forces, either on the CPU or on the coprocessor,
			when using a USER-INTEL supported <A HREF = "pair_style.html">pair style</A>. It
			can take a value of <I>single</I>, <I>mixed</I> which is the default, or
			<I>double</I>. <I>Single</I> means single precision is used for the entire
			force calculation. <I>Mixed</I> means forces between a pair of atoms are
			computed in single precision, but accumulated and stored in double
			precision, including storage of forces, torques, energies, and virial
			quantities. <I>Double</I> means double precision is used for the entire
			force calculation.
			</P>
			<P>The <I>balance</I> keyword sets the fraction of <A HREF = "pair_style.html">pair
			style</A> work offloaded to the coprocessor for split
			values between 0.0 and 1.0 inclusive. While this fraction of work is
			running on the coprocessor, other calculations will run on the host,
			including neighbor and pair calculations that are not offloaded, as
			well as angle, bond, dihedral, kspace, and some MPI communications.
			If <I>split</I> is set to -1, the fraction of work is dynamically adjusted
			automatically throughout the run. This typically give performance
			within 5 to 10 percent of the optimal fixed fraction.
			</P>
			<P>The <I>ghost</I> keyword determines whether or not ghost atoms, i.e. atoms
			at the boundaries of proessor sub-domains, are offloaded for neighbor
			and force calculations. When the value = "no", ghost atoms are not
			offloaded. This option can reduce the amount of data transfer with
			the coprocessor and can also overlap MPI communication of forces with
			computation on the coprocessor when the <A HREF = "newton.html">newton pair</A>
			setting is "on". When the value = "yes", ghost atoms are offloaded.
			In some cases this can provide better performance, especially if the
			<I>balance</I> fraction is high.
			</P>
			<P>The <I>tpc</I> keyword sets the max # of coprocessor threads <I>Ntpc</I> that
			will run on each core of the coprocessor. The default value = 4,
			which is the number of hardware threads per core supported by the
			current generation Xeon Phi chips.
			</P>
			<P>The <I>tptask</I> keyword sets the max # of coprocessor threads (Ntptask</I>
			assigned to each MPI task. The default value = 240, which is the
			total # of threads an entire current generation Xeon Phi chip can run
			(240 = 60 cores * 4 threads/core). This means each MPI task assigned
			to the Phi will enough threads for the chip to run the max allowed,
			even if only 1 MPI task is assigned. If 8 MPI tasks are assigned to
			the Phi, each will run with 30 threads. If you wish to limit the
			number of threads per MPI task, set <I>tptask</I> to a smaller value.
			E.g. for <I>tptask</I> = 16, if 8 MPI tasks are assigned, each will run
			with 16 threads, for a total of 128.
			</P>
			<P>Note that the default settings for <I>tpc</I> and <I>tptask</I> are fine for
			most problems, regardless of how many MPI tasks you assign to a Phi.
			</P>
			<P>The <I>no_affinity</I> keyword will turn off automatic setting of core
			affinity for MPI tasks and OpenMP threads on the host when using
			offload to a coprocessor. Affinity settings are used when possible
			to prevent MPI tasks and OpenMP threads from being on separate NUMA
			domains and to prevent offload threads from interfering with other
			processes/threads used for LAMMPS.
			</P>
			<HR>

			<P>The <I>kokkos</I> style invokes settings associated with the use of the
			KOKKOS package.
			</P>
			<P>All of the settings are optional keyword/value pairs. Each has a
			default value as listed below.
			</P>
			<P>The <I>neigh</I> keyword determines how neighbor lists are built. A value
			of <I>half</I> uses half-neighbor lists, the same as used by most pair
			styles in LAMMPS. A value of <I>half/thread</I> uses a thread-safe variant
			of the half-neighbor list. It should be used instead of <I>half</I> when
			running with more than 1 threads per MPI task on a CPU. A value of
			<I>n2</I> uses an O(N^2) algorithm to build the neighbor list without
			binning, where N = # of atoms on a processor. It is typically slower
			than the other methods, which use binning.
			</P>
			<P>A value of <I>full</I> uses a full neighbor lists and is the default. This
			performs twice as much computation as the <I>half</I> option, however that
			is often a win because it is thread-safe and doesn't require atomic
			operations in the calculation of pair forces. For that reason, <I>full</I>
			is the default setting. However, when running in MPI-only mode with 1
			thread per MPI task, <I>half</I> neighbor lists will typically be faster,
			just as it is for non-accelerated pair styles.
			</P>
			<P>A value of <I>full/cluster</I> is an experimental neighbor style, where
			particles interact with all particles within a small cluster, if at
			least one of the clusters particles is within the neighbor cutoff
			range. This potentially allows for better vectorization on
			architectures such as the Intel Phi. If also reduces the size of the
			neighbor list by roughly a factor of the cluster size, thus reducing
			the total memory footprint considerably.
			</P>
			<P>The <I>newton</I> keyword sets the Newton flags for pairwise and bonded
			interactions to <I>off</I> or <I>on</I>, the same as the <A HREF = "newton.html">newton</A>
			command allows. The default is <I>off</I> because this will almost always
			give better performance for the KOKKOS package. This means more
			computation is done, but less communication. However, when running in
			MPI-only mode with 1 thread per MPI task, a value of <I>on</I> will
			typically be faster, just as it is for non-accelerated pair styles.
			</P>
			<P>The <I>binsize</I> keyword sets the size of bins used to bin atoms in
			neighbor list builds. The same value can be set by the <A HREF = "neigh_modify.html">neigh_modify
			binsize</A> command. Making it an option in the
			package kokkos command allows it to be set from the command line. The
			default value is 0.0, which means the LAMMPS default will be used,
			which is bins = 1/2 the size of the pairwise cutoff + neighbor skin
			distance. This is fine when neighbor lists are built on the CPU. For
			GPU builds, a 2x larger binsize equal to the pairwise cutoff +
			neighbor skin, is often faster, which can be set by this keyword.
			Note that if you use a longer-than-usual pairwise cutoff, e.g. to
			allow for a smaller fraction of KSpace work with a <A HREF = "kspace_style.html">long-range
			Coulombic solver</A> because the GPU is faster at
			performing pairwise interactions, then this rule of thumb may give too
			large a binsize.
			</P>
			<P>The <I>comm</I> and <I>comm/exchange</I> and <I>comm/forward</I> keywords determine
			whether the host or device performs the packing and unpacking of data
			when communicating per-atom data between processors. "Exchange"
			communication happens only on timesteps that neighbor lists are
			rebuilt. The data is only for atoms that migrate to new processors.
			"Forward" communication happens every timestep. The data is for atom
			coordinates and any other atom properties that needs to be updated for
			ghost atoms owned by each processor.
			</P>
			<P>The <I>comm</I> keyword is simply a short-cut to set the same value
			for both the <I>comm/exchange</I> and <I>comm/forward</I> keywords.
			</P>
			<P>The value options for all 3 keywords are <I>no</I> or <I>host</I> or <I>device</I>.
			A value of <I>no</I> means to use the standard non-KOKKOS method of
			packing/unpacking data for the communication. A value of <I>host</I> means
			to use the host, typically a multi-core CPU, and perform the
			packing/unpacking in parallel with threads. A value of <I>device</I> means
			to use the device, typically a GPU, to perform the packing/unpacking
			operation.
			</P>
			<P>The optimal choice for these keywords depends on the input script and
			the hardware used. The <I>no</I> value is useful for verifying that the
			Kokkos-based <I>host</I> and <I>device</I> values are working correctly. It may
			also be the fastest choice when using Kokkos styles in MPI-only mode
			(i.e. with a thread count of 1).
			</P>
			<P>When running on CPUs or Xeon Phi, the <I>host</I> and <I>device</I> values work
			identically. When using GPUs, the <I>device</I> value will typically be
			optimal if all of your styles used in your input script are supported
			by the KOKKOS package. In this case data can stay on the GPU for many
			timesteps without being moved between the host and GPU, if you use the
			<I>device</I> value. This requires that your MPI is able to access GPU
			memory directly. Currently that is true for OpenMPI 1.8 (or later
			versions), Mvapich2 1.9 (or later), and CrayMPI. If your script uses
			styles (e.g. fixes) which are not yet supported by the KOKKOS package,
			then data has to be move between the host and device anyway, so it is
			typically faster to let the host handle communication, by using the
			<I>host</I> value. Using <I>host</I> instead of <I>no</I> will enable use of
			multiple threads to pack/unpack communicated data.
			</P>
			<HR>

			<P>The <I>omp</I> style invokes settings associated with the use of the
			USER-OMP package.
			</P>
			<P>The <I>Nthread</I> argument sets the number of OpenMP threads allocated for
			each MPI task. For example, if your system has nodes with dual
			quad-core processors, it has a total of 8 cores per node. You could
			use two MPI tasks per node (e.g. using the -ppn option of the mpirun
			command in MPICH or -npernode in OpenMPI), and set <I>Nthreads</I> = 4.
			This would use all 8 cores on each node. Note that the product of MPI
			tasks * threads/task should not exceed the physical number of cores
			(on a node), otherwise performance will suffer.
			</P>
			<P>Setting <I>Nthread</I> = 0 instructs LAMMPS to use whatever value is the
			default for the given OpenMP environment. This is usually determined
			via the <I>OMP_NUM_THREADS</I> environment variable or the compiler
			runtime. Note that in most cases the default for OpenMP capable
			compilers is to use one thread for each available CPU core when
			<I>OMP_NUM_THREADS</I> is not explicitly set, which can lead to poor
			performance.
			</P>
			<P>Here are examples of how to set the environment variable when
			launching LAMMPS:
			</P>
			<PRE>env OMP_NUM_THREADS=4 lmp_machine -sf omp -in in.script
			env OMP_NUM_THREADS=2 mpirun -np 2 lmp_machine -sf omp -in in.script
			mpirun -x OMP_NUM_THREADS=2 -np 2 lmp_machine -sf omp -in in.script
			</PRE>
			<P>or you can set it permanently in your shell's start-up script.
			All three of these examples use a total of 4 CPU cores.
			</P>
			<P>Note that different MPI implementations have different ways of passing
			the OMP_NUM_THREADS environment variable to all MPI processes. The
			2nd example line above is for MPICH; the 3rd example line with -x is
			for OpenMPI. Check your MPI documentation for additional details.
			</P>
			<P>What combination of threads and MPI tasks gives the best performance
			is difficult to predict and can depend on many components of your
			input. Not all features of LAMMPS support OpenMP threading via the
			USER-OMP packaage and the parallel efficiency can be very different,
			too.
			</P>
			<P>Optional keyword/value pairs can also be specified. Each has a
			default value as listed below.
			</P>
			<P>The <I>neigh</I> keyword specifies whether neighbor list building will be
			multi-threaded in addition to force calculations. If <I>neigh</I> is set
			to <I>no</I> then neighbor list calculation is performed only by MPI tasks
			with no OpenMP threading. If <I>mode</I> is <I>yes</I> (the default), a
			multi-threaded neighbor list build is used. Using <I>neigh</I> = <I>yes</I> is
			almost always faster and should produce idential neighbor lists at the
			expense of using more memory. Specifically, neighbor list pages are
			allocated for all threads at the same time and each thread works
			within its own pages.
			</P>
			<HR>

			<P><B>Restrictions:</B>
			</P>
			<P>This command cannot be used after the simulation box is defined by a
			<A HREF = "read_data.html">read_data</A> or <A HREF = "create_box.html">create_box</A> command.
			</P>
			<P>The cuda style of this command can only be invoked if LAMMPS was built
			with the USER-CUDA package. See the <A HREF = "Section_start.html#start_3">Making
			LAMMPS</A> section for more info.
			</P>
			<P>The gpu style of this command can only be invoked if LAMMPS was built
			with the GPU package. See the <A HREF = "Section_start.html#start_3">Making
			LAMMPS</A> section for more info.
			</P>
			<P>The intel style of this command can only be invoked if LAMMPS was
			built with the USER-INTEL package. See the <A HREF = "Section_start.html#start_3">Making
			LAMMPS</A> section for more info.
			</P>
			<P>The kk style of this command can only be invoked if LAMMPS was built
			with the KOKKOS package. See the <A HREF = "Section_start.html#start_3">Making
			LAMMPS</A> section for more info.
			</P>
			<P>The omp style of this command can only be invoked if LAMMPS was built
			with the USER-OMP package. See the <A HREF = "Section_start.html#start_3">Making
			LAMMPS</A> section for more info.
			</P>
			<P><B>Related commands:</B>
			</P>
			<P><A HREF = "suffix.html">suffix</A>, "-pk" <A HREF = "Section_start.html#start_7">command-line
			setting</A>
			</P>
			<P><B>Default:</B>
			</P>
			<P>For the USER-CUDA package, the default is Ngpu = 1 and the option
			defaults are newton = off, gpuID = 0 to Ngpu-1, timing = not enabled,
			test = not enabled, and thread = auto. These settings are made
			automatically by the required "-c on" <A HREF = "Section_start.html#start_7">command-line
			switch</A>. You can change them bu using the
			package cuda command in your input script or via the "-pk cuda"
			<A HREF = "Section_start.html#start_7">command-line switch</A>.
			</P>
			<P>For the GPU package, the default is Ngpu = 1 and the option defaults
			are neigh = yes, newton = off, binsize = 0.0, split = 1.0, gpuID = 0
			to Ngpu-1, tpa = 1, and device = not used. These settings are made
			automatically if the "-sf gpu" <A HREF = "Section_start.html#start_7">command-line
			switch</A> is used. If it is not used, you
			must invoke the package gpu command in your input script or via the
			"-pk gpu" <A HREF = "Section_start.html#start_7">command-line switch</A>.
			</P>
			<P>For the USER-INTEL package, the default is Nphi = 1 and the option
			defaults are omp = 0, mode = mixed, balance = -1, tpc = 4, tptask =
			240. The default ghost option is determined by the pair style being
			used. This value is output to the screen in the offload report at the
			end of each run. Note that all of these settings, except "omp" and
			"mode", are ignored if LAMMPS was not built with Xeon Phi coprocessor
			support. These settings are made automatically if the "-sf intel"
			<A HREF = "Section_start.html#start_7">command-line switch</A> is used. If it is
			not used, you must invoke the package intel command in your input
			script or or via the "-pk intel" <A HREF = "Section_start.html#start_7">command-line
			switch</A>.
			</P>
			<P>For the KOKKOS package, the option defaults neigh = full, newton =
			off, binsize = 0.0, and comm = host. These settings are made
			automatically by the required "-k on" <A HREF = "Section_start.html#start_7">command-line
			switch</A>. You can change them bu using the
			package kokkos command in your input script or via the "-pk kokkos"
			<A HREF = "Section_start.html#start_7">command-line switch</A>.
			</P>
			<P>For the OMP package, the default is Nthreads = 0 and the option
			defaults are neigh = yes. These settings are made automatically if
			the "-sf omp" <A HREF = "Section_start.html#start_7">command-line switch</A> is
			used. If it is not used, you must invoke the package omp command in
			your input script or via the "-pk omp" <A HREF = "Section_start.html#start_7">command-line
			switch</A>.
			</P>
			</HTML>

lammps/doc/doc2/package.htmld38957c1c0b5runner

package.htmlView Options

lammps/doc/doc2/package.html
d38957c1c0b5runner

package.html
View Options