Homec4science
Diffusion elastic-yarn (master) Browse

R3704/2.6.3/ELASTIC_CONFIG.txt
9a0e89b0dbdcmaster

Last Change
Raw File

ELASTIC_CONFIG.txt
View Options

** REAL JOBS (EXISTING OPTIONS) **
yarn.application.classpath="$YARN_CLASSPATH"
** REAL JOBS (NEW OPTIONS) **
mapreduce.elastic.{map|reduce}.duration-ms=<map_duration_estimate> # PER-JOB
mapreduce.elastic.{map|reduce}.minimum-allocation-mb=<minimum_memory_value_for_mappers> # PER-JOB ; Overrides global value set on the RM
mapreduce.elastic.{map|reduce}.memory.min-ratio=<minimum_percentage_of_memory_acceptable> # PER-JOB ; Overrides global value set on the RM
mapreduce.elastic.{map|reduce}.penalty.model={NONE|POWER1|POWER2|SQUAREROOT|STEP|FLAT|SAWTOOTH|SPARKWC}
mapreduce.elastic.{map|reduce}.penalty.ib=<initial_bump_for_penalty>
mapreduce.elastic.{map|reduce}.penalty.elastic-disk-rate-mb=<estimated_disk_rate_when_elastic> # PER-JOB
mapreduce.elastic.job.disabled-priorities=<comma_separated_list_of_priority_values> # PER-JOB ; Values are added (not overwritten) to the global value set on the RM
mapreduce.elastic.job.strategies={RESERVED,GREEDY,SMARTG}
mapreduce.elastic.job.direct-io-spills={false|true}
yarn.nodemanager.elastic.cgroups.delta-mb=<cgroups_delta_value|-1>
yarn.nodemanager.elastic.cgroups.enabled=false
** SLS (EXISTING OPTIONS) **
yarn.sls.nm.heartbeat.interval.ms=1000
yarn.sls.nm.memory.mb=10000
yarn.sls.nm.vcores=1000
yarn.sls.am.heartbeat.interval.ms=1000
yarn.sls.am.container.resource.memory.mb=100
yarn.resourcemanager.container-tokens.master-key-rolling-interval-secs=10368000
yarn.resourcemanager.am-rm-tokens.master-key-rolling-interval-secs=120
yarn.resourcemanager.nm-tokens.master-key-rolling-interval-secs=3801600000
yarn.nm.liveness-monitor.expiry-interval-ms=1900800000
yarn.am.liveness-monitor.expiry-interval-ms=1900800000
** SLS (NEW OPTIONS) **
yarn.sls.elastic.run-error-percentage=<percentage_error_to_inject_in_running_time>
yarn.sls.elastic.startport=<startport_to_avoid_port_overlaps>
yarn.nodes.disable-dns-lookup=true
** COMMON **
yarn.elastic.strategies={RESERVED,GREEDY,SMARTG,RACE,RACENG,RACE_LOCKSTEP,PEEK} # PEEK needs to be used together with GREEDY/SMARTG
yarn.elastic.penalty.model={NONE|POWER1|POWER2|SQUAREROOT|STEP|FLAT|SAWTOOTH|SPARKWC}
yarn.elastic.penalty.ib=<initial_bump_for_penalty>
yarn.elastic.memory.min-ratio=<minimum_percentage_of_memory_acceptable>
yarn.elastic.disabled-priorities={0, 1} # Implicitly disable elasticity for AM containers
yarn.elastic.max-elastic-per-node=<max_number_of_concurrent_elastic_containers_per_node>
yarn.elastic.max-disk-rate-per-node-mb=<max_disk_rate_of_concurrent_elastic_containers_per_node>
yarn.elastic.race.binary-simulation.timeout-ms=<timeout_in_millis>
yarn.elastic.race.min-improvement-percent=<min_percentage_elastic_improvement_to_use_elastic>
yarn.elastic.fill-reserved-gaps={false|true} # ONLY USE WITH PEEK!
yarn.scheduler.increment-allocation-mb=100
yarn.scheduler.minimum-allocation-mb=100
c4science · Help