Homec4science
Diffusion elastic-yarn (master) Browse

R3704/2.3.0/hadoop-2.3.0/share/hadoop/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html
6c2cef95801emaster

Last Change
Raw File

CapacityScheduler.html
View Options

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<!-- Generated by Apache Maven Doxia at 2014-02-11 -->
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Apache Hadoop 2.3.0 - Hadoop Map Reduce Next Generation-2.3.0 - Capacity Scheduler</title>
<style type="text/css" media="all">
@import url("./css/maven-base.css");
@import url("./css/maven-theme.css");
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
<meta name="Date-Revision-yyyymmdd" content="20140211" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</head>
<body class="composite">
<div id="banner">
<a href="http://hadoop.apache.org/" id="bannerLeft">
<img src="http://hadoop.apache.org/images/hadoop-logo.jpg" alt="" />
</a>
<a href="http://www.apache.org/" id="bannerRight">
<img src="http://www.apache.org/images/asf_logo_wide.png" alt="" />
</a>
<div class="clear">
<hr/>
</div>
</div>
<div id="breadcrumbs">
<div class="xleft">
<a href="http://www.apache.org/" class="externalLink">Apache</a>
&gt;
<a href="http://hadoop.apache.org/" class="externalLink">Hadoop</a>
&gt;
<a href="../">hadoop-yarn</a>
&gt;
Apache Hadoop 2.3.0
</div>
<div class="xright"> <a href="http://wiki.apache.org/hadoop" class="externalLink">Wiki</a>
|
<a href="https://svn.apache.org/repos/asf/hadoop/" class="externalLink">SVN</a>
|
<a href="http://hadoop.apache.org/" class="externalLink">Apache Hadoop</a>
&nbsp;| Last Published: 2014-02-11
&nbsp;| Version: 2.3.0
</div>
<div class="clear">
<hr/>
</div>
</div>
<div id="leftColumn">
<div id="navcolumn">
<h5>General</h5>
<ul>
<li class="none">
<a href="../../index.html">Overview</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/SingleCluster.html">Single Node Setup</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/ClusterSetup.html">Cluster Setup</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/CommandsManual.html">Hadoop Commands Reference</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/FileSystemShell.html">File System Shell</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/Compatibility.html">Hadoop Compatibility</a>
</li>
</ul>
<h5>Common</h5>
<ul>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/CLIMiniCluster.html">CLI Mini Cluster</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/NativeLibraries.html">Native Libraries</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/Superusers.html">Superusers</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/SecureMode.html">Secure Mode</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/ServiceLevelAuth.html">Service Level Authorization</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/HttpAuthentication.html">HTTP Authentication</a>
</li>
</ul>
<h5>HDFS</h5>
<ul>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html">HDFS User Guide</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/HDFSHighAvailabilityWithQJM.html">High Availability With QJM</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/HDFSHighAvailabilityWithNFS.html">High Availability With NFS</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/Federation.html">Federation</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsSnapshots.html">HDFS Snapshots</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsDesign.html">HDFS Architecture</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsEditsViewer.html">Edits Viewer</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsImageViewer.html">Image Viewer</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html">Permissions and HDFS</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsQuotaAdminGuide.html">Quotas and HDFS</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/Hftp.html">HFTP</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/LibHdfs.html">C API libhdfs</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/WebHDFS.html">WebHDFS REST API</a>
</li>
<li class="none">
<a href="../../hadoop-hdfs-httpfs/index.html">HttpFS Gateway</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html">Short Circuit Local Reads</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.html">Centralized Cache Management</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html">HDFS NFS Gateway</a>
</li>
</ul>
<h5>MapReduce</h5>
<ul>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduce_Compatibility_Hadoop1_Hadoop2.html">Compatibilty between Hadoop 1.x and Hadoop 2.x</a>
</li>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html">Encrypted Shuffle</a>
</li>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/PluggableShuffleAndPluggableSort.html">Pluggable Shuffle/Sort</a>
</li>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistributedCacheDeploy.html">Distributed Cache Deploy</a>
</li>
</ul>
<h5>YARN</h5>
<ul>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/YARN.html">YARN Architecture</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html">Writing YARN Applications</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html">Capacity Scheduler</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/FairScheduler.html">Fair Scheduler</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/WebApplicationProxy.html">Web Application Proxy</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/YarnCommands.html">YARN Commands</a>
</li>
<li class="none">
<a href="../../hadoop-sls/SchedulerLoadSimulator.html">Scheduler Load Simulator</a>
</li>
</ul>
<h5>YARN REST APIs</h5>
<ul>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html">Introduction</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html">Resource Manager</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/NodeManagerRest.html">Node Manager</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/MapredAppMasterRest.html">MR Application Master</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/HistoryServerRest.html">History Server</a>
</li>
</ul>
<h5>Auth</h5>
<ul>
<li class="none">
<a href="../../hadoop-auth/index.html">Overview</a>
</li>
<li class="none">
<a href="../../hadoop-auth/Examples.html">Examples</a>
</li>
<li class="none">
<a href="../../hadoop-auth/Configuration.html">Configuration</a>
</li>
<li class="none">
<a href="../../hadoop-auth/BuildingIt.html">Building</a>
</li>
</ul>
<h5>Reference</h5>
<ul>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/releasenotes.html">Release Notes</a>
</li>
<li class="none">
<a href="../../api/index.html">API docs</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/CHANGES.txt">Common CHANGES.txt</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/CHANGES.txt">HDFS CHANGES.txt</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-mapreduce/CHANGES.txt">MapReduce CHANGES.txt</a>
</li>
</ul>
<h5>Configuration</h5>
<ul>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/core-default.xml">core-default.xml</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/hdfs-default.xml">hdfs-default.xml</a>
</li>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml">mapred-default.xml</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-common/yarn-default.xml">yarn-default.xml</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/DeprecatedProperties.html">Deprecated Properties</a>
</li>
</ul>
<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
<img alt="Built by Maven" src="./images/logos/maven-feather.png"/>
</a>
</div>
</div>
<div id="bodyColumn">
<div id="contentBox">
<!-- Licensed under the Apache License, Version 2.0 (the "License"); --><!-- you may not use this file except in compliance with the License. --><!-- You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. See accompanying LICENSE file. --><div class="section">
<h2>Hadoop MapReduce Next Generation - Capacity Scheduler<a name="Hadoop_MapReduce_Next_Generation_-_Capacity_Scheduler"></a></h2>
<p>[ <a href="./index.html">Go Back</a> ]</p>
<ul>
<li><a href="#Hadoop_MapReduce_Next_Generation_-_Capacity_Scheduler">Hadoop MapReduce Next Generation - Capacity Scheduler</a>
<ul>
<li><a href="#Purpose">Purpose </a></li>
<li><a href="#Overview">Overview</a></li>
<li><a href="#Features">Features</a></li>
<li><a href="#Configuration">Configuration</a></li>
<li><a href="#Changing_Queue_Configuration">Changing Queue Configuration</a></li></ul></li></ul>
<div class="section">
<h3><a name="Purpose">Purpose</a> </h3>
<p>This document describes the <tt>CapacityScheduler</tt>, a pluggable scheduler for Hadoop which allows for multiple-tenants to securely share a large cluster such that their applications are allocated resources in a timely manner under constraints of allocated capacities.</p></div>
<div class="section">
<h3><a name="Overview">Overview</a></h3>
<p>The <tt>CapacityScheduler</tt> is designed to run Hadoop applications as a shared, multi-tenant cluster in an operator-friendly manner while maximizing the throughput and the utilization of the cluster.</p>
<p>Traditionally each organization has it own private set of compute resources that have sufficient capacity to meet the organization's SLA under peak or near peak conditions. This generally leads to poor average utilization and overhead of managing multiple independent clusters, one per each organization. Sharing clusters between organizations is a cost-effective manner of running large Hadoop installations since this allows them to reap benefits of economies of scale without creating private clusters. However, organizations are concerned about sharing a cluster because they are worried about others using the resources that are critical for their SLAs. </p>
<p>The <tt>CapacityScheduler</tt> is designed to allow sharing a large cluster while giving each organization capacity guarantees. The central idea is that the available resources in the Hadoop cluster are shared among multiple organizations who collectively fund the cluster based on their computing needs. There is an added benefit that an organization can access any excess capacity not being used by others. This provides elasticity for the organizations in a cost-effective manner.</p>
<p>Sharing clusters across organizations necessitates strong support for multi-tenancy since each organization must be guaranteed capacity and safe-guards to ensure the shared cluster is impervious to single rouge application or user or sets thereof. The <tt>CapacityScheduler</tt> provides a stringent set of limits to ensure that a single application or user or queue cannot consume disproportionate amount of resources in the cluster. Also, the <tt>CapacityScheduler</tt> provides limits on initialized/pending applications from a single user and queue to ensure fairness and stability of the cluster.</p>
<p>The primary abstraction provided by the <tt>CapacityScheduler</tt> is the concept of <i>queues</i>. These queues are typically setup by administrators to reflect the economics of the shared cluster. </p>
<p>To provide further control and predictability on sharing of resources, the <tt>CapacityScheduler</tt> supports <i>hierarchical queues</i> to ensure resources are shared among the sub-queues of an organization before other queues are allowed to use free resources, there-by providing <i>affinity</i> for sharing free resources among applications of a given organization.</p></div>
<div class="section">
<h3><a name="Features">Features</a></h3>
<p>The <tt>CapacityScheduler</tt> supports the following features:</p>
<ul>
<li>Hierarchical Queues - Hierarchy of queues is supported to ensure resources are shared among the sub-queues of an organization before other queues are allowed to use free resources, there-by providing more control and predictability.</li>
<li>Capacity Guarantees - Queues are allocated a fraction of the capacity of the grid in the sense that a certain capacity of resources will be at their disposal. All applications submitted to a queue will have access to the capacity allocated to the queue. Adminstrators can configure soft limits and optional hard limits on the capacity allocated to each queue.</li>
<li>Security - Each queue has strict ACLs which controls which users can submit applications to individual queues. Also, there are safe-guards to ensure that users cannot view and/or modify applications from other users. Also, per-queue and system administrator roles are supported.</li>
<li>Elasticity - Free resources can be allocated to any queue beyond it's capacity. When there is demand for these resources from queues running below capacity at a future point in time, as tasks scheduled on these resources complete, they will be assigned to applications on queues running below the capacity (pre-emption is not supported). This ensures that resources are available in a predictable and elastic manner to queues, thus preventing artifical silos of resources in the cluster which helps utilization.</li>
<li>Multi-tenancy - Comprehensive set of limits are provided to prevent a single application, user and queue from monopolizing resources of the queue or the cluster as a whole to ensure that the cluster isn't overwhelmed.</li>
<li>Operability
<ul>
<li>Runtime Configuration - The queue definitions and properties such as capacity, ACLs can be changed, at runtime, by administrators in a secure manner to minimize disruption to users. Also, a console is provided for users and administrators to view current allocation of resources to various queues in the system. Administrators can <i>add additional queues</i> at runtime, but queues cannot be <i>deleted</i> at runtime.</li>
<li>Drain applications - Administrators can <i>stop</i> queues at runtime to ensure that while existing applications run to completion, no new applications can be submitted. If a queue is in <tt>STOPPED</tt> state, new applications cannot be submitted to <i>itself</i> or <i>any of its child queueus</i>. Existing applications continue to completion, thus the queue can be <i>drained</i> gracefully. Administrators can also <i>start</i> the stopped queues. </li></ul></li>
<li>Resource-based Scheduling - Support for resource-intensive applications, where-in a application can optionally specify higher resource-requirements than the default, there-by accomodating applications with differing resource requirements. Currently, <i>memory</i> is the the resource requirement supported.</li></ul></div>
<div class="section">
<h3><a name="Configuration">Configuration</a></h3>
<ul>
<li>Setting up <tt>ResourceManager</tt> to use <tt>CapacityScheduler</tt>
<p>To configure the <tt>ResourceManager</tt> to use the <tt>CapacityScheduler</tt>, set the following property in the <b>conf/yarn-site.xml</b>:</p>
<table border="1" class="bodyTable">
<tr class="a">
<th align="left">Property</th>
<th align="left">Value</th></tr>
<tr class="b">
<td align="left"><tt>yarn.resourcemanager.scheduler.class</tt></td>
<td align="left"><tt>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</tt></td></tr></table></li>
<li>Setting up <i>queues</i>
<p><b>conf/capacity-scheduler.xml</b> is the configuration file for the <tt>CapacityScheduler</tt>. </p>
<p>The <tt>CapacityScheduler</tt> has a pre-defined queue called <i>root</i>. All queueus in the system are children of the root queue.</p>
<p>Further queues can be setup by configuring <tt>yarn.scheduler.capacity.root.queues</tt> with a list of comma-separated child queues.</p>
<p>The configuration for <tt>CapacityScheduler</tt> uses a concept called <i>queue path</i> to configure the hierarchy of queues. The <i>queue path</i> is the full path of the queue's hierarchy, starting at <i>root</i>, with . (dot) as the delimiter.</p>
<p>A given queue's children can be defined with the configuration knob: <tt>yarn.scheduler.capacity.&lt;queue-path&gt;.queues</tt>. Children do not inherit properties directly from the parent unless otherwise noted.</p>
<p>Here is an example with three top-level child-queues <tt>a</tt>, <tt>b</tt> and <tt>c</tt> and some sub-queues for <tt>a</tt> and <tt>b</tt>:</p>
<div>
<pre>&lt;property&gt;
&lt;name&gt;yarn.scheduler.capacity.root.queues&lt;/name&gt;
&lt;value&gt;a,b,c&lt;/value&gt;
&lt;description&gt;The queues at the this level (root is the root queue).
&lt;/description&gt;
&lt;/property&gt;
&lt;property&gt;
&lt;name&gt;yarn.scheduler.capacity.root.a.queues&lt;/name&gt;
&lt;value&gt;a1,a2&lt;/value&gt;
&lt;description&gt;The queues at the this level (root is the root queue).
&lt;/description&gt;
&lt;/property&gt;
&lt;property&gt;
&lt;name&gt;yarn.scheduler.capacity.root.b.queues&lt;/name&gt;
&lt;value&gt;b1,b2,b3&lt;/value&gt;
&lt;description&gt;The queues at the this level (root is the root queue).
&lt;/description&gt;
&lt;/property&gt;</pre></div></li>
<li>Queue Properties
<ul>
<li>Resource Allocation
<table border="1" class="bodyTable">
<tr class="a">
<th align="left">Property</th>
<th align="left">Description</th></tr>
<tr class="b">
<td align="left"><tt>yarn.scheduler.capacity.&lt;queue-path&gt;.capacity</tt></td>
<td align="left">Queue <i>capacity</i> in percentage (%) as a float (e.g. 12.5). The sum of capacities for all queues, at each level, must be equal to 100. Applications in the queue may consume more resources than the queue's capacity if there are free resources, providing elasticity.</td></tr>
<tr class="a">
<td align="left"><tt>yarn.scheduler.capacity.&lt;queue-path&gt;.maximum-capacity</tt></td>
<td align="left">Maximum queue capacity in percentage (%) as a float. This limits the <i>elasticity</i> for applications in the queue. Defaults to -1 which disables it.</td></tr>
<tr class="b">
<td align="left"><tt>yarn.scheduler.capacity.&lt;queue-path&gt;.minimum-user-limit-percent</tt></td>
<td align="left">Each queue enforces a limit on the percentage of resources allocated to a user at any given time, if there is demand for resources. The user limit can vary between a minimum and maximum value. The the former (the minimum value) is set to this property value and the latter (the maximum value) depends on the number of users who have submitted applications. For e.g., suppose the value of this property is 25. If two users have submitted applications to a queue, no single user can use more than 50% of the queue resources. If a third user submits an application, no single user can use more than 33% of the queue resources. With 4 or more users, no user can use more than 25% of the queues resources. A value of 100 implies no user limits are imposed. The default is 100. Value is specified as a integer.</td></tr>
<tr class="a">
<td align="left"><tt>yarn.scheduler.capacity.&lt;queue-path&gt;.user-limit-factor</tt></td>
<td align="left">The multiple of the queue capacity which can be configured to allow a single user to acquire more resources. By default this is set to 1 which ensures that a single user can never take more than the queue's configured capacity irrespective of how idle th cluster is. Value is specified as a float.</td></tr></table></li>
<li>Running and Pending Application Limits
<p>The <tt>CapacityScheduler</tt> supports the following parameters to control the running and pending applications:</p>
<table border="1" class="bodyTable">
<tr class="a">
<th align="left">Property</th>
<th align="left">Description</th></tr>
<tr class="b">
<td align="left"><tt>yarn.scheduler.capacity.maximum-applications</tt> / <tt>yarn.scheduler.capacity.&lt;queue-path&gt;.maximum-applications</tt></td>
<td align="left">Maximum number of applications in the system which can be concurrently active both running and pending. Limits on each queue are directly proportional to their queue capacities and user limits. This is a hard limit and any applications submitted when this limit is reached will be rejected. Default is 10000. This can be set for all queues with <tt>yarn.scheduler.capacity.maximum-applications</tt> and can also be overridden on a per queue basis by setting <tt>yarn.scheduler.capacity.&lt;queue-path&gt;.maximum-applications</tt>. Integer value expected.</td></tr>
<tr class="a">
<td align="left"><tt>yarn.scheduler.capacity.maximum-am-resource-percent</tt> / <tt>yarn.scheduler.capacity.&lt;queue-path&gt;.maximum-am-resource-percent</tt></td>
<td align="left">Maximum percent of resources in the cluster which can be used to run application masters - controls number of concurrent active applications. Limits on each queue are directly proportional to their queue capacities and user limits. Specified as a float - ie 0.5 = 50%. Default is 10%. This can be set for all queues with <tt>yarn.scheduler.capacity.maximum-am-resource-percent</tt> and can also be overridden on a per queue basis by setting <tt>yarn.scheduler.capacity.&lt;queue-path&gt;.maximum-am-resource-percent</tt></td></tr></table></li>
<li>Queue Administration &amp; Permissions
<p>The <tt>CapacityScheduler</tt> supports the following parameters to the administer the queues:</p>
<table border="1" class="bodyTable">
<tr class="a">
<th align="left">Property</th>
<th align="left">Description</th></tr>
<tr class="b">
<td align="left"><tt>yarn.scheduler.capacity.&lt;queue-path&gt;.state</tt></td>
<td align="left">The <i>state</i> of the queue. Can be one of <tt>RUNNING</tt> or <tt>STOPPED</tt>. If a queue is in <tt>STOPPED</tt> state, new applications cannot be submitted to <i>itself</i> or <i>any of its child queues</i>. Thus, if the <i>root</i> queue is <tt>STOPPED</tt> no applications can be submitted to the entire cluster. Existing applications continue to completion, thus the queue can be <i>drained</i> gracefully. Value is specified as Enumeration.</td></tr>
<tr class="a">
<td align="left"><tt>yarn.scheduler.capacity.root.&lt;queue-path&gt;.acl_submit_applications</tt></td>
<td align="left">The <i>ACL</i> which controls who can <i>submit</i> applications to the given queue. If the given user/group has necessary ACLs on the given queue or <i>one of the parent queues in the hierarchy</i> they can submit applications. <i>ACLs</i> for this property <i>are</i> inherited from the parent queue if not specified.</td></tr>
<tr class="b">
<td align="left"><tt>yarn.scheduler.capacity.root.&lt;queue-path&gt;.acl_administer_queue</tt></td>
<td align="left">The <i>ACL</i> which controls who can <i>administer</i> applications on the given queue. If the given user/group has necessary ACLs on the given queue or <i>one of the parent queues in the hierarchy</i> they can administer applications. <i>ACLs</i> for this property <i>are</i> inherited from the parent queue if not specified.</td></tr></table>
<p><i>Note:</i> An <i>ACL</i> is of the form <i>user1</i>, <i>user2</i><i>space</i><i>group1</i>, <i>group2</i>. The special value of <b>*</b> implies <i>anyone</i>. The special value of <i>space</i> implies <i>no one</i>. The default is <b>*</b> for the root queue if not specified.</p></li>
<li>Reviewing the configuration of the CapacityScheduler
<p>Once the installation and configuration is completed, you can review it after starting the YARN cluster from the web-ui.</p>
<ul>
<li>Start the YARN cluster in the normal manner.</li>
<li>Open the <tt>ResourceManager</tt> web UI.</li>
<li>The <i>/scheduler</i> web-page should show the resource usages of individual queues.</li></ul></li></ul></li></ul></div>
<div class="section">
<h3><a name="Changing_Queue_Configuration">Changing Queue Configuration</a></h3>
<p>Changing queue properties and adding new queues is very simple. You need to edit <b>conf/capacity-scheduler.xml</b> and run <i>yarn rmadmin -refreshQueues</i>.</p>
<div>
<pre>$ vi $HADOOP_CONF_DIR/capacity-scheduler.xml
$ $HADOOP_YARN_HOME/bin/yarn rmadmin -refreshQueues</pre></div>
<p><i>Note:</i> Queues cannot be <i>deleted</i>, only addition of new queues is supported - the updated queue configuration should be a valid one i.e. queue-capacity at each <i>level</i> should be equal to 100%.</p></div></div>
</div>
</div>
<div class="clear">
<hr/>
</div>
<div id="footer">
<div class="xright">&#169; 2014
Apache Software Foundation
- <a href="http://maven.apache.org/privacy-policy.html">Privacy Policy</a></div>
<div class="clear">
<hr/>
</div>
</div>
</body>
</html>
c4science · Help