Homec4science
Diffusion elastic-yarn (future) Browse

R3704/hadoop-2.3.0/share/hadoop/hadoop-project-dist/hadoop-common/ServiceLevelAuth.html
c0a03c32f340future

Last Change
Raw File

ServiceLevelAuth.html
View Options

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<!-- Generated by Apache Maven Doxia at 2014-02-11 -->
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Apache Hadoop 2.3.0 - Service Level Authorization Guide</title>
<style type="text/css" media="all">
@import url("./css/maven-base.css");
@import url("./css/maven-theme.css");
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
<meta name="Date-Revision-yyyymmdd" content="20140211" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</head>
<body class="composite">
<div id="banner">
<a href="http://hadoop.apache.org/" id="bannerLeft">
<img src="http://hadoop.apache.org/images/hadoop-logo.jpg" alt="" />
</a>
<a href="http://www.apache.org/" id="bannerRight">
<img src="http://www.apache.org/images/asf_logo_wide.png" alt="" />
</a>
<div class="clear">
<hr/>
</div>
</div>
<div id="breadcrumbs">
<div class="xleft">
<a href="http://www.apache.org/" class="externalLink">Apache</a>
&gt;
<a href="http://hadoop.apache.org/" class="externalLink">Hadoop</a>
&gt;
<a href="../">Apache Hadoop Project Dist POM</a>
&gt;
Apache Hadoop 2.3.0
</div>
<div class="xright"> <a href="http://wiki.apache.org/hadoop" class="externalLink">Wiki</a>
|
<a href="https://svn.apache.org/repos/asf/hadoop/" class="externalLink">SVN</a>
|
<a href="http://hadoop.apache.org/" class="externalLink">Apache Hadoop</a>
&nbsp;| Last Published: 2014-02-11
&nbsp;| Version: 2.3.0
</div>
<div class="clear">
<hr/>
</div>
</div>
<div id="leftColumn">
<div id="navcolumn">
<h5>General</h5>
<ul>
<li class="none">
<a href="../../index.html">Overview</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/SingleCluster.html">Single Node Setup</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/ClusterSetup.html">Cluster Setup</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/CommandsManual.html">Hadoop Commands Reference</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/FileSystemShell.html">File System Shell</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/Compatibility.html">Hadoop Compatibility</a>
</li>
</ul>
<h5>Common</h5>
<ul>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/CLIMiniCluster.html">CLI Mini Cluster</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/NativeLibraries.html">Native Libraries</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/Superusers.html">Superusers</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/SecureMode.html">Secure Mode</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/ServiceLevelAuth.html">Service Level Authorization</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/HttpAuthentication.html">HTTP Authentication</a>
</li>
</ul>
<h5>HDFS</h5>
<ul>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html">HDFS User Guide</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/HDFSHighAvailabilityWithQJM.html">High Availability With QJM</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/HDFSHighAvailabilityWithNFS.html">High Availability With NFS</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/Federation.html">Federation</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsSnapshots.html">HDFS Snapshots</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsDesign.html">HDFS Architecture</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsEditsViewer.html">Edits Viewer</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsImageViewer.html">Image Viewer</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html">Permissions and HDFS</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsQuotaAdminGuide.html">Quotas and HDFS</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/Hftp.html">HFTP</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/LibHdfs.html">C API libhdfs</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/WebHDFS.html">WebHDFS REST API</a>
</li>
<li class="none">
<a href="../../hadoop-hdfs-httpfs/index.html">HttpFS Gateway</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html">Short Circuit Local Reads</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.html">Centralized Cache Management</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html">HDFS NFS Gateway</a>
</li>
</ul>
<h5>MapReduce</h5>
<ul>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduce_Compatibility_Hadoop1_Hadoop2.html">Compatibilty between Hadoop 1.x and Hadoop 2.x</a>
</li>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html">Encrypted Shuffle</a>
</li>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/PluggableShuffleAndPluggableSort.html">Pluggable Shuffle/Sort</a>
</li>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistributedCacheDeploy.html">Distributed Cache Deploy</a>
</li>
</ul>
<h5>YARN</h5>
<ul>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/YARN.html">YARN Architecture</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html">Writing YARN Applications</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html">Capacity Scheduler</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/FairScheduler.html">Fair Scheduler</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/WebApplicationProxy.html">Web Application Proxy</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/YarnCommands.html">YARN Commands</a>
</li>
<li class="none">
<a href="../../hadoop-sls/SchedulerLoadSimulator.html">Scheduler Load Simulator</a>
</li>
</ul>
<h5>YARN REST APIs</h5>
<ul>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html">Introduction</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html">Resource Manager</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/NodeManagerRest.html">Node Manager</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/MapredAppMasterRest.html">MR Application Master</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/HistoryServerRest.html">History Server</a>
</li>
</ul>
<h5>Auth</h5>
<ul>
<li class="none">
<a href="../../hadoop-auth/index.html">Overview</a>
</li>
<li class="none">
<a href="../../hadoop-auth/Examples.html">Examples</a>
</li>
<li class="none">
<a href="../../hadoop-auth/Configuration.html">Configuration</a>
</li>
<li class="none">
<a href="../../hadoop-auth/BuildingIt.html">Building</a>
</li>
</ul>
<h5>Reference</h5>
<ul>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/releasenotes.html">Release Notes</a>
</li>
<li class="none">
<a href="../../api/index.html">API docs</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/CHANGES.txt">Common CHANGES.txt</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/CHANGES.txt">HDFS CHANGES.txt</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-mapreduce/CHANGES.txt">MapReduce CHANGES.txt</a>
</li>
</ul>
<h5>Configuration</h5>
<ul>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/core-default.xml">core-default.xml</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/hdfs-default.xml">hdfs-default.xml</a>
</li>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml">mapred-default.xml</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-common/yarn-default.xml">yarn-default.xml</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/DeprecatedProperties.html">Deprecated Properties</a>
</li>
</ul>
<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
<img alt="Built by Maven" src="./images/logos/maven-feather.png"/>
</a>
</div>
</div>
<div id="bodyColumn">
<div id="contentBox">
<!-- Licensed under the Apache License, Version 2.0 (the "License"); --><!-- you may not use this file except in compliance with the License. --><!-- You may obtain a copy of the License at --><!-- --><!-- http://www.apache.org/licenses/LICENSE-2.0 --><!-- --><!-- Unless required by applicable law or agreed to in writing, software --><!-- distributed under the License is distributed on an "AS IS" BASIS, --><!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --><!-- See the License for the specific language governing permissions and --><!-- limitations under the License. See accompanying LICENSE file. --><div class="section">
<h2>Service Level Authorization Guide<a name="Service_Level_Authorization_Guide"></a></h2>
<ul>
<li><a href="#Service_Level_Authorization_Guide">Service Level Authorization Guide</a>
<ul>
<li><a href="#Purpose">Purpose</a></li>
<li><a href="#Prerequisites">Prerequisites</a></li>
<li><a href="#Overview">Overview</a></li>
<li><a href="#Configuration">Configuration</a>
<ul>
<li><a href="#Enable_Service_Level_Authorization">Enable Service Level Authorization</a></li>
<li><a href="#Hadoop_Services_and_Configuration_Properties">Hadoop Services and Configuration Properties</a></li>
<li><a href="#Access_Control_Lists">Access Control Lists</a></li>
<li><a href="#Refreshing_Service_Level_Authorization_Configuration">Refreshing Service Level Authorization Configuration</a></li>
<li><a href="#Examples">Examples</a></li></ul></li></ul></li></ul>
<div class="section">
<h3>Purpose<a name="Purpose"></a></h3>
<p>This document describes how to configure and manage Service Level Authorization for Hadoop.</p></div>
<div class="section">
<h3>Prerequisites<a name="Prerequisites"></a></h3>
<p>Make sure Hadoop is installed, configured and setup correctly. For more information see:</p>
<ul>
<li><a href="./SingleCluster.html">Single Node Setup</a> for first-time users.</li>
<li><a href="./ClusterSetup.html">Cluster Setup</a> for large, distributed clusters.</li></ul></div>
<div class="section">
<h3>Overview<a name="Overview"></a></h3>
<p>Service Level Authorization is the initial authorization mechanism to ensure clients connecting to a particular Hadoop service have the necessary, pre-configured, permissions and are authorized to access the given service. For example, a MapReduce cluster can use this mechanism to allow a configured list of users/groups to submit jobs.</p>
<p>The <tt>$<a name="HADOOP_CONF_DIR">HADOOP_CONF_DIR</a>/hadoop-policy.xml</tt> configuration file is used to define the access control lists for various Hadoop services.</p>
<p>Service Level Authorization is performed much before to other access control checks such as file-permission checks, access control on job queues etc.</p></div>
<div class="section">
<h3>Configuration<a name="Configuration"></a></h3>
<p>This section describes how to configure service-level authorization via the configuration file <tt>$<a name="HADOOP_CONF_DIR">HADOOP_CONF_DIR</a>/hadoop-policy.xml</tt>.</p>
<div class="section">
<h4>Enable Service Level Authorization<a name="Enable_Service_Level_Authorization"></a></h4>
<p>By default, service-level authorization is disabled for Hadoop. To enable it set the configuration property hadoop.security.authorization to true in <tt>$<a name="HADOOP_CONF_DIR">HADOOP_CONF_DIR</a>/core-site.xml</tt>.</p></div>
<div class="section">
<h4>Hadoop Services and Configuration Properties<a name="Hadoop_Services_and_Configuration_Properties"></a></h4>
<p>This section lists the various Hadoop services and their configuration knobs:</p>
<table border="1" class="bodyTable">
<tr class="a">
<th align="left">Property</th>
<th align="left">Service</th></tr>
<tr class="b">
<td align="left">security.client.protocol.acl</td>
<td align="left">ACL for ClientProtocol, which is used by user code via the DistributedFileSystem.</td></tr>
<tr class="a">
<td align="left">security.client.datanode.protocol.acl</td>
<td align="left">ACL for ClientDatanodeProtocol, the client-to-datanode protocol for block recovery.</td></tr>
<tr class="b">
<td align="left">security.datanode.protocol.acl</td>
<td align="left">ACL for DatanodeProtocol, which is used by datanodes to communicate with the namenode.</td></tr>
<tr class="a">
<td align="left">security.inter.datanode.protocol.acl</td>
<td align="left">ACL for InterDatanodeProtocol, the inter-datanode protocol for updating generation timestamp.</td></tr>
<tr class="b">
<td align="left">security.namenode.protocol.acl</td>
<td align="left">ACL for NamenodeProtocol, the protocol used by the secondary namenode to communicate with the namenode.</td></tr>
<tr class="a">
<td align="left">security.inter.tracker.protocol.acl</td>
<td align="left">ACL for InterTrackerProtocol, used by the tasktrackers to communicate with the jobtracker.</td></tr>
<tr class="b">
<td align="left">security.job.submission.protocol.acl</td>
<td align="left">ACL for JobSubmissionProtocol, used by job clients to communciate with the jobtracker for job submission, querying job status etc.</td></tr>
<tr class="a">
<td align="left">security.task.umbilical.protocol.acl</td>
<td align="left">ACL for TaskUmbilicalProtocol, used by the map and reduce tasks to communicate with the parent tasktracker.</td></tr>
<tr class="b">
<td align="left">security.refresh.policy.protocol.acl</td>
<td align="left">ACL for RefreshAuthorizationPolicyProtocol, used by the dfsadmin and mradmin commands to refresh the security policy in-effect.</td></tr>
<tr class="a">
<td align="left">security.ha.service.protocol.acl</td>
<td align="left">ACL for HAService protocol used by HAAdmin to manage the active and stand-by states of namenode.</td></tr></table></div>
<div class="section">
<h4>Access Control Lists<a name="Access_Control_Lists"></a></h4>
<p><tt>$<a name="HADOOP_CONF_DIR">HADOOP_CONF_DIR</a>/hadoop-policy.xml</tt> defines an access control list for each Hadoop service. Every access control list has a simple format:</p>
<p>The list of users and groups are both comma separated list of names. The two lists are separated by a space.</p>
<p>Example: <tt>user1,user2 group1,group2</tt>.</p>
<p>Add a blank at the beginning of the line if only a list of groups is to be provided, equivalently a comman-separated list of users followed by a space or nothing implies only a set of given users.</p>
<p>A special value of <tt>*</tt> implies that all users are allowed to access the service.</p></div>
<div class="section">
<h4>Refreshing Service Level Authorization Configuration<a name="Refreshing_Service_Level_Authorization_Configuration"></a></h4>
<p>The service-level authorization configuration for the NameNode and JobTracker can be changed without restarting either of the Hadoop master daemons. The cluster administrator can change <tt>$<a name="HADOOP_CONF_DIR">HADOOP_CONF_DIR</a>/hadoop-policy.xml</tt> on the master nodes and instruct the NameNode and JobTracker to reload their respective configurations via the <tt>-refreshServiceAcl</tt> switch to <tt>dfsadmin</tt> and <tt>mradmin</tt> commands respectively.</p>
<p>Refresh the service-level authorization configuration for the NameNode:</p>
<div>
<pre> $ bin/hadoop dfsadmin -refreshServiceAcl</pre></div>
<p>Refresh the service-level authorization configuration for the JobTracker:</p>
<div>
<pre> $ bin/hadoop mradmin -refreshServiceAcl</pre></div>
<p>Of course, one can use the <tt>security.refresh.policy.protocol.acl</tt> property in <tt>$<a name="HADOOP_CONF_DIR">HADOOP_CONF_DIR</a>/hadoop-policy.xml</tt> to restrict access to the ability to refresh the service-level authorization configuration to certain users/groups.</p></div>
<div class="section">
<h4>Examples<a name="Examples"></a></h4>
<p>Allow only users <tt>alice</tt>, <tt>bob</tt> and users in the <tt>mapreduce</tt> group to submit jobs to the MapReduce cluster:</p>
<div>
<pre>&lt;property&gt;
&lt;name&gt;security.job.submission.protocol.acl&lt;/name&gt;
&lt;value&gt;alice,bob mapreduce&lt;/value&gt;
&lt;/property&gt;</pre></div>
<p>Allow only DataNodes running as the users who belong to the group datanodes to communicate with the NameNode:</p>
<div>
<pre>&lt;property&gt;
&lt;name&gt;security.datanode.protocol.acl&lt;/name&gt;
&lt;value&gt;datanodes&lt;/value&gt;
&lt;/property&gt;</pre></div>
<p>Allow any user to talk to the HDFS cluster as a DFSClient:</p>
<div>
<pre>&lt;property&gt;
&lt;name&gt;security.client.protocol.acl&lt;/name&gt;
&lt;value&gt;*&lt;/value&gt;
&lt;/property&gt;</pre></div></div></div></div>
</div>
</div>
<div class="clear">
<hr/>
</div>
<div id="footer">
<div class="xright">&#169; 2014
Apache Software Foundation
- <a href="http://maven.apache.org/privacy-policy.html">Privacy Policy</a></div>
<div class="clear">
<hr/>
</div>
</div>
</body>
</html>
c4science · Help