Hive MetaStore 认证和权限模型

hive.metastore.pre.event.listeners

hive.metastore.pre.event.listeners 是 meta store 实现权限的主要途径。可以设置多个，用 ‘,’ 分开。

    hive.metastore.pre.event.listeners
    
    List of comma separated listeners for metastore events.
  

HivemetaStore$HMSHandler#init

在 handler 的 init 方法里，创建 preListeners

preListeners = metaStoreUtils.getmetaStoreListeners(metaStorePreEventListener.class,
          conf, metastoreConf.getVar(conf, ConfVars.PRE_EVENT_LISTENERS));
preListeners.add(0, new TransactionalValidationListener(conf));

get_database

当 HMSHandler 处理请求时，调用 firePreEvent，传递相应的 event，以 get_database 的代码如下：

@Override
    public Database get_database(final String name) throws NoSuchObjectException, metaException {
      startFunction("get_database", ": " + name);
      Database db = null;
      Exception ex = null;
      try {
        String[] parsedDbName = parseDbName(name, conf);
        db = get_database_core(parsedDbName[CAT_NAME], parsedDbName[DB_NAME]);
        firePreEvent(new PreReadDatabaseEvent(db, this));
      } catch (metaException|NoSuchObjectException e) {
        ex = e;
        throw e;
      } finally {
        endFunction("get_database", db != null, ex);
      }
      return db;
    }

PreReadDatabaseEvent

Event type 是 PreEventType.READ_DATAbase。

public class PreReadDatabaseEvent extends PreEventContext {
  private final Database db;

  public PreReadDatabaseEvent(Database db, IHMSHandler handler) {
    super(PreEventType.READ_DATAbase, handler);
    this.db = db;
  }
}

firePreEvent

firePreEvent 调用

private void firePreEvent(PreEventContext event) throws metaException {
      for (metaStorePreEventListener listener : preListeners) {
        try {
          listener.onEvent(event);
        } catch (NoSuchObjectException e) {
        // ...
      }
    }

认证是获取用户信息的方式

默认是 通过 UGI 获取 用户信息。 HadoopDefaultmetastoreAuthenticator

    hive.security.metastore.authenticator.manager
    org.apache.hadoop.hive.ql.security.HadoopDefaultmetastoreAuthenticator
    
      authenticator manager class name to be used in the metastore for authentication. 
      The user defined authenticator should implement interface org.apache.hadoop.hive.ql.security.HiveAuthenticationProvider.
    
  

AuthorizationPreEventListener

当 listeners 配置 时，在 meta store 端实现权限控制。

      hive.metastore.pre.event.listeners
      org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener

AuthorizationPreEventListener 用配置 hive.security.metastore.authenticator.manager 获取用户信息。用 hive.security.metastore.authorization.manager 实现鉴权。

  hive.security.metastore.authenticator.manager
  org.apache.hadoop.hive.ql.security.HadoopDefaultmetastoreAuthenticator
  
    authenticator manager class name to be used in the metastore for authentication. 
    The user defined authenticator should implement interface org.apache.hadoop.hive.ql.security.HiveAuthenticationProvider.
  


  hive.security.metastore.authorization.manager
  org.apache.hadoop.hive.ql.security.authorization.DefaultHivemetastoreAuthorizationProvider
  
    Names of authorization manager classes (comma separated) to be used in the metastore
    for authorization. The user defined authorization class should implement interface
    org.apache.hadoop.hive.ql.security.authorization.HivemetastoreAuthorizationProvider.
    All authorization manager classes have to successfully authorize the metastore API
    call for the command execution to be allowed.
  

AuthorizationPreEventListener 使用线程本地变量存储数据

private static final ThreadLocal tConfig = new ThreadLocal() {
    @Override
    protected Configuration initialValue() {
      return new HiveConf(AuthorizationPreEventListener.class);
    }
  };

  private static final ThreadLocal tAuthenticator
      = new ThreadLocal() {
    @Override
    protected HivemetastoreAuthenticationProvider initialValue() {
      try {
        return  (HivemetastoreAuthenticationProvider) HiveUtils.getAuthenticator(
            tConfig.get(), HiveConf.ConfVars.HIVE_metaSTORE_AUTHENTICATOR_MANAGER);
      } catch (HiveException he) {
        throw new IllegalStateException("Authentication provider instantiation failure",he);
      }
    }
  };

  private static final ThreadLocal> tAuthorizers
      = new ThreadLocal>() {
    @Override
    protected List initialValue() {
      try {
        return  HiveUtils.getmetaStoreAuthorizeProviderManagers(
            tConfig.get(), HiveConf.ConfVars.HIVE_metaSTORE_AUTHORIZATION_MANAGER, tAuthenticator.get());
      } catch (HiveException he) {
        throw new IllegalStateException("Authorization provider instantiation failure",he);
      }
    }
  };

  private static final ThreadLocal tConfigSetOnAuths = new ThreadLocal() {
    @Override
    protected Boolean initialValue() {
      return false;
    }
  };

AuthorizationPreEventListener#onEvent 实现

public void onEvent(PreEventContext context) throws metaException, NoSuchObjectException,
      InvalidOperationException {
      switch (context.getEventType()) {
    case CREATE_TABLE:
      authorizeCreateTable((PreCreateTableEvent)context);
      break;
      // ...
}      

authorizeCreateTable

authorizeCreateTable 调用每个 authorizer 的 authorize 方法进行鉴权。

private void authorizeCreateTable(PreCreateTableEvent context)
      throws InvalidOperationException, metaException {
    try {
      org.apache.hadoop.hive.ql.metadata.Table wrappedTable = new TableWrapper(context.getTable());
      for (HivemetastoreAuthorizationProvider authorizer : tAuthorizers.get()) {
        authorizer.authorize(wrappedTable,
            HiveOperation.CREATETABLE.getInputRequiredPrivileges(),
            HiveOperation.CREATETABLE.getOutputRequiredPrivileges());
      }
    } catch (AuthorizationException e) {
      throw invalidOperationException(e);
    } catch (HiveException e) {
      throw metaException(e);
    }
  }

HivemetastoreAuthorizationProvider 的可选配置项

org.apache.hadoop.hive.ql.security.authorization.metaStoreAuthzAPIAuthorizerEmbedOnlyorg.apache.hadoop.hive.ql.security.authorization.DefaultHivemetastoreAuthorizationProviderorg.apache.hadoop.hive.ql.security.authorization.StoragebasedAuthorizationProvider metaStoreAuthzAPIAuthorizerEmbedonly

对任何操作都不做鉴权，用于把 meta store 和 hive server 或者 hive cli 启动在同一 JVM 内。

DefaultHivemetastoreAuthorizationProvider

DefaultHivemetastoreAuthorizationProvider 继承 BitSetCheckedAuthorizationProvider

public class DefaultHivemetastoreAuthorizationProvider extends BitSetCheckedAuthorizationProvider
  implements HivemetastoreAuthorizationProvider {
}

public abstract class BitSetCheckedAuthorizationProvider extends
    HiveAuthorizationProviderbase {
}

BitSetChecker

静态内部类 BitSetChecker，并且有创建 BitSetChecker 的静态方法。 getBitSetChecker

static class BitSetChecker {

    boolean[] inputCheck = null;
    boolean[] outputCheck = null;

    public static BitSetChecker getBitSetChecker(Privilege[] inputRequiredPriv,
        Privilege[] outputRequiredPriv) {
      BitSetChecker checker = new BitSetChecker();
      if (inputRequiredPriv != null) {
        checker.inputCheck = new boolean[inputRequiredPriv.length];
        for (int i = 0; i < checker.inputCheck.length; i++) {
          checker.inputCheck[i] = false;
        }
      }
      if (outputRequiredPriv != null) {
        checker.outputCheck = new boolean[outputRequiredPriv.length];
        for (int i = 0; i < checker.outputCheck.length; i++) {
          checker.outputCheck[i] = false;
        }
      }
      return checker;
    }
  }

authorize 进行全局权限判断

@Override
  public void authorize(Privilege[] inputRequiredPriv,
      Privilege[] outputRequiredPriv) throws HiveException, AuthorizationException {

    BitSetChecker checker = BitSetChecker.getBitSetChecker(inputRequiredPriv,
        outputRequiredPriv);
    boolean[] inputCheck = checker.inputCheck;
    boolean[] outputCheck = checker.outputCheck;

    authorizeUserPriv(inputRequiredPriv, inputCheck, outputRequiredPriv,
        outputCheck);
    checkAndThrowAuthorizationException(inputRequiredPriv, outputRequiredPriv,
        inputCheck, outputCheck, null, null, null, null);
  }