Ranger2.1编译笔记

本文记录了 ranger2.1版本的编译过程,与遇到的一些bug的解决方法。

准备

1 外部工具依赖

需要linux环境,保证python命令可用,如果只装了python3,推荐

1
2
apt install -y python-is-python3

另外还需要安装gcc

1
2
apt install gcc -y

2 排除kylin模块

kylin2.6.4依赖calcite-linq4j-1.16.0-kylin-r2
这个jar包在外部maven仓库是找不到的,导致build失败,参考:
https://issues.apache.org/jira/browse/RANGER-2994
https://issues.apache.org/jira/browse/RANGER-2999

因为是kylin本身的依赖问题,不是ranger的锅,这里谴责一下kylin然后对其选择性放弃
放弃方法:在pom.xml将其所有的module注释掉,如下

1
2
<!--                <module>plugin-kylin</module>-->
<!-- <module>ranger-kylin-plugin-shim</module>-->

3 ranger-examples-distro错误

参考:https://github.com/apache/ranger/pull/71
根据issue,修改 sampleapp.xml 和 plugin-sampleapp.xml

4 ranger client

官方的rangerClient有bug,新版本已修复,但还没发布,需要自行编译
步骤:

  1. 参考官方仓库 ranger/intg/src/main/java 工程,可直接搬运过来,原来的目录要先删掉
  2. 将 ranger/intg/pom.xml 中 ranger-plugins-common 的版本从${project.version} 改为2.1.0(即公网仓库能找到的最新版本)

执行编译

1 编译各个组件 tar.gz包

在ranger工程下执行,执行成功可以在target下看到包

1
2
mvn clean compile package assembly:assembly -DskipTests -DskipJSTests -Dpmd.skip=true

日志参考:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for ranger 2.1.1-SNAPSHOT:
[INFO]
[INFO] ranger ............................................. SUCCESS [02:31 min]
[INFO] Jdbc SQL Connector ................................. SUCCESS [ 12.635 s]
[INFO] Credential Support ................................. SUCCESS [ 14.241 s]
[INFO] Audit Component .................................... SUCCESS [ 40.494 s]
[INFO] ranger-plugin-classloader .......................... SUCCESS [ 8.902 s]
[INFO] Common library for Plugins ......................... SUCCESS [02:43 min]
[INFO] ranger-intg ........................................ SUCCESS [ 15.121 s]
[INFO] Installer Support Component ........................ SUCCESS [ 4.590 s]
[INFO] Credential Builder ................................. SUCCESS [ 7.686 s]
[INFO] Embedded Web Server Invoker ........................ SUCCESS [ 12.830 s]
[INFO] Key Management Service ............................. SUCCESS [ 52.264 s]
[INFO] HBase Security Plugin Shim ......................... SUCCESS [ 15.598 s]
[INFO] HBase Security Plugin .............................. SUCCESS [ 32.827 s]
[INFO] Hdfs Security Plugin ............................... SUCCESS [ 34.502 s]
[INFO] Hive Security Plugin ............................... SUCCESS [ 35.109 s]
[INFO] Knox Security Plugin Shim .......................... SUCCESS [ 11.906 s]
[INFO] Knox Security Plugin ............................... SUCCESS [ 49.254 s]
[INFO] Storm Security Plugin .............................. SUCCESS [ 23.171 s]
[INFO] YARN Security Plugin ............................... SUCCESS [ 16.011 s]
[INFO] Ozone Security Plugin .............................. SUCCESS [ 16.418 s]
[INFO] Ranger Util ........................................ SUCCESS [01:43 min]
[INFO] Unix Authentication Client ......................... SUCCESS [ 8.165 s]
[INFO] Security Admin Web Application ..................... SUCCESS [21:33 min]
[INFO] KAFKA Security Plugin .............................. SUCCESS [ 21.139 s]
[INFO] SOLR Security Plugin ............................... SUCCESS [ 16.487 s]
[INFO] NiFi Security Plugin ............................... SUCCESS [ 18.495 s]
[INFO] NiFi Registry Security Plugin ...................... SUCCESS [ 15.196 s]
[INFO] Kudu Security Plugin ............................... SUCCESS [ 9.105 s]
[INFO] Unix User Group Synchronizer ....................... SUCCESS [ 47.248 s]
[INFO] Ldap Config Check Tool ............................. SUCCESS [ 8.631 s]
[INFO] Unix Authentication Service ........................ SUCCESS [ 9.789 s]
[INFO] Unix Native Authenticator .......................... SUCCESS [ 4.242 s]
[INFO] KMS Security Plugin ................................ SUCCESS [ 34.582 s]
[INFO] Tag Synchronizer ................................... SUCCESS [ 28.172 s]
[INFO] Hdfs Security Plugin Shim .......................... SUCCESS [ 7.847 s]
[INFO] Hive Security Plugin Shim .......................... SUCCESS [ 13.040 s]
[INFO] YARN Security Plugin Shim .......................... SUCCESS [ 10.582 s]
[INFO] OZONE Security Plugin Shim ......................... SUCCESS [ 16.250 s]
[INFO] Storm Security Plugin shim ......................... SUCCESS [ 10.807 s]
[INFO] KAFKA Security Plugin Shim ......................... SUCCESS [ 8.435 s]
[INFO] SOLR Security Plugin Shim .......................... SUCCESS [ 10.037 s]
[INFO] Atlas Security Plugin Shim ......................... SUCCESS [ 9.938 s]
[INFO] KMS Security Plugin Shim ........................... SUCCESS [ 13.735 s]
[INFO] ranger-examples .................................... SUCCESS [ 2.577 s]
[INFO] Ranger Examples - Conditions and ContextEnrichers .. SUCCESS [ 13.963 s]
[INFO] Ranger Examples - SampleApp ........................ SUCCESS [ 14.751 s]
[INFO] Ranger Examples - Ranger Plugin for SampleApp ...... SUCCESS [ 9.067 s]
[INFO] sample-client ...................................... SUCCESS [ 21.965 s]
[INFO] Apache Ranger Examples Distribution ................ SUCCESS [ 17.195 s]
[INFO] Ranger Tools ....................................... SUCCESS [ 32.274 s]
[INFO] Atlas Security Plugin .............................. SUCCESS [ 15.610 s]
[INFO] SchemaRegistry Security Plugin ..................... SUCCESS [ 33.643 s]
[INFO] Sqoop Security Plugin .............................. SUCCESS [ 32.979 s]
[INFO] Sqoop Security Plugin Shim ......................... SUCCESS [ 8.851 s]
[INFO] Presto Security Plugin ............................. SUCCESS [ 36.558 s]
[INFO] Presto Security Plugin Shim ........................ SUCCESS [01:06 min]
[INFO] Elasticsearch Security Plugin Shim ................. SUCCESS [ 14.989 s]
[INFO] Elasticsearch Security Plugin ...................... SUCCESS [ 13.180 s]
[INFO] Apache Ranger Distribution ......................... SUCCESS [38:46 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:24 h
[INFO] Finished at: 2021-05-27T15:47:58+08:00
[INFO] ------------------------------------------------------------------------

2 编译 ranger_client jar包

在ranger/intg下执行

1
mvn clean compile package install -DskipTests

可以在 ranger/intg/target 看到生成后的jar包


Ranger2.1编译笔记
https://linshenkx.github.io/ranger-compile/
作者
林泽浩
发布于
2021年6月21日
许可协议