基于ARM(aarch64)平台编译Protobuf 2.5支持Apache Hive

背景

目前很多公司都开始使用基于ARM(aarch64)平台的服务器用来降低硬件成本，随着平台的变更相应的后端服务、中间件和各类集群的基础环境都需要支持aarch64平台。

大数据集群是一个相对来说较重的资产，而且现有的x64架构服务器集群已经支持的非常完善了，而ARM平台的支持还没有太完善，本文将基于ARM(aarch64)平台展示如何编译Protobuf以支持Apache Hive。

编译Apache Hive

[root@workstation apache-hive-4.0.0]# mvn -B -e clean package -Pdist -DskipTests

直接执行Hive编译命令会报下面的错误：

[ERROR] Failed to execute goal com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project hive-standalone-metastore-common: Error resolving artifact: com.google.protobuf:protoc:2.5.0: Could not find artifact com.google.protobuf:protoc:exe:linux-aarch_64:2.6.1-build3 in central (https://repo.maven.apache.org/maven2)

这是由于Apache Hive依赖的protoc-2.5.0库在Maven的central repo中没有ARM64版本的导致的，为此可以提供两种方式来规避上述问题：

• 手动修改和编译源码，支持aarch64；
• 修改依赖库的版本，高版本protobuf已支持aarch64；

本文将选择第一种方式，通过修改Protobuf-2.5.0官方源码来支持Hive编译。

编译Google Protobuf

首先安装支持ARM(aarch64)平台的基础编译工具，比如gcc、autoconf、automake和boost-devel等：

[root@workstation packages]# yum install -y autoconf automake libtool boost-devel

因为protobuf-2.5.0发布的时候本身就没有对ARM(aarch64)进行支持，因此即使在ARM平台下直接编译也会报错，需要对源码进行修改：

在源码包中执行Protobuf 2.5.0（ARM）编译命令：

[root@workstation packages]# ./autogen.sh
[root@workstation packages]# ./configure --prefix=/usr/bin/
[root@workstation packages]# make
[root@workstation packages]# make installpopdldconfig
[root@workstation packages]# /usr/bin/protoc --version
libprotoc 2.5.0
[root@workstation bin]# file protoc
protoc: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 3.7.0, BuildID[sha1]=7349d04dcf99ece9e54eebcce961524a1b42b25d, not stripped

重新编译Apache Hive

[root@workstation apache-hive-4.0.0]# mvn -B -e clean package -Pdist -DskipTests
[INFO] Hive Storage API 2.7.0-SNAPSHOT .................... SUCCESS [ 35.316 s]
[INFO] Hive 4.0.0-SNAPSHOT ................................ SUCCESS [  3.588 s]
[INFO] Hive Classifications 4.0.0-SNAPSHOT ................ SUCCESS [  2.908 s]
[INFO] Hive Shims Common 4.0.0-SNAPSHOT ................... SUCCESS [ 13.182 s]
[INFO] Hive Shims 0.23 4.0.0-SNAPSHOT ..................... SUCCESS [ 21.488 s]
[INFO] Hive Shims Scheduler 4.0.0-SNAPSHOT ................ SUCCESS [  8.195 s]
[INFO] Hive Shims 4.0.0-SNAPSHOT .......................... SUCCESS [  7.796 s]
[INFO] Hive Standalone Metastore 4.0.0-SNAPSHOT ........... SUCCESS [  7.144 s]
[INFO] Hive Standalone Metastore Common Code 4.0.0-SNAPSHOT SUCCESS [01:31 min]
[INFO] Hive Common 4.0.0-SNAPSHOT ......................... SUCCESS [ 26.369 s]
[INFO] Hive Service RPC 4.0.0-SNAPSHOT .................... SUCCESS [ 10.166 s]
[INFO] Hive Serde 4.0.0-SNAPSHOT .......................... SUCCESS [ 22.236 s]
[INFO] Hive Metastore 4.0.0-SNAPSHOT ...................... SUCCESS [ 15.598 s]
[INFO] Hive Vector-Code-Gen Utilities 4.0.0-SNAPSHOT ...... SUCCESS [  1.390 s]
[INFO] Hive Llap Common 4.0.0-SNAPSHOT .................... SUCCESS [ 12.416 s]
[INFO] Hive Llap Client 4.0.0-SNAPSHOT .................... SUCCESS [  7.765 s]
[INFO] Hive Llap Tez 4.0.0-SNAPSHOT ....................... SUCCESS [  9.947 s]
[INFO] Hive Spark Remote Client 4.0.0-SNAPSHOT ............ SUCCESS [ 31.137 s]
[INFO] Hive Metastore Server 4.0.0-SNAPSHOT ............... SUCCESS [01:22 min]
[INFO] Hive Query Language 4.0.0-SNAPSHOT ................. SUCCESS [04:30 min]
[INFO] Hive Llap Server 4.0.0-SNAPSHOT .................... SUCCESS [ 34.095 s]
[INFO] Hive Service 4.0.0-SNAPSHOT ........................ SUCCESS [ 39.474 s]
[INFO] Hive Accumulo Handler 4.0.0-SNAPSHOT ............... SUCCESS [ 32.670 s]
[INFO] Hive JDBC 4.0.0-SNAPSHOT ........................... SUCCESS [01:33 min]
[INFO] Hive Beeline 4.0.0-SNAPSHOT ........................ SUCCESS [ 23.243 s]
[INFO] Hive CLI 4.0.0-SNAPSHOT ............................ SUCCESS [ 13.128 s]
[INFO] Hive Contrib 4.0.0-SNAPSHOT ........................ SUCCESS [  7.813 s]
[INFO] Hive Druid Handler 4.0.0-SNAPSHOT .................. SUCCESS [05:31 min]
[INFO] Hive HBase Handler 4.0.0-SNAPSHOT .................. SUCCESS [ 21.637 s]
[INFO] Hive JDBC Handler 4.0.0-SNAPSHOT ................... SUCCESS [  8.978 s]
[INFO] Hive HCatalog 4.0.0-SNAPSHOT ....................... SUCCESS [  5.312 s]
[INFO] Hive HCatalog Core 4.0.0-SNAPSHOT .................. SUCCESS [ 20.405 s]
[INFO] Hive HCatalog Pig Adapter 4.0.0-SNAPSHOT ........... SUCCESS [ 14.177 s]
[INFO] Hive HCatalog Server Extensions 4.0.0-SNAPSHOT ..... SUCCESS [ 13.020 s]
[INFO] Hive HCatalog Webhcat Java Client 4.0.0-SNAPSHOT ... SUCCESS [ 13.702 s]
[INFO] Hive HCatalog Webhcat 4.0.0-SNAPSHOT ............... SUCCESS [ 51.169 s]
[INFO] Hive HCatalog Streaming 4.0.0-SNAPSHOT ............. SUCCESS [ 15.080 s]
[INFO] Hive HPL/SQL 4.0.0-SNAPSHOT ........................ SUCCESS [ 18.665 s]
[INFO] Hive Streaming 4.0.0-SNAPSHOT ...................... SUCCESS [ 10.964 s]
[INFO] Hive Llap External Client 4.0.0-SNAPSHOT ........... SUCCESS [ 10.416 s]
[INFO] Hive Shims Aggregator 4.0.0-SNAPSHOT ............... SUCCESS [  0.345 s]
[INFO] Hive Kryo Registrator 4.0.0-SNAPSHOT ............... SUCCESS [ 11.091 s]
[INFO] Hive TestUtils 4.0.0-SNAPSHOT ...................... SUCCESS [  0.563 s]
[INFO] Hive Kafka Storage Handler 4.0.0-SNAPSHOT .......... SUCCESS [ 29.490 s]
[INFO] Hive Packaging 4.0.0-SNAPSHOT ...................... SUCCESS [03:31 min]
[INFO] Hive Metastore Tools 4.0.0-SNAPSHOT ................ SUCCESS [  0.037 s]
[INFO] Hive Metastore Tools common libraries 4.0.0-SNAPSHOT SUCCESS [ 53.578 s]
[INFO] Hive metastore benchmarks 4.0.0-SNAPSHOT ........... SUCCESS [03:06 min]
[INFO] Hive Upgrade Acid 4.0.0-SNAPSHOT ................... SUCCESS [  0.043 s]
[INFO] Hive Pre Upgrade Acid 4.0.0-SNAPSHOT ............... SUCCESS [ 21.233 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  48:30 min
[INFO] Finished at: 2022-10-03T11:09:12Z