【原创】ARM版CentOS部署单节点Doris（ARM版麒麟适用）

da11

ARM版CentOS部署单节点Doris（ARM版麒麟适用）

1. 环境准备：
因国产化要求，需要在ARM架构上部署Doris，jdk这里也使用了国产jdk。

系统自带JDK如果存在就卸载
yum -y remove java-1.8.0*
yum -y remove java-11*


1.1. 毕昇JDK 8 安装
cd /data/
wget https://mirrors.huaweicloud.com/ ... inux-aarch64.tar.gz
tar -zxf bisheng-jdk-8u382-linux-aarch64.tar.gz
cd /data/bisheng-jdk1.8.0_382/


vim /etc/profile

在最后加上下面两行配置
export JAVA_HOME=/data/bisheng-jdk1.8.0_382
export PATH=$PATH:$JAVA_HOME/bin


source /etc/profile

# 查看java版本，显示出版本既安装成功。
java -version



1.2. Mysql安装

警告：如果需要后续使用同步任务到doris，还是建议安装5.6+的mysql版本，rpm包安装教程可查看 https://www.swztdza.cn/dalt/forum.php?mod=viewthread&tid=510&highlight=mysql
Mysql官网下载地址：https://downloads.mysql.com/archives/community/
ARM版本只能下载8.0版本以上，所以直接选择8.0版本以上即可。



因为Doris是基于mysql客户端通信，所以原理上只要安装mysql客户端即可，但因业务原因也安装了服务端，并且使用mariadb代替mysql。

# 使用yum源安装mariadb，一般yum安装的mariadb-server版本是5.5.68
yum -y install mariadb mariadb-server

systemctl restart mariadb

# 使用root进入mysql（能成功进入代表安装成功）
mysql -uroot

# （mysql环境）修改root密码
UPDATE user SET password = PASSWORD('123456') WHERE user = 'root' AND host = '127.0.0.1';
# （mysql环境）修改完成后记得刷新权限才能生效
FLUSH PRIVILEGES;

1.3. gcc版本验证
GCC 版本要求 4.8.2 及以上
gcc -v




1.4. 设置系统最大打开文件句柄数
vim /etc/security/limits.conf

# 添加以下配置（如有就忽略）
root soft nofile 65535
root hard nofile 65535
* soft nofile 65535
* hard nofile 65535





1.5. 永久关闭交换分区
sed -i '/swap/s/^/#/' /etc/fstab
swapoff -a


1.6. 配置时钟同步
Doris 的元数据要求时间精度要小于5000ms，所以所有集群所有机器要进行时钟同步，避免因为时钟问题引发的元数据不一致导致服务出现异常。这里因为单节点部署，所以可以忽略


1.7. 防火墙配置
测试部署时推荐关闭防火墙

systemctl stop firewalld


1.8. 关闭SELINUX
测试部署时推荐关闭SELINUX

setenforce 0

2. Doris安装
2.1. 创建doris用户

adduser doris

2.2. 创建相关目录

# FE 元数据目录 FE节点需要执行
mkdir -p /data/doris-meta
chown -R doris:doris /data/doris-meta
# BE 数据目录 BE 节点需要执行
mkdir -p /data/doris-BE/
chown -R doris:doris /data/doris-BE/
# 日志数据 所有节点需要执行
mkdir -p /var/log/doris/sys
mkdir -p /var/log/doris/audit
chown -R doris:doris /var/log/doris


2.3. 下载安装包并解压安装
cd /data/
wget https://apache-doris-releases.os ... .2-bin-arm64.tar.gz
tar -zxf apache-doris-2.0.2-bin-arm64.tar.gz

mv apache-doris-2.0.2-bin-arm64/ apache-doris-2.0.2/
chown -R doris:doris /data/apache-doris-2.0.2/

2.4 配置FE
只需要在FE节点操作即可，在doris用户下执行


su - doris
vim /data/apache-doris-2.0.2/fe/conf/fe.conf

# 红色字体为需要修改的地方
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.

#####################################################################
## The uppercase properties are read and exported by bin/start_fe.sh.
## To see all Frontend configurations,
## see fe/src/org/apache/doris/common/Config.java
#####################################################################

CUR_DATE=`date +%Y%m%d-%H%M%S`

# the output dir of stderr and stdout
# LOG_DIR = ${DORIS_HOME}/log
LOG_DIR = /var/log/doris

JAVA_OPTS="-Djavax.security.auth.useSubjectCredsOnly=false -Xss4m -Xmx4096m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:/var/log/doris/fe.gc.log.$CUR_DATE"

# For jdk 9+, this JAVA_OPTS will be used as default JVM options
JAVA_OPTS_FOR_JDK_9="-Djavax.security.auth.useSubjectCredsOnly=false -Xss4m -Xmx4096m -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xlog:gc*:/var/log/doris/fe.gc.log.$CUR_DATE:time"

##
## the lowercase properties are read by main program.
##

# INFO, WARN, ERROR, FATAL
sys_log_level = INFO

# NORMAL, BRIEF, ASYNC
sys_log_mode = NORMAL

# store metadata, must be created before start FE.
# Default value is ${DORIS_HOME}/doris-meta
# meta_dir = ${DORIS_HOME}/doris-meta
meta_dir = /data/doris-meta

# Default dirs to put jdbc drivers,default value is ${DORIS_HOME}/jdbc_drivers
# jdbc_drivers_dir = ${DORIS_HOME}/jdbc_drivers

http_port = 8030
rpc_port = 9020
query_port = 9030
edit_log_port = 9010

# Choose one if there are more than one ip except loopback address.
# Note that there should at most one ip match this list.
# If no ip match this rule, will choose one randomly.
# use CIDR format, e.g. 10.10.10.0/24 or IP format, e.g. 10.10.10.1
# Default value is empty.
# priority_networks = 10.10.10.0/24;192.168.0.0/16
priority_networks = 192.168.0.0/24

# Advanced configurations
# log_roll_size_mb = 1024
# sys_log_dir = ${DORIS_HOME}/log
sys_log_dir = /var/log/doris/sys
# sys_log_roll_num = 10
# sys_log_verbose_modules = org.apache.doris
# audit_log_dir = ${DORIS_HOME}/log
audit_log_dir = /var/log/doris/audit
# audit_log_modules = slow_query, query
# audit_log_roll_num = 10
# meta_delay_toleration_second = 10
# qe_max_connection = 1024
# qe_query_timeout_second = 300
# qe_slow_log_ms = 5000


保存并退出






2.5. 配置BE
只需要在BE节点操作即可，在doris用户下执行


su - doris
vim /data/apache-doris-2.0.2/be/conf/be.conf

# 红色字体为需要修改的地方
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.

CUR_DATE=`date +%Y%m%d-%H%M%S`

PPROF_TMPDIR="/var/log/doris/"

# 只要修改到/var/log/doris/路径即可，后面具体的log名称不需要修改
JAVA_OPTS="-Xmx1024m -DlogPath=/var/log/doris/jni.log -Xloggc:/var/log/doris/be.gc.log.$CUR_DATE -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.java.command=DorisBE -XX:-CriticalJNINatives -DJDBC_MIN_POOL=1 -DJDBC_MAX_POOL=100 -DJDBC_MAX_IDLE_TIME=300000 -DJDBC_MAX_WAIT_TIME=5000"

# For jdk 9+, this JAVA_OPTS will be used as default JVM options
JAVA_OPTS_FOR_JDK_9="-Xmx1024m -DlogPath=/var/log/doris/jni.log -Xlog:gc:/var/log/doris/be.gc.log.$CUR_DATE -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.java.command=DorisBE -XX:-CriticalJNINatives -DJDBC_MIN_POOL=1 -DJDBC_MAX_POOL=100 -DJDBC_MAX_IDLE_TIME=300000 -DJDBC_MAX_WAIT_TIME=5000"

# since 1.2, the JAVA_HOME need to be set to run BE process.
# JAVA_HOME=/path/to/jdk/
JAVA_HOME=/data/bisheng-jdk1.8.0_382/

# https://github.com/apache/doris/ ... malloc-heap-profile
# https://jemalloc.net/jemalloc.3.html
JEMALLOC_CONF="percpu_arena:percpu,background_thread:true,metadata_thp:auto,muzzy_decay_ms:15000,dirty_decay_ms:15000,oversize_threshold:0,lg_tcache_max:20,prof:false,lg_prof_interval:32,lg_prof_sample:19,prof_gdump:false,prof_accum:false,prof_leak:false,prof_final:false"
JEMALLOC_PROF_PRFIX=""

# INFO, WARNING, ERROR, FATAL
sys_log_level = INFO

# ports for admin, web, heartbeat service
be_port = 9060
webserver_port = 8040
heartbeat_service_port = 9050
brpc_port = 8060

# HTTPS configures
enable_https = false
# path of certificate in PEM format.
ssl_certificate_path = "$DORIS_HOME/conf/cert.pem"
# path of private key in PEM format.
ssl_private_key_path = "$DORIS_HOME/conf/key.pem"

# enable auth check
enable_auth = false

# Choose one if there are more than one ip except loopback address.
# Note that there should at most one ip match this list.
# If no ip match this rule, will choose one randomly.
# use CIDR format, e.g. 10.10.10.0/24 or IP format, e.g. 10.10.10.1
# Default value is empty.
# priority_networks = 10.10.10.0/24;192.168.0.0/16
priority_networks = 192.168.0.0/24

# data root path, separate by ';'
# you can specify the storage medium of each root path, HDD or SSD
# you can add capacity limit at the end of each root path, separate by ','
# eg:
# storage_root_path = /home/disk1/doris.HDD,50;/home/disk2/doris.SSD,1;/home/disk2/doris
# /home/disk1/doris.HDD, capacity limit is 50GB, HDD;
# /home/disk2/doris.SSD, capacity limit is 1GB, SSD;
# /home/disk2/doris, capacity limit is disk capacity, HDD(default)
#
# you also can specify the properties by setting '<property>:<value>', separate by ','
# property 'medium' has a higher priority than the extension of path
#
# Default value is ${DORIS_HOME}/storage, you should create it by hand.
# storage_root_path = ${DORIS_HOME}/storage
storage_root_path = /data/doris-BE,medium:hdd

# Default dirs to put jdbc drivers,default value is ${DORIS_HOME}/jdbc_drivers
# jdbc_drivers_dir = ${DORIS_HOME}/jdbc_drivers

# Advanced configurations
# sys_log_dir = ${DORIS_HOME}/log
sys_log_dir = /var/log/doris/sys
# sys_log_roll_mode = SIZE-MB-1024
# sys_log_roll_num = 10
# sys_log_verbose_modules = *
# log_buffer_level = -1
# palo_cgroups


保存并退出

2.6. 启动 FE
只需要在FE节点操作即可，在doris用户下执行 。这里只有一个FE

cd /data/apache-doris-2.0.2/fe/


./bin/start_fe.sh --daemon

# 通过 MySQL 客户端 查看FE是否启动成功
mysql -uroot -P9030 -h127.0.0.1

SHOW PROC '/frontends';




2.7. 启动BE
2.7.1. BE还需要额外进行Linux 内核参数配置，使用root用户

echo "vm.max_map_count=2000000" >> /etc/sysctl.conf
sudo sysctl -p


tee -a /etc/security/limits.d/90-doris.conf <<EOF
> doris soft nofile 65536
> doris hard nofile 65536
> * soft nofile 65536
> * hard nofile 65536
> EOF


sudo sysctl -p



# 如上述设置后，启动BE时还是提示：Please set the maximum number of open file descriptors to be 65536 using 'ulimit -n 65536'.


则需要修改be启动脚本，强制设置句柄
vim /data/apache-doris-2.0.2/be/bin/start_be.sh

# 添加如下行配置：
[ $(ulimit -n) -lt 65536 ] && ulimit -n 65536




2.7.2. 在doris-fe添加所有BE节点
在 MySQL 客户端操作
mysql -uroot -P9030 -h127.0.0.1

# 添加当前IP的BE节点
ALTER SYSTEM ADD BACKEND "192.168.0.95:9050";


commit;

# 检查节点
SHOW BACKENDS;



2.7.3. 设置句柄和添加节点后，启动be，在doris用户中执行。

cd /data/apache-doris-2.0.2/be/

./bin/start_be.sh --daemon




2.7.4. 通过MySQL 客户端查看BE节点状态

SHOW PROC '/backends';

这是可以看到BE节点是激活的，就代表BE节点成功启动了



3. 访问Web UI

IP:8030即可访问Doris-WebUI界面



默认用户名是root和admin，默认无密码，高风险，建议马上修改密码！修改密码请参考以下帖子：
https://www.swztdza.cn/dalt/forum.php?mod=viewthread&tid=1202&extra=



至此Doris单节点部署完成~