Keepalived与Haproxy安装配置
LiuSw Lv6
 2022-12-20 15:02:38      1.2k 字  5 分钟  

Keepalived与Haproxy安装配置

1.Keepalived安装

keepalived与k8s系统的master安装在相同服务器，那么根据描述我们可以使用VIP访问master，当master1不可用的时候，VIP漂移到master2上，保障了K8S集群的高可用性

Keepalived主要的功能模块有三大块，分别为：虚拟IP、服务监控、虚拟服务器

  • ①虚拟IP：即为上面架构中提到的核心功能

  • ②服务监控：当本服务器的相应服务出现问题的时候降低keepalived优先级，实现IP漂移

  • ③虚拟服务器：keepalived可以为其他服务器做代理，实现负载均衡

Keepalived安装比较简单，根据yum查询在centos7上的安装依赖

解压后使用下面命令依次安装rpm包

1
2
3
4
tar -zxvf keepalived.tar.gz
rpm -ivh net-snmp-libs-5.7.2-48.el7_8.1.x86_64.rpm
rpm -ivh --nodeps  net-snmp-agent-libs-5.7.2-48.el7_8.1.x86_64.rpm
rpm -ivh keepalived-1.3.5-16.el7.x86_64.rpm

# 注意：以前的教程中有提到rpm解决不了软件依赖关系的弊端，如果安装过程中还有依赖提示，只能慢慢摸索`

yum 安装

1
yum install -y keepalived

配置文件修改

keepalived只有一个配置文件，位置在/etc/keepalived/keepalived.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
vi /etc/keepalived/keepalived.conf
cat >/etc/keepalived/keepalived.conf <<EOF

# 全局配置 主要是配置故障发生时的通知对象以及机器标识
global_defs {
  router_id k8s-master1  #修改为自己的hostname
}

# 自定义VRRP实例健康检查脚本 keepalived只能做到对自身问题和网络故障的监控，Script可以增加其他的监控来判定是否需要切换主备
vrrp_script check_haproxy {
  script "/etc/keepalived/check_haproxy.sh"  #检查脚本
  interval 3
  fall 10
  timeout 9
  rise 2
}

# VRRP实例 定义对外提供服务的VIP区域及其相关属性
vrrp_instance VI_1 {
  state MASTER     #备服务器上改为BACKUP
  interface ens33    #改为自己的接口
  virtual_router_id 51
  priority 100     #优先级，备服务器上改为小于100的数字，90,80
  advert_int 1
  mcast_src_ip 192.168.150.11  #本机IP
  nopreempt
  authentication {
    auth_type PASS
    auth_pass 1111
  }
  unicast_peer {
    192.168.150.12    #除本机外其余两个master的IP节点
    192.168.150.13
  }
  virtual_ipaddress {
    192.168.150.10     #虚拟vip，自己设定
  }
  track_script {
    check_haproxy
  }
}

# 监测虚拟IP端口
# 不检测端口可以将此段注释
virtual_server 192.168.11.135 30880 {

    delay_loop 6
    lb_algo rr
    lb_kind NAT
    persistence_timeout 50
    protocol TCP

    # 实际IP端口
    real_server 192.168.11.51 30880 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        connect_port 30880
        }
    }
}



EOF

生成检查脚本

1
2
3
4
5
6
7
8
9
cat >/etc/keepalived/check_haproxy.sh <<EOF
#!/bin/bash

# 若检查其他服务 将haproxy替换
A=\`ps -C haproxy --no-header | wc -l\`
if [ \$A -eq 0 ];then
  systemctl stop keepalived
fi
EOF

给脚本执行权限

1
chmod +x /etc/keepalived/check_haproxy.sh

设置服务

1
2
3
4
5
systemctl start keepalived.service
systemctl stop keepalived.service
systemctl restart keepalived.service
systemctl status keepalived.service
systemctl enable keepalived.service

2.Haproxy安装

我们使用keepalvied系统对三台K8S集群的master做了虚拟IP，使得K8S集群的访问具备了高可用性，但是并没有解决服务负载的问题

本章节我们在keepalived和K8S中间加入了haproxy负载均衡系统

  • ①keepalived不再监控K8S的状态，改为监控haproxy的状态

  • ②由haproxy去分配访问业务流到三台K8S的master

  • ③同时haproxy支持后台业务的健康检查，使得业务同时具备高可用性和负载均衡特性

解压依赖

1
tar -zxvf haproxy_deps.tar.gz

rpm安装

1
2
3
4
5
6
rpm -ivh --nodeps libcom_err-devel-1.42.9-17.el7.x86_64.rpm
rpm -ivh --nodeps krb5-devel-1.15.1-46.el7.x86_64.rpm
rpm -ivh --nodeps pcre-devel-8.32-17.el7.x86_64.rpm
rpm -ivh --nodeps zlib-devel-1.2.7-18.el7.x86_64.rpm
rpm -ivh --nodeps openssl-devel-1.0.2k-19.el7.x86_64.rpm
rpm -ivh --nodeps systemd-devel-219-73.el7_8.9.x86_64.rpm

安装haproxy

1
2
3
4
tar -zxvf haproxy-1.8.26.tar.gz
cd haproxy-1.8.26.tar.gz
make  ARCH=x86_64 TARGET=linux2628 USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 USE_SYSTEMD=1  USE_CPU_AFFINITY=1  PREFIX=/usr/local/haproxy
make install PREFIX=/usr/local/haproxy

添加haproxy命令到sbin

1
2
cp /usr/local/haproxy/sbin/ /usr/sbin
haproxy -v

软件使用文件夹创建

1
2
3
mkdir  /etc/haproxy
mkdir  /etc/haproxy/conf
mkdir  /usr/local/haproxy/run

配置文件修改和说明

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
vim /etc/haproxy/haproxy.cfg
# /etc/haproxy/haproxy.cfg
global
  log     127.0.0.1 local2
  chroot   /var/lib/haproxy
  pidfile   /var/run/haproxy.pid
  maxconn   4000
  user    haproxy
  group    haproxy
  daemon
 
defaults
  mode          tcp
  log           global
  retries         3
  timeout connect     10s
  timeout client     1m
  timeout server     1m
 
frontend kubernetes
  bind *:8443
  mode tcp
  option tcplog
  default_backend kubernetes-apiserver
 
backend kubernetes-apiserver
  mode tcp
  balance roundrobin
  server k8s-master1 192.168.150.11:6443 check maxconn 2000
  server k8s-master2 192.168.150.12:6443 check maxconn 2000
  server k8s-master3 192.168.150.13:6443 check maxconn 2000

修改内核参数

1
2
3
vi /etc/sysctl.conf
# 设置net.ipv4.ip_nonlocal_bind = 1
# 设置net.ipv4.ip_forward = 1

使修改生效

1
sysctl -p

生成service

1
2
3
4
5
6
7
8
9
10
11
12
13
14
cat > /usr/lib/systemd/system/haproxy.service <<EOF
[Unit]
Description=HAProxy Load Balancer
After=syslog.target network.target

[Service]
#支持多配置文件读取，类似于从侧面是实现配置文件的include功能。
ExecStartPre=/usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -f /etc/haproxy/conf -c -q
ExecStart=/usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -f /etc/haproxy/conf -p /run/haproxy.pid
ExecReload=/bin/kill -USR2 $MAINPID

[Install]
WantedBy=multi-user.target
EOF

设置服务

1
2
3
4
5
systemctl start haproxy.service
systemctl stop haproxy.service
systemctl restart haproxy.service
systemctl status haproxy.service
systemctl enable haproxy.service
 评论