标签： 负载均衡

CentOS Linux release 7.9.2009 编译安装最新版HAProxy 2.8 LTS长期支持版，并创建Shell监听脚本

• CentOS、工具、网络 • 评论关闭

编译安装HAProxy 2.8 LTS版本，官方源码包下载地址：http://www.haproxy.org/download/

由于CentOS7 之前版本自带的lua版本比较低并不符合HAProxy要求的lua最低版本(5.3)的要求，因此需要编译安装较新版本的lua环境，然后才能编译安装HAProxy。

1、查看现有的lua版本，如果版本太旧需要先更新lua

lua -v 
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio

2、安装基础命令及编译依赖环境

yum install gcc readline-devel

3、下载和编译lua-5.4.6

cd /usr/local/src
wget --no-check-certificate https://www.lua.org/ftp/lua-5.4.6.tar.gz
tar xvf lua-5.4.6.tar.gz

开始编译

cd  lua-5.4.6
make linux test

查看编译安装的版本
src/lua -v
Lua 5.4.6  Copyright (C) 1994-2023 Lua.org, PUC-Rio

把旧版本移走，换成新的版本
mv /usr/bin/lua /usr/bin/lua.old
cp /usr/local/src/lua-5.4.6/src/lua /usr/bin/lua

确认现在的版本
lua -v 
Lua 5.4.6  Copyright (C) 1994-2023 Lua.org, PUC-Rio

4、安装Haproxy

创建用户
useradd -r -s /sbin/nologin haproxy

5、安装依赖

yum -y install gcc openssl-devel pcre-devel systemd-devel

6、下载和编译

wget --no-check-certificate  https://www.haproxy.org/download/2.8/src/haproxy-2.8.9.tar.gz
tar xvf haproxy-2.8.9.tar.gz
cd haproxy-2.8.9/


make TARGET=linux-glibc USE_OPENSSL=1 USE_PCRE=1 USE_SYSTEMD=1 USE_LUA=1 LUA_INC=/usr/local/src/lua-5.4.6/src LUA_LIB=/usr/local/src/lua-5.4.6/src
make install PREFIX=/etc/haproxy

创建软连接
ln -s /etc/haproxy/sbin/haproxy /usr/bin/haproxy
www.zhangfangzhou.cn
查看版本
haproxy -v
HAProxy version 2.8.9-1842fd0 2024/04/05 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2028.
Known bugs: http://www.haproxy.org/bugs/bugs-2.8.9.html
Running on: Linux 3.10.0-1160.105.1.el7.x86_64 #1 SMP Thu Dec 7 15:39:45 UTC 2023 x86_64

7、创建配置文件

vi /etc/haproxy/haproxy.cfg
global
  maxconn 100000
  chroot /etc/haproxy
  stats socket /var/tmp/haproxy.sock mode 600 level admin
  user haproxy
  group haproxy
  daemon
  pidfile /var/tmp/haproxy.pid

defaults
  option http-keep-alive
  option forwardfor
  maxconn 100000
  mode http
  timeout connect 300000ms
  timeout client 300000ms
  timeout server 300000ms

listen stats
  mode http
  bind 0.0.0.0:3306
  stats enable
  log global
  stats uri /haproxy-status
  stats auth haadmin:Admin@2018..

listen db_port
  bind :3306
  mode tcp
  log global
  balance roundrobin
  option tcplog
  server db1 10.53.123.104:3306 check inter 3000 fall 2 rise 3
  server db2 10.53.123.105:3306 check inter 3000 fall 2 rise 3
  server db3 10.53.123.106:3306 check inter 3000 fall 2 rise 3


8、创建service服务

vi /usr/lib/systemd/system/haproxy.service
[Unit]
Description=HAProxy Load Balancer
After=syslog.target network.target

[Service]
ExecStartPre=/usr/bin/haproxy -f /etc/haproxy/haproxy.cfg -c -q
ExecStart=/usr/bin/haproxy -Ws -f /etc/haproxy/haproxy.cfg
ExecReload=/bin/kill -USR2 $MAINPID
LimitNOFILE=100000

[Install]
WantedBy=multi-user.target

systemctl enable haproxy
systemctl start haproxy
9、设置监听脚本

vi ha_watchdog.sh
#!/bin/bash
#www.zhangfangzhou.cn
# 定义日志文件路径
LOG_FILE="/var/log/haproxy_watchdog.log"

# 定义 HAProxy 进程的名称
HAPROXY_PROCESS="haproxy"

# 检查 HAProxy 进程是否在运行
check_haproxy_process() {
    # 使用 pgrep 命令检查是否存在名为 haproxy 的进程
    pgrep -x $HAPROXY_PROCESS > /dev/null
}

# 启动 HAProxy 进程
start_haproxy() {
    echo "$(date +'%Y-%m-%d %H:%M:%S') - Starting HAProxy..." >> $LOG_FILE
    systemctl start haproxy
}

# 主循环
while true; do
    # 检查 HAProxy 进程是否在运行
    if ! check_haproxy_process; then
        # 如果进程不存在，则重新启动 HAProxy 并记录日志
        start_haproxy
    fi
    # 休眠一段时间后再次检查
    sleep 60
done


#设置开机启动
chmod +x ha_watchdog.sh

vi /etc/rc.local
nohup /root/ha_watchdog.sh &

Nginx编译加载使用动态模块（Dynamic Shared Object）（DSO）

• LNMP、Nginx • 评论关闭

Nginx编译加载使用动态模块（Dynamic Shared Object）（DSO）
Nginx版本必须>=1.9.11

查看支持的动态模块
[root@localhost nginx-1.12.2]# ./configure --help | grep dynamic
--with-http_xslt_module=dynamic enable dynamic ngx_http_xslt_module
--with-http_image_filter_module=dynamic
enable dynamic ngx_http_image_filter_module
--with-http_geoip_module=dynamic enable dynamic ngx_http_geoip_module
--with-http_perl_module=dynamic enable dynamic ngx_http_perl_module
--with-mail=dynamic enable dynamic POP3/IMAP4/SMTP proxy module
--with-stream=dynamic enable dynamic TCP/UDP proxy module
--with-stream_geoip_module=dynamic enable dynamic ngx_stream_geoip_module
--add-dynamic-module=PATH enable dynamic external module
--with-compat dynamic modules compatibility

编译时候的参数
--with-stream \
--with-stream=dynamic \
--with-stream_ssl_module \
--with-stream_realip_module \
--with-stream_ssl_preread_module

使用stream动态模块
user www www;
worker_processes auto;
worker_cpu_affinity auto;

load_module "modules/ngx_stream_module.so";

error_log /home/wwwlogs/error_nginx.log warn;#debug | info | notice | warn | error | crit ]
pid /usr/local/nginx/logs/nginx.pid;
worker_rlimit_nofile 51200;

events {
use epoll;
worker_connections 51200;
multi_accept on;
}

stream {
server {
listen 20020;
proxy_pass 192.168.122.214:3389;
}
server {
listen 20021;
proxy_pass 192.168.122.200:3389;
}
}
......
......

Linux CentOS 7 多网卡配置bond模式 bond1 bond5 bond6

• CentOS、Linux • 评论关闭

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Networking_Guide/sec-Network_Bonding_Using_the_Command_Line_Interface.html

Linux CentOS 7 多网卡配置bond模式 bond1 bond5 bond6

网卡绑定mode共有七种(0~6) bond0、bond1、bond2、bond3、bond4、bond5、bond6
常用的有四种
mode=0：(balance-rr) Round-robin policy（平衡抡循环策略）平衡负载模式，有自动备援，但需要”Switch”支援及设定。
mode=1，(active-backup) Active-backup policy（主-备份策略）只有一个设备处于活动状态，当一个宕掉另一个马上由备份转换为主设备。mac地址是外部可见得，从外面看来，bond的MAC地址是唯一的。
mode=5，(balance-tlb) Adaptive transmit load balancing（适配器传输负载均衡）不需要任何特别的switch(交换机)支持的通道bonding。
mode=6，(balance-alb) Adaptive load balancing（适配器适应性负载均衡）该模式包含了balance-tlb模式，同时加上针对IPV4流量的接收负载均衡(receive load balance, rlb)，而且不需要任何switch(交换机)的支持。接收负载均衡是通过ARP协商实现的

mode=1，(active-backup) Active-backup policy（主-备份策略）

在Centos 7中，我们可以使用modinfo bonding 命令查看bonding模块的信息，默认情况下bonding模块没有被加载

以root用户登录，输入如下的命令进行开启加载bonding模块
modprobe --first-time bonding //--first-time Fail if module already inserted or removed

cat /etc/sysconfig/network-scripts/ifcfg-bond1
DEVICE=bond1
BONDING_OPTS="resend_igmp=1 updelay=0 use_carrier=1 miimon=100 downdelay=0 xmit_hash_policy=0 primary_reselect=0 fail_over_mac=0 arp_validate=0 mode=active-backup primary=eno16777736 arp_interval=0 ad_select=0"
TYPE=Bond
BONDING_MASTER=yes
BOOTPROTO=none
IPADDR=192.168.199.235
PREFIX=24
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME="Bond connection 1"
UUID=e110d205-848e-4527-8cb1-2b3ac76cfb9b
ONBOOT=yes

cat /etc/sysconfig/network-scripts/ifcfg-eno33554960
HWADDR=00:0C:29:1B:57:14
TYPE=Ethernet
BOOTPROTO=none 修改
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=eno33554960
UUID=ef35e777-0bc3-4afa-ad54-490e1543242a
ONBOOT=yes 修改

[root@www.zhangfangzhou.cn ~]# cat /proc/net/bonding/bond1
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: fault-tolerance (active-backup)
Primary Slave: eno16777736 (primary_reselect always)
Currently Active Slave: eno16777736
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eno16777736
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:1b:57:0a
Slave queue ID: 0

Slave Interface: eno33554960
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:1b:57:14
Slave queue ID: 0

Slave Interface: eno50332184
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:1b:57:1e
Slave queue ID: 0

Slave Interface: eno67109408
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:1b:57:28
Slave queue ID: 0
------------------------------------------------------------------------------------------------------------------------
CentOS7手动配置bond6(balance-alb) Adaptive load balancing（适配器适应性负载均衡）

1、在Centos 7中，我们可以使用modinfo bonding 命令查看bonding模块的信息，默认情况下bonding模块没有被加载

以root用户登录，输入如下的命令进行开启加载bonding模块
modprobe --first-time bonding //--first-time Fail if module already inserted or removed

2、创建bond0配置文件，创建/etc/sysconfig/network-scripts/ifcfg-bond0文件，加入如下内容
cat /etc/sysconfig/network-scripts/ifcfg-bond0

DEVICE=bond0
TYPE=Bond
BONDING_MASTER=yes
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME="bond0"
IPADDR=192.168.199.223
PREFIX=24
ONBOOT=yes
BONDING_OPTS="resend_igmp=1 updelay=0 use_carrier=1 miimon=100 downdelay=0 xmit_hash_policy=0 primary_reselect=0 fail_over_mac=0 arp_validate=0 mode=balance-alb arp_interval=0 ad_select=0"
PEERDNS=yes
PEERROUTES=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes

其中BONDING_OPTS指定bonding的模式，其中10.101.230.30/27将作为bond0的ip地址

3、配置e网卡配置文件，文件位于/etc/sysconfig/network-scripts/目录

vi ifcfg-eno16777736

TYPE=Ethernet
BOOTPROTO=none 修改
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=eno1
UUID=26ea0bd1-0837-4b1a-9039-f147abd19632
DEVICE=eno16777736
ONBOOT=yes 修改
#以下两项为新增配置选项
MASTER=bond0
SLAVE=yes

systemctl restart network.service //重启网络服务

[root@www.zhangfangzhou.cn~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: adaptive load balancing
Primary Slave: None
Currently Active Slave: eno33554960
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eno16777736
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 1
Permanent HW addr: 00:0c:29:1b:57:0a
Slave queue ID: 0

Slave Interface: eno33554960
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:1b:57:14
Slave queue ID: 0

Slave Interface: eno50332184
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:1b:57:1e
Slave queue ID: 0

Slave Interface: eno67109408
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:1b:57:28
Slave queue ID: 0

------------------------------------------------------------
nutui配置bond6 (balance-alb) Adaptive load balancing（适配器适应性负载均衡）
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Networking_Guide/sec-Configure_Bonding_Using_the_Text_User_Interface_nmtui.html

[root@www.zhangfangzhou.cn network-scripts]# cat ifcfg-Bond_connection_1
DEVICE=bond0
TYPE=Bond
BONDING_MASTER=yes
BOOTPROTO=dhcp
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME="Bond connection 1"
UUID=5aa07976-3cc3-4db2-b9a8-af598906d263
ONBOOT=yes
BONDING_OPTS="resend_igmp=1 updelay=0 use_carrier=1 miimon=100 downdelay=0 xmit_hash_policy=0 primary_reselect=0 fail_over_mac=0 arp_validate=0 mode=balance-alb arp_interval=0 ad_select=0"
PEERDNS=yes
PEERROUTES=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes

[root@www.zhangfangzhou.cn network-scripts]# cat ifcfg-eno16777736
HWADDR=00:0C:29:00:50:D4
TYPE=Ethernet
BOOTPROTO=dhcp
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=eno16777736
UUID=600f243a-ef20-4b91-8e5c-02a21b49cfb3
ONBOOT=no

[root@www.zhangfangzhou.cn network-scripts]# cat ifcfg-Ethernet_connection_1
TYPE=Ethernet
NAME="Ethernet connection 1"
UUID=a7f3051a-0aac-45fd-a019-75b9db495736
DEVICE=eno16777736
ONBOOT=yes
MASTER=5aa07976-3cc3-4db2-b9a8-af598906d263
SLAVE=yes

------------------------------------------------------------------------------------------------------------------------
配置mode=5，(balance-tlb) Adaptive transmit load balancing（适配器传输负载均衡）
cat /etc/sysconfig/network-scripts/ifcfg-bond5

bond5
DEVICE=bond5
TYPE=Bond
NAME=bond5
BONDING_MASTER=yes
BOOTPROTO=none
ONBOOT=yes
IPADDR=10.216.13.3
NETMASK=255.255.255.0
BONDING_OPTS="resend_igmp=1 updelay=0 use_carrier=1 miimon=100 downdelay=0 xmit_hash_policy=0 primary_reselect=0 fail_over_mac=0 arp_validate=0 mode=balance-tlb arp_interval=0 ad_select=0"

Nginx1.90 nginx_stream 做TCP代理和协议负载均衡

• LNMP、Nginx • 评论关闭

Nginx1.90做TCP代理和协议负载均衡的功能
nginx从1.9.0开始增加了stream模块（ngx_stream_core_module），默认configure不包含该模块，需要在configure的时候加上--with-stream

./configure --prefix=/usr/local/nginx --user=www --group=www \
--add-module=/root/ngx_http_google_filter_module \
--add-module=/root/ngx_http_substitutions_filter_module \
--with-http_stub_status_module \
--with-http_v2_module \
--with-http_ssl_module \
--with-ipv6 \
--with-http_gzip_static_module \
--with-http_realip_module \
--with-http_flv_module \
--with-http_sub_module \
--with-stream

配置文件很简单，最基本的

.....................
events {
use epoll;
worker_connections 51200;
}
stream {
server {
listen 2002;
proxy_pass 123.123.123.123:3389;
}

server {
listen 2003;
proxy_pass 123.123.123.123:22;
}
}

http {
.....................

需要在防火墙允许相应的端口通过。这样可以反代远程桌面3389端口或者其他固定的TCP端口，比iptables转发或者虚拟专用网络连接来管理国外Windows或者Linux服务器要方便不少。

当然，该模块最重要的功能是支持TCP负载均衡，比如远程多台mysql负载均衡。

stream {
upstream mysql {
server 1.1.1.1:3306;
server 2.2.2.2:3306;
server 3.3.3.3:3306;
}
server {
listen 3306;
proxy_pass mysql;
}
}

官方文档http://nginx.org/en/docs/stream/ngx_stream_core_module.html