标签： 负载均衡

Nginx编译加载使用动态模块（Dynamic Shared Object）（DSO）

• LNMP、Nginx • 评论关闭

Nginx编译加载使用动态模块（Dynamic Shared Object）（DSO）
Nginx版本必须>=1.9.11

查看支持的动态模块
[root@localhost nginx-1.12.2]# ./configure --help | grep dynamic
--with-http_xslt_module=dynamic enable dynamic ngx_http_xslt_module
--with-http_image_filter_module=dynamic
enable dynamic ngx_http_image_filter_module
--with-http_geoip_module=dynamic enable dynamic ngx_http_geoip_module
--with-http_perl_module=dynamic enable dynamic ngx_http_perl_module
--with-mail=dynamic enable dynamic POP3/IMAP4/SMTP proxy module
--with-stream=dynamic enable dynamic TCP/UDP proxy module
--with-stream_geoip_module=dynamic enable dynamic ngx_stream_geoip_module
--add-dynamic-module=PATH enable dynamic external module
--with-compat dynamic modules compatibility

编译时候的参数
--with-stream \
--with-stream=dynamic \
--with-stream_ssl_module \
--with-stream_realip_module \
--with-stream_ssl_preread_module

使用stream动态模块
user www www;
worker_processes auto;
worker_cpu_affinity auto;

load_module "modules/ngx_stream_module.so";

error_log /home/wwwlogs/error_nginx.log warn;#debug | info | notice | warn | error | crit ]
pid /usr/local/nginx/logs/nginx.pid;
worker_rlimit_nofile 51200;

events {
use epoll;
worker_connections 51200;
multi_accept on;
}

stream {
server {
listen 20020;
proxy_pass 192.168.122.214:3389;
}
server {
listen 20021;
proxy_pass 192.168.122.200:3389;
}
}
......
......

Linux CentOS 7 多网卡配置bond模式 bond1 bond5 bond6

• CentOS、Linux • 评论关闭

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Networking_Guide/sec-Network_Bonding_Using_the_Command_Line_Interface.html

Linux CentOS 7 多网卡配置bond模式 bond1 bond5 bond6

网卡绑定mode共有七种(0~6) bond0、bond1、bond2、bond3、bond4、bond5、bond6
常用的有四种
mode=0：(balance-rr) Round-robin policy（平衡抡循环策略）平衡负载模式，有自动备援，但需要”Switch”支援及设定。
mode=1，(active-backup) Active-backup policy（主-备份策略）只有一个设备处于活动状态，当一个宕掉另一个马上由备份转换为主设备。mac地址是外部可见得，从外面看来，bond的MAC地址是唯一的。
mode=5，(balance-tlb) Adaptive transmit load balancing（适配器传输负载均衡）不需要任何特别的switch(交换机)支持的通道bonding。
mode=6，(balance-alb) Adaptive load balancing（适配器适应性负载均衡）该模式包含了balance-tlb模式，同时加上针对IPV4流量的接收负载均衡(receive load balance, rlb)，而且不需要任何switch(交换机)的支持。接收负载均衡是通过ARP协商实现的

mode=1，(active-backup) Active-backup policy（主-备份策略）

在Centos 7中，我们可以使用modinfo bonding 命令查看bonding模块的信息，默认情况下bonding模块没有被加载

以root用户登录，输入如下的命令进行开启加载bonding模块
modprobe --first-time bonding //--first-time Fail if module already inserted or removed

cat /etc/sysconfig/network-scripts/ifcfg-bond1
DEVICE=bond1
BONDING_OPTS="resend_igmp=1 updelay=0 use_carrier=1 miimon=100 downdelay=0 xmit_hash_policy=0 primary_reselect=0 fail_over_mac=0 arp_validate=0 mode=active-backup primary=eno16777736 arp_interval=0 ad_select=0"
TYPE=Bond
BONDING_MASTER=yes
BOOTPROTO=none
IPADDR=192.168.199.235
PREFIX=24
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME="Bond connection 1"
UUID=e110d205-848e-4527-8cb1-2b3ac76cfb9b
ONBOOT=yes

cat /etc/sysconfig/network-scripts/ifcfg-eno33554960
HWADDR=00:0C:29:1B:57:14
TYPE=Ethernet
BOOTPROTO=none 修改
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=eno33554960
UUID=ef35e777-0bc3-4afa-ad54-490e1543242a
ONBOOT=yes 修改

[root@www.zhangfangzhou.cn ~]# cat /proc/net/bonding/bond1
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: fault-tolerance (active-backup)
Primary Slave: eno16777736 (primary_reselect always)
Currently Active Slave: eno16777736
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eno16777736
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:1b:57:0a
Slave queue ID: 0

Slave Interface: eno33554960
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:1b:57:14
Slave queue ID: 0

Slave Interface: eno50332184
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:1b:57:1e
Slave queue ID: 0

Slave Interface: eno67109408
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:1b:57:28
Slave queue ID: 0
------------------------------------------------------------------------------------------------------------------------
CentOS7手动配置bond6(balance-alb) Adaptive load balancing（适配器适应性负载均衡）

1、在Centos 7中，我们可以使用modinfo bonding 命令查看bonding模块的信息，默认情况下bonding模块没有被加载

以root用户登录，输入如下的命令进行开启加载bonding模块
modprobe --first-time bonding //--first-time Fail if module already inserted or removed

2、创建bond0配置文件，创建/etc/sysconfig/network-scripts/ifcfg-bond0文件，加入如下内容
cat /etc/sysconfig/network-scripts/ifcfg-bond0

DEVICE=bond0
TYPE=Bond
BONDING_MASTER=yes
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME="bond0"
IPADDR=192.168.199.223
PREFIX=24
ONBOOT=yes
BONDING_OPTS="resend_igmp=1 updelay=0 use_carrier=1 miimon=100 downdelay=0 xmit_hash_policy=0 primary_reselect=0 fail_over_mac=0 arp_validate=0 mode=balance-alb arp_interval=0 ad_select=0"
PEERDNS=yes
PEERROUTES=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes

其中BONDING_OPTS指定bonding的模式，其中10.101.230.30/27将作为bond0的ip地址

3、配置e网卡配置文件，文件位于/etc/sysconfig/network-scripts/目录

vi ifcfg-eno16777736

TYPE=Ethernet
BOOTPROTO=none 修改
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=eno1
UUID=26ea0bd1-0837-4b1a-9039-f147abd19632
DEVICE=eno16777736
ONBOOT=yes 修改
#以下两项为新增配置选项
MASTER=bond0
SLAVE=yes

systemctl restart network.service //重启网络服务

[root@www.zhangfangzhou.cn~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: adaptive load balancing
Primary Slave: None
Currently Active Slave: eno33554960
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eno16777736
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 1
Permanent HW addr: 00:0c:29:1b:57:0a
Slave queue ID: 0

Slave Interface: eno33554960
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:1b:57:14
Slave queue ID: 0

Slave Interface: eno50332184
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:1b:57:1e
Slave queue ID: 0

Slave Interface: eno67109408
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0c:29:1b:57:28
Slave queue ID: 0

------------------------------------------------------------
nutui配置bond6 (balance-alb) Adaptive load balancing（适配器适应性负载均衡）
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Networking_Guide/sec-Configure_Bonding_Using_the_Text_User_Interface_nmtui.html

[root@www.zhangfangzhou.cn network-scripts]# cat ifcfg-Bond_connection_1
DEVICE=bond0
TYPE=Bond
BONDING_MASTER=yes
BOOTPROTO=dhcp
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME="Bond connection 1"
UUID=5aa07976-3cc3-4db2-b9a8-af598906d263
ONBOOT=yes
BONDING_OPTS="resend_igmp=1 updelay=0 use_carrier=1 miimon=100 downdelay=0 xmit_hash_policy=0 primary_reselect=0 fail_over_mac=0 arp_validate=0 mode=balance-alb arp_interval=0 ad_select=0"
PEERDNS=yes
PEERROUTES=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes

[root@www.zhangfangzhou.cn network-scripts]# cat ifcfg-eno16777736
HWADDR=00:0C:29:00:50:D4
TYPE=Ethernet
BOOTPROTO=dhcp
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=eno16777736
UUID=600f243a-ef20-4b91-8e5c-02a21b49cfb3
ONBOOT=no

[root@www.zhangfangzhou.cn network-scripts]# cat ifcfg-Ethernet_connection_1
TYPE=Ethernet
NAME="Ethernet connection 1"
UUID=a7f3051a-0aac-45fd-a019-75b9db495736
DEVICE=eno16777736
ONBOOT=yes
MASTER=5aa07976-3cc3-4db2-b9a8-af598906d263
SLAVE=yes

------------------------------------------------------------------------------------------------------------------------
配置mode=5，(balance-tlb) Adaptive transmit load balancing（适配器传输负载均衡）
cat /etc/sysconfig/network-scripts/ifcfg-bond5

bond5
DEVICE=bond5
TYPE=Bond
NAME=bond5
BONDING_MASTER=yes
BOOTPROTO=none
ONBOOT=yes
IPADDR=10.216.13.3
NETMASK=255.255.255.0
BONDING_OPTS="resend_igmp=1 updelay=0 use_carrier=1 miimon=100 downdelay=0 xmit_hash_policy=0 primary_reselect=0 fail_over_mac=0 arp_validate=0 mode=balance-tlb arp_interval=0 ad_select=0"

Nginx1.90 nginx_stream 做TCP代理和协议负载均衡

• LNMP、Nginx • 评论关闭

Nginx1.90做TCP代理和协议负载均衡的功能
nginx从1.9.0开始增加了stream模块（ngx_stream_core_module），默认configure不包含该模块，需要在configure的时候加上--with-stream

./configure --prefix=/usr/local/nginx --user=www --group=www \
--add-module=/root/ngx_http_google_filter_module \
--add-module=/root/ngx_http_substitutions_filter_module \
--with-http_stub_status_module \
--with-http_v2_module \
--with-http_ssl_module \
--with-ipv6 \
--with-http_gzip_static_module \
--with-http_realip_module \
--with-http_flv_module \
--with-http_sub_module \
--with-stream

配置文件很简单，最基本的

.....................
events {
use epoll;
worker_connections 51200;
}
stream {
server {
listen 2002;
proxy_pass 123.123.123.123:3389;
}

server {
listen 2003;
proxy_pass 123.123.123.123:22;
}
}

http {
.....................

需要在防火墙允许相应的端口通过。这样可以反代远程桌面3389端口或者其他固定的TCP端口，比iptables转发或者虚拟专用网络连接来管理国外Windows或者Linux服务器要方便不少。

当然，该模块最重要的功能是支持TCP负载均衡，比如远程多台mysql负载均衡。

stream {
upstream mysql {
server 1.1.1.1:3306;
server 2.2.2.2:3306;
server 3.3.3.3:3306;
}
server {
listen 3306;
proxy_pass mysql;
}
}

官方文档http://nginx.org/en/docs/stream/ngx_stream_core_module.html