最新消息:

部署LVS+Keepalived集群

集群 admin 3378浏览 0评论

系统:CentOS 5.8 X86_64

软件:ipvsadm-1.24.tar.gz keepalived-1.2.7.tar.gz
拓扑:
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_1196329391

 

一、安装ipvsadm(Master DR和Backup DR都安装)
1.查看系统内核版本,以下载相应的ipvsadm软件包(或使用系统的自带rpm安装)
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_11998277111
2.下载内核对应的ipvsadm软件包
# wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
3.创建内核文件的软链接
# ln -s /usr/src/kernels/2.6.18-308.el5-x86_64/ /usr/src/linux
如没有创建软链接,在编译ipvsadm时将出现下面的错误:
# make
make -C libipvs
make[1]: Entering directory `/usr/local/src/ipvsadm-1.24/libipvs’
gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include  -DHAVE_NET_IP_VS_H -c -o libipvs.o libipvs.c
In file included from libipvs.c:23:
libipvs.h:14:23: error: net/ip_vs.h: No such file or directory
In file included from libipvs.c:23:
libipvs.h:119: error: expected ‘)’ before ‘fwmark’
libipvs.c:27: error: field ‘svc’ has incomplete type
libipvs.c:28: error: field ‘dest’ has incomplete type
libipvs.c: In function ‘ipvs_init’:
4.安装ipvsadm
# tar xvf ipvsadm-1.24.tar.gz -C /usr/local/src/
# cd /usr/local/src/ipvsadm-1.24/
# make && make install
二、安装keepalived(Master DR和Backup DR都安装)
1.下载keepalived
# wget http://www.keepalived.org/software/keepalived-1.2.7.tar.gz
2.安装keepalived
# tar xvf keepalived-1.2.7.tar.gz -C /usr/local/src/
# cd /usr/local/src/keepalived-1.2.7/
# ./configure && make && make install
注:编译时出现下面的警告
configure: WARNING: keepalived will be built without libnl support.
可以通过安装libnl-devel解决,需要根据keepalived的版本安装相应版本的libnl包
3.拷贝配置文件、启动脚本等
# cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/
# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
# mkdir /etc/keepalived
# cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
# cp /usr/local/sbin/keepalived /usr/sbin/
三、根据lvs的调度算法、工作模式定义keepalived配置
这里使用DR模式、rr调度算法;
MASTER DR:
# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
#全局定义:设置keepalived的通知机制和标识
global_defs {
   notification_email {  #定义接收邮件的地址,一行对应一个
     qiuyx@app.moonbasagroup.com
   }
   notification_email_from admin@app.moonbasagroup.com
   smtp_server 10.0.65.29 #指定smtp服务器地址
   smtp_connect_timeout 30
   router_id LVS_DEVEL  #定义运行keepalived机器的标识
}
#VRRP实例组定义:VRRP实例就表示在上面开启了VRRP协议,这个实例说明了VRRP的一些特性,如主从、VRID等,可以在每个接口上开启一个实例。
vrrp_instance VI_1 {
    state MASTER #指定实例的初始状态,高优先级的会竞选为master,
    interface eth0 #实例绑定的网卡
    virtual_router_id 51 #VRID标记(0…255)
    priority 100 #数字越高基本越高,master要高于backup至少50
    advert_int 1 #检查间隔,默认1s
    authentication {  #设置认证
        auth_type PASS  #认证方式,支持PASS和AH
        auth_pass 1111  #认证的密码
    }
    virtual_ipaddress {  #里面指定漂移地址(VIP)
        10.0.37.10
    }
}
#LVS配置
virtual_server 10.0.37.10 80 {
    delay_loop 6 #服务轮询的延时时间
    lb_algo rr #lvs的调度算法
    lb_kind DR #lvs集群模式
    nat_mask 255.255.255.0
    #persistence_timeout 50 #会话保持时间(秒),测试方便取消设置
    protocol TCP #使用的协议是TCP
    real_server 10.0.37.5 80 {
        weight 1  #默认为1,0为失效
        TCP_CHECK { #后端服务器健康检查
            connect_timeout 3 #连接超时时间
            nb_get_retry 3 #重连次数
            delay_before_retry 3 #重连间隔时间(秒)
            connect_port 80
        }
    }
    real_server 10.0.37.6 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
}
BACKUP DR:
! Configuration File for keepalived
global_defs {
   notification_email {
     qiuyx@app.moonbasagroup.com
   }
   notification_email_from admin@app.moonbasagroup.com
   smtp_server 10.0.65.29
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
vrrp_instance VI_1 {
    state BACKUP  #备机使用BACKUP
    interface eth0
    virtual_router_id 51
    priority 50  #备机的优先级要低于MASTER
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.0.37.10
    }
}
virtual_server 10.0.37.10 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    nat_mask 255.255.255.0
    #persistence_timeout 50
    protocol TCP
    real_server 10.0.37.5 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
    real_server 10.0.37.6 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
}
MASTER DR和BACKUP DR都开启路由转发:
echo “1″ > /proc/sys/net/ipv4/ip_forward
固定到配置文件:
# vim /etc/sysctl.conf
net.ipv4.ip_forward = 1
四、启动keepalived服务(Master DR和Backup DR都启动)
# /etc/init.d/keepalived start
查看日志:
# tail -f /var/log/messages
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_1199533651
查看绑定的VIP地址:
# ip add show eth0
::__IHACKLOG_REMOTE_IMAGE_AUTODOWN_BLOCK__::2
查看lvs的状态:
# ipvsadm -Ln
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_1200111631
五、LVS DR模式下配置后端服务器
Linux服务器:
1.绑定VIP地址
# ifconfig lo:0 10.0.37.10 netmask 255.255.255.255 broadcast 10.0.37.10
# route add -host 10.0.37.10 dev lo:0
2.解决ARP问题
# echo “1″ >/proc/sys/net/ipv4/conf/lo/arp_ignore
# echo “2″ >/proc/sys/net/ipv4/conf/lo/arp_announce
# echo “1″ >/proc/sys/net/ipv4/conf/all/arp_ignore
# echo “2″ >/proc/sys/net/ipv4/conf/all/arp_announce
永久的配置:
1.绑定VIP地址
# vim /etc/sysconfig/network-scripts/ifcfg-lo:0
DEVICE=lo:0
IPADDR=10.0.37.10
NETMASK=255.255.255.255
ONBOOT=yes
NAME=loopback
2.解决ARP问题
# vim /etc/sysctl.conf
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.lo.arp_ignore = 1
Windows服务器(server 2008):
1.”控制面板”–>”设备管理器”–>随意选择一个设备,选择”操作”–>”添加过时硬件”,如下图:
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_1201085081
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_1201766961
点击”下一步”–>选择”安装我手动从列表选择的硬件(高级)”,如下图:
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_1201997841
选择”网络适配器”–>选择”下一步”,如下图:
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_1202423721
在厂商列表中选择”Microsoft”–>在网络适配器列表中选择”Microsoft Loopback Adapter”–>点击”下一步”,如下图:
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_1203326191
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_1193370981
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_1193479401
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_1193580491
“控制面板”–>”网络和共享中心”–>”更改适配器设置”,进行loopback网卡的ip地址设置,如下图:
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_1205962601
在”开始”–>”运行”–>输入”regedit”打开注册表,修改掩码为”255.255.255.255″(通过搜索vip地址可以找到相应的键值),如下图:
我这里的键值路径为”HKEY_LOCAL_MACHINESYSTEMControlSet001servicesTcpipParametersInterfaces{7BFC1EF6-F328-4018-8607-197492DECFEA}”
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_1194964071
开启端口的weakhostsend and weakhostreceive:
netsh interface ipv4 set interface “本地连接 2″ weakhostreceive=enabled
netsh interface ipv4 set interface “本地连接” weakhostreceive=enabled
netsh interface ipv4 set interface “本地连接” weakhostsend=enabled
注:本地连接为网卡设备的名称,在”控制面板”–”网络和共享中心”–>”更改适配器设置”页面可以看到。
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_1187356991
六、测试
1.使用客户端浏览器测试
因是设置的轮询模式,使用浏览器不停刷新时,将会是在不同的后端服务器之间切换。
2.手工停止MASTER DR上的Keepalived服务
停止keepalived服务:
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_235481321
查看日志情况:
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_235726551
ping vip地址情况,延时加大,没有出现丢包:
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_235876941
3.重启启动MASTER DR上的keepalived服务
MASTER DR将重新接管VIP地址:
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_236968631
ping vip地址情况:
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_237161761
4.模拟MASTER DR上的网络出现问题
在MASTER DR上断开网络:
# cat test.sh
#!/bin/sh
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
ifdown eth0
sleep 60
ifup eth0
查看BACKUP DR日志,已经成功接管VIP地址:
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_238802431
但是MASTER DR网络恢复后,不能自动切换到主,日志如下:
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_242006841
5.后端服务器故障
停止10.0.37.6的http服务,查看keepalived的日志情况:
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_248043301
查看LVS状态,已经成功将10.0.37.6剔除(也可以通过设置降低权重,而不删除):
wpid-bdbf776dca5b97dd1d9059a449cb6d6b_1076512041

转载请注明:爱开源 » 部署LVS+Keepalived集群

您必须 登录 才能发表评论!