系统：CentOS 5.8 X86_64
软件：nginx-1.2.4.tar.gz keepalived-1.2.7.tar.gz
当前nginx最新稳定版本为：nginx 1.2.4
拓扑：
一、安装Nginx(MASTER DR和BACKUP DR都安装)
1.安装pcre
# tar xvf pcre-8.31.tar.gz
# cd pcre-8.31
# ./configure –prefix=/usr
# make
# make install
# ldconfig
2.安装Nginx
# tar xvf nginx-1.2.4.tar.gz
# ./configure –prefix=/usr/local/nginx –user=www –group=www –with-http_ssl_module –with-http_realip_module –with-http_gzip_static_module –with-http_stub_status_module
# make
# make install
3.添加用户
# useradd -M -s /sbin/nologin www
4.上传启动脚本
# cp nginx.sh /etc/init.d/nginx
# chkconfig –add nginx
# chkconfig –level 2345 nginx on
二、安装Keepalived(MASTER DR和BACKUP DR都安装)
1.下载keepalived
2.安装keepalived
# tar xvf keepalived-1.2.7.tar.gz -C /usr/local/src/
# cd /usr/local/src/keepalived-1.2.7/
# ./configure –disable-lvs-syncd –disable-lvs && make && make install
注：编译时出现下面的警告
configure: WARNING: keepalived will be built without libnl support.
可以通过安装libnl-devel解决，需要根据keepalived的版本安装相应版本的libnl包
3.拷贝配置文件、启动脚本等
# cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/
# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
# mkdir /etc/keepalived
# cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
# cp /usr/local/sbin/keepalived /usr/sbin/
# chkconfig –add keepalived
# chkconfig –level 2345 keepalived on
三、配置Nginx(MASTER DR和BACKUP DR都一样)
# vim /usr/local/nginx/conf/nginx.conf
user www www; #启动用户和组
worker_processes 2; #
worker_rlimit_nofile 65535;
events {
worker_connections 65535;
}
http {
include mime.types;
default_type application/octet-stream;
server_names_hash_bucket_size 128;
client_header_buffer_size 32k;
large_client_header_buffers 4 32k;
client_max_body_size 8m;
sendfile on;
tcp_nopush on;
keepalive_timeout 60;
tcp_nodelay on;
gzip on;
gzip_min_length 1k;
gzip_buffers 4 16k;
gzip_http_version 1.0;
gzip_comp_level 2;
gzip_types text/plain application/x-javascript text/css application/xml;
gzip_vary on;
log_format main ‘$remote_addr – $remote_user [$time_local] “$request” ‘
‘$status $body_bytes_sent “$http_referer” ‘
‘”$http_user_agent” “$http_x_forwarded_for”‘;
access_log logs/access.log main;
upstream web1 {
server 10.0.37.5:80;
server 10.0.37.6:80;
}
server {
listen 80;
server_name localhost;
#charset koi8-r;
location / {
#proxy_redirect off;
#proxy_set_header Host $host;
#proxy_set_header X-Real-IP $remote_addr;
#proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://web1;
}
error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
四、配置Keepalived(MASTER DR和BACKUP DR都配置，有个别变化)
MASTER DR：
# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
qiuyx@app.moonbasagroup.com
}
notification_email_from admin@app.moonbasagroup.com
smtp_server 10.0.65.29
smtp_connect_timeout 30
router_id Nginx_LB1
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
#mcast_src_ip 10.0.37.3
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.37.7
}
}
BACKUP DR:
! Configuration File for keepalived
global_defs {
notification_email {
qiuyx@app.moonbasagroup.com
}
notification_email_from admin@app.moonbasagroup.com
smtp_server 10.0.65.29
smtp_connect_timeout 30
router_id Nginx_LB2
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 50
#mcast_src_ip 10.0.37.4
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.37.7
}
}
五、测试
1.手工停止MASTER DR的Keepalived服务
# /etc/init.d/keepalived stop
查看日志：
# tail -f /var/log/messages
在BACKUP DR上检查日志和查看VIP绑定：
# tail -f /var/log/messages
# ip add show eth0
Ping VIP情况：
2.MASTER DR上的网络故障
# vim down_network.sh
#!/bin/sh
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
ifdown eth0
sleep 60
ifup eth0
查看BACKUP DR服务器的日志：
查看MASTER DR服务器的日志：
查看BACKUP DR服务器的VIP绑定：
查看MASTER DR服务器的VIP绑定：
ping VIP地址情况:
3.后端服务器故障
停止10.0.37.5的httpd服务，使用浏览器访问VIP地址(10.0.37.7)，在MASTER DR上查看nginx的错误日志，发现访问VIP地址(10.0.37.7)，发现10.0.37.5这台服务器错误：
# tail -f /usr/local/nginx/logs/error.log
默认nginx upstream模块有错误检查的功能，语法如：server address [parameters]
Nginx在检测到后端服务器故障后，nginx依然会把请求转向该服务器，当nginx发现timeout或者refused后，会把改请求会分发到upstream的其它节点，直到获得正常数据后，nginx才会把数据返回给用户。address可以为ip地址、域名、unix socket；域名也可以解析为多个IP地址。
weight：设置服务器权重，默认为1，数字越高，级别越高；
max_fails：设置的最大失败尝试次数，默认为1，0关闭检查；
fail_timeout：默认为10秒，在fail_timeout时间内与后端服务器通信失败的次数超过max_fails设定的次数，将后端服务器标记为不可用，在fail_timeout时间内，nginx不再将请求分给失效的后端服务器；
down：标志服务器作为永久离线，用于ip_hash指令
backup：如果所有服务器都down或忙，将使用backup服务器，不能用于ip_hash指令下。
为了更好的进行后端服务器的状态检查，安装一个后端状态检查补丁：
# unzip yaoweibin-nginx_upstream_check_module-8ec8024.zip
# cd nginx-1.3.5
# patch -p1 < ../yaoweibin-nginx_upstream_check_module-8ec8024/check_1.2.1.patch
# ./configure –prefix=/usr/local/nginx –user=www –group=www –with-http_ssl_module –with-http_realip_module –with-http_gzip_static_module –with-http_stub_status_module –add-module=../yaoweibin-nginx_upstream_check_module-8ec8024/
# make && make install
修改nginx配置文件：
在upstream下增加后台状态检查的语法：
upstream web1 {
server 10.0.37.5:80;
server 10.0.37.6:80;
check interval=3000 rise=2 fall=5 timeout=1000 type=http;
check_http_send “GET / HTTP/1.0rnrn”;
}
停止10.0.37.5的httpd服务，查看MASTER DR服务器上的nginx的错误日志：
停止10.0.37.6的iis服务，查看nginx的错误日志：
4.nginx本身出现故障
通过增加nginx检查页，结合脚本来完成nginx本身的监控状态检查；
新建检查页面：
# vim /usr/local/nginx/conf/nginx.conf
#新增以下段
server {
listen 80;
server_name ngxcheck.test.com;
location / {
root html;
index index.html;
}
}
配置hosts：
# vim /etc/hosts
#新增段
10.0.37.3 ngxcheck.test.com
编写检查脚本，依据返回的状态码进行nginx的健康判断：
# vim ngx_check.sh
#!/bin/bash
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
CHECK_URL=””
REV_CODE=`curl -o /dev/null -s -w %{http_code} $CHECK_URL`
if [[ “$REV_CODE” -ne “200” ]];then
/etc/init.d/keepalived stop
else
exit
fi
给脚本添加执行权限：
# chmod +x /root/ngx_check.sh
修改keepalived配置文件，加载nginx检查脚本：
! Configuration File for keepalived
global_defs {
notification_email {
qiuyx@app.moonbasagroup.com
}
notification_email_from admin@app.moonbasagroup.com
smtp_server 10.0.65.29
smtp_connect_timeout 30
router_id Nginx_LB1
}
vrrp_script ngx_check {
script “/root/ngx_check.sh” #/root/ngx_check.sh为脚本路径
interval 1
weight 1
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
#mcast_src_ip 10.0.37.3
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script { #调用脚本
ngx_check
}
virtual_ipaddress {
10.0.37.7
}
}
启动nginx和keepalived：
# /etc/init.d/nginx reload
# /etc/init.d/keepalived reload
测试nginx故障，删除html的index.html页面：
# rm -f /usr/local/nginx/html/index.html
查看MASTER DR的日志：
查看BACKUP DR的日志：
Ping VIP地址情况：
转载请注明：爱开源 » 部署Nginx+Keepalived集群