最新消息:

部署Nginx+Keepalived集群

nginx admin 4009浏览 0评论
系统:CentOS 5.8 X86_64
软件:nginx-1.2.4.tar.gz keepalived-1.2.7.tar.gz
当前nginx最新稳定版本为:nginx 1.2.4
拓扑:
wpid-04fb4f1ee374ec1e4b97646e5ed26207_2604810152

 

一、安装Nginx(MASTER DR和BACKUP DR都安装)
1.安装pcre
# wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.31.tar.gz
# tar xvf pcre-8.31.tar.gz
# cd pcre-8.31
# ./configure –prefix=/usr
# make
# make install
# ldconfig
2.安装Nginx
# wget http://nginx.org/download/nginx-1.2.4.tar.gz
# tar xvf nginx-1.2.4.tar.gz
# ./configure –prefix=/usr/local/nginx –user=www –group=www –with-http_ssl_module –with-http_realip_module –with-http_gzip_static_module –with-http_stub_status_module
# make
# make install
3.添加用户
# useradd -M -s /sbin/nologin www
4.上传启动脚本
# cp nginx.sh /etc/init.d/nginx
# chkconfig –add nginx
# chkconfig –level 2345 nginx on
二、安装Keepalived(MASTER DR和BACKUP DR都安装)
1.下载keepalived
2.安装keepalived
# tar xvf keepalived-1.2.7.tar.gz -C /usr/local/src/
# cd /usr/local/src/keepalived-1.2.7/
# ./configure –disable-lvs-syncd –disable-lvs && make && make install
注:编译时出现下面的警告
configure: WARNING: keepalived will be built without libnl support.
可以通过安装libnl-devel解决,需要根据keepalived的版本安装相应版本的libnl包
3.拷贝配置文件、启动脚本等
# cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/
# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
# mkdir /etc/keepalived
# cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
# cp /usr/local/sbin/keepalived /usr/sbin/
# chkconfig –add keepalived
# chkconfig –level 2345 keepalived on
三、配置Nginx(MASTER DR和BACKUP DR都一样)
# vim /usr/local/nginx/conf/nginx.conf
user  www www;  #启动用户和组
worker_processes  2;  #
worker_rlimit_nofile 65535;
events {
    worker_connections  65535;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    server_names_hash_bucket_size 128;
    client_header_buffer_size 32k;
    large_client_header_buffers 4 32k;
    client_max_body_size 8m;
    sendfile on;
    tcp_nopush     on;
    keepalive_timeout 60;
    tcp_nodelay on;
    gzip on;
    gzip_min_length 1k;
    gzip_buffers     4 16k;
    gzip_http_version 1.0;
    gzip_comp_level 2;
    gzip_types       text/plain application/x-javascript text/css application/xml;
    gzip_vary on;
    log_format  main  ‘$remote_addr – $remote_user [$time_local] “$request” ‘
                      ‘$status $body_bytes_sent “$http_referer” ‘
                      ‘”$http_user_agent” “$http_x_forwarded_for”‘;
    access_log  logs/access.log  main;
    upstream web1 {
        server 10.0.37.5:80;
        server 10.0.37.6:80;
    }
    server {
        listen       80;
        server_name  localhost;
        #charset koi8-r;
        location / {
            #proxy_redirect off;
            #proxy_set_header Host $host;
            #proxy_set_header X-Real-IP $remote_addr;
            #proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_pass http://web1;
        }
        error_page  404              /404.html;
        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}
四、配置Keepalived(MASTER DR和BACKUP DR都配置,有个别变化)
MASTER DR:
# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
   notification_email {
     qiuyx@app.moonbasagroup.com
   }
   notification_email_from admin@app.moonbasagroup.com
   smtp_server 10.0.65.29
   smtp_connect_timeout 30
   router_id Nginx_LB1
}
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    #mcast_src_ip 10.0.37.3
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.0.37.7
    }
}
BACKUP DR:
! Configuration File for keepalived
global_defs {
   notification_email {
     qiuyx@app.moonbasagroup.com
   }
   notification_email_from admin@app.moonbasagroup.com
   smtp_server 10.0.65.29
   smtp_connect_timeout 30
   router_id Nginx_LB2
}
vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 50
    #mcast_src_ip 10.0.37.4
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.0.37.7
    }
}
五、测试
1.手工停止MASTER DR的Keepalived服务
# /etc/init.d/keepalived stop
查看日志:
# tail -f /var/log/messages
wpid-04fb4f1ee374ec1e4b97646e5ed26207_1930231413
在BACKUP DR上检查日志和查看VIP绑定:
# tail -f /var/log/messages
wpid-04fb4f1ee374ec1e4b97646e5ed26207_1932526963
# ip add show eth0
wpid-04fb4f1ee374ec1e4b97646e5ed26207_1933116963
Ping VIP情况:
wpid-04fb4f1ee374ec1e4b97646e5ed26207_1933779503
2.MASTER DR上的网络故障
# vim down_network.sh
#!/bin/sh
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
ifdown eth0
sleep 60
ifup eth0
查看BACKUP DR服务器的日志:
wpid-04fb4f1ee374ec1e4b97646e5ed26207_1942462823
查看MASTER DR服务器的日志:
wpid-04fb4f1ee374ec1e4b97646e5ed26207_1943166233
查看BACKUP DR服务器的VIP绑定:
wpid-04fb4f1ee374ec1e4b97646e5ed26207_1943624723
查看MASTER DR服务器的VIP绑定:
wpid-04fb4f1ee374ec1e4b97646e5ed26207_1943896163
ping VIP地址情况:
wpid-04fb4f1ee374ec1e4b97646e5ed26207_1944161213
3.后端服务器故障
停止10.0.37.5的httpd服务,使用浏览器访问VIP地址(10.0.37.7),在MASTER DR上查看nginx的错误日志,发现访问VIP地址(10.0.37.7),发现10.0.37.5这台服务器错误:
# tail -f /usr/local/nginx/logs/error.log
wpid-04fb4f1ee374ec1e4b97646e5ed26207_1953960883
默认nginx upstream模块有错误检查的功能,语法如:server address [parameters]
Nginx在检测到后端服务器故障后,nginx依然会把请求转向该服务器,当nginx发现timeout或者refused后,会把改请求会分发到upstream的其它节点,直到获得正常数据后,nginx才会把数据返回给用户。address可以为ip地址、域名、unix socket;域名也可以解析为多个IP地址。
weight:设置服务器权重,默认为1,数字越高,级别越高;
max_fails:设置的最大失败尝试次数,默认为1,0关闭检查;
fail_timeout:默认为10秒,在fail_timeout时间内与后端服务器通信失败的次数超过max_fails设定的次数,将后端服务器标记为不可用,在fail_timeout时间内,nginx不再将请求分给失效的后端服务器;
down:标志服务器作为永久离线,用于ip_hash指令
backup:如果所有服务器都down或忙,将使用backup服务器,不能用于ip_hash指令下。
为了更好的进行后端服务器的状态检查,安装一个后端状态检查补丁:
# unzip yaoweibin-nginx_upstream_check_module-8ec8024.zip
# cd nginx-1.3.5
# patch -p1 < ../yaoweibin-nginx_upstream_check_module-8ec8024/check_1.2.1.patch
wpid-04fb4f1ee374ec1e4b97646e5ed26207_2003916923
# ./configure –prefix=/usr/local/nginx –user=www –group=www –with-http_ssl_module –with-http_realip_module –with-http_gzip_static_module –with-http_stub_status_module –add-module=../yaoweibin-nginx_upstream_check_module-8ec8024/
wpid-04fb4f1ee374ec1e4b97646e5ed26207_1994285583
# make && make install
修改nginx配置文件:
在upstream下增加后台状态检查的语法:
upstream web1 {
        server 10.0.37.5:80;
        server 10.0.37.6:80;
        check interval=3000 rise=2 fall=5 timeout=1000 type=http;
        check_http_send “GET / HTTP/1.0rnrn”;
    }
停止10.0.37.5的httpd服务,查看MASTER DR服务器上的nginx的错误日志:
wpid-04fb4f1ee374ec1e4b97646e5ed26207_2006880163
停止10.0.37.6的iis服务,查看nginx的错误日志:
wpid-04fb4f1ee374ec1e4b97646e5ed26207_2009993013
4.nginx本身出现故障
通过增加nginx检查页,结合脚本来完成nginx本身的监控状态检查;
新建检查页面:
# vim /usr/local/nginx/conf/nginx.conf
#新增以下段
    server {
                listen 80;
                server_name ngxcheck.test.com;
                location / {
                        root html;
                        index index.html;
                }
        }
配置hosts:
# vim /etc/hosts
#新增段
10.0.37.3       ngxcheck.test.com
编写检查脚本,依据返回的状态码进行nginx的健康判断:
# vim ngx_check.sh
#!/bin/bash
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
CHECK_URL=””
REV_CODE=`curl -o /dev/null -s -w %{http_code} $CHECK_URL`
if [[ “$REV_CODE” -ne “200” ]];then
        /etc/init.d/keepalived stop
else
        exit
fi
给脚本添加执行权限:
# chmod +x /root/ngx_check.sh
修改keepalived配置文件,加载nginx检查脚本:
! Configuration File for keepalived
global_defs {
   notification_email {
     qiuyx@app.moonbasagroup.com
   }
   notification_email_from admin@app.moonbasagroup.com
   smtp_server 10.0.65.29
   smtp_connect_timeout 30
   router_id Nginx_LB1
}
vrrp_script ngx_check {
                script “/root/ngx_check.sh”   #/root/ngx_check.sh为脚本路径
                interval 1
                weight 1
}
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    #mcast_src_ip 10.0.37.3
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
track_script {  #调用脚本
        ngx_check
   }
    virtual_ipaddress {
        10.0.37.7
    }
}
启动nginx和keepalived:
# /etc/init.d/nginx reload
# /etc/init.d/keepalived reload
测试nginx故障,删除html的index.html页面:
# rm -f /usr/local/nginx/html/index.html
wpid-04fb4f1ee374ec1e4b97646e5ed26207_2078840613
查看MASTER DR的日志:
wpid-04fb4f1ee374ec1e4b97646e5ed26207_2079895963
查看BACKUP DR的日志:
wpid-04fb4f1ee374ec1e4b97646e5ed26207_2080767233
Ping VIP地址情况:
wpid-04fb4f1ee374ec1e4b97646e5ed26207_2081078143

转载请注明:爱开源 » 部署Nginx+Keepalived集群

您必须 登录 才能发表评论!