2010年12月23日 星期四

Nginx+Keepalived(HA+LoadBalance)



 4台伺服器,一台做主LB,一台做備用LB,另2台做web伺服器;
1、主調度器的ip10.10.10.1 0/24 192.168.1.25/24
2、備用調度器的ip: 10.10.10.20 /24192.168.1.26/24
3、 外部vip ip10.10.10.1 
4、兩台web伺服器的ip分別為:192.168.1.1/24192.168.1.2/24
5MASTERBACKUP :nginx+keepalived;  WEB SERVER可以是IIS, Apache, nginx, Lighttpd…等等
6MASTERBACKUP部份, 目前是作為Reverse proxy(反向代理)同時加上nginx本身cache功能,並配合keepalived HA功能!


在主調度器上(MASTER)設定
在安裝Keepalived軟體前,首先檢查你的系统是否已經安裝好以下套件:
Kernel-devel, gcc, openssl, openssl-devel,
一般DVD光碟就有,沒有就安裝它吧!!
例如:
#rpm -ivh kernel-devel-2.6.18-194.el5.i686.rpm

設定kernel source soft link因等下編譯要用到
#ln -s /usr/src/kernels/2.6.18-194.el5-i686/ /usr/src/linux

下載keepalive

安裝keepalived
# tar zxvf keepalived-1.2.1.tar.gz
# cd keepalived-1.2.1
# ./configure --prefix=/usr/local/keepalived --sysconfdir=/etc
# make && make install
# ln -s /usr/local/keepalived/sbin/keepalived /usr/sbin/

編輯keepalived 檔案
內容如下
# vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id LVS_DEVEL
}
vrrp_sync_group VG1 {
   group {
VI_1
   }
}
vrrp_instance VI_1 {
    state MASTER        //主調度器
    interface eth0
    virtual_router_id 61
    priority 150         //優先權
    advert_int 1

authentication {       //認證
auth_type PASS
auth_pass 1111
    }
virtual_ipaddress {
10.10.10.1         //這裡是設定多VIP地方
}
}

啟動keepalived
# service keepalived start
Starting keepalived: [  OK  ]
                                       
備註: 1. # service keepalived
Usage: /etc/init.d/keepalived {start|stop|reload|restart|condrestart|status}
  
OK!  MASTER 主調度器已設定完成, 接下來設定備用調度器, 設定和MASTER差不多一樣只有Keepalived.conf小小改變就好,

在備用調度器上(BACKUP)設定
keepalived 套件方式同MASTER, 只有keepalived.conf檔案有些許要改就好, 只有stateBACKUPpriority改成比MASTER低就好, 如下所示

編輯keepalived 檔案
vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id LVS_DEVEL
}
vrrp_sync_group VG1 {
   group {
       VI_1
    }
}
vrrp_instance VI_1 {
    state BACKUP        //備用調度器
    interface eth0
    virtual_router_id 61
    priority 100         //優先權
    advert_int 1

authentication {      //認證
auth_type PASS
auth_pass 1111
    }
virtual_ipaddress {
10.10.10.1         //這裡是設定多VIP地方
}
}

自動執行
MASTER BACKUP  2,都同樣設定如下

# chkconfig --add keepalived
# chkconfig --level 3 keepalived on
# chkconfig  keepalived --list
keepalived     0:off1:off2:off3:on4:off5:off6:off

安裝及設定Nginx Reverse Proxy&Cache
請參考我另一文章- Nginx - Reverse Proxy+Web Cache
注意: nginx.conflisten 10.10.10.1:80; 要改成 :80就好,不要帶有IP, 要不然會和原VIP衝突!!

設定簡易iptables ( firewall )
內容如下
# vi /usr/local/ipfw.sh
#!/bin/bash

/sbin/iptables -F
/sbin/iptables -X
/sbin/iptables -Z
/sbin/iptables -P INPUT DROP
/sbin/iptables -P OUTPUT ACCEPT
/sbin/iptables -P FORWARD ACCEPT
#----------------------------------------------------------------------------
/sbin/iptables -A INPUT -i lo -j ACCEPT
/sbin/iptables -A INPUT -i eth1 -s 192.168.1.0/24 -j ACCEPT
/sbin/iptables -A INPUT -i eth0 -p tcp --dport 80 -j ACCEPT
/sbin/iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
/sbin/iptables -A INPUT -m state --state INVALID -j DROP
/sbin/iptables -A INPUT -i eth0 -p tcp -j REJECT --reject-with tcp-reset
/sbin/iptables -A INPUT -i eth0 -p tcp -j REJECT --reject-with icmp-port-unreachable
存檔退出即可

#chmod +x /usr/local/ipfw.sh

加入 /etc/rc.local
#vi /etc/rc.local
/usr/local/ipfw.sh

優化 kernel 一下(Optional)
#vi /etc/sysctl.conf
net.ipv4.tcp_max_syn_backlog=8192
net.ipv4.tcp_fin_timeout=30
net.ipv4.tcp_rmem=4096 87380 16777216
net.ipv4.tcp_wmem=4096 65536 16777216
net.core.netdev_max_backlog=300
net.core.rmem_max=16777216
net.core.wmem_max=16777216

#sysctl –p   //使sysctl.conf生效
大功告成, 測試沒問題可以自動切換!!!

沒有留言:

張貼留言