安装软件
apt-get -y install swift swift-proxy swift-account swift-container swift-object xfsprogs curl python-pastedeploy
分区
我安装系统的时候,有一个专门的分区给swift使用。分区前,先umount
umount /dev/sda6
格式化分区
mkfs.xfs -f -i size=1024 /dev/sda6
创建挂载点
mkdir /mnt/swift_backend
修改/etc/fstab, 原来是采用uuid,注释掉,加上
/dev/sda6 /mnt/swift_backend xfs noatime,nodiratime,nobarrier,logbufs=8 0 0
检查修改是否正确
mount -a
如果fstab有错误,会进行提示。没错误,就会把目录挂载上。
目录设置
pushd /mnt/swift_backend mkdir node1 node2 node3 node4 popd chown swift.swift /mnt/swift_backend/* for i in {1..4}; do sudo ln -s /mnt/swift_backend/node$i /srv/node$i; done; mkdir -p /etc/swift/account-server /etc/swift/container-server /etc/swift/object-server /srv/node1/device /srv/node2/device /srv/node3/device /srv/node4/device mkdir /run/swift chown -L -R swift.swift /etc/swift /srv/node[1-4]/ /run/swift
为了在系统启动时启动Swift服务,需要把如下两行命令写入 /etc/rc.local里,位置在“exit 0;”之前:
sudo mkdir /run/swift sudo chown swift.swift /run/swift
配置rsync
编辑 /etc/default/rsync文件
sed -i 's/RSYNC_ENABLE=false/RSYNC_ENABLE=true/g' /etc/default/rsync
创建 /etc/rsyncd.conf
cat > /etc/rsyncd.conf <<EOF # General stuff uid = swift gid = swift log file = /var/log/rsyncd.log pid file = /run/rsyncd.pid address = 127.0.0.1 # Account Server replication settings [account6012] max connections = 25 path = /srv/node1/ read only = false lock file = /run/lock/account6012.lock [account6022] max connections = 25 path = /srv/node2/ read only = false lock file = /run/lock/account6022.lock [account6032] max connections = 25 path = /srv/node3/ read only = false lock file = /run/lock/account6032.lock [account6042] max connections = 25 path = /srv/node4/ read only = false lock file = /run/lock/account6042.lock # Container server replication settings [container6011] max connections = 25 path = /srv/node1/ read only = false lock file = /run/lock/container6011.lock [container6021] max connections = 25 path = /srv/node2/ read only = false lock file = /run/lock/container6021.lock [container6031] max connections = 25 path = /srv/node3/ read only = false lock file = /run/lock/container6031.lock [container6041] max connections = 25 path = /srv/node4/ read only = false lock file = /run/lock/container6041.lock # Object Server replication settings [object6010] max connections = 25 path = /srv/node1/ read only = false lock file = /run/lock/object6010.lock [object6020] max connections = 25 path = /srv/node2/ read only = false lock file = /run/lock/object6020.lock [object6030] max connections = 25 path = /srv/node3/ read only = false lock file = /run/lock/object6030.lock [object6040] max connections = 25 path = /srv/node4/ read only = false lock file = /run/lock/object6040.lock EOF
重启rsync服务
service rsync restart
Swift
Swift配置文件
cat >/etc/swift/swift.conf <<EOF [swift-hash] # random unique string that can never change (DO NOT LOSE) swift_hash_path_suffix = `od -t x8 -N 8 -A n </dev/random` EOF
Proxy Server
创建 /etc/swift/proxy-server.conf
cat > /etc/swift/proxy-server.conf <<EOF [DEFAULT] bind_port = 8080 #bind_port = 443 #cert_file = /etc/swift/cert.crt #key_file = /etc/swift/cert.key workers = 8 user = swift log_facility = LOG_LOCAL1 [pipeline:main] pipeline = catch_errors healthcheck cache authtoken keystone proxy-server [app:proxy-server] use = egg:swift#proxy account_autocreate = true [filter:healthcheck] use = egg:swift#healthcheck [filter:cache] use = egg:swift#memcache memcache_servers = 127.0.0.1:11211 [filter:keystone] paste.filter_factory = keystone.middleware.swift_auth:filter_factory operator_roles = Member,admin [filter:authtoken] paste.filter_factory = keystone.middleware.auth_token:filter_factory service_port = 5000 service_host = $MASTER auth_port = 35357 auth_host = $MASTER auth_protocol = http auth_token = $SERVICE_TOKEN admin_token = $SERVICE_TOKEN admin_tenant_name = service admin_user = swift admin_password = $SERVICE_PASSWORD cache = swift.cache [filter:catch_errors] use = egg:swift#catch_errors [filter:swift3] use = egg:swift#swift3 EOF
Account Server, Container Server, Object Server
过程比较复杂,所以就考虑用脚本来搞定
for x in {1..4}; do cat > /etc/swift/account-server/$x.conf <<EOF [DEFAULT] devices = /srv/node$x mount_check = false bind_port = 60${x}2 user = swift log_facility = LOG_LOCAL2 [pipeline:main] pipeline = account-server [app:account-server] use = egg:swift#account [account-replicator] vm_test_mode = no [account-auditor] [account-reaper] EOF cat >/etc/swift/container-server/$x.conf <<EOF [DEFAULT] devices = /srv/node$x mount_check = false bind_ip = 0.0.0.0 bind_port = 60${x}1 user = swift log_facility = LOG_LOCAL2 [pipeline:main] pipeline = container-server [app:container-server] use = egg:swift#container [container-replicator] vm_test_mode = no [container-updater] [container-auditor] [container-sync] EOF cat > /etc/swift/object-server/${x}.conf <<EOF [DEFAULT] devices = /srv/node${x} mount_check = false bind_port = 60${x}0 user = swift log_facility = LOG_LOCAL2 [pipeline:main] pipeline = object-server [app:object-server] use = egg:swift#object [object-replicator] vm_test_mode = no [object-updater] [object-auditor] EOF cat <<EOF >>/etc/swift/container-server.conf [container-sync] EOF done
设置日志
sed -i 's/LOCAL2/LOCAL3/g' /etc/swift/account-server/2.conf sed -i 's/LOCAL2/LOCAL4/g' /etc/swift/account-server/3.conf sed -i 's/LOCAL2/LOCAL5/g' /etc/swift/account-server/4.conf sed -i 's/LOCAL2/LOCAL3/g' /etc/swift/container-server/2.conf sed -i 's/LOCAL2/LOCAL4/g' /etc/swift/container-server/3.conf sed -i 's/LOCAL2/LOCAL5/g' /etc/swift/container-server/4.conf sed -i 's/LOCAL2/LOCAL3/g' /etc/swift/object-server/2.conf sed -i 's/LOCAL2/LOCAL4/g' /etc/swift/object-server/3.conf sed -i 's/LOCAL2/LOCAL5/g' /etc/swift/object-server/4.conf
Ring Server
pushd /etc/swift swift-ring-builder object.builder create 18 3 1 swift-ring-builder container.builder create 18 3 1 swift-ring-builder account.builder create 18 3 1 swift-ring-builder object.builder add z1-127.0.0.1:6010/device 1 swift-ring-builder object.builder add z2-127.0.0.1:6020/device 1 swift-ring-builder object.builder add z3-127.0.0.1:6030/device 1 swift-ring-builder object.builder add z4-127.0.0.1:6040/device 1 swift-ring-builder object.builder rebalance swift-ring-builder container.builder add z1-127.0.0.1:6011/device 1 swift-ring-builder container.builder add z2-127.0.0.1:6021/device 1 swift-ring-builder container.builder add z3-127.0.0.1:6031/device 1 swift-ring-builder container.builder add z4-127.0.0.1:6041/device 1 swift-ring-builder container.builder rebalance swift-ring-builder account.builder add z1-127.0.0.1:6012/device 1 swift-ring-builder account.builder add z2-127.0.0.1:6022/device 1 swift-ring-builder account.builder add z3-127.0.0.1:6032/device 1 swift-ring-builder account.builder add z4-127.0.0.1:6042/device 1 swift-ring-builder account.builder rebalance
启动相关服务
设置目录权限
chown -R swift.swift /etc/swift
启动swift服务
swift-init main start swift-init rest start
验证
-k,是swift账号的密码
swift -v -V 2.0 -A http://127.0.0.1:5000/v2.0/ -U service:swift -K $SERVICE_PASSWORD stat
StorageURL: http://10.1.199.17:8080/v1/AUTH_a8b0b44cb5db4da39b053eabac6d3ed7 Auth Token: 3f85c92d6860444e90bf0e1bedc4b45a Account: AUTH_a8b0b44cb5db4da39b053eabac6d3ed7 Containers: 0 Objects: 0 Bytes: 0 Accept-Ranges: bytes X-Trans-Id: txea28887460ff4f1d84e9e826e5514711
你也可以直接运行 swift stat. 这时候是直接采用 租户/用户 admin/admin 去查询swift。因为我们设置了环境变量。
swift stat Account: AUTH_eb68709e74314aa59c449510a91f8d56 Containers: 0 Objects: 0 Bytes: 0 Accept-Ranges: bytes X-Trans-Id: txc5a3afa7f228471698c96fd561830a3d
Glance集成Swift
编辑 /etc/glance/glance-api.conf
#default_store = file default_store = swift #swift_store_auth_address = 127.0.0.1:35357/v2.0/ swift_store_auth_address = http://10.1.199.8:5000/v2.0/ #swift_store_user = jdoe:jdoe swift_store_user = service:swift #swift_store_key = a86850deb2742ec3cb41518e26aa2d89 swift_store_key = password #swift_store_create_container_on_put = False swift_store_create_container_on_put = True
说明
- swift_store_auth_addres 不能去掉http,否则会导致认证失败
- swift_store_key , 我理解就是swift的密码,也就是 租户 service,用户 swift的密码。
可以直接运行下面命令实现修改
sed -i "/default_store/s/file/swift/; /swift_store_auth_address/s/127.0.0.1:35357/$MASTER:5000/; /swift_store_user/s/jdoe:jdoe/service:swift/; /swift_store_key/s/a86850deb2742ec3cb41518e26aa2d89/$SERVICE_PASSWORD/; /swift_store_create_container_on_put/s/False/True/" /etc/glance/glance-api.conf
重启glance服务
service glance-api restart && service glance-registry restart
检测
这个时候,image就会传到swift上。在dashboard里,也可以上传文件。并且snapshot可以上传到swift上。
swift -V 2 -A <a href="http://%24master:5000/v2.0">http://$MASTER:5000/v2.0</a> -U service:swift -K $SERVICE_PASSWORD stat swift -V 2 -A <a href="http://%24master:5000/v2.0">http://$MASTER:5000/v2.0</a> -U service:swift -K $SERVICE_PASSWORD list
上面命令可以查看上传的image
没上传镜像前
# swift -V 2 -A http://$MASTER:5000/v2.0 -U service:swift -K $SERVICE_PASSWORD stat Account: AUTH_678c42aa31114faeb18add84615b4e83 Containers: 0 Objects: 0 Bytes: 0 Accept-Ranges: bytes X-Trans-Id: tx72707ce7086c4bf0bc72ff7ec2813a27 # swift -V 2 -A http://$MASTER:5000/v2.0 -U service:swift -K $SERVICE_PASSWORD list
上传镜像后
# swift -V 2 -A http://$MASTER:5000/v2.0 -U service:swift -K $SERVICE_PASSWORD stat Account: AUTH_678c42aa31114faeb18add84615b4e83 Containers: 1 Objects: 0 Bytes: 0 Accept-Ranges: bytes X-Trans-Id: tx65d1d1ee502b4960839f8196b76813f6 # swift -V 2 -A http://$MASTER:5000/v2.0 -U service:swift -K $SERVICE_PASSWORD list glance
其中:-V 2 指示为keystone验证; IP为keystone节点IP;service:swift为tanent:user ;-K为password
swift -V 2 -A <a href="http://%24master:5000/v2.0">http://$MASTER:5000/v2.0</a> -U admin:admin -K $OS_PASSWORD upload test /root/CentOS-6.2-x86_64-bin-DVD1.iso