背景:
去年(2018年)正好赶上数据库需要存储扩容。在各方协调下,我们借到并测试了 NetApp A700 All Flash Arrays。
因为测试得出的性能报告,是根据我们特定的业务场景和具体的硬件设备测试所得,所以我这里测试得出的存储的性能报告未必对大家都受用。也就不公开分享了。
存储性能的三大关键指标(IOPS、 Throughput 吞吐量、 Latency 访问时延) IOPS: 和盘的数量、数据块大小有关。 Throughput 吞吐量: 和数据块大小也是相关的。 Latency 访问时延: 是指块存储处理一个I/O需要的时间。 所以,存储的性能数据需要有个基准,一般是需要具体测试的。
虽说由于种种原因最终没有采购使用,但还是非常感谢当时的领导、同事以及 NetApp 厂商、代理商的支持。
整理此文「NetApp A700 存储 使用 multipath 多路径管理软件」与大家分享。
Revision V1.4
No. | Date | Author/Modifier | Comments |
---|---|---|---|
1.0 | 2018-07-20 | 谈权 | 初稿 |
1.1 | 2018-08-23 | 谈权 | 完善自动生成各个配置文件的脚本 |
1.2 | 2018-08-24 | 谈权 | 增加【附2:配置 multipath 基本使用方法】 |
1.3 | 2018-09-03 | 谈权 | 增加【附3:删除LUN的操作】 |
1.4 | 2018-09-05 | 谈权 | 增加【附4:网卡配置中添加 hotplug=no 参数,避免 start_udev 命令导致Oracle RAC 的vip漂移问题】 |
Table of Contents
步骤1: 生成 lun_info.txt 文件。(格式化 sanlun lun show
输出,见附1)
# sanlun lun show | awk '{a[$2]=$3;b[$2]+=!c[$3]++;d[$2]=$(NF-1)}END{for(i in a){print i,a[i],b[i],d[i]}}' | sort -n | grep -v filename | grep -v device | grep -v '[\-]' | grep -v unknown > /root/test/20180801/lun_info_20180801.txt
步骤2: 生成多路径配置文件 /etc/multipath.conf
generate_multipath.sh
生成多路径配置文件( /etc/multipath.conf
)
#!/bin/bash
lun_file=/root/test/20180801/lun_info_20180801.txt
echo "
# NetApp A700 `date '+%Y-%m-%d %H:%M:%S'`
defaults {
user_friendly_names no
max_fds max
flush_on_last_del yes
queue_without_daemon no
}
# All data under blacklist must be specific to your system.
blacklist {
devnode \"^hd[a-z]\"
devnode \"^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*\"
devnode \"^cciss.*\"
}
devices {
device {
vendor \"NETAPP\"
product \"LUN\"
path_grouping_policy group_by_prio
features \"3 queue_if_no_path pg_init_retries 50\"
prio \"alua\"
path_checker tur
failback immediate
path_selector \"round-robin 0\"
hardware_handler \"1 alua\"
rr_weight uniform
rr_min_io 128
getuid_callout \"/lib/udev/scsi_id -g -u -d /dev/%n\"
}
}
"
{
echo "multipaths {"
cat $lun_file | awk '{gsub(/.*\//,"",$1);print $1,$2}' | while read t1 t2;do a=$(/lib/udev/scsi_id --whitelisted --device=$t2); echo "
multipath {
wwid ${a}
alias netapp-${t1}
}";done
echo "}"
} | grep -v "^$"
生成 /etc/multipath.conf
多路径配置文件
[root@dbtan21: ~/test/20180801]# sh generate_multipath.sh > /etc/multipath.conf
步骤2-1: 格式化输出 multipath -ll
信息,方便查看链路状态
format_multipath-ll.sh
[root@dbtan22: ~/test/20180808]# cat format_multipath-ll.sh
#!/bin/bash
multipath_file=/root/test/20180808/dbtan22_multipath-ll.txt
cat ${multipath_file} |awk $'{
if($0~/^netapp/)
{
line=$0
getline tmp
line=line" "tmp
gsub(/features=.*wp=/,"wp=",line)
if(status!="")
{
print status" active_count="int(active_count)" failed_count="int(failed_count)
}
status=""
active_count=""
failed_count=""
print line
}
if($0~/status=active/ || $0~/status=enabled/)
{
if(status!="")
{
print status" active_count="int(active_count)" failed_count="int(failed_count)
}
status=$0
active_count=""
failed_count=""
}
if($0~/active ready running/)
{
active_count++
}
if($0~/failed faulty running/)
{
failed_count++
}
}
END{
if(status!="")
{
print status" active_count="int(active_count)" failed_count="int(failed_count)
}
}'
[root@dbtan22: ~/test/20180808]#
执行 format_multipath-ll.sh
脚本,统计每个 LUN 的主/备(active/enabled) 的活跃/失败(active_count/failed_count) 链路个数
[root@dbtan22: ~/test/20180808]# sh format_multipath-ll.sh
netapp-lun19 (3600a09803830475a4c2b4d3059494955) dm-10 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun36 (3600a09803830475a4d3f4d3072414248) dm-28 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun21 (3600a09803830475a4c2b4d3059494957) dm-12 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun68 (3600a09803830475a4c2b4d305949496a) dm-59 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun53 (3600a09803830475a4d3f4d3072414259) dm-44 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun70 (3600a09803830475a4c2b4d305949496c) dm-61 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lunvd12 (3600a09803830475a4d3f4d3072414273) dm-84 NETAPP,LUN C-Mode size=80G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun18 (3600a09803830475a4c2b4d3059494954) dm-7 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun35 (3600a09803830475a4d3f4d3072414247) dm-27 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun20 (3600a09803830475a4c2b4d3059494956) dm-11 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun67 (3600a09803830475a4d3f4d307241426c) dm-58 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun52 (3600a09803830475a4d3f4d3072414258) dm-43 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lunvd09 (3600a09803830475a4d3f4d3072414271) dm-80 NETAPP,LUN C-Mode size=80G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lunvd11 (3600a09803830475a4d3f4d3072414274) dm-83 NETAPP,LUN C-Mode size=80G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun17 (3600a09803830475a4c2b4d3059494953) dm-9 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun49 (3600a09803830475a4d3f4d3072414255) dm-40 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun34 (3600a09803830475a4c2b4d3059494969) dm-26 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun66 (3600a09803830475a4d3f4d307241426b) dm-57 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun51 (3600a09803830475a4d3f4d3072414257) dm-42 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lunvd08 (3600a09803830475a4d3f4d3072414270) dm-79 NETAPP,LUN C-Mode size=80G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lunvd10 (3600a09803830475a4d3f4d3072414272) dm-81 NETAPP,LUN C-Mode size=80G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun16 (3600a09803830475a4c2b4d3059494952) dm-8 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun48 (3600a09803830475a4d3f4d3072414254) dm-39 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun33 (3600a09803830475a4c2b4d3059494968) dm-24 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun65 (3600a09803830475a4d3f4d307241426a) dm-56 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun50 (3600a09803830475a4d3f4d3072414256) dm-41 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lunvd07 (3600a09803830475a4d3f4d307241426f) dm-78 NETAPP,LUN C-Mode size=80G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun15 (3600a09803830475a4c2b4d3059494951) dm-6 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun47 (3600a09803830475a4d3f4d3072414253) dm-38 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun32 (3600a09803830475a4c2b4d3059494967) dm-25 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun64 (3600a09803830475a4d3f4d3072414269) dm-55 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lunvd06 (3600a09803830475a4c2b4d305949497a) dm-77 NETAPP,LUN C-Mode size=80G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun29 (3600a09803830475a4c2b4d3059494964) dm-22 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun14 (3600a09803830475a4c2b4d3059494950) dm-5 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun46 (3600a09803830475a4d3f4d3072414252) dm-37 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun31 (3600a09803830475a4c2b4d3059494966) dm-23 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun63 (3600a09803830475a4d3f4d3072414268) dm-54 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lunvd05 (3600a09803830475a4c2b4d3059494979) dm-76 NETAPP,LUN C-Mode size=80G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun28 (3600a09803830475a4c2b4d3059494963) dm-19 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun13 (3600a09803830475a4c2b4d305949494f) dm-3 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun45 (3600a09803830475a4d3f4d3072414251) dm-36 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun30 (3600a09803830475a4c2b4d3059494965) dm-21 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun103 (3600a09803830475a4d3f4d307241426e) dm-71 NETAPP,LUN C-Mode size=10G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun77 (3600a09803830475a4c2b4d3059494973) dm-68 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun62 (3600a09803830475a4d3f4d3072414267) dm-53 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lunvd04 (3600a09803830475a4c2b4d3059494978) dm-75 NETAPP,LUN C-Mode size=80G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun27 (3600a09803830475a4c2b4d3059494962) dm-18 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun12 (3600a09803830475a4c2b4d305949494e) dm-2 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun59 (3600a09803830475a4d3f4d3072414264) dm-50 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun44 (3600a09803830475a4d3f4d3072414250) dm-35 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun102 (3600a09803830475a4d3f4d307241426d) dm-70 NETAPP,LUN C-Mode size=10G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun76 (3600a09803830475a4c2b4d3059494972) dm-67 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun61 (3600a09803830475a4d3f4d3072414266) dm-52 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lunvd03 (3600a09803830475a4c2b4d3059494977) dm-74 NETAPP,LUN C-Mode size=80G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun26 (3600a09803830475a4c2b4d3059494961) dm-17 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun11 (3600a09803830475a4c2b4d305949494d) dm-4 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun58 (3600a09803830475a4d3f4d3072414263) dm-49 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun43 (3600a09803830475a4d3f4d307241424f) dm-34 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun101 (3600a09803830475a4c2b4d3059494974) dm-69 NETAPP,LUN C-Mode size=10G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun75 (3600a09803830475a4c2b4d3059494971) dm-66 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun60 (3600a09803830475a4d3f4d3072414265) dm-51 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lunvd02 (3600a09803830475a4c2b4d3059494976) dm-73 NETAPP,LUN C-Mode size=80G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun25 (3600a09803830475a4c2b4d305949492f) dm-13 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun57 (3600a09803830475a4d3f4d3072414262) dm-48 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun42 (3600a09803830475a4d3f4d307241424e) dm-33 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun74 (3600a09803830475a4c2b4d3059494970) dm-65 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lunvd01 (3600a09803830475a4c2b4d3059494975) dm-72 NETAPP,LUN C-Mode size=80G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun39 (3600a09803830475a4d3f4d307241424b) dm-20 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun24 (3600a09803830475a4c2b4d305949495a) dm-16 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun56 (3600a09803830475a4d3f4d3072414261) dm-47 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun41 (3600a09803830475a4d3f4d307241424d) dm-32 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun73 (3600a09803830475a4c2b4d305949496f) dm-64 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun38 (3600a09803830475a4d3f4d307241424a) dm-29 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun23 (3600a09803830475a4c2b4d3059494959) dm-15 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun55 (3600a09803830475a4d3f4d307241422f) dm-46 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun40 (3600a09803830475a4d3f4d307241424c) dm-31 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun72 (3600a09803830475a4c2b4d305949496e) dm-63 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun37 (3600a09803830475a4d3f4d3072414249) dm-30 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun22 (3600a09803830475a4c2b4d3059494958) dm-14 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun69 (3600a09803830475a4c2b4d305949496b) dm-60 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun54 (3600a09803830475a4d3f4d307241425a) dm-45 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
netapp-lun71 (3600a09803830475a4c2b4d305949496d) dm-62 NETAPP,LUN C-Mode size=400G wp=rw
|-+- policy='round-robin 0' prio=50 status=active active_count=4 failed_count=0
`-+- policy='round-robin 0' prio=10 status=enabled active_count=4 failed_count=0
[root@dbtan22: ~/test/20180808]#
步骤3:生成绑定文件 /etc/multipath/bindings
(可不做,由于多路径配置文件中 user_friendly_names no
)
cat generate_bindings.sh
生成绑定文件 (/etc/multipath/bindings
)
#!/bin/bash
lun_file=/root/test/20180801/lun_info_20180801.txt
{
echo "# NetApp A700 `date '+%Y-%m-%d %H:%M:%S'`"
cat $lun_file | awk '{gsub(/.*\//,"",$1);print $1,$2}' | while read t1 t2;do a=$(/lib/udev/scsi_id --whitelisted --device=$t2); echo "
mpath${t1} ${a}
";done
} | grep -v "^$"
生成 /etc/multipath/bindings
多路径绑定配置文件
[root@dbtan21: ~/test/20180801]# sh generate_bindings.sh >> /etc/multipath/bindings
步骤4:生成 99-oracle-asmdevices.rules
规则
generate_99-oracle-asmdevices.rules.sh
#!/bin/bash
lun_file=/root/test/20180801/lun_info_20180801.txt
{
cat $lun_file | awk '{gsub(/.*\//,"",$1);print $1,$2}' | while read t1 t2;do a=$(/lib/udev/scsi_id --whitelisted --device=$t2); echo "
KERNEL==\"dm*\",SUBSYSTEM==\"block\", PROGRAM==\"/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/\$name\", RESULT==\"$a\", NAME=\"asm-$t1\", OWNER=\"grid\", GROUP=\"asmadmin\", MODE=\"0660\"
";done
} | grep -v "^$"
生成 /etc/udev/rules.d/99-oracle-asmdevices.rules
规则
[root@dbtan21: ~/test/20180801]# sh generate_99-oracle-asmdevices.rules.sh | grep -v asm-lunvd > /etc/udev/rules.d/99-oracle-asmdevices.rules
[root@dbtan21: ~/test/20180801]#
步骤5:对比2个节点的 /dev/asm-lunX
块设备(block)的 wwid
generate_ASM_wwid.sh
#!/bin/bash
lun_file=/root/test/20180801/lun_info_20180801.txt
{
cat $lun_file | awk '{gsub(/.*\//,"",$1);print $1,$2}' | while read t1 t2;do a=$(/lib/udev/scsi_id --whitelisted --device=$t2); echo "
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-$t1
`/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-$t1`
";done
} | grep -v "^$"
生成节点1(dbtan21) /dev/asm-lunX 块设备(block)的 wwid
[root@dbtan21: ~/test/20180801]# sh generate_ASM_wwid.sh
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun101
3600a09803830475a4c2b4d3059494974
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun102
3600a09803830475a4d3f4d307241426d
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun103
3600a09803830475a4d3f4d307241426e
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun11
3600a09803830475a4c2b4d305949494d
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun12
3600a09803830475a4c2b4d305949494e
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun13
3600a09803830475a4c2b4d305949494f
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun14
3600a09803830475a4c2b4d3059494950
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun15
3600a09803830475a4c2b4d3059494951
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun16
3600a09803830475a4c2b4d3059494952
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun17
3600a09803830475a4c2b4d3059494953
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun18
3600a09803830475a4c2b4d3059494954
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun19
3600a09803830475a4c2b4d3059494955
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun20
3600a09803830475a4c2b4d3059494956
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun21
3600a09803830475a4c2b4d3059494957
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun22
3600a09803830475a4c2b4d3059494958
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun23
3600a09803830475a4c2b4d3059494959
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun24
3600a09803830475a4c2b4d305949495a
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun25
3600a09803830475a4c2b4d305949492f
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun26
3600a09803830475a4c2b4d3059494961
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun27
3600a09803830475a4c2b4d3059494962
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun28
3600a09803830475a4c2b4d3059494963
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun29
3600a09803830475a4c2b4d3059494964
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun30
3600a09803830475a4c2b4d3059494965
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun31
3600a09803830475a4c2b4d3059494966
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun32
3600a09803830475a4c2b4d3059494967
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun33
3600a09803830475a4c2b4d3059494968
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun34
3600a09803830475a4c2b4d3059494969
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun35
3600a09803830475a4d3f4d3072414247
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun36
3600a09803830475a4d3f4d3072414248
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun37
3600a09803830475a4d3f4d3072414249
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun38
3600a09803830475a4d3f4d307241424a
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun39
3600a09803830475a4d3f4d307241424b
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun40
3600a09803830475a4d3f4d307241424c
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun41
3600a09803830475a4d3f4d307241424d
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun42
3600a09803830475a4d3f4d307241424e
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun43
3600a09803830475a4d3f4d307241424f
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun44
3600a09803830475a4d3f4d3072414250
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun45
3600a09803830475a4d3f4d3072414251
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun46
3600a09803830475a4d3f4d3072414252
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun47
3600a09803830475a4d3f4d3072414253
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun48
3600a09803830475a4d3f4d3072414254
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun49
3600a09803830475a4d3f4d3072414255
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun50
3600a09803830475a4d3f4d3072414256
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun51
3600a09803830475a4d3f4d3072414257
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun52
3600a09803830475a4d3f4d3072414258
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun53
3600a09803830475a4d3f4d3072414259
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun54
3600a09803830475a4d3f4d307241425a
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun55
3600a09803830475a4d3f4d307241422f
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun56
3600a09803830475a4d3f4d3072414261
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun57
3600a09803830475a4d3f4d3072414262
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun58
3600a09803830475a4d3f4d3072414263
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun59
3600a09803830475a4d3f4d3072414264
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun60
3600a09803830475a4d3f4d3072414265
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun61
3600a09803830475a4d3f4d3072414266
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun62
3600a09803830475a4d3f4d3072414267
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun63
3600a09803830475a4d3f4d3072414268
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun64
3600a09803830475a4d3f4d3072414269
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun65
3600a09803830475a4d3f4d307241426a
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun66
3600a09803830475a4d3f4d307241426b
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun67
3600a09803830475a4d3f4d307241426c
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun68
3600a09803830475a4c2b4d305949496a
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun69
3600a09803830475a4c2b4d305949496b
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun70
3600a09803830475a4c2b4d305949496c
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun71
3600a09803830475a4c2b4d305949496d
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun72
3600a09803830475a4c2b4d305949496e
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun73
3600a09803830475a4c2b4d305949496f
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun74
3600a09803830475a4c2b4d3059494970
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun75
3600a09803830475a4c2b4d3059494971
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun76
3600a09803830475a4c2b4d3059494972
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun77
3600a09803830475a4c2b4d3059494973
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd01
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd02
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd03
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd04
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd05
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd06
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd07
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd08
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd09
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd10
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd11
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd12
[root@dbtan21: ~/test/20180801]#
生成节点2(dbtan22) /dev/asm-lunX 块设备(block)的 wwid
[root@dbtan22: ~/test/20180801]# sh generate_ASM_wwid.sh
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun101
3600a09803830475a4c2b4d3059494974
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun102
3600a09803830475a4d3f4d307241426d
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun103
3600a09803830475a4d3f4d307241426e
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun11
3600a09803830475a4c2b4d305949494d
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun12
3600a09803830475a4c2b4d305949494e
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun13
3600a09803830475a4c2b4d305949494f
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun14
3600a09803830475a4c2b4d3059494950
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun15
3600a09803830475a4c2b4d3059494951
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun16
3600a09803830475a4c2b4d3059494952
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun17
3600a09803830475a4c2b4d3059494953
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun18
3600a09803830475a4c2b4d3059494954
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun19
3600a09803830475a4c2b4d3059494955
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun20
3600a09803830475a4c2b4d3059494956
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun21
3600a09803830475a4c2b4d3059494957
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun22
3600a09803830475a4c2b4d3059494958
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun23
3600a09803830475a4c2b4d3059494959
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun24
3600a09803830475a4c2b4d305949495a
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun25
3600a09803830475a4c2b4d305949492f
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun26
3600a09803830475a4c2b4d3059494961
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun27
3600a09803830475a4c2b4d3059494962
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun28
3600a09803830475a4c2b4d3059494963
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun29
3600a09803830475a4c2b4d3059494964
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun30
3600a09803830475a4c2b4d3059494965
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun31
3600a09803830475a4c2b4d3059494966
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun32
3600a09803830475a4c2b4d3059494967
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun33
3600a09803830475a4c2b4d3059494968
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun34
3600a09803830475a4c2b4d3059494969
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun35
3600a09803830475a4d3f4d3072414247
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun36
3600a09803830475a4d3f4d3072414248
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun37
3600a09803830475a4d3f4d3072414249
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun38
3600a09803830475a4d3f4d307241424a
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun39
3600a09803830475a4d3f4d307241424b
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun40
3600a09803830475a4d3f4d307241424c
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun41
3600a09803830475a4d3f4d307241424d
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun42
3600a09803830475a4d3f4d307241424e
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun43
3600a09803830475a4d3f4d307241424f
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun44
3600a09803830475a4d3f4d3072414250
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun45
3600a09803830475a4d3f4d3072414251
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun46
3600a09803830475a4d3f4d3072414252
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun47
3600a09803830475a4d3f4d3072414253
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun48
3600a09803830475a4d3f4d3072414254
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun49
3600a09803830475a4d3f4d3072414255
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun50
3600a09803830475a4d3f4d3072414256
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun51
3600a09803830475a4d3f4d3072414257
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun52
3600a09803830475a4d3f4d3072414258
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun53
3600a09803830475a4d3f4d3072414259
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun54
3600a09803830475a4d3f4d307241425a
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun55
3600a09803830475a4d3f4d307241422f
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun56
3600a09803830475a4d3f4d3072414261
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun57
3600a09803830475a4d3f4d3072414262
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun58
3600a09803830475a4d3f4d3072414263
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun59
3600a09803830475a4d3f4d3072414264
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun60
3600a09803830475a4d3f4d3072414265
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun61
3600a09803830475a4d3f4d3072414266
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun62
3600a09803830475a4d3f4d3072414267
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun63
3600a09803830475a4d3f4d3072414268
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun64
3600a09803830475a4d3f4d3072414269
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun65
3600a09803830475a4d3f4d307241426a
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun66
3600a09803830475a4d3f4d307241426b
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun67
3600a09803830475a4d3f4d307241426c
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun68
3600a09803830475a4c2b4d305949496a
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun69
3600a09803830475a4c2b4d305949496b
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun70
3600a09803830475a4c2b4d305949496c
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun71
3600a09803830475a4c2b4d305949496d
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun72
3600a09803830475a4c2b4d305949496e
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun73
3600a09803830475a4c2b4d305949496f
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun74
3600a09803830475a4c2b4d3059494970
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun75
3600a09803830475a4c2b4d3059494971
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun76
3600a09803830475a4c2b4d3059494972
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lun77
3600a09803830475a4c2b4d3059494973
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd01
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd02
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd03
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd04
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd05
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd06
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd07
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd08
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd09
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd10
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd11
/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/asm-lunvd12
[root@dbtan22: ~/test/20180801]#
附1:查看格式化后的 sanlun lun show
输出
[root@dbtan22: ~/test/20180801]# cat lun_info_20180801.txt
/vol/dbtan2122_ocr1/lun101 /dev/sdbr 8 10g
/vol/dbtan2122_ocr2/lun102 /dev/sdbv 8 10g
/vol/dbtan2122_ocr3/lun103 /dev/sdbw 8 10g
/vol/dbtan2122_vol11/lun11 /dev/sdb 8 400.0g
/vol/dbtan2122_vol12/lun12 /dev/sdd 8 400.0g
/vol/dbtan2122_vol13/lun13 /dev/sde 8 400.0g
/vol/dbtan2122_vol14/lun14 /dev/sdf 8 400.0g
/vol/dbtan2122_vol15/lun15 /dev/sdg 8 400.0g
/vol/dbtan2122_vol16/lun16 /dev/sdh 8 400.0g
/vol/dbtan2122_vol17/lun17 /dev/sdi 8 400.0g
/vol/dbtan2122_vol18/lun18 /dev/sdj 8 400.0g
/vol/dbtan2122_vol19/lun19 /dev/sdk 8 400.0g
/vol/dbtan2122_vol20/lun20 /dev/sdl 8 400.0g
/vol/dbtan2122_vol21/lun21 /dev/sdm 8 400.0g
/vol/dbtan2122_vol22/lun22 /dev/sdn 8 400.0g
/vol/dbtan2122_vol23/lun23 /dev/sdo 8 400.0g
/vol/dbtan2122_vol24/lun24 /dev/sdp 8 400.0g
/vol/dbtan2122_vol25/lun25 /dev/sdq 8 400.0g
/vol/dbtan2122_vol26/lun26 /dev/sdr 8 400.0g
/vol/dbtan2122_vol27/lun27 /dev/sds 8 400.0g
/vol/dbtan2122_vol28/lun28 /dev/sdt 8 400.0g
/vol/dbtan2122_vol29/lun29 /dev/sdu 8 400.0g
/vol/dbtan2122_vol30/lun30 /dev/sdv 8 400.0g
/vol/dbtan2122_vol31/lun31 /dev/sdw 8 400.0g
/vol/dbtan2122_vol32/lun32 /dev/sdx 8 400.0g
/vol/dbtan2122_vol33/lun33 /dev/sdy 8 400.0g
/vol/dbtan2122_vol34/lun34 /dev/sdz 8 400.0g
/vol/dbtan2122_vol35/lun35 /dev/sdaa 8 400.0g
/vol/dbtan2122_vol36/lun36 /dev/sdab 8 400.0g
/vol/dbtan2122_vol37/lun37 /dev/sdac 8 400.0g
/vol/dbtan2122_vol38/lun38 /dev/sdad 8 400.0g
/vol/dbtan2122_vol39/lun39 /dev/sdae 8 400.0g
/vol/dbtan2122_vol40/lun40 /dev/sdaf 8 400.0g
/vol/dbtan2122_vol41/lun41 /dev/sdag 8 400.0g
/vol/dbtan2122_vol42/lun42 /dev/sdah 8 400.0g
/vol/dbtan2122_vol43/lun43 /dev/sdai 8 400.0g
/vol/dbtan2122_vol44/lun44 /dev/sdaj 8 400.0g
/vol/dbtan2122_vol45/lun45 /dev/sdak 8 400.0g
/vol/dbtan2122_vol46/lun46 /dev/sdal 8 400.0g
/vol/dbtan2122_vol47/lun47 /dev/sdam 8 400.0g
/vol/dbtan2122_vol48/lun48 /dev/sdan 8 400.0g
/vol/dbtan2122_vol49/lun49 /dev/sdao 8 400.0g
/vol/dbtan2122_vol50/lun50 /dev/sdap 8 400.0g
/vol/dbtan2122_vol51/lun51 /dev/sdaq 8 400.0g
/vol/dbtan2122_vol52/lun52 /dev/sdar 8 400.0g
/vol/dbtan2122_vol53/lun53 /dev/sdas 8 400.0g
/vol/dbtan2122_vol54/lun54 /dev/sdat 8 400.0g
/vol/dbtan2122_vol55/lun55 /dev/sdau 8 400.0g
/vol/dbtan2122_vol56/lun56 /dev/sdav 8 400.0g
/vol/dbtan2122_vol57/lun57 /dev/sdaw 8 400.0g
/vol/dbtan2122_vol58/lun58 /dev/sdax 8 400.0g
/vol/dbtan2122_vol59/lun59 /dev/sday 8 400.0g
/vol/dbtan2122_vol60/lun60 /dev/sdaz 8 400.0g
/vol/dbtan2122_vol61/lun61 /dev/sdbb 8 400.0g
/vol/dbtan2122_vol62/lun62 /dev/sdbc 8 400.0g
/vol/dbtan2122_vol63/lun63 /dev/sdbd 8 400.0g
/vol/dbtan2122_vol64/lun64 /dev/sdbe 8 400.0g
/vol/dbtan2122_vol65/lun65 /dev/sdbg 8 400.0g
/vol/dbtan2122_vol66/lun66 /dev/sdbh 8 400.0g
/vol/dbtan2122_vol67/lun67 /dev/sdbi 8 400.0g
/vol/dbtan2122_vol68/lun68 /dev/sdbj 8 400.0g
/vol/dbtan2122_vol69/lun69 /dev/sdbk 8 400.0g
/vol/dbtan2122_vol70/lun70 /dev/sdbl 8 400.0g
/vol/dbtan2122_vol71/lun71 /dev/sdbm 8 400.0g
/vol/dbtan2122_vol72/lun72 /dev/sdbn 8 400.0g
/vol/dbtan2122_vol73/lun73 /dev/sdbo 8 400.0g
/vol/dbtan2122_vol74/lun74 /dev/sdbp 8 400.0g
/vol/dbtan2122_vol75/lun75 /dev/sdc 8 400.0g
/vol/dbtan2122_vol76/lun76 /dev/sdba 8 400.0g
/vol/dbtan2122_vol77/lun77 /dev/sdbf 8 400.0g
/vol/vdbench01/lunvd01 /dev/sdbx 8 80.0g
/vol/vdbench02/lunvd02 /dev/sdby 8 80.0g
/vol/vdbench03/lunvd03 /dev/sdbz 8 80.0g
/vol/vdbench04/lunvd04 /dev/sdca 8 80.0g
/vol/vdbench05/lunvd05 /dev/sdcb 8 80.0g
/vol/vdbench06/lunvd06 /dev/sdcc 8 80.0g
/vol/vdbench07/lunvd07 /dev/sdcd 8 80.0g
/vol/vdbench08/lunvd08 /dev/sdce 8 80.0g
/vol/vdbench09/lunvd09 /dev/sdcf 8 80.0g
/vol/vdbench10/lunvd10 /dev/sdcg 8 80.0g
/vol/vdbench11/lunvd11 /dev/sdxr 8 80.0g
/vol/vdbench12/lunvd12 /dev/sdxs 8 80.0g
[root@dbtan22: ~/test/20180801]#
附2:配置 multipath
基本使用方法
1. 查看主机或者存储交换机上的WWN号,在存储上将LUN映射给需要的主机
[root@dbtan21: ~]# cat /sys/class/fc_host/host*/port_name
0x100000109b1b2c72
0x100000109b1b2c73
0x100000109b176552
0x100000109b176553
[root@dbtan21: ~]#
2. 在系统内执行扫盘命令,没有命令先安装 sg3_utils
# yum install sg3_utils
rescan-scsi-bus.sh
[root@dbtan21: ~]# rpm -qa | grep sg3_utils
sg3_utils-1.28-13.el6.x86_64
sg3_utils-libs-1.28-13.el6.x86_64
[root@dbtan21: ~]#
[root@dbtan21: ~]# ll /usr/bin/rescan-scsi-bus.sh
-rwxr-xr-x. 1 root root 33968 Jun 19 23:22 /usr/bin/rescan-scsi-bus.sh
[root@dbtan21: ~]#
3. 查看是否映射到对应的
# fdisk -l
# lsblk -f
4. 查看是否安装了 multipath
# yum install device-mapper-multipath
[root@dbtan21: ~]# rpm -qa | grep device-mapper
device-mapper-libs-1.02.117-7.el6.x86_64
device-mapper-event-1.02.117-7.el6.x86_64
device-mapper-persistent-data-0.6.2-0.1.rc7.el6.x86_64
device-mapper-1.02.117-7.el6.x86_64
device-mapper-multipath-libs-0.4.9-93.el6.x86_64
device-mapper-multipath-0.4.9-93.el6.x86_64
device-mapper-event-libs-1.02.117-7.el6.x86_64
[root@dbtan21: ~]#
5. 拷贝默认的 multipath.conf
到 /etc
目录下,也可以使用mpathconf命令创建默认模板
# cp /usr/share/doc/device-mapper-multipath-0.4.9/multipath.conf /etc/multipath.conf
# mpathconf --enable --with_multipathd y
6. 查看本地存储wwid
[root@dbtan21: ~]# /lib/udev/scsi_id --whitelisted --device=/dev/sda
36101b5442bcc700022bf914a0cca39f5
[root@dbtan21: ~]# /lib/udev/scsi_id --whitelisted --device=/dev/mapper/netapp-lun11
3600a09803830475a4c2b4d305949494d
[root@dbtan21: ~]#
[root@dbtan21: ~]# ll /dev/mapper/*
crw-rw---- 1 root root 10, 58 Aug 13 18:38 /dev/mapper/control
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun101 -> ../dm-69
lrwxrwxrwx 1 root root 8 Aug 16 20:50 /dev/mapper/netapp-lun102 -> ../dm-70
lrwxrwxrwx 1 root root 8 Aug 16 20:50 /dev/mapper/netapp-lun103 -> ../dm-71
lrwxrwxrwx 1 root root 7 Aug 16 20:28 /dev/mapper/netapp-lun11 -> ../dm-3
lrwxrwxrwx 1 root root 7 Aug 16 20:28 /dev/mapper/netapp-lun12 -> ../dm-4
lrwxrwxrwx 1 root root 7 Aug 16 20:28 /dev/mapper/netapp-lun13 -> ../dm-5
lrwxrwxrwx 1 root root 7 Aug 16 20:28 /dev/mapper/netapp-lun14 -> ../dm-6
lrwxrwxrwx 1 root root 7 Aug 16 20:28 /dev/mapper/netapp-lun15 -> ../dm-9
lrwxrwxrwx 1 root root 7 Aug 16 20:28 /dev/mapper/netapp-lun16 -> ../dm-7
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun17 -> ../dm-14
lrwxrwxrwx 1 root root 7 Aug 16 20:28 /dev/mapper/netapp-lun18 -> ../dm-8
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun19 -> ../dm-10
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun20 -> ../dm-15
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun21 -> ../dm-11
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun22 -> ../dm-13
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun23 -> ../dm-12
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun24 -> ../dm-17
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun25 -> ../dm-16
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun26 -> ../dm-18
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun27 -> ../dm-20
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun28 -> ../dm-19
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun29 -> ../dm-21
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun30 -> ../dm-23
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun31 -> ../dm-22
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun32 -> ../dm-24
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun33 -> ../dm-25
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun34 -> ../dm-26
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun35 -> ../dm-27
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun36 -> ../dm-28
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun37 -> ../dm-30
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun38 -> ../dm-29
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun39 -> ../dm-31
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun40 -> ../dm-34
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun41 -> ../dm-33
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun42 -> ../dm-32
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun43 -> ../dm-36
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun44 -> ../dm-38
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun45 -> ../dm-35
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun46 -> ../dm-39
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun47 -> ../dm-44
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun48 -> ../dm-41
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun49 -> ../dm-37
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun50 -> ../dm-40
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun51 -> ../dm-42
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun52 -> ../dm-43
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun53 -> ../dm-46
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun54 -> ../dm-45
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun55 -> ../dm-47
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun56 -> ../dm-49
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun57 -> ../dm-48
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun58 -> ../dm-50
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun59 -> ../dm-51
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun60 -> ../dm-52
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun61 -> ../dm-54
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun62 -> ../dm-55
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun63 -> ../dm-56
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun64 -> ../dm-58
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun65 -> ../dm-59
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun66 -> ../dm-60
lrwxrwxrwx 1 root root 8 Aug 16 20:30 /dev/mapper/netapp-lun67 -> ../dm-66
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun68 -> ../dm-62
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun69 -> ../dm-63
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun70 -> ../dm-61
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun71 -> ../dm-64
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun72 -> ../dm-65
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun73 -> ../dm-67
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun74 -> ../dm-68
lrwxrwxrwx 1 root root 7 Aug 16 20:28 /dev/mapper/netapp-lun75 -> ../dm-2
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun76 -> ../dm-53
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun77 -> ../dm-57
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun78 -> ../dm-93
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun79 -> ../dm-94
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun80 -> ../dm-95
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun81 -> ../dm-96
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun82 -> ../dm-97
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun83 -> ../dm-98
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lun84 -> ../dm-99
lrwxrwxrwx 1 root root 9 Aug 16 20:28 /dev/mapper/netapp-lun85 -> ../dm-100
lrwxrwxrwx 1 root root 9 Aug 16 20:28 /dev/mapper/netapp-lun86 -> ../dm-101
lrwxrwxrwx 1 root root 9 Aug 16 20:28 /dev/mapper/netapp-lun87 -> ../dm-102
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lunvd01 -> ../dm-72
lrwxrwxrwx 1 root root 18 Aug 16 20:28 /dev/mapper/netapp-lunvd02 -> ../vdbench-lunvd02
lrwxrwxrwx 1 root root 18 Aug 16 20:28 /dev/mapper/netapp-lunvd03 -> ../vdbench-lunvd03
lrwxrwxrwx 1 root root 18 Aug 16 20:28 /dev/mapper/netapp-lunvd04 -> ../vdbench-lunvd04
lrwxrwxrwx 1 root root 18 Aug 16 20:28 /dev/mapper/netapp-lunvd05 -> ../vdbench-lunvd05
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lunvd06 -> ../dm-77
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lunvd07 -> ../dm-78
lrwxrwxrwx 1 root root 18 Aug 16 20:28 /dev/mapper/netapp-lunvd08 -> ../vdbench-lunvd08
lrwxrwxrwx 1 root root 18 Aug 16 20:28 /dev/mapper/netapp-lunvd09 -> ../vdbench-lunvd09
lrwxrwxrwx 1 root root 18 Aug 16 20:28 /dev/mapper/netapp-lunvd10 -> ../vdbench-lunvd10
lrwxrwxrwx 1 root root 8 Aug 16 20:28 /dev/mapper/netapp-lunvd11 -> ../dm-83
lrwxrwxrwx 1 root root 8 Aug 16 20:29 /dev/mapper/netapp-lunvd12 -> ../dm-84
lrwxrwxrwx 1 root root 18 Aug 16 20:29 /dev/mapper/netapp-lunvd13 -> ../vdbench-lunvd13
lrwxrwxrwx 1 root root 18 Aug 16 20:29 /dev/mapper/netapp-lunvd14 -> ../vdbench-lunvd14
lrwxrwxrwx 1 root root 18 Aug 16 20:29 /dev/mapper/netapp-lunvd15 -> ../vdbench-lunvd15
lrwxrwxrwx 1 root root 18 Aug 16 20:29 /dev/mapper/netapp-lunvd16 -> ../vdbench-lunvd16
lrwxrwxrwx 1 root root 8 Aug 16 20:29 /dev/mapper/netapp-lunvd17 -> ../dm-89
lrwxrwxrwx 1 root root 18 Aug 16 20:29 /dev/mapper/netapp-lunvd18 -> ../vdbench-lunvd18
lrwxrwxrwx 1 root root 8 Aug 16 20:29 /dev/mapper/netapp-lunvd19 -> ../dm-91
lrwxrwxrwx 1 root root 8 Aug 16 20:29 /dev/mapper/netapp-lunvd20 -> ../dm-92
lrwxrwxrwx 1 root root 7 Aug 13 18:38 /dev/mapper/VolGroup-LogVol00 -> ../dm-1
lrwxrwxrwx 1 root root 7 Aug 13 18:38 /dev/mapper/VolGroup-LogVol01 -> ../dm-0
lrwxrwxrwx 1 root root 8 Aug 13 18:38 /dev/mapper/VolGroup-lv_app -> ../dm-82
[root@dbtan21: ~]#
7. 查看存储参数
[root@dbtan21: ~]# cat /sys/block/sdb/device/vendor
NETAPP
[root@dbtan21: ~]# cat /sys/block/sdb/device/model
LUN C-Mode
[root@dbtan21: ~]#
8. 添加本地盘到 blacklist
内,不同的存储和系统参考官方的最佳实践
已经可以脚本自动生成 multipath 配置文件,参见:
【步骤2: 生成多路径配置文件 /etc/multipath.conf
】
【步骤3:生成绑定文件 /etc/multipath/bindings (可不做,由于多路径配置文件中 user_friendly_names no )】
9. 清空已有的 multipath
记录
# multipath -F
说明:
multipath -F
不会清空使用中的链路。下列举例,
multipath -F
全部清理了,是因为此时所以链路都已经没有在使用了。清理后,重启
multipath
服务(/etc/init.d/multipathd restart
),即可重新聚合链路。我们可以发现,在写好
/etc/udev/rules.d/99-oracle-asmdevices.rules
规则后,每次重启multipath
服务,/dev/mapper/netapp-lun
的符号链接会切换指向/dev/dm-
和/dev/asm-lun
的块设备(block)。通过分析,
/dev/mapper/netapp-lun
的符号链接会切换,是与执行partprobe
有关。(例:partprobe /dev/mapper/netapp-lun101
)
[root@dbtan21: ~]# multipath -F
[root@dbtan21: ~]# multipath -ll
[root@dbtan21: ~]# ll /dev/mapper/
total 0
crw-rw---- 1 root root 10, 58 Aug 13 18:38 control
lrwxrwxrwx 1 root root 7 Aug 13 18:38 VolGroup-LogVol00 -> ../dm-1
lrwxrwxrwx 1 root root 7 Aug 13 18:38 VolGroup-LogVol01 -> ../dm-0
lrwxrwxrwx 1 root root 8 Aug 13 18:38 VolGroup-lv_app -> ../dm-82
[root@dbtan21: ~]# /etc/init.d/multipathd restart
ok
Stopping multipathd daemon: [ OK ]
Starting multipathd daemon: [ OK ]
[root@dbtan21: ~]# ll /dev/mapper/
total 0
crw-rw---- 1 root root 10, 58 Aug 13 18:38 control
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun102 -> ../dm-80
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun103 -> ../dm-81
lrwxrwxrwx 1 root root 7 Aug 24 15:43 netapp-lun11 -> ../dm-2
lrwxrwxrwx 1 root root 7 Aug 24 15:43 netapp-lun12 -> ../dm-3
lrwxrwxrwx 1 root root 7 Aug 24 15:43 netapp-lun13 -> ../dm-4
lrwxrwxrwx 1 root root 7 Aug 24 15:43 netapp-lun14 -> ../dm-5
lrwxrwxrwx 1 root root 7 Aug 24 15:43 netapp-lun15 -> ../dm-6
lrwxrwxrwx 1 root root 7 Aug 24 15:43 netapp-lun16 -> ../dm-7
lrwxrwxrwx 1 root root 7 Aug 24 15:43 netapp-lun17 -> ../dm-8
lrwxrwxrwx 1 root root 7 Aug 24 15:43 netapp-lun18 -> ../dm-9
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun19 -> ../dm-10
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun20 -> ../dm-11
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun21 -> ../dm-12
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun22 -> ../dm-13
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun23 -> ../dm-14
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun24 -> ../dm-15
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun25 -> ../dm-16
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun26 -> ../dm-17
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun27 -> ../dm-18
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun28 -> ../dm-19
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun29 -> ../dm-20
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun30 -> ../dm-21
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun31 -> ../dm-22
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun32 -> ../dm-23
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun33 -> ../dm-24
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun34 -> ../dm-25
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun35 -> ../dm-26
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun36 -> ../dm-27
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun37 -> ../dm-28
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun38 -> ../dm-29
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun39 -> ../dm-30
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun40 -> ../dm-31
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun41 -> ../dm-32
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun42 -> ../dm-33
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun43 -> ../dm-34
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun44 -> ../dm-35
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun45 -> ../dm-36
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun46 -> ../dm-37
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun47 -> ../dm-38
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun48 -> ../dm-39
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun49 -> ../dm-40
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun50 -> ../dm-41
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun51 -> ../dm-42
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun52 -> ../dm-43
lrwxrwxrwx 1 root root 12 Aug 24 15:43 netapp-lun53 -> ../asm-lun53
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun54 -> ../dm-45
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun55 -> ../dm-46
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun56 -> ../dm-47
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun57 -> ../dm-48
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun58 -> ../dm-49
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun59 -> ../dm-50
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun60 -> ../dm-51
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun61 -> ../dm-52
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun62 -> ../dm-53
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun63 -> ../dm-54
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun64 -> ../dm-55
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun65 -> ../dm-56
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun66 -> ../dm-57
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun67 -> ../dm-58
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun68 -> ../dm-59
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun69 -> ../dm-60
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun70 -> ../dm-61
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun71 -> ../dm-62
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun72 -> ../dm-63
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun73 -> ../dm-64
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun74 -> ../dm-65
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun75 -> ../dm-66
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun76 -> ../dm-67
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun77 -> ../dm-68
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun78 -> ../dm-69
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun79 -> ../dm-70
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun80 -> ../dm-71
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun81 -> ../dm-72
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun83 -> ../dm-74
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun84 -> ../dm-75
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun85 -> ../dm-76
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lun87 -> ../dm-78
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lunvd01 -> ../dm-83
lrwxrwxrwx 1 root root 8 Aug 24 15:43 netapp-lunvd03 -> ../dm-85
lrwxrwxrwx 1 root root 7 Aug 13 18:38 VolGroup-LogVol00 -> ../dm-1
lrwxrwxrwx 1 root root 7 Aug 13 18:38 VolGroup-LogVol01 -> ../dm-0
lrwxrwxrwx 1 root root 8 Aug 13 18:38 VolGroup-lv_app -> ../dm-82
[root@dbtan21: ~]#
[root@dbtan21: ~]# /etc/init.d/multipathd restart
ok
Stopping multipathd daemon: [ OK ]
Starting multipathd daemon: [ OK ]
[root@dbtan21: ~]#
[root@dbtan21: ~]# ll /dev/mapper/
total 0
crw-rw---- 1 root root 10, 58 Aug 13 18:38 control
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lun101 -> ../asm-lun78
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lun102 -> ../asm-lun79
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lun103 -> ../asm-lun80
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun11 -> ../asm-lun18
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun12 -> ../asm-lun18
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun13 -> ../asm-lun18
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun14 -> ../asm-lun18
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun15 -> ../asm-lun18
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun16 -> ../asm-lun16
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun17 -> ../asm-lun23
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun18 -> ../asm-lun18
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun19 -> ../asm-lun19
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun20 -> ../asm-lun40
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun21 -> ../asm-lun21
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun22 -> ../asm-lun22
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun23 -> ../asm-lun23
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun24 -> ../asm-lun40
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun25 -> ../asm-lun25
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun26 -> ../asm-lun40
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun27 -> ../asm-lun40
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun28 -> ../asm-lun28
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun29 -> ../asm-lun29
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun30 -> ../asm-lun40
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun31 -> ../asm-lun31
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun32 -> ../asm-lun32
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun33 -> ../asm-lun40
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun34 -> ../asm-lun40
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun35 -> ../asm-lun35
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun36 -> ../asm-lun36
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun37 -> ../asm-lun40
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun38 -> ../asm-lun38
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun39 -> ../asm-lun39
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun40 -> ../asm-lun40
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun41 -> ../asm-lun41
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun42 -> ../asm-lun42
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun43 -> ../asm-lun43
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun44 -> ../asm-lun53
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun45 -> ../asm-lun45
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun46 -> ../asm-lun50
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun47 -> ../asm-lun53
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun48 -> ../asm-lun50
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun49 -> ../asm-lun49
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun50 -> ../asm-lun50
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun51 -> ../asm-lun51
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun52 -> ../asm-lun52
lrwxrwxrwx 1 root root 8 Aug 24 15:46 netapp-lun53 -> ../dm-44
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun54 -> ../asm-lun54
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun55 -> ../asm-lun56
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun56 -> ../asm-lun56
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun57 -> ../asm-lun57
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun58 -> ../asm-lun76
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun59 -> ../asm-lun76
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun60 -> ../asm-lun76
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun61 -> ../asm-lun76
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun62 -> ../asm-lun75
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun63 -> ../asm-lun76
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun64 -> ../asm-lun75
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun65 -> ../asm-lun76
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun66 -> ../asm-lun77
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun67 -> ../asm-lun75
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun68 -> ../asm-lun76
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun69 -> ../asm-lun77
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun70 -> ../asm-lun70
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun71 -> ../asm-lun76
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun72 -> ../asm-lun77
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun73 -> ../asm-lun76
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun74 -> ../asm-lun77
lrwxrwxrwx 1 root root 12 Aug 24 15:46 netapp-lun75 -> ../asm-lun75
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lun76 -> ../asm-lun76
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lun77 -> ../asm-lun77
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lun78 -> ../asm-lun78
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lun79 -> ../asm-lun79
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lun80 -> ../asm-lun80
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lun81 -> ../asm-lun81
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lun82 -> ../asm-lun82
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lun83 -> ../asm-lun83
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lun84 -> ../asm-lun84
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lun85 -> ../asm-lun85
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lun86 -> ../asm-lun86
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lun87 -> ../asm-lun87
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lunvd01 -> ../asm-lun81
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lunvd02 -> ../asm-lun82
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lunvd03 -> ../asm-lun83
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lunvd04 -> ../asm-lun84
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lunvd05 -> ../asm-lun85
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lunvd06 -> ../asm-lun86
lrwxrwxrwx 1 root root 12 Aug 24 15:47 netapp-lunvd07 -> ../asm-lun87
lrwxrwxrwx 1 root root 18 Aug 24 15:47 netapp-lunvd08 -> ../vdbench-lunvd08
lrwxrwxrwx 1 root root 18 Aug 24 15:47 netapp-lunvd09 -> ../vdbench-lunvd09
lrwxrwxrwx 1 root root 18 Aug 24 15:47 netapp-lunvd10 -> ../vdbench-lunvd10
lrwxrwxrwx 1 root root 18 Aug 24 15:47 netapp-lunvd11 -> ../vdbench-lunvd11
lrwxrwxrwx 1 root root 18 Aug 24 15:47 netapp-lunvd12 -> ../vdbench-lunvd12
lrwxrwxrwx 1 root root 18 Aug 24 15:47 netapp-lunvd13 -> ../vdbench-lunvd13
lrwxrwxrwx 1 root root 18 Aug 24 15:47 netapp-lunvd14 -> ../vdbench-lunvd14
lrwxrwxrwx 1 root root 18 Aug 24 15:47 netapp-lunvd15 -> ../vdbench-lunvd15
lrwxrwxrwx 1 root root 18 Aug 24 15:47 netapp-lunvd16 -> ../vdbench-lunvd16
lrwxrwxrwx 1 root root 18 Aug 24 15:47 netapp-lunvd17 -> ../vdbench-lunvd17
lrwxrwxrwx 1 root root 18 Aug 24 15:47 netapp-lunvd18 -> ../vdbench-lunvd18
lrwxrwxrwx 1 root root 18 Aug 24 15:47 netapp-lunvd19 -> ../vdbench-lunvd19
lrwxrwxrwx 1 root root 18 Aug 24 15:47 netapp-lunvd20 -> ../vdbench-lunvd20
lrwxrwxrwx 1 root root 7 Aug 13 18:38 VolGroup-LogVol00 -> ../dm-1
lrwxrwxrwx 1 root root 7 Aug 13 18:38 VolGroup-LogVol01 -> ../dm-0
lrwxrwxrwx 1 root root 8 Aug 13 18:38 VolGroup-lv_app -> ../dm-82
[root@dbtan21: ~]#
[root@dbtan21: ~]# ll /dev/mapper/netapp-lun10*
lrwxrwxrwx 1 root root 12 Aug 24 15:47 /dev/mapper/netapp-lun101 -> ../asm-lun78
lrwxrwxrwx 1 root root 12 Aug 24 15:47 /dev/mapper/netapp-lun102 -> ../asm-lun79
lrwxrwxrwx 1 root root 12 Aug 24 15:47 /dev/mapper/netapp-lun103 -> ../asm-lun80
[root@dbtan21: ~]# partprobe /dev/mapper/netapp-lun101
[root@dbtan21: ~]# partprobe /dev/mapper/netapp-lun102
[root@dbtan21: ~]# partprobe /dev/mapper/netapp-lun103
[root@dbtan21: ~]#
[root@dbtan21: ~]# ll /dev/mapper/netapp-lun10*
lrwxrwxrwx 1 root root 8 Aug 24 15:52 /dev/mapper/netapp-lun101 -> ../dm-79
lrwxrwxrwx 1 root root 8 Aug 24 15:52 /dev/mapper/netapp-lun102 -> ../dm-80
lrwxrwxrwx 1 root root 8 Aug 24 15:52 /dev/mapper/netapp-lun103 -> ../dm-81
[root@dbtan21: ~]#
10. 打印诊断信息
# multipath -v3
11. 启用多路径守护程序以在引导时启动
# mpathconf --enable
12. 启动多路径服务
# service multipathd start
或
# /etc/init.d/multipathd restart
13. 如果在启动 multipath
守护程序后更改多路径配置文件,请运行以下命令以使更改生效。
# service multipathd reload
14. 重启系统测试
说明:首次配置好
multipath
后,需要重启服务器。
# init 6
15. 查看多路径当前状态
# multipath -ll
附3:删除LUN的操作
通过匹配
lsblk -s
的结果分别执行echo 1 > /sys/block/sdX/device/delete multipath -f mpathc
从而达到删除LUN设备的目的。
下例,在 dbtan21/dbtan22 主机删除12个测试LUN( netapp-lunvd01
~ netapp-lunvd12
)
dbtan21 节点:
-- dbtan21
echo 1 > /sys/block/sdbx/device/delete
echo 1 > /sys/block/sddl/device/delete
echo 1 > /sys/block/sdiq/device/delete
echo 1 > /sys/block/sduf/device/delete
echo 1 > /sys/block/sdfg/device/delete
echo 1 > /sys/block/sdrd/device/delete
echo 1 > /sys/block/sdnp/device/delete
echo 1 > /sys/block/sdxh/device/delete
multipath -f netapp-lunvd01
echo 1 > /sys/block/sdby/device/delete
echo 1 > /sys/block/sddw/device/delete
echo 1 > /sys/block/sdis/device/delete
echo 1 > /sys/block/sdug/device/delete
echo 1 > /sys/block/sdfh/device/delete
echo 1 > /sys/block/sdre/device/delete
echo 1 > /sys/block/sdnr/device/delete
echo 1 > /sys/block/sdxi/device/delete
multipath -f netapp-lunvd02
echo 1 > /sys/block/sdbz/device/delete
echo 1 > /sys/block/sdeh/device/delete
echo 1 > /sys/block/sdit/device/delete
echo 1 > /sys/block/sduh/device/delete
echo 1 > /sys/block/sdfi/device/delete
echo 1 > /sys/block/sdrf/device/delete
echo 1 > /sys/block/sdns/device/delete
echo 1 > /sys/block/sdxj/device/delete
multipath -f netapp-lunvd03
echo 1 > /sys/block/sdca/device/delete
echo 1 > /sys/block/sder/device/delete
echo 1 > /sys/block/sdiu/device/delete
echo 1 > /sys/block/sdui/device/delete
echo 1 > /sys/block/sdfj/device/delete
echo 1 > /sys/block/sdrg/device/delete
echo 1 > /sys/block/sdnu/device/delete
echo 1 > /sys/block/sdxk/device/delete
multipath -f netapp-lunvd04
echo 1 > /sys/block/sdcb/device/delete
echo 1 > /sys/block/sdfd/device/delete
echo 1 > /sys/block/sdiw/device/delete
echo 1 > /sys/block/sduj/device/delete
echo 1 > /sys/block/sdfk/device/delete
echo 1 > /sys/block/sdrh/device/delete
echo 1 > /sys/block/sdnv/device/delete
echo 1 > /sys/block/sdxl/device/delete
multipath -f netapp-lunvd05
echo 1 > /sys/block/sdcc/device/delete
echo 1 > /sys/block/sdfq/device/delete
echo 1 > /sys/block/sdix/device/delete
echo 1 > /sys/block/sduk/device/delete
echo 1 > /sys/block/sdfl/device/delete
echo 1 > /sys/block/sdri/device/delete
echo 1 > /sys/block/sdnw/device/delete
echo 1 > /sys/block/sdxm/device/delete
multipath -f netapp-lunvd06
echo 1 > /sys/block/sdfm/device/delete
echo 1 > /sys/block/sdrj/device/delete
echo 1 > /sys/block/sdnz/device/delete
echo 1 > /sys/block/sdxn/device/delete
echo 1 > /sys/block/sdcd/device/delete
echo 1 > /sys/block/sdgc/device/delete
echo 1 > /sys/block/sdja/device/delete
echo 1 > /sys/block/sdul/device/delete
multipath -f netapp-lunvd07
echo 1 > /sys/block/sdfn/device/delete
echo 1 > /sys/block/sdrk/device/delete
echo 1 > /sys/block/sdoa/device/delete
echo 1 > /sys/block/sdxo/device/delete
echo 1 > /sys/block/sdce/device/delete
echo 1 > /sys/block/sdgo/device/delete
echo 1 > /sys/block/sdjb/device/delete
echo 1 > /sys/block/sdum/device/delete
multipath -f netapp-lunvd08
echo 1 > /sys/block/sdfo/device/delete
echo 1 > /sys/block/sdrl/device/delete
echo 1 > /sys/block/sdob/device/delete
echo 1 > /sys/block/sdxp/device/delete
echo 1 > /sys/block/sdcf/device/delete
echo 1 > /sys/block/sdhb/device/delete
echo 1 > /sys/block/sdjd/device/delete
echo 1 > /sys/block/sdun/device/delete
multipath -f netapp-lunvd09
echo 1 > /sys/block/sdfp/device/delete
echo 1 > /sys/block/sdrm/device/delete
echo 1 > /sys/block/sdod/device/delete
echo 1 > /sys/block/sdxq/device/delete
echo 1 > /sys/block/sdcg/device/delete
echo 1 > /sys/block/sdhn/device/delete
echo 1 > /sys/block/sdje/device/delete
echo 1 > /sys/block/sduo/device/delete
multipath -f netapp-lunvd10
echo 1 > /sys/block/sdxt/device/delete
echo 1 > /sys/block/sdyb/device/delete
echo 1 > /sys/block/sdxx/device/delete
echo 1 > /sys/block/sdyf/device/delete
echo 1 > /sys/block/sdxr/device/delete
echo 1 > /sys/block/sdxz/device/delete
echo 1 > /sys/block/sdxv/device/delete
echo 1 > /sys/block/sdyd/device/delete
multipath -f netapp-lunvd11
echo 1 > /sys/block/sdxu/device/delete
echo 1 > /sys/block/sdyc/device/delete
echo 1 > /sys/block/sdxy/device/delete
echo 1 > /sys/block/sdyg/device/delete
echo 1 > /sys/block/sdxs/device/delete
echo 1 > /sys/block/sdya/device/delete
echo 1 > /sys/block/sdxw/device/delete
echo 1 > /sys/block/sdye/device/delete
multipath -f netapp-lunvd12
dbtan22 节点:
-- dbtan22
echo 1 > /sys/block/sdbt/device/delete
echo 1 > /sys/block/sdob/device/delete
echo 1 > /sys/block/sdhx/device/delete
echo 1 > /sys/block/sduf/device/delete
echo 1 > /sys/block/sdev/device/delete
echo 1 > /sys/block/sdrd/device/delete
echo 1 > /sys/block/sdkz/device/delete
echo 1 > /sys/block/sdxh/device/delete
multipath -f netapp-lunvd01
echo 1 > /sys/block/sdbu/device/delete
echo 1 > /sys/block/sdoc/device/delete
echo 1 > /sys/block/sdhy/device/delete
echo 1 > /sys/block/sdug/device/delete
echo 1 > /sys/block/sdew/device/delete
echo 1 > /sys/block/sdre/device/delete
echo 1 > /sys/block/sdla/device/delete
echo 1 > /sys/block/sdxi/device/delete
multipath -f netapp-lunvd02
echo 1 > /sys/block/sdbv/device/delete
echo 1 > /sys/block/sdod/device/delete
echo 1 > /sys/block/sdhz/device/delete
echo 1 > /sys/block/sduh/device/delete
echo 1 > /sys/block/sdex/device/delete
echo 1 > /sys/block/sdrf/device/delete
echo 1 > /sys/block/sdlb/device/delete
echo 1 > /sys/block/sdxj/device/delete
multipath -f netapp-lunvd03
echo 1 > /sys/block/sdbw/device/delete
echo 1 > /sys/block/sdoe/device/delete
echo 1 > /sys/block/sdia/device/delete
echo 1 > /sys/block/sdui/device/delete
echo 1 > /sys/block/sdey/device/delete
echo 1 > /sys/block/sdrg/device/delete
echo 1 > /sys/block/sdlc/device/delete
echo 1 > /sys/block/sdxk/device/delete
multipath -f netapp-lunvd04
echo 1 > /sys/block/sdbx/device/delete
echo 1 > /sys/block/sdof/device/delete
echo 1 > /sys/block/sdib/device/delete
echo 1 > /sys/block/sduj/device/delete
echo 1 > /sys/block/sdez/device/delete
echo 1 > /sys/block/sdrh/device/delete
echo 1 > /sys/block/sdld/device/delete
echo 1 > /sys/block/sdxl/device/delete
multipath -f netapp-lunvd05
echo 1 > /sys/block/sdby/device/delete
echo 1 > /sys/block/sdog/device/delete
echo 1 > /sys/block/sdic/device/delete
echo 1 > /sys/block/sduk/device/delete
echo 1 > /sys/block/sdfa/device/delete
echo 1 > /sys/block/sdri/device/delete
echo 1 > /sys/block/sdle/device/delete
echo 1 > /sys/block/sdxm/device/delete
multipath -f netapp-lunvd06
echo 1 > /sys/block/sdfb/device/delete
echo 1 > /sys/block/sdrj/device/delete
echo 1 > /sys/block/sdlf/device/delete
echo 1 > /sys/block/sdxn/device/delete
echo 1 > /sys/block/sdbz/device/delete
echo 1 > /sys/block/sdoh/device/delete
echo 1 > /sys/block/sdid/device/delete
echo 1 > /sys/block/sdul/device/delete
multipath -f netapp-lunvd07
echo 1 > /sys/block/sdfc/device/delete
echo 1 > /sys/block/sdrk/device/delete
echo 1 > /sys/block/sdlg/device/delete
echo 1 > /sys/block/sdxo/device/delete
echo 1 > /sys/block/sdca/device/delete
echo 1 > /sys/block/sdoi/device/delete
echo 1 > /sys/block/sdie/device/delete
echo 1 > /sys/block/sdum/device/delete
multipath -f netapp-lunvd08
echo 1 > /sys/block/sdfd/device/delete
echo 1 > /sys/block/sdrl/device/delete
echo 1 > /sys/block/sdlh/device/delete
echo 1 > /sys/block/sdxp/device/delete
echo 1 > /sys/block/sdcb/device/delete
echo 1 > /sys/block/sdoj/device/delete
echo 1 > /sys/block/sdif/device/delete
echo 1 > /sys/block/sdun/device/delete
multipath -f netapp-lunvd09
echo 1 > /sys/block/sdfe/device/delete
echo 1 > /sys/block/sdrm/device/delete
echo 1 > /sys/block/sdli/device/delete
echo 1 > /sys/block/sdxq/device/delete
echo 1 > /sys/block/sdcc/device/delete
echo 1 > /sys/block/sdok/device/delete
echo 1 > /sys/block/sdig/device/delete
echo 1 > /sys/block/sduo/device/delete
multipath -f netapp-lunvd10
echo 1 > /sys/block/sdxt/device/delete
echo 1 > /sys/block/sdyb/device/delete
echo 1 > /sys/block/sdxx/device/delete
echo 1 > /sys/block/sdyf/device/delete
echo 1 > /sys/block/sdxr/device/delete
echo 1 > /sys/block/sdxz/device/delete
echo 1 > /sys/block/sdxv/device/delete
echo 1 > /sys/block/sdyd/device/delete
multipath -f netapp-lunvd11
echo 1 > /sys/block/sdxu/device/delete
echo 1 > /sys/block/sdyc/device/delete
echo 1 > /sys/block/sdxy/device/delete
echo 1 > /sys/block/sdyg/device/delete
echo 1 > /sys/block/sdxs/device/delete
echo 1 > /sys/block/sdya/device/delete
echo 1 > /sys/block/sdxw/device/delete
echo 1 > /sys/block/sdye/device/delete
multipath -f netapp-lunvd12
附4:网卡配置中添加 hotplug=no
参数,避免start_udev
命令导致Oracle RAC 的vip漂移问题
start_udev
命令导致网卡重启从而导致vip漂移
解决办法:网卡配置中添加 hotplug=no
参数
注意:如果是使用网卡绑定,比如绑定后的网卡为bond0
,则要在bond0的配置文件里添加hotplug配置信息,在eth0里添加不起作用。
参考:
dbtan21 测试节点
[root@dbtan21: /etc/sysconfig/network-scripts]# cat ifcfg-bond0
DEVICE=bond0
BOOTPROTO=static
ONBOOT=yes
IPADDR=192.168.18.21
NETMASK=255.255.255.0
GATEWAY=192.168.18.1
HOTPLUG="no"
[root@dbtan21: /etc/sysconfig/network-scripts]# cat ifcfg-bond1
DEVICE=bond1
BOOTPROTO=static
ONBOOT=yes
IPADDR=192.168.118.21
NETMASK=255.255.255.0
HOTPLUG="no"
[root@dbtan21: /etc/sysconfig/network-scripts]# cat ifcfg-bond2
DEVICE=bond2
BOOTPROTO=static
ONBOOT=yes
IPADDR=172.16.18.21
NETMASK=255.255.254.0
HOTPLUG="no"
[root@dbtan21: /etc/sysconfig/network-scripts]#
网卡绑定信息
[root@dbtan21: /etc/modprobe.d]# cat modprobe.conf
alias bond0 bonding
options bond0 miimon=100 mode=4
alias bond1 bonding
options bond1 miimon=100 mode=1
alias bond2 bonding
options bond2 miimon=100 mode=4
[root@dbtan21: /etc/modprobe.d]#
[root@dbtan21: /etc/modprobe.d]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
Aggregator ID: 41
Number of ports: 2
Actor Key: 11
Partner Key: 32816
Partner Mac Address: 00:23:04:ee:be:64
Slave Interface: eth6
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: e4:c2:d1:f4:5f:dc
Aggregator ID: 41
Slave queue ID: 0
Slave Interface: eth8
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: e4:c2:d1:f4:5f:de
Aggregator ID: 41
Slave queue ID: 0
[root@dbtan21: /etc/modprobe.d]# cat /proc/net/bonding/bond1
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
Aggregator ID: 41
Number of ports: 1
Actor Key: 11
Partner Key: 1
Partner Mac Address: 00:00:00:00:00:00
Slave Interface: eth10
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 44:6a:2e:ee:18:f0
Aggregator ID: 41
Slave queue ID: 0
Slave Interface: eth11
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 44:6a:2e:ee:18:f1
Aggregator ID: 42
Slave queue ID: 0
[root@dbtan21: /etc/modprobe.d]# cat /proc/net/bonding/bond2
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
Aggregator ID: 41
Number of ports: 2
Actor Key: 11
Partner Key: 833
Partner Mac Address: 70:79:90:a8:40:91
Slave Interface: eth7
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: e4:c2:d1:f4:5f:dd
Aggregator ID: 41
Slave queue ID: 0
Slave Interface: eth9
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: e4:c2:d1:f4:5f:df
Aggregator ID: 41
Slave queue ID: 0
[root@dbtan21: /etc/modprobe.d]# ethtool bond0
Settings for bond0:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Speed: 20000Mb/s
Duplex: Full
Port: Other
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
Link detected: yes
[root@dbtan21: /etc/modprobe.d]# ethtool bond1
Settings for bond1:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Speed: 10000Mb/s
Duplex: Full
Port: Other
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
Link detected: yes
[root@dbtan21: /etc/modprobe.d]# ethtool bond2
Settings for bond2:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Speed: 20000Mb/s
Duplex: Full
Port: Other
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
Link detected: yes
[root@dbtan21: /etc/modprobe.d]#
Reload and Restart the udev rules
[root@dbtan21: ~]# udevadm control --reload-rules
[root@dbtan21: ~]# udevadm trigger
[root@dbtan21: ~]# udevadm trigger subsystem-match=block
[root@dbtan21: ~]# udevadm trigger subsystem-nomatch=net
[root@dbtan21: ~]# start_udev
Starting udev: [ OK ]
[root@dbtan21: ~]#
udevadm trigger [options]
接收内核发送来的设备事件。主要用于重放coldplug事件信息
(译者补充:内核在启动时已经检测到了系统的硬件设备,并把硬件设备信息通过sysfs内核虚拟文件系统导出。udev扫描sysfs文件系统,根据硬件设备信息生成热插拔(hotplug)事件,udev再读取这些事件,生成对应的硬件设备文件。由于没有实际的硬件插拔动作,所以这一过程被称为coldplug。)--verbose 输出将要被触发的设备列表。 --dry-run 不真的触发事件 --type=type 触发一个特殊的设备。合法的类型:devices,subsystem,failed.默认是devices --action=action 被触发的事件,默认是change --subsystem-match=subsystem 触发匹配子系统的设备事件。这个选项可以被多次指定,并且支持shell模式匹配。 --attr-match=attribute=value 触发匹配sysfs属性的设备事件。如果属性值和属性一起指定,属性的值可以使用shell模式匹配。如果没有指定值,会重新确认现有属性。这个选项可以被多次指定。 --attr-nomatch=attribute=value 不要触发匹配属性的设备事件。如果可以使用模式匹配。也可以多次指定 --property-match=property=value 匹配属性吻合的设备。可以多次指定支持模式匹配 --tag-match=property 匹配标签吻合的设备。可以多次指定。 --sysname-match=name
-- The End --