Part 1
Overview:
I like IBM storage and, while I don't use their storage exclusively, am rather fond of the DS3500 (traditional low-midrange integrated controller storage array) and the DCS3700 (midrange high-density storage arrays) solutions. Both units are easily expandable with SAS expansion shelves and the 3700 series in particular receives frequent line refreshes that boost performance and scalability.
However, a minor hassle is that the IBM DS Storage Manager and Remote Management software are not made available to Linux platforms other than Red Hat and SUSE Enterprise. I run a Debian-only shop (for historical rather than religious reasons) and was not prepared to make exceptions for our storage servers for the sake of convenience.
Here's a quick how-to for installing and configuring a Debian system for use with the aforementioned IBM storage arrays. These steps apply to both 64-bit Debian 6 and 7 (32-bit should be the same). It is most probable that these instructions will fare well for other IBM storage systems (both SAS and FC) that work with IBM's DS Storage Manager software.
Set-up:
IBM x3650 M4 server with 2x QLogic 8Gb dual-port FC HBA for IBM System X (QLE2562)
IBM DCS3700 dual FC controllers
1x connection from each HBA to each disk controller
In-band management and out-band SNMP and mail alerts.
Process:
1) Download the latest IBM DS Storage Manager Software and, optionally, Remote Manager Software from IBM. At the time of writing this can be found by doing a product search at ibm.com -> support & downloads -> downloads -> fixed, updates & drivers
The current file is SM10.86_Linux_64bit_x86-64_SMIA-10.86.x5.43.tar
2) Some packages to install before the journey:
apt-get install multipath-tools sysfsutils qlogic-tools alien
3) Ensure that the installed HBA is properly detected by the OS. Some command variations:
$ dmesg | grep qla
[ 7.712817] qla2xxx [0000:20:00.1]-00fa:10: QLogic Fibre Channed HBA Driver: 8.03.07.12-k.
...
$ lspci | grep -i qlogic
11:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
...
$ lsmod | grep qla
qla2xxx 304826 5
scsi_transport_fc 39180 1 qla2xxx
$ systool -c fc_host -v
$ systool -m qla2xxx -v
4) Untar the IBM DS Storage Manager software and begin installation (follow the prompts for the default install):
$ cd Linux_x64_10p86/Linux_x86-64
$ chmod +x SMIA-LINUXX64-10.86.0A05.0043.bin
$ ./SMIA-LINUXX64-10.86.0A05.0043.bin
5) The installation will install dump RPMs and rest of the software in /opt/IBM_DS. Go there are convert those RPMs into DEBs:
$ alien --scripts SMagent.rpm
$ alien --scripts SMutil.rpm
$ alien --scripts SMesm.rpm
$ alien --scripts SMruntime.rpm
$ alien --scripts SMclient.rpm
Then dpkg -i each of the resulting .deb packages
SMagent and SMmonitor daemons will have init scripts created and should start automatically after installation.
6) Configure multipathing.
Create a file named /etc/multipath.conf. Since this is an IBM array we're going to use the RDAC driver for multipath I/O.
But first we need to get the WWN:
/lib/udev/scsi_id -g -u /dev/sdc (or whatever sd device the storage mapped to)
My output was: 360080e5000324c18000000a852d67c5e
Copy the WWN for use in /etc/multipath.conf:
#####################################################
defaults {
udev_dir /dev
path_checker rdac
user_friendly_names yes
}
blacklist {
# devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^(ram|raw|loop|fd|md|sr|scd|st)[0-9]*"
devnode "^hd[a-z][[0-9]*]"
devnode "^sd[a-b]"
}
multipaths {
multipath {
wwid 360080e5000324c18000000a852d67c5e
alias 3700-array-1
}
}
devices {
device {
vendor "IBM"
product "1818"
path_grouping_policy group_by_prio
getuid_callout "/lib/udev/scsi_id -g -u /dev/%n"
path_checker rdac
prio rdac
hardware_handler "1 rdac"
path_selector "round-robin 0"
failback immediate
features "1 queue_if_no_path"
rr_weight uniform
rr_min_io 100
no_path_retry queue
}
}
#####################################################
7) Restart the multipath-tools daemon:
$ /etc/init.d/multipath-tools restart
8) List the multipath devices:
$ multipath -ll
9) Launch the SMclient utility to configure your disk array:
$ /opt/IBM_DS/client/SMclient &
