Multipathing Overview

What is Device Mapper

Device Mapper is a kernel driver that provides a framework for volume management. It provides a generic way of creating mapped devices, which may be used as logical volumes. It provides the foundation for a number of higher-level technologies. In addition to LVM, Device-Mapper multipath and the dmraid command use the Device Mapper. The user interface is the dmsetup command. It forms the foundation of LVM2, software RAIDs and dm-crypt disk encryption, and offers additional features such as file system snapshots

What is Multipathing

A path is a connection between a server and the underlying storage. The path can be severed due to many reasons like faulty HBA, faulty cable etc. To avoid such single point of failures, multipathing exists. Multipathing ensures that the system uses multiple physical paths to provide redundancy and increased throughput. There are many vendor specific multipathing implementations like EMC’s powerpath and Symantecs VxDMP.

What is Device Mapper multipath

Device Mapper Multipathing (or DM-multipathing) is a Linux native multipath tool, which allows you to configure multiple I/O paths between server nodes and storage arrays into a single device. These I/O paths are physical SAN connections that can include separate cables, switches, and controllers. Multipathing aggregates the I/O paths, creating a new device that consists of the aggregated paths. Regardless of the vendor hardware in use, device mapper creates a block device under /dev/mapper/ for each LUN attached to the system.

Device Mapper components

The important components of Device Mapper multipathing are :

a) dm-multipath - kernel module responsible for making routing decisions under normal/failure conditions

b) multipath Command used for viewing/listing multipath devices and for initial configuration

c) multipathd - daemon that moitors path, marks failed paths, reactivates restored paths, adds/removes device files as needed.

d) Kpartx command used to create device mapper entries for partitions on multipathed LUN. It is invoked automatically when multipath command is used.

How to verify if DMMP is installed and configured

1. Check whether device-mapper is installed.

# rpm -qa |grep device-mapper

device-mapper-1.02.39-1.el5

device-mapper-multipath-0.4.7-34.el5

device-mapper-1.02.39-1.el5

device-mapper-event-1.02.39-1.el5

2. Check that the following device mapper modules are loaded.

# lsmod |grep dm_multipath

dm_multipath           56921  2 dm_round_robin

scsi_dh                42177  2 scsi_dh_rdac,dm_multipath

dm_mod                101649  11 dm_mirror,dm_multipath,dm_raid45,dm_log

3. If above conditions are met, check whether the file /etc/multipath.conf is configured. Make sure the lines in bold are commented out in order to enable device mapper.

# This is a basic configuration file with some examples, for device mapper multipath

......

# Blacklist all devices by default. Remove this to enable multipathing

# on the default devices.

#blacklist {

#        devnode "*"

#}

......

4. Check whether multipathd is running.

# /etc/init.d/multipathd status

 "multipathd (pid  11405) is running..."

5. If yes, check any devices listed using the command below.

# multipath -v2 or #multipath -ll

mpath15 (3600a0b8000473abc0000bafc52fac127) dm-14 SUN,STK6580_6780

[size=10G][features=0][hwhandler=0][rw]

_ round-robin 0 [prio=1][enabled]

 _ 8:0:0:2  sds 65:32 [active][ready]

_ round-robin 0 [prio=0][enabled]

 _ 9:0:0:2  sdu 65:64 [active][faulty]

mpath13 (3600a0b8000473abc0000bb74530aa7da) dm-12 SUN,STK6580_6780

[size=931G][features=0][hwhandler=0][rw]

_ round-robin 0 [prio=1][enabled]

 _ 9:0:0:0  sdp 8:240 [active][ready]

_ round-robin 0 [prio=0][enabled]

 _ 8:0:0:0  sdo 8:224 [active][faulty]

If all the above steps succeed, the system is configured for DMMP.

Multipathing Configuration

Before starting to configure the multipathing, make sure the device-mapper-multipath package is installed. If not installed, install it using yum :

# yum -y install device-mapper-multipath

The device mapper multipathing uses the configuration file /etc/multipath.conf for the configuration. If you make any changes to this file the multipath command must be run in order to reconfigure the multipathed devices. The easiest way to create this file is to use the mpathconf utility. If there is an existing configuration file mpathconf will edit it, if no such file exists it will copy /usr/share/doc/device-mapper-multipath-*/multipath.conf.

#mpathconf --enable --with_multipathd y --with_chkconfig y

Verifying Configuration

The multipath command can be used to verify the multipath configuration. To list the information about multipathed devices :

#multipath -ll

mpath0 (3600a0b8000473abc0000bafc52fac127) dm-14 SUN,STK6580_6780

[size=10G][features=0][hwhandler=0][rw]

_ round-robin 0 [prio=1][enabled]

 _ 8:0:0:2  sds 65:32 [active][ready]

_ round-robin 0 [prio=0][enabled]

 _ 9:0:0:2  sdu 65:64 [active][faulty]

The output shows a multipathed LUN, mpath0. The number following it is the LUN’s WWID. The status active/ready indicates that the path is ready for I/O. If the path is showing faulty/failed then it needs to be repaired before using it for I/O. After the configuration is completed, we can start the multipathd persistently :

# /etc/init.d/multipathd start

# chkconfig multipathd on

User Friendly Device Names

In order to troubleshoot efficiently, device-mapper can be configured to have human readable, user friendly device names under /dev/mapper instead of using the WWIDs. The user friendly names like /dev/mapper/mpath0 can be created by enabling the user_friendly_names option in /etc/multipath.conf file :

defaults {

    user_friendly_names yes

}

You can also control the name for a particular LUN by using the alias option :

multipaths {

    multipath {

            wwid     3600a0b8000473abc0000bafc52fac127 

            alias    mdisk001

              }

}

Removing Multipath

After removing the all the paths for a multipathed device, run the below command to remove the multipath device completely.

# multipath -f [device]

To flush all the multipathed device after stopping the multipahtd daemon :

# multipath -F