About LUN's and Multipath

What is a LUN

In computer storage, a Logical Unit Number, or LUN, is a number used to identify a logical unit, which is a device addressed by SAN. LUN vs. SCSI Device ID: The LUN is not the only way to identify a logical unit. There is also the SCSI Device ID, which identifies a logical unit uniquely in the world

How to scan LUN in the server

For FC Host

# echo "1" > /sys/class/fc_host/host/issue_lip

Above command performs a Loop Initialization Protocol (LIP) and then scans the interconnect and causes the SCSI layer to be updated to reflect the devices currently on the bus. A LIP is, essentially, a bus reset,  and will cause device addition and removal. This procedure is necessary to configure a new SCSI target on a Fibre Channel interconnect. Bear in mind that issue_lip is an asynchronous operation.

The command may complete before the entire scan has completed. You must monitor /var/log/messages to determine when it is done. The lpfc and qla2xxx drivers support issue_lip

For SCSI Host

# echo "- - -" > /sys/class/scsi_host/host0/scan

It means that you are echoing a wildcard value of "channel target and lun" (CTL), and the operating system will rescan the device path.

The Universally Unique Identifier can be used to identify a device independent form its mount point or device name. This is more and more important as many devices today support hot-plugging or are external anyway. Therefore it makes sometimes sense to access a device (for example in fstab) not by device name but by the UUID.

There are several ways to get the UUID. The first one uses the /dev/ directory. While you are on is you might want to check other by-* directories, I never knew of them.

The reason for using links is because kernel device names (e.g. /dev/sda, /dev/sdb) may change depending on where or when the disk is plugged in, whereas the links in /dev/disk/by-* will always point to the same drive no matter what, and therefore are safer to use.

How to find the Number of HBA's in the server

# lspci | grep -i HBA

04:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)

05:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)

Find out how many host bus adapter configured in the Linux box. You can use “systool -fc_host -v” to verify available FC in the system.

3.If the system virtual memory is too low ,then do not proceed further.If you have enough free virtual memory,then you can proceed with below command to scan new LUNS.

pwd

/sys/class/fc_host/host1

-r--r--r-- 1 root root 4096 Apr  7 03:06 fabric_name

--w------- 1 root root 4096 Apr  7 03:06 issue_lip

-r--r--r-- 1 root root 4096 Apr  7 03:06 node_name

-r--r--r-- 1 root root 4096 Apr  7 03:06 port_id

-r--r--r-- 1 root root 4096 Apr  5 08:40 port_name

-r--r--r-- 1 root root 4096 Apr  5 08:40 port_state

-r--r--r-- 1 root root 4096 Apr  7 03:06 port_type

-r--r--r-- 1 root root 4096 Apr  7 03:06 speed

drwxr-xr-x 2 root root    0 Apr  4 18:22 statistics

lrwxrwxrwx 1 root root    0 Apr  4 18:24 subsystem -> ../../../class/fc_host

-r--r--r-- 1 root root 4096 Apr  7 03:06 supported_classes

-r--r--r-- 1 root root 4096 Apr  7 03:06 supported_speeds

-r--r--r-- 1 root root 4096 Apr  7 03:06 symbolic_name

-rw-r--r-- 1 root root 4096 Apr  7 03:06 system_hostname

-rw-r--r-- 1 root root 4096 Apr  7 03:06 tgtid_bind_type

--w------- 1 root root 4096 Apr  4 18:23 uevent

2.Find out how many SCSI controller configured.

[root@mylinz1 ~]# ls /sys/class/scsi_host/host

host0 host1 host2

In this case,you need to scan host0,host1 & host2.

How to check the HBA driver version

Method 1

[test@linux:NODB /proc/scsi] # lspci | grep -i HBA

04:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)

[test@linux:NODB /proc/scsi] # lspci -v -s 04:00.0

04:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)

        Subsystem: QLogic Corp. Device 015c

        Flags: bus master, fast devsel, latency 0, IRQ 98

        I/O ports at ec00 [size=256]

        Memory at df2fc000 (64-bit, non-prefetchable) [size=16K]

        Expansion ROM at df200000 [disabled] [size=256K]

        Capabilities: [44] Power Management version 3

        Capabilities: [4c] Express Endpoint, MSI 00

        Capabilities: [88] MSI: Enable- Count=1/32 Maskable- 64bit+

        Capabilities: [98] Vital Product Data

        Capabilities: [a0] MSI-X: Enable+ Count=2 Masked-

        Capabilities: [100] Advanced Error Reporting

        Capabilities: [138] Power Budgeting <?>

        Kernel driver in use: qla2xxx

        Kernel modules: qla2xxx

# modinfo qla2xxx | grep version

version:        8.03.07.15.05.09-k

Method 2

[test@linux:NODB /proc/scsi] # systool -c scsi_host -v | grep 'driver_version'

Method 3   : 

#dmesg | grep -i hba

How to check the HBA driver version


Method 1: 

# systool -c scsi_host -v | grep -w fw_version

Method 2:

# dmesg | grep -i fw

How to check the wwn number 

# cat /sys/class/scsi_host/host1/device/fc_host:host1/port_name

# cat /sys/class/scsi_host/host*/device/fc_host/host*/node_name

[XXXX@XXXX ~]# modinfo qla2xxx | grep version
version:        8.03.01.04.05.05-k
srcversion:     6CD3A3C1F687EACB007F2CD

l2105 ~]# cat /sys/class/fc_host/host*/node_name

0x5001438028cbd4c5

0x5001438028cbd4c7

0x5001438028cbd2c5

0x5001438028cbd2c7

[root@l2105 ~]# lspci | grep -i hba

04:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)

04:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)

27:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)

27:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)

@test ~]# modinfo qla2xxx | grep version

version:        8.07.00.26.06.8-k

srcversion:     50E0219D762F696CEE5A2D3

vermagic:       2.6.32-642.13.1.el6.x86_64 SMP mod_unload modversions

[root@l2105 ~]# lsmod

Module                  Size  Used by

iptable_filter          2793  0

ip_tables              17895  1 iptable_filter

sunrpc                267620  1

bonding               132885  0

ipv6                  336282  0

power_meter             9009  0

acpi_ipmi               3745  1 power_meter

ipmi_si                46440  2 acpi_ipmi

ipmi_msghandler        40044  2 acpi_ipmi,ipmi_si

iTCO_wdt                7278  0

iTCO_vendor_support     3056  1 iTCO_wdt

joydev                 10480  0

hpilo                   7821  8

hpwdt                   7062  0

sg                     29318  0

tg3                   163674  0

bnx2x                 727465  0

ptp                     9614  2 tg3,bnx2x

pps_core               10690  1 ptp

libcrc32c               1246  1 bnx2x

mdio                    4769  1 bnx2x

serio_raw               4626  0

lpc_ich                13571  0

mfd_core                1927  1 lpc_ich

shpchp                 29130  0

ext4                  379559  10

jbd2                   93252  1 ext4

mbcache                 8193  1 ext4

dm_round_robin          2525  4

sd_mod                 37158  8

crc_t10dif              1209  1 sd_mod

qla2xxx               502213  16

How to check LUN ID's in a server

Method 1:

# multipath -ll - Will give the LUN ID's

Method 2: Native Command

# scsi_id -g -s /block/sd<>

-g  - Treat the device as white listed. The -g option must be specified on the command line or in the scsi_id configuration file for scsi_id to generate any output

-s - Generate an id for the sysfs-device.  The sysfs mount point must not be included. For 

example, use /block/sd, not /sys/block/sd.

For each and every LUN there will be a drive letter managed by sd as the drives are managed by sd driver. which can found in /dev/disk/by-id/ - Folder will also have the scsi_id. Else we can get the drive letters by executing fdisk -l and grep for sd then give the sd device name in scsi id command to find the LUN ID. For Eg: if we get output as /dev/sdqp1 then with scsi_id execute scsi_id -g -s /block/sdqp

In the above folder we can four files with same id, that may be because of multipath. Even under /sys/block/sd four devices can have same id may be because of multipathing.