Fiberchannel Solaris Part 3: SAN Diagnostics
1. Kernel modules loaded?
Check whether the needed kernel modules/drivers are loaded:
The needed modules are printed in bold. The last line just matches because it also contains "fc".
2. Which FC interfaces do I have?
Use "fcinfo":
Result: Two ports available, one port per fiberchannel card. The WWPNs are printed in bold.
There's also the port type (N_port in this case) and the current data bit rate.
3. Which WWPNs (storage devices) are seen from my fiberchannel interface ports?
Use "fcinfo remote-port -p" with the interface port WWPN found in step 2:
"SCSI" types are storage disks (storage volumes).
"SCSI,IP" types are devices capable of talking IP and SCSI (storage). In this case this is a Solaris host with IP configured.
4. Are all disks configured as solaris disks (usable by Solaris)?
Use "cfgadm -al" (this will configure also newly added fc disks):
The relevant lines are printed in bold.
5. Is multipathing working?
Use "luxadm display" with the target WWPN of the storage disk device to list the paths:
There are three LUNs here: 0,1 and 2; in Solaris "FC Device Address" syntax they are defined as wwpn,lun.
6. A simple MPxIO overview: vhci_stat
To have a short output of MPxIO SAN status, you may use vhci_stat.
This is a little utility written by myself, available for download here: http://southbrain.com/south/software/vhci-stat.html
In this example you see also ZFS filesystem information and SCSI inquiry data.
Check whether the needed kernel modules/drivers are loaded:
# modinfo|grep fc
93 ffffffffef17d000 19ac0 87 1 fcp (SunFC FCP v20090210-1.134)
94 ffffffffef196000 a2a0 - 1 fctl (SunFC Transport v20090210-1.59)
157 ffffffffef3e8000 9620 83 1 fcip (SunFC FCIP v20090210-1.50)
160 ffffffffef494850 5610 92 1 fcsm (Sun FC SAN Management v20070116)
166 ffffffffef4b5bd8 1fc0 151 1 ippctl (IP Policy Configuration Driver )
The needed modules are printed in bold. The last line just matches because it also contains "fc".
2. Which FC interfaces do I have?
Use "fcinfo":
# fcinfo hba-port
HBA Port WWN: 2100001b321fbd9c
OS Device Name: /dev/cfg/c4
Manufacturer: QLogic Corp.
Model: 375-3355-01
Firmware Version: 4.04.01
FCode/BIOS Version: BIOS: 1.24; fcode: 1.24; EFI: 1.8;
Serial Number: 0402R00-0725369970
Driver Name: qlc
Driver Version: 20081115-2.29
Type: N-port
State: online
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: 4Gb
Node WWN: 2000001b321fbd9c
HBA Port WWN: 2100001b321fe69b
OS Device Name: /dev/cfg/c5
Manufacturer: QLogic Corp.
Model: 375-3355-01
Firmware Version: 4.04.01
FCode/BIOS Version: BIOS: 1.24; fcode: 1.24; EFI: 1.8;
Serial Number: 0402R00-0725369921
Driver Name: qlc
Driver Version: 20081115-2.29
Type: N-port
State: online
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: 4Gb
Node WWN: 2000001b321fe69b
Result: Two ports available, one port per fiberchannel card. The WWPNs are printed in bold.
There's also the port type (N_port in this case) and the current data bit rate.
3. Which WWPNs (storage devices) are seen from my fiberchannel interface ports?
Use "fcinfo remote-port -p" with the interface port WWPN found in step 2:
# fcinfo remote-port -p 2100001b321fbd9c
Remote Port WWN: 210000e08b0d82c6
Active FC4 Types: SCSI,IP
SCSI Target: no
Node WWN: 200000e08b0d82c6
Remote Port WWN: 2100001b321fb79d
Active FC4 Types: SCSI,IP
SCSI Target: no
Node WWN: 2000001b321fb79d
Remote Port WWN: 210000d0231c50ab
Active FC4 Types: SCSI
SCSI Target: yes
Node WWN: 200000d0231c50ab
Remote Port WWN: 220000d0232c50be
Active FC4 Types: SCSI
SCSI Target: yes
Node WWN: 200000d0232c50be
"SCSI" types are storage disks (storage volumes).
"SCSI,IP" types are devices capable of talking IP and SCSI (storage). In this case this is a Solaris host with IP configured.
4. Are all disks configured as solaris disks (usable by Solaris)?
Use "cfgadm -al" (this will configure also newly added fc disks):
# cfgadm -al
Ap_Id Type Receptacle Occupant Condition
c3 scsi-bus connected configured unknown
c3::dsk/c3t0d0 disk connected configured unknown
c4 fc-fabric connected configured unknown
c4::2100001b321fb79d unknown connected unconfigured unknown
c4::210000d0231c50ab disk connected configured unknown
c4::210000e08b0d82c6 unknown connected unconfigured unknown
c4::220000d0232c50be disk connected configured unknown
c5 fc-fabric connected configured unknown
c5::2100001b321f9eb3 unknown connected unconfigured unknown
c5::210000d0231c50be disk connected configured unknown
c5::220000d0232c50ab disk connected configured unknown
usb0/1 unknown empty unconfigured ok
usb0/2 unknown empty unconfigured ok
usb0/3 usb-device connected configured ok
usb0/4 usb-storage connected configured ok
usb0/5 usb-floppy connected configured ok
usb0/6 unknown empty unconfigured ok
usb0/7 unknown empty unconfigured ok
usb1/1 unknown empty unconfigured ok
usb1/2 unknown empty unconfigured ok
usb1/3 unknown empty unconfigured ok
usb1/4 unknown empty unconfigured ok
usb1/5 unknown empty unconfigured ok
usb1/6 usb-hub connected configured ok
usb1/6.1 unknown empty unconfigured ok
usb1/6.2 unknown empty unconfigured ok
usb1/6.3 unknown empty unconfigured ok
usb1/6.4 unknown empty unconfigured ok
usb1/7 unknown empty unconfigured ok
The relevant lines are printed in bold.
5. Is multipathing working?
Use "luxadm display" with the target WWPN of the storage disk device to list the paths:
# luxadm display 220000d0232c50be
DEVICE PROPERTIES for disk: /dev/rdsk/c6t600D0230006C1C4C0C50BE16FF1FE200d0s2
Vendor: ADVUNI
Product ID: OXYGENRAID 416F4
Revision: 347B
Serial Num: 0C50BE16FF1FE200
Unformatted capacity: 0.000 MBytes
Write Cache: Enabled
Read Cache: Enabled
Minimum prefetch: 0x0
Maximum prefetch: 0xffff
Device Type: Disk device
Path(s):
/dev/rdsk/c6t600D0230006C1C4C0C50BE16FF1FE200d0s2
/devices/scsi_vhci/disk@g600d0230006c1c4c0c50be16ff1fe200:c,raw
Controller /dev/cfg/c4
Device Address 220000d0232c50be,2
Host controller port WWN 2100001b321fbd9c
Class primary
State ONLINE
Controller /dev/cfg/c5
Device Address 210000d0231c50be,2
Host controller port WWN 2100001b321fe69b
Class primary
State ONLINE
DEVICE PROPERTIES for disk: /dev/rdsk/c6t600D0230006C1C4C0C50BE4DFE511B00d0s2
Vendor: ADVUNI
Product ID: OXYGENRAID 416F4
Revision: 347B
Serial Num: 0C50BE4DFE511B00
Unformatted capacity: 715144.000 MBytes
Write Cache: Enabled
Read Cache: Enabled
Minimum prefetch: 0x0
Maximum prefetch: 0xffff
Device Type: Disk device
Path(s):
/dev/rdsk/c6t600D0230006C1C4C0C50BE4DFE511B00d0s2
/devices/scsi_vhci/disk@g600d0230006c1c4c0c50be4dfe511b00:c,raw
Controller /dev/cfg/c4
Device Address 220000d0232c50be,1
Host controller port WWN 2100001b321fbd9c
Class primary
State ONLINE
Controller /dev/cfg/c5
Device Address 210000d0231c50be,1
Host controller port WWN 2100001b321fe69b
Class primary
State ONLINE
DEVICE PROPERTIES for disk: /dev/rdsk/c6t600D0230006C1C4C0C50BE57396E9F00d0s2
Vendor: ADVUNI
Product ID: OXYGENRAID 416F4
Revision: 347B
Serial Num: 0C50BE57396E9F00
Unformatted capacity: 0.000 MBytes
Write Cache: Enabled
Read Cache: Enabled
Minimum prefetch: 0x0
Maximum prefetch: 0xffff
Device Type: Disk device
Path(s):
/dev/rdsk/c6t600D0230006C1C4C0C50BE57396E9F00d0s2
/devices/scsi_vhci/disk@g600d0230006c1c4c0c50be57396e9f00:c,raw
Controller /dev/cfg/c4
Device Address 220000d0232c50be,0
Host controller port WWN 2100001b321fbd9c
Class primary
State ONLINE
Controller /dev/cfg/c5
Device Address 210000d0231c50be,0
Host controller port WWN 2100001b321fe69b
Class primary
State ONLINE
There are three LUNs here: 0,1 and 2; in Solaris "FC Device Address" syntax they are defined as wwpn,lun.
6. A simple MPxIO overview: vhci_stat
To have a short output of MPxIO SAN status, you may use vhci_stat.
This is a little utility written by myself, available for download here: http://southbrain.com/south/software/vhci-stat.html
# vhci_stat -iz
vhci_stat
1.3.1; Pascal Gienger <pascal@southbrain.com>; http://southbrain.com/south/
Host: atlanta
PATH WWPN,LUN STATE
------------------------------------------------- ------------------- -----
/scsi_vhci/disk@g600d0230006b66680c50ab5675f91300
(Vendor = ADVUNI Model = OXYGENRAID 416F4 Rev. = 347B VolID = 5675F913)
ZFS: pool name = mail, ZFS Version: 10
ZFS: size = 3.41T, member type = mirror
/pci@7b,0/pci10de,5d@e/pci1077,142@0/fp@0,0 220000d0232c50ab,2 ONLINE
/pci@7b,0/pci10de,5d@d/pci1077,142@0/fp@0,0 210000d0231c50ab,2 ONLINE
/scsi_vhci/disk@g600d0230006b66680c50ab4f92f61000
(Vendor = ADVUNI Model = OXYGENRAID 416F4 Rev. = 347B VolID = 4F92F610)
ZFS: pool name = cyrus, ZFS Version: 10
ZFS: size = 698.37G, member type = mirror
/pci@7b,0/pci10de,5d@e/pci1077,142@0/fp@0,0 220000d0232c50ab,1 ONLINE
/pci@7b,0/pci10de,5d@d/pci1077,142@0/fp@0,0 210000d0231c50ab,1 ONLINE
/scsi_vhci/disk@g600d0230006b66680c50ab0f36adf100
(Vendor = ADVUNI Model = OXYGENRAID 416F4 Rev. = 347B VolID = 0F36ADF1)
ZFS: pool name = mail, ZFS Version: 10
ZFS: size = 3.41T, member type = mirror
/pci@7b,0/pci10de,5d@e/pci1077,142@0/fp@0,0 220000d0232c50ab,0 ONLINE
/pci@7b,0/pci10de,5d@d/pci1077,142@0/fp@0,0 210000d0231c50ab,0 ONLINE
/scsi_vhci/disk@g600d0230006c1c4c0c50be16ff1fe200
(Vendor = ADVUNI Model = OXYGENRAID 416F4 Rev. = 347B VolID = 16FF1FE2)
ZFS: pool name = mail, ZFS Version: 10
ZFS: size = 3.41T, member type = mirror
/pci@7b,0/pci10de,5d@e/pci1077,142@0/fp@0,0 210000d0231c50be,2 ONLINE
/pci@7b,0/pci10de,5d@d/pci1077,142@0/fp@0,0 220000d0232c50be,2 ONLINE
/scsi_vhci/disk@g600d0230006c1c4c0c50be4dfe511b00
(Vendor = ADVUNI Model = OXYGENRAID 416F4 Rev. = 347B VolID = 4DFE511B)
ZFS: pool name = cyrus, ZFS Version: 10
ZFS: size = 698.37G, member type = mirror
/pci@7b,0/pci10de,5d@e/pci1077,142@0/fp@0,0 210000d0231c50be,1 ONLINE
/pci@7b,0/pci10de,5d@d/pci1077,142@0/fp@0,0 220000d0232c50be,1 ONLINE
/scsi_vhci/disk@g600d0230006c1c4c0c50be57396e9f00
(Vendor = ADVUNI Model = OXYGENRAID 416F4 Rev. = 347B VolID = 57396E9F)
ZFS: pool name = mail, ZFS Version: 10
ZFS: size = 3.41T, member type = mirror
/pci@7b,0/pci10de,5d@e/pci1077,142@0/fp@0,0 210000d0231c50be,0 ONLINE
/pci@7b,0/pci10de,5d@d/pci1077,142@0/fp@0,0 220000d0232c50be,0 ONLINE
In this example you see also ZFS filesystem information and SCSI inquiry data.
0 TrackBacks
Listed below are links to blogs that reference this entry: Fiberchannel Solaris Part 3: SAN Diagnostics.
TrackBack URL for this entry: http://southbrain.com/mt/mt-tb.cgi/117
Hi Pascal,
This guide of yours is excellent!! Very simple and to the point. I was greatly benefited by this tutorial. Some of my doubts were cleared. Keep writing such things. God bless you!!!!
Thanks,
Ajay