[OpenIndiana-discuss] Mapping devices in OI to physical devices

[real-men-dont-click at gmx.net]

-----Original message-----
From:	Len Zaifman <leonardz at sickkids.ca>
Sent:	Tue 22-01-2013 18:29
Subject:	[OpenIndiana-discuss] Mapping devices in OI to physical devices
To:	openindiana-discuss at openindiana.org; 
> We have just had a major system meltdown and it took several days to fix.
> 
> What we would have liked is 2 things we had on thumpers (Old SUN ZFS systems)
> 
> 1) A tool to show the mapping of a solaris device name to a physical location
> 2) A tool to turn on the light on a disk via its solaris device name.
> 
> The process below is too painful, and we have other devices whose disks may go 
> bad. Does either 1 or 2 above exist in openindiana? I could not find it, if it 
> does.
> 
> Thanks.
> 
> The issue was:
> 
> OI (OpenIndiana Development oi_151a X86) reported:
> 
> 
> Jan 22 10:57:43 archivea scsi: [ID 107833 kern.warning] WARNING: 
> /pci at 7a,0/pci8086,3408 at 1/pci1000,3040 at 0 (mpt_sas10):
> Jan 22 10:57:43 archivea        Disconnected command timeout for Target 18
> Jan 22 10:57:43 archivea scsi: [ID 365881 kern.info] 
> /pci at 7a,0/pci8086,3408 at 1/pci1000,3040 at 0 (mpt_sas10):
> Jan 22 10:57:43 archivea        Log info 0x31140000 received for target 18.
> Jan 22 10:57:43 archivea        scsi_status=0x0, ioc_status=0x8048, 
> scsi_state=0xc
> 
> zfs performance went through the floor  and was intolerable(< 1 mb/sec where we 
> had hundreds of MB/sec for resilver/scrubs and 100 MB/sec through the 
> filesystem).
> 
> The defective disk was one of 45 disks in a Supermicro Jbod system 
> (SC847E26-RJBOD1)
> 
> We finally found which disk it was by comparing serial numbers reported by 
> iostat, disks that reported errors and the actual disk serial number (we pulled 
> all 45 disks out to do this mapping). we do not want to repeat this process for 
> our other devices.
> 
> 
>

Hi Len,

we had similar problems in the past, so I did some (google) research for already available tools addressing the issue. What I came across was the following:

dskinfo (quite powerfull and maybe a good bet for you): http://sparcv9.blogspot.de/2011/06/solaris-dskinfo-utility.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+UnixSystem+(It's+a+UNIX+system!)

vhci_stat (interesting if you are using mpxio): http://southbrain.com/south/software/vhci-stat.html


What we always use to identify a disk (flash the led) is something like:

dd if=/dev/dsk/$DISK of=/dev/null

in S11express fmdump shows the affected enclosure slot with SuperMicro JBODs as well (didn't have a broken disk in an OI installation so far, so I don't know if OI works the same way):

Oct 15 16:16:24.1188 0bf2d7d3-4fcf-ee05-d3ed-dc8ffc0ec433 FMD-8000-4M Repaired
  100%  fault.io.disk.predictive-failure        Repair Attempted

        Problem in: hc://:product-id=LSI-CORP-SAS2X36:server-id=:chassis-id=5003048000992b7f:serial=3SL0RFGF00009103VC79:part=SEAGATE-ST3600057SS:revision=0006/ses-enclosure=0/bay=3/disk=0
           Affects: dev:///:devid=id1,sd@n5000c50023f6af03//scsi_vhci/disk@g5000c50023f6af03
               FRU: hc://:product-id=LSI-CORP-SAS2X36:server-id=:chassis-id=5003048000992b7f:serial=3SL0RFGF00009103VC79:part=SEAGATE-ST3600057SS:revision=0006/ses-enclosure=0/bay=3/disk=0
          Location: Slot 04


What I'm missing , too is a completely integrated tool for that....

cu

Carsten