[OpenIndiana-discuss] Only one SSD detected on expander

Scott Marcy oi at mscott.org
Wed Oct 17 13:30:48 UTC 2012


So one more data point that seems to suggest that I'm getting duplicate names from these drives. When I attach at least one of the Samsung SSDs to each expander (front and back), I get this:

$ cfgadm -al
Ap_Id                          Type         Receptacle   Occupant     Condition
c6                             scsi-sas     connected    configured   unknown
c6::es/ses0                    ESI          connected    configured   unknown
c6::smp/expd0                  smp          connected    configured   unknown
c6::w5002538043584d30,0        disk-path    connected    configured   unknown  <-- SSD
c6::w5000c500513a0efe,0        disk-path    connected    configured   unknown
c6::w5000c5005168b2f0,0        disk-path    connected    configured   unknown
c6::w5000c5005169a612,0        disk-path    connected    configured   unknown
c6::w5000c5005169cc6a,0        disk-path    connected    configured   unknown
c6::w5000c500515199bd,0        disk-path    connected    configured   unknown
c6::w5000c500517042d9,0        disk-path    connected    configured   unknown
c8                             scsi-sas     connected    configured   unknown
c8::es/ses1                    ESI          connected    configured   unknown
c8::smp/expd1                  smp          connected    configured   unknown
c8::w5002538043584d30,0        disk-path    connected    configured   unknown  <-- SSD
c9                             scsi-sas     connected    unconfigured unknown
sata0/0::dsk/c4t0d0            disk         connected    configured   ok
sata0/1::dsk/c4t1d0            disk         connected    configured   ok
sata0/2                        sata-port    empty        unconfigured ok
sata0/3                        sata-port    empty        unconfigured ok
sata0/4                        sata-port    empty        unconfigured ok
sata0/5                        sata-port    empty        unconfigured ok

Note that both c6 and c8 have the exact same name for these drives. I *CAN* access the two drives separately, but why on earth am I getting these duplicates? This means I can only reliably access one Samsung SSD per expander.

Thanks.

-Scott


On Oct 16, 2012, at 7:29 PM, Scott Marcy <oi at mscott.org> wrote:

> Hi all,
> 
> I'm new to the list and OI, but I've been running a Solaris box with some version of ZFS on it for a couple years. I recently acquired a Supermicro 6047R-E1R36N. It has an X9DRi-LN4F+ motherboard, dual E5-2620s and 48GB of RAM. I soon learned that I needed a different HBA (it comes with an LSI 2108-based card that doesn't have any JBOD passthrough mode), so I replaced that with an LSI SAS9211-8i, flashed with the latest "IT" firmware.
> 
> Being new to SAS expanders (which this server has two of), I quickly learned that some older 2TB drives I had on my older Solaris box simply will not work reliably on the expander. (They would very quickly lock up the entire file system once the load got heavy.) However, I've had much better luck (so far) with some newer 3TB SATA drives (Seagate ST3000DM001). I'm still not convinced that these are going to be reliable in the long run, having now read about a few SATA-on-SAS expander horror stories (and my own experience with the 2TB drives), but this isn't a production system (yet) so I'm willing to see how it goes.
> 
> But the one problem I can't seem to figure out are the Samsung 830 SSD drives I have for ZIL and ARC. I have 2 mirrored 128GB 830s as the boot pool, but they're on the built-in SATA ports, not the expander ports. Then I have four other 128GB 830s which I installed into 3.5" hot-swap adapters and these are the ones causing problems. The HBA sees the drives just fine—they're listed during POST and I can access them via the card's built-in BIOS. But once OI (151a5) is up and running, it only sees ONE of the 830s. And here's where it gets weird: if I remove all but one of the SSDs from the expander ports, it will see whichever one SSD is connected—it does not matter which port the drives are on. Also, regardless of the port, the attachment path always seems to be the same in the OS. It's like the OS thinks all four drives are the exact same physical drive.
> 
> $ cfgadm -al
> Ap_Id                          Type         Receptacle   Occupant     Condition
> c6                             scsi-sas     connected    configured   unknown
> c6::es/ses0                    ESI          connected    configured   unknown
> c6::smp/expd0                  smp          connected    configured   unknown
> c6::w5002538043584d30,0        disk-path    connected    configured   unknown  <-- this is the SSD
> c6::w5000c500513a0efe,0        disk-path    connected    configured   unknown
> c6::w5000c5005168b2f0,0        disk-path    connected    configured   unknown
> c6::w5000c5005169a612,0        disk-path    connected    configured   unknown
> c6::w5000c5005169cc6a,0        disk-path    connected    configured   unknown
> c6::w5000c500515199bd,0        disk-path    connected    configured   unknown
> c6::w5000c500517042d9,0        disk-path    connected    configured   unknown
> c8                             scsi-sas     connected    configured   unknown
> c8::es/ses1                    ESI          connected    configured   unknown
> c8::smp/expd1                  smp          connected    configured   unknown
> c9                             scsi-sas     connected    unconfigured unknown
> sata0/0::dsk/c4t0d0            disk         connected    configured   ok
> sata0/1::dsk/c4t1d0            disk         connected    configured   ok
> sata0/2                        sata-port    empty        unconfigured ok
> sata0/3                        sata-port    empty        unconfigured ok
> sata0/4                        sata-port    empty        unconfigured ok
> sata0/5                        sata-port    empty        unconfigured ok
> 
> 
> $ format
> Searching for disks...done
> 
> 
> AVAILABLE DISK SELECTIONS:
>       0. c4t0d0 <ATA-SAMSUNGSSD830-3B1Q cyl 19932 alt 2 hd 224 sec 56>
>          /pci at 0,0/pci15d9,626 at 1f,2/disk at 0,0
>       1. c4t1d0 <ATA-SAMSUNGSSD830-3B1Q cyl 19932 alt 2 hd 224 sec 56>
>          /pci at 0,0/pci15d9,626 at 1f,2/disk at 1,0
>       2. c7t5000C500513A0EFEd0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB>
>          /scsi_vhci/disk at g5000c500513a0efe
>       3. c7t5000C5005168B2F0d0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB>
>          /scsi_vhci/disk at g5000c5005168b2f0
>       4. c7t5000C5005169A612d0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB>
>          /scsi_vhci/disk at g5000c5005169a612
>       5. c7t5000C5005169CC6Ad0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB>
>          /scsi_vhci/disk at g5000c5005169cc6a
>       6. c7t5000C500515199BDd0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB>
>          /scsi_vhci/disk at g5000c500515199bd
>       7. c7t5000C500517042D9d0 <ATA-ST3000DM001-9YN1-CC4B-2.73TB>
>          /scsi_vhci/disk at g5000c500517042d9
>       8. c7t5002538043584D30d0 <ATA-SAMSUNG SSD 830-3B1Q-119.24GB>
>          /scsi_vhci/disk at g5002538043584d30
> 
> This last one is the SSD. If I remove all the SSDs from the expander ports, disk #8 does not show up (nor does it show in cfgadm). If I put one SSD into any expander port, disk 8 is back. If I add additional SSDs, I still only have disk 8. If I remove the first SSD (while others are still attached), disk 8 still appears. In all cases, I can run a read test on the disk (format -> analyze -> test) without any errors, so I think it's actually accessing one of the physical drives.
> 
> How do I get OI to see the other 3 SSDs?
> 
> Any pointers are greatly appreciated. Thanks!
> 
> -Scott
> 
> 
> _______________________________________________
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss at openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss



More information about the OpenIndiana-discuss mailing list