[OpenIndiana-discuss] An interesting experience with sata drives on an HBA

[Dan Swartzendruber]

I have an M1015 (rebadged LSI HBA with two 8087 connectors.)  One of 
them connects to a 3.5" jbod chassis with 4 SAS nearline drives (tank 
pool.)  I also have two samsung 830 SSD (sata) connected to the 2nd port 
on the HBA with a forward breakout cable.  Works just fine.  Apparently 
some SATA drives (WD blues at least) *do* seem to have a WWN assigned.  
Others (like the SSDs) don't - I infer the HBA fakes up WWNs for them.  
So I spent several hours trying to salvage my virtualized OI install to 
move it to the bare-metal server.  The first hurdle was a hang at boot.  
Booting with '-kv' showed the last message being 'audio0 is 
/pseudo/audio0' and nothing after that.  I did try 'touch /reconfigure' 
via the live CD but that didn't help.  Not sure what else to do, and 
google was not real helpful.  I really hoped OI wouldn't be as finicky 
as windows, but alas :( I decided to do a fresh install and fix up stuff 
later.  So I do a live CD install to get the GUI desktop so I can run 
virtualbox.  Comes up just fine, except one of the two SSDs shows as 
faulted.  After much experimentation (and I apologize that I forgot to 
save output from the format command - it was 1:30AM by that point and I 
was exhausted.)  Anyway, here is the output from the virtualized system 
(back to that now):

        4. c5t5000C50041BD3E85d0 <SEAGATE-ST1000NM0001-0002-931.51GB>
           
/pci at 0,0/pci15ad,7a0 at 15/pci1000,3040 at 0/iport at f/disk at w5000c50041bd3e85,0
        5. c5t5000C50041BD703Dd0 <SEAGATE-ST1000NM0001-0002-931.51GB>
           
/pci at 0,0/pci15ad,7a0 at 15/pci1000,3040 at 0/iport at f/disk at w5000c50041bd703d,0

Note the '@w5000xxxxxxxx' stuff?  In my fail case, it was like 
'c65000xxxxxxx' and 'c75000xxxxxxxxx' because the two SSDs were on 
different SATA connectors of the breakout cable.  Here is the gotcha 
though: the part *after* the 'c6' or 'c7' was identical!  And it was 
identical because the part after the '@w5000' was identical!  I can only 
infer this is some kind of glitch with the HBA where it got confused and 
assigned duplicate WWN for the two SATA SSDs, and this led OI to think 
they were in fact the same device, so trying to add the 2nd SSD to the 
pool would get a message that it is already in the pool.  Confused the 
heck out of me until I noticed they were duplicates :)  Even if I could 
jigger the HBA into 'fixing' this, no guarantee it wouldn't bork later.  
For now, I put the two SSDs on motherboard sata ports, so they get the 
canonical cXtYdZ.  It's odd that OI seemed to think they were the same 
device, but different.  Hmmmm...