[OpenIndiana-discuss] sata port multiplier? dedup tables overflowing RAM D510MO

[Ray Arachelian]

On 05/28/2011 08:45 AM, Matt Connolly wrote:
> On 28/05/2011, at 7:08 AM, Ray Arachelian <ray at arachelian.com> wrote:
>   
>> On 05/25/2011 05:21 PM, Matt Connolly wrote:
>>     
>>> Sounds like an almost identical setup to what I have. Although I'm using a Si3114 card. I have AHCI enabled in the bios, so my drivers are:
>>>
>>> Silicon Image si3114 = pci-ide
>>> Intel SATA Controller = ahci.
>>>
>>> And, if you do find a miniPCIe SSD, please do share, I'd be interested in this too.
>>>
>>>       
>> I bought this one: 
>> http://www.newegg.com/Product/Product.aspx?Item=N82E16820167039
>>
>> I haven't yet installed it, but I'll do it this weekend. :)
>>
>>     

Bad news.  This card requires mSATA support, which of course, the D510,
even though it being also an Intel product, and even though the SSD also
being an Intel product, does not have.
Neither Solaris, nor the BIOS, nor Linux see the SSD.  These are
intended for eeePC's. 

I'm now looking for a mini PCIe SATA card, and a standard SATA SSD. 
Grrr Arg... (insert other angry sounds here.)  I don't see any mini PCIe
SATA cards at newegg, so I'll have to look deeper I guess.  I did see
another mini PCIe SATA card that doesnt mention mSATA, but it won't be
in stock at newegg until the end of July, so I'm not sure I want to wait
for it.

I think I saw some mini PCIe to PCIe adapter, not sure, but even so, I'm
not sure it will fit in the case either.


So incase anyone else has a D510MO, please don't buy that card, it won't
work.

I initially thought I had to reflash the BIOS, which I did.  I went up
from some low end version 145 or so to 516.  Sadly you can't directly
flash that way, you have to download a copy of every major version and
flash from 1XX to 2XX to 3XX to 4XX to 5XX.  It's painful as it's just
copying a bunch of .BIO files and pressing repeatedly F7 and rebooting,
so only a little bit annoying.


> Sounds just like what I've done recently, except I zfs send straight from one drive (was previously mirrored) into a freshly made raidz pool with 4k alignment. I'd consider not using dedup given the cheap price of storage these days combined with limited ram in on that mobo. As in, the performance hit for the saved disk space wouldnt be worth it for my needs.
>   

I'm sure the dedup would work just fine for casual duplicates, but as
usual good sanitary practices are more important.
I wish there was some warning to /var/adm/messages saying "your dedup
tables are over 25% of your RAM, clean up your filesystem." :)

Is there a way to safely turn off dedup?  What happens to the already
deduped files and the dedup tables?

Or better yet find out which inodes contain duplicate blocks?  What I
did was very time consuming - took more than a week to go through all
the files and obtain MD5 sums of them.  Looking on sourceforge for
programs that find duplicate files wasn't of much help - they're just
not suited for this kind of massive task.

I guess if I were to write something like this, I'd not use sort and
instead dump the md5's out to sqlite or mysql and then add some fields
in there for mtime/ctime/inode to also handle moving files around.... 
Anyone interested in this, other than just myself?  Should I bother to
invest the effort?