[OpenIndiana-discuss] [zfs] mask pci-e errors

[jason matthews]

On Jan 14, 2014, at 4:27 PM, "Garrett D'Amore" <garrett at damore.org> wrote:

Thanks for the response.

> 
> I'm surprised to see *both* the X1 and the 910, in particular.  I imagine the X1 is your SLOG, in which case you are probably using the 910 for L2ARC?  If so, why not just have that on normal SAS connected SSD?  It would have probably been cheaper and since they have separate power supplies (SAS / SATA power are not routed over the data bus, but use separate power lines that go right to the PSU), you'd be free of this problem. 
> 

The systems run multiple master and slave postgres systems, one per zone. The masters run on the 910s and the slaves run against two nds-2441 disk shelves that they share with a sister system. Each system gets 1/2 of each shelf, configured as mirrors. So twelve sets of mirrors of 15k rpm drives for each system.

The systems were initially deployed with the x1 as slog for the spinning rust. Over time, the spinning rust was over run by i/o - the latency just got to high. Postgres now does in excess of 1M inserts/hour/zone and is limited by the software used to feed postgres the data. The 910s were added and the masters were migrated to the 910s. The slaves stayed on the shelves.  

The 910s are now running out of space and I will be moving to a mirrored dcs3700 setup as soon as i can complete a data center expansion. Basically, I am a efficient (cheap?) bastard and cram the most into a system I can. The power whips all run exactly at 80% - nothing goes to waste ;-)


> Anyway, my thought is that you're over power, or so close to the limit of the specification that peak loads are killing you. 

I guess it is possible. The power budget section of the motherboard technical spec would lead one to a different conclusion. 

i have support contracts on the gear in Ashburn. i will open a case on one of those and see if i can get a determination from Intel.

thanks,
j.