LSI will not support the 840 or 840 pro with LSI megaraid, 9260 through 9271 .
The problem manifests itself in two specific drives, oddly these two drives are both competitors to a company called Sandforce.
Samsung 840/840 pro - arguably the best ssd from a stable company (sorry OCZ but your reputation is tarnished, great drive the vector is).
Intel DC3700.
Odd how these two competitors are the biggest threat to sandforce, but Intel is a huge LSI reseller no less.
So - What's the problem?
1. Controller will not let you alter the Disk Cache policy. So if it is off , the massive 512meg cache on the samsung 840 pro (large sized) is not enabled*
2. Strange performance - typically when building a raid set using CTRL-H the raid will perform on AS SSD 4K WRITE at 1-2MB/s taking 7 minutes to complete that portion of the test.
3. Using the 5.6 firmware,MSM, and Drivers (all dated Mar 02 2013) with windows 2008 R2 we could replicate this. We consulted Webhostingtalk and lowendtalk and this is a known problem manifesting the two drive makers' product. LSI knew about this in the last set of drivers and did not update anything on the Mar 02 2013 release of firmware and driver.
4. Controller resets to RA randomly on reboot.Why the heck would I set the SSD to Read Ahead, that's silly and against the fastpath guidelines. More than that gets reset but I won't say more.
5. Creating the raid volume via MSM however may not cause this problem (WTF!). Benchmarks after creating a volume with the 9266 as secondary and the 9260 as primary show normal performance. The 9266 is about twice as fast as the 9260 without fastpath. Fastpath doesn't really seem to work that hot with 2 drives (4 drive raid-10). More on this**
.
6. You must cold shutdown this controller between changes. Firmware, settings. Shutdown and pull the power plug for a minute. Seriously. Everytime.
7. Why would this affect the DC3700 drive as well? it's on the list of approved. Watch it fall off the list soon. LSI we have google and we can go back in time to retrieve your older approved lists. the 840 pro was on the list a few months ago.
The symptoms are quite simple. Flakey. The AS SSD 4K write taking 7 minutes is slower than a 5400 rpm drive. The drive lights are lit solid. This can be replicated on the 9260 by "ENABLE DISK CACHE POLICY: DISABLE" - but the option is greyed out on the 9266. Both controllers use the same 5.6 driver, and 5.6 MSM. Matter of fact, doesn't the 9260 and 9266 share the same firmware nowadays? I don't know that answer.
The symptom also presents as "BLOCK SSD WRITE CACHE CHANGE: YES" when these drives are present. That's a pretty straight up obvious message (storecli show all, or megacli adpaLInfo ) - this message pretty much goes with the "Enable Disk Cache Policy" being greyed out on the MSM.
** We applied fastpath/cc 2 trial key to the card and it then would allow change of policy on the card. however you should not have to buy this key?
* the 9260 had a dead battery, the 9266 had no CV module, but the cache offload (cachevault) was enabled in advanced software options.
So really, LSI, why don't you go back and fix your firmware and controllers. The folks are webhostingtalk are probably some of your larger customers. The original firmware that Samsung 840/840 pro had did have GC issues, but was replaced a month or so ago with new firmware. The DC3700 should have no such problems, especially since it has super capacitor.
The megaraid controllers are pretty interesting. If you build a raid-1 or 1+0 you can kind of see what's going on. It's like it is tuning itself based on the type of reads, Queue Depth, latency, and linear/random, to use the drives as read ahead. Most RAID 1+0 systems will read from 4 drives, if you build a 4 drive raid. It seems that LSI is dynamically choosing to read from 2 , 3 [blip], 3 , or 4 drives based on the block size, type of read, and read/write mix. Unless I have the read/write thing backwards in which case its reading from all 4 but writing to 2,3, or 4 disks. You can replicate this by building a raid-10 and watching ATTO (QD4,QD10), AS SSD, and CDM - all popular benchmarks. This is very different from every other raid manufacturer and I wonder if it part of their secret sauce or part of their problem?
For reference for my above opinion, I believe LSI needs to work on their drivers (win2k8R2) and firmware. We noticed you could change the C: or E: drive cache policy in windows with the 9266. The 9260 would refuse when I touched this. Likewise the P400,P410,P420 HP [latter two are PMC Adaptec] would refuse to allow us to change these settings. This is very normal since the raid card should be in control of these settings.
Back to the reference:
9260 w/fastpath is just as fast as 9266 without, and quite honestly it is stable with the 840 pro/840 and cheap refurb/used.
PMC-Adaptec PM8020 based HP P420 - I got a deal on these, nobody wants them because they have tiny heatsinks and overheat quickly and go offline, nothing a little arctic goop and a creative copped bridge or active cooling can't fix [idle the P420 is 58C, the 9266 idles at 68C in the same slot on a dl380 G7 - put that in your pipe and smoke it].
So the P420, a rather new card, not designed for the G7, with 0% read-ahead, and 100% write back outruns the 9266. I suspect with more drives it would outrun it further. It can enable the drive write cache on the 840/840 pro. It does not mind that I'm running the non-hp drives. I am sure the 1GB supercap cache helps with writes, but quite honestly, at the end of the day it really is which controller is consistent and performs well.
Folks might notice the 71605e (raid 1/10/1e and job at same time) or 71605h (hba only) use a newer version of this chip. It has 16 ports per card standard. It requires no fastpath to go faster because it has 16 ports - 16 840 pro's will go faster than the 9266/9271 w/fastpath. The Adaptec(PMC) cards also support JBOD+RAID at the same time on the raid. And Cachekade on their expensive 71605q. [The hp P420 enables raid-6/60 and cacheKAde 1.0 read only with the SAAP 2.0 key].
There are other options but all know that pretty much HP uses PMC-adaptec in their servers now, everyone else on the planet uses LSI.
I hope someone else reads this, perhaps someone at LSI, and definitely anyone buying samsung 840/840 pro's and an expensive Megaraid card to save them the time and trouble i've wasted.
I am not the only one. google "840 pro 9266 webhostingtalk" or "840 pro cachecade webhostingtalk" or look around here. Folks are having to cross-flash their LSI branded cards to PERC to get them to perform? How odd is that?.
As always, have a nice day!
