[PLUG] megaraid_sas write caching?

Brent Jones brent at servuhome.net
Mon Dec 17 19:20:43 UTC 2007


On 12/17/07, Aaron Ten Clay <aaron at madebyai.com> wrote:
>
>
> On Dec 15, 2007, at 10:24 AM, Aaron Burt wrote:
>
> > On Sat, Dec 15, 2007 at 02:34:32AM -0800, Aaron Ten Clay wrote:
> >> Hey all,
> >>
> >> Just got Gentoo installed on a shiny new Dell PowerEdge 2950 server
> >> with an LSI Logic MegaRAID SAS RAID controller. Everything is great
> >> except this:
> >>
> >> [   81.028864] sd 0:2:0:0: [sda] 1951399936 512-byte hardware sectors
> >> (999117 MB)
> >> [   81.028991] sd 0:2:0:0: [sda] Write Protect is off
> >> [   81.029059] sd 0:2:0:0: [sda] Mode Sense: 1f 00 10 08
> >> [   81.029121] sd 0:2:0:0: [sda] Write cache: disabled, read cache:
> >> disabled, supports DPO and FUA
> >>
> >> As the kernel reports, both write- and read-caching are disabled.
> >> This
> >> is grossly obvious during file copies of a few GB or more - the
> >> entire
> >> system grinds to a halt for 5+ minutes per GB. Even the console is
> >> unresponsive.
> >
> > One usually disables write-caching with RAID.  It makes it much less
> > likely that you'll end up with an inconsistent RAID volume if
> > there's a
> > power-cut.
> >
> > That said, the little caches on the drives won't help you much,
> > anyway.
> > Depending on what technique is used for the copying, you're running
> > into
> > a problem with some implementations of the block caching, readahead
> > and/or scheduler systems.  Basically, it tries to read as many files
> > as it can into RAM, then swaps madly, while writing files out, while
> > it's trying to read more, and trips over its own feet.  I don't know
> > what kernel config ad optimizations you're using in Gentoo, but poor
> > choices there could make it worse.
> >
> > The cp command, I understand, does things in little chunks to prevent
> > this.
> >
> > _______________________________________________
> > PLUG mailing list
> > PLUG at lists.pdxlinux.org
> > http://lists.pdxlinux.org/mailman/listinfo/plug
>
> Thanks for the reply. The server is UPS-backed and the RAID controller
> has it's own battery backup unit. I don't do anything special with the
> kernel, I just like Gentoo's package management. I've tried the copy
> with both cp and rsync, in both cases the servers console becomes
> unusable. I've never seen this before - I have several systems running
> with 3ware raid controllers that respond just fine under heavy disk
> utilization. The kernel also doesn't say write caching is disabled on
> those boxes.
>
> I think what's happening is the kernel isn't buffering writes to the
> RAID controller - irrespective of whether the RAID controller is
> disabling it's own write cache or the caches on each disk.
>
> -Aaron
> _______________________________________________
> PLUG mailing list
> PLUG at lists.pdxlinux.org
> http://lists.pdxlinux.org/mailman/listinfo/plug
>

Install the LSI Linux raid management software, and you can then view/set
the options your kernel is using.
http://www.lsi.com/support/downloads/


-- 
Brent Jones
brent at servuhome.net



More information about the PLUG mailing list