I went thru the same hardware config gymnastics (confounded by bad disks, a
bad controller, bad hotswap cages, and some SW config issues - see:
The 3ware info is partially reported here:
and I believe I also sent a similar report to this list, but google doesn't
Here's the results of my bonnie tests on a similar system:
8x250GB WD SD series disks on a 3ware 9500S-8, on an IWILL 2x Opteron mobo,
4GB RAM, with XFS.
The XFS parameters are with a 64K stripe to match the RAID card; other params
more or less vanilla.
Below are some bonnie timing results with differing filesystems (1 run with
ext3, 3 with XFS with differnt file sizes - unwrap in an editor to compare in
+++, 3248, 16,1718,15,+++++,+++,750, 4
ext3 sand, 7000M,40682,91, 47732,23,25432,9,38027,72,179352,27,311.1,0,80,
+++, 3753, 17,1913,17,+++++,+++,449, 2
+++++,+++, 4565, 22,1700,16,+++++,+++,793, 4
From my reading (see URL above for resourcelist), XFS is quite bad for tiny
files - we use it for very large files (>GB size); using XFS for this would
generally be a bad thing. We do NOT get tremendous performance out of it;
but the performance is much better than with ext3 and the CPU usage is lower,
sometimes dramtically so. Real life experience with some benchmarks confirms
this - we get approximately ~ the same real life thruput as we do on a IBM
SP2 8way module with a direct attach disk.
We are now considering adding a local PVFS2 system to a small cluster for very
fast IO under MPI
On Monday 25 July 2005 18:11, Gaspar Bakos wrote:
> Dear all,
> The purpose of this email is twofold:
> - to share the results of the many tests I performed with
> a 3ware RAID card + RAID-5 + XFS, pushing for better file I/O,
> - and to initiate some brainstorming on what parameters can be tuned for
> getting a good performance out of this hardware under 2.6.* kernels.
> I started all these tests because the performance was quite poor, meaning
> that the write speed was slow, the read speed was barely acceptable, and
> the system load went very high (10.0) during bonnie++ tests.
> My questions are marked below with "Q".