[Top] [All Lists]

Re: nfs/local performance with software raid5 and xfs and SMP

To: Tru Huynh <tru@xxxxxxxxxx>
Subject: Re: nfs/local performance with software raid5 and xfs and SMP
From: Seth Mos <knuffie@xxxxxxxxx>
Date: Wed, 18 Jul 2001 21:53:58 +0200 (CEST)
Cc: "linux-xfs@xxxxxxxxxxx" <linux-xfs@xxxxxxxxxxx>
In-reply-to: <3B55E49E.5B4B6CEF@xxxxxxxxxx>
Sender: owner-linux-xfs@xxxxxxxxxxx
On Wed, 18 Jul 2001, Tru Huynh wrote:

> Hello,
> I would like to add to the xfs performance with sw raid discussion.
> I encounter the same kind of degraded performance with the
> following hardware/software combination.
> Although the degradation can be feel with the local access too,
> but there are no errors messages :( 
> hardware config
> The nfs client is a plain redhat 7.1 
> hardware: K7-1.2GHz 2x256PC133 Abit KT7A motherboard
> with 2 3com 3c905C NICs (one to an isolated HP procurve 
> switch and the other to a Cisco for the main campus network)
> The server is a dual P3-866MHz with 4x256PC133
> (rocky 3732 mb - chipset Via VT82C694X/VT82C686B)
> with 1 ide-scsi controller 3ware 3w6800 with 8 IBM-DTLA-307075 72G HDs
> 20G system boot disk redhat 7.1+SGI xfs 1.0.1
> 3com 3c905C NICs to an isolated HP procurve switch
> 3com 3c905C NICs to a Cisco for the main campus network
> I tried with both kernel smp 2.4.3-2 from xfs-1.0.1 and 2.4.5 with xfs
> /dev/hda6   is a scratch partition on the system disk (ext2/xfs)
> /dev/mdo is the software raid device (450GB with xfs)
> when copying large directory fron the client to the server
> on the raid5 partition the following error is recorded in
> the syslog file (on either network HP/Cisco):
> Jul 18 20:19:44 jason.bis.pasteur.fr kernel: nfs: server nfs.cluster not
> responding, still trying
> Jul 18 20:20:08 jason.bis.pasteur.fr last message repeated 2 times
> Jul 18 20:20:08 jason.bis.pasteur.fr kernel: nfs: server nfs.cluster OK

This looks like packets getting lost. Do you see this one network in
particular? I have had this exact problem with a rotten network cable.
Yours might be fine but something is giving. It might be the driver for
your network card dropping packets or something on the switch.

I have used a 3c950b and a intel eepro 100 for respectively server and
client. Have you tried running a bonnie over NFS and see what througput
you are seeing? Does the NFS server become unavailable when running
multiple bonnies?

Something might be generating so much interrupts that it can't handle all
of them resulting in packet loss.

> 1) When doing the very same copy to the nfs exported scratch partition
> either ext2 or xfs the copy is really fast and no error message are
> found.

This could be that your 3ware controller is interfering with your system.
Did you try the latest cvs and firmware for your 3ware card?

> 2) Doing the same experiments with the UP kernel 2.4.5-xfs /2.4.3-xfs
> give no errors (xfs on sw raid5 and (xfs,ext2) on scracth partition)

The 3ware driver does not like SMP perhaps?

> 3) with or without noapic does not help either...

Maybe you have found a combination of factors which your hardware or the
linux kernel does not like.


<Prev in Thread] Current Thread [Next in Thread>