>Received: from mobile.sauter-bc.com (unknown [10.1.6.21])
	by basel1.sauter-bc.com (Postfix) with ESMTP
	id C621657306; Thu, 19 Jul 2001 08:37:28 +0200 (CEST)
Received: from ch.sauter-bc.com (support.cad.sba [10.1.200.117])
	by mobile.sauter-bc.com (Postfix) with ESMTP
	id 3DB7C25835; Thu, 19 Jul 2001 08:45:21 +0200 (CEST)
Message-ID: <3B567DE2.90F09F41@ch.sauter-bc.com>
Date: Thu, 19 Jul 2001 08:27:46 +0200
From: Simon Matter <simon.matter@ch.sauter-bc.com>
Organization: Sauter AG, Basel
X-Mailer: Mozilla 4.77 [de] (X11; U; Linux 2.2.19-6.2.7 i686)
X-Accept-Language: de-CH, en
MIME-Version: 1.0
To: Tru Huynh <tru@pasteur.fr>
Cc: "linux-xfs@oss.sgi.com" <linux-xfs@oss.sgi.com>
Subject: Re: nfs/local performance with software raid5 and xfs and SMP
References: <3B55E49E.5B4B6CEF@pasteur.fr>
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=us-ascii
Sender: owner-linux-xfs@oss.sgi.com
Precedence: bulk

Tru Huynh schrieb:
> 
> Hello,
> 
> I would like to add to the xfs performance with sw raid discussion.
> I encounter the same kind of degraded performance with the
> following hardware/software combination.
> Although the degradation can be feel with the local access too,
> but there are no errors messages :(
> 
> hardware config
> 
> The nfs client is a plain redhat 7.1
> hardware: K7-1.2GHz 2x256PC133 Abit KT7A motherboard
> with 2 3com 3c905C NICs (one to an isolated HP procurve
> switch and the other to a Cisco for the main campus network)
> 
> The server is a dual P3-866MHz with 4x256PC133
> (rocky 3732 mb - chipset Via VT82C694X/VT82C686B)
> with 1 ide-scsi controller 3ware 3w6800 with 8 IBM-DTLA-307075 72G HDs

Could you try RAID5/XFS with the onboard IDE controller and SMP Kernel?
This way we could find out whether it's the 3ware / SMP / RAID5
combination that makes the problem. I'm running several servers here
with RAID5 on IDE and SCSI with good results but all of them are
single CPU systems.

> 20G system boot disk redhat 7.1+SGI xfs 1.0.1
> 3com 3c905C NICs to an isolated HP procurve switch
> 3com 3c905C NICs to a Cisco for the main campus network

3c905C NIC? I was having performance problems with them while the
3c905B were doing very fine. 

> 
> I tried with both kernel smp 2.4.3-2 from xfs-1.0.1 and 2.4.5 with xfs
> 
> /dev/hda6   is a scratch partition on the system disk (ext2/xfs)
> /dev/mdo is the software raid device (450GB with xfs)
> 
> when copying large directory fron the client to the server
> on the raid5 partition the following error is recorded in
> the syslog file (on either network HP/Cisco):
> Jul 18 20:19:44 jason.bis.pasteur.fr kernel: nfs: server nfs.cluster not
> responding, still trying
> Jul 18 20:20:08 jason.bis.pasteur.fr last message repeated 2 times
> Jul 18 20:20:08 jason.bis.pasteur.fr kernel: nfs: server nfs.cluster OK
> 
> 1) When doing the very same copy to the nfs exported scratch partition
> either ext2 or xfs the copy is really fast and no error message are
> found.
> 
> 2) Doing the same experiments with the UP kernel 2.4.5-xfs /2.4.3-xfs
> give no errors (xfs on sw raid5 and (xfs,ext2) on scracth partition)
> 
> 3) with or without noapic does not help either...
> 
> conclusions:
> ============
> 
> smp+swraid5+xfs : not ok
>  up+swraid5+xfs : ok
> 
> I will try the cvs version tomorrow
> 
> Thank you for listening.
> 
> Tru
> 
> details:
> --------
> /etc/raidtab file:
> -----------------
> # Sample raid-5 configuration
> #
> raiddev                 /dev/md0
> raid-level              5
> nr-raid-disks           7
> # Spare disks for hot reconstruction
> nr-spare-disks          1
> persistent-superblock   1
> 
> chunk-size              64
> 
> # the best one for maximum performance:
> #
> parity-algorithm        left-symmetric
> 
> #parity-algorithm       right-asymmetric
> #parity-algorithm       right-symmetric
> 
> 
> device                  /dev/sdb1
> raid-disk               0
> device                  /dev/sdc1
> raid-disk               1
> device                  /dev/sdd1
> raid-disk               2
> device                  /dev/sde1
> raid-disk               3
> device                  /dev/sdf1
> raid-disk               4
> device                  /dev/sdg1
> raid-disk               5
> device                  /dev/sdh1
> raid-disk               6
> #
> device                  /dev/sda1
> spare-disk              0
> 
> UP-2.4.5-xfs (rpm --rebuild --target=i686
> kernel-2.4.5-SGI_XFS_1.0.1.src.rpm)
> [tru@sheridan ~]$ uname -a
> Linux sheridan.bis.pasteur.fr 2.4.5-SGI_XFS_1.0.1 #1 Wed Jul 18 15:03:15
> CEST 2001 i686 unknown
> [tru@sheridan ~]$ cat /proc/cmdline
> auto BOOT_IMAGE=1-245xfs-101 ro root=301
> BOOT_FILE=/boot/vmlinuz-2.4.5-SGI_XFS_1.0.1 ramdisk_size=2500
> devfs=nomount
> 
> SMP-2.4.5-xfs (rpm --rebuild --target=i686
> kernel-2.4.5-SGI_XFS_1.0.1.src.rpm)
> [tru@sheridan ~]$ uname -a
> Linux sheridan.bis.pasteur.fr 2.4.5-SGI_XFS_1.0.1smp #1 SMP Wed Jul 18
> 14:50:17
> [tru@sheridan tru]$ cat /proc/cmdline
> auto BOOT_IMAGE=1-xfs-101smp ro root=301
> BOOT_FILE=/boot/vmlinuz-2.4.3-SGI_XFS_1.0.1smp ramdisk_size=2500
> devfs=nomount
> 
> UP-2.4.3-2 from the 1.0.2 iso
> [tru@sheridan ~]$ uname -a
> Linux sheridan.bis.pasteur.fr 2.4.3-SGI_XFS_1.0.1 #1 Mon Jul 9 14:27:56
> CDT 2001 i686 unknown
> [tru@sheridan ~]$ cat /proc/cmdline
> auto BOOT_IMAGE=1-xfs-101 ro root=301
> BOOT_FILE=/boot/vmlinuz-2.4.3-SGI_XFS_1.0.1 ramdisk_size=2500
> devfs=nomount
> [tru@sheridan ~]$ cat /proc/version
> Linux version 2.4.3-SGI_XFS_1.0.1 (root@linux-xfs2.americas.sgi.com)
> (gcc version egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)) #1 Mon
> Jul 9 14:27:56 CDT 2001
> 
> SMP-2.4.3-2 from the 1.0.2 iso
> [tru@sheridan ~]$ uname -a
> Linux sheridan.bis.pasteur.fr 2.4.3-SGI_XFS_1.0.1smp #1 SMP Mon Jul 9
> 14:03:40 CDT 2001 i686 unknown
> [tru@sheridan ~]$ cat /proc/cmdline
> auto BOOT_IMAGE=1-xfs-101smp ro root=301
> BOOT_FILE=/boot/vmlinuz-2.4.3-SGI_XFS_1.0.1smp ramdisk_size=2500
> devfs=nomount
> --
> Dr Tru Huynh          | Bioinformatique Structurale
> mailto:tru@pasteur.fr | tel/fax +33 1 45 68 87 37/19
> Institut Pasteur, 25-28 rue du Docteur Roux, 75724 Paris CEDEX 15 France

-- 
Simon Matter              Tel:  +41 61 695 57 35
Fr.Sauter AG / CIT        Fax:  +41 61 695 53 30
Im Surinam 55
CH-4016 Basel             [mailto:simon.matter@ch.sauter-bc.com]