MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Message-ID: <17264.52374.316530.749268@base.ty.sabi.co.UK>
Date: Tue, 8 Nov 2005 16:04:38 +0000
Precedence: air-mail
To: linux-xfs@oss.sgi.com
Subject: Re: XFS unstable with little memory; OOPS in prune_dcache()
In-Reply-To: <200511081425.06695.philipp.reisner@linbit.com>
References: <200511081425.06695.philipp.reisner@linbit.com>
From: pg_xfs@xfs.for.sabi.co.UK (Peter Grandi)
Content-Transfer-Encoding: 8bit
Sender: linux-xfs-bounce@oss.sgi.com
Errors-to: linux-xfs-bounce@oss.sgi.com
Precedence: bulk
Content-Length: 2178
Lines: 59

>>> On Tue, 8 Nov 2005 14:25:06 +0100, Philipp Reisner
>>> <philipp.reisner@linbit.com> said:

philipp.reisner> Hi XFS gurus, I experienced a series of kernel
philipp.reisner> crashes, that are triggered by:

philipp.reisner> o XFS ( Kernel vanilla 2.6.13.4 )
philipp.reisner> o uniprocessor (compiled for K7) / no HIGHMEM
philipp.reisner> o little memory ( 256 MB Ram )
philipp.reisner> o 200GB XFS filesystem, 138GB used
philipp.reisner> o   4378907 inodes   ( 4 m)
philipp.reisner> o  30349548 dentries (30 m)

The crashes are not a big deal, as they don't happen with more
RAM as you say.

What would worry me about having 2656MB is the ability and time
taken to check that filesystem with so many inodes and directory
entries might, or at last could take a _long_ while, even if
perhaps not the 75 days reported for a large 'ext3' filesystem
here:

  http://UKAI.org/b/log/debian/snapshot/1_month_fsck-2005-07-22-00-00.html
  http://UKAI.org/b/log/debian/snapshot/fsck_completed_but-2005-09-04-15-00.html

Couple of threads on RAM (and address space) usage:

  http://OSS.SGI.com/archives/linux-xfs/2005-09/msg00101.html
  http://OSS.SGI.com/archives/linux-xfs/2005-08/msg00031.html

for example, as to checking:

  http://OSS.SGI.com/archives/linux-xfs/2005-08/msg00045.html

   ŤFrom some quick tests I just ran, for 32bit binaries xfs_check
    needs around 1GiB RAM per TiB of filesystem plus about 100MiB
    RAM per 1million inodes in the filesystem (more if you have
    lots of fragmented files).ť

More generally, the rule for most sw, and freee sw too, is ''it
works for me'', where ''me'' is the developer or the employer of
the developer.

So if your system is very different from those used by the
developers, bad luck. Right now most Linux kernel/fs developer
are employed by large corporates and it is easy to imagine that
they have >2GB of memory installed.

Also, XFS is designed/targeted to handle very large filesystems
on very large computers. On the scale of systems for which XFS
was designed, a 256MB PC is an embedded system.

[ ... ]

philipp.reisner> [ See rsbak3 at http://oss.linbit.com/ ]

or http://WWW.rsnapshot.org/ which seems similar...