[Top] [All Lists]

Dealing with a very large file -1,5TB

To: linux-xfs@xxxxxxxxxxx
Subject: Dealing with a very large file -1,5TB
From: Christoph Klocker <cklocker@xxxxxx>
Date: Thu, 25 Sep 2003 14:43:00 +0200
Sender: linux-xfs-bounce@xxxxxxxxxxx
User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5b) Gecko/20030723 Thunderbird/0.1

I am developing a system to stream very large uncompressed videofiles which are about 1,5TB.
I need a bandwith of 195MB/s.
my system is now:
kernel: 2.4.18
xeon 2,4GHz
3ware escalade controller 12 s-ata harddisks, seagate barracuda 7200.7 - 120GB - SCSI

when I do different tests with bonnie++ I get very good results for a 2GB file, readperformance 249mb/s,
but when the files get larger the speed is going down significantly.
at 10GB - 216MB/s
at 100GB - 187MB/s
at 1TB - 161MB/s
the write performance stays at 183MB/s any time, up to 1TB.
As my harddisk have a sustained rate of 32-50 MB/s it should not be the case that they influence the results. I am now thinking maybe the filesystem influences the result when the disks are getting fuller. I thought also about that the logfiles get bigger and bigger and that maybe a reason.

I had a look if its possible to switch of the logging completely (there is no reason to have it because I just have one file) , but I think its not possible with xfs.

Is there a special parameter configuration for having just one file at this size (1,5TB)


Klocker Christoph
10044 Stockholm

<Prev in Thread] Current Thread [Next in Thread>