Subject: Re: more experiments with realtime subvolumes.
From: Eric Sandeen <sandeen@sgi.com>
To: Ramesh K <kram@astron.nl>
Cc: linux-xfs@oss.sgi.com
In-Reply-To: <Pine.LNX.4.44.0402131420170.8002-100000@wop10.nfra.nl>
References: <Pine.LNX.4.44.0402131420170.8002-100000@wop10.nfra.nl>
Content-Type: text/plain
Organization: Eric Conspiracy Secret Labs
Message-Id: <1076693494.10710.44.camel@stout.americas.sgi.com>
Mime-Version: 1.0
Date: 13 Feb 2004 11:31:35 -0600
Content-Transfer-Encoding: 7bit
Sender: linux-xfs-bounce@oss.sgi.com
Errors-to: linux-xfs-bounce@oss.sgi.com
Precedence: bulk

Oops, sorry, ok so you've abandoned the ramdisk idea, missed that.  :)

On Fri, 2004-02-13 at 08:48, Ramesh K wrote:
> Hello:
> 	Some more experiements with the realtime subvolumes. Please do pass your
> comments. Before we start, my system (OS - SuSE 8.1, kernel 2.4.19-64GB-SMP,
> hardware: Transtec, dual-xeon 2.8 GHz, with 3ware Raid Controller - 8*120GB IDE
> disks on controller), doesn't seem to have O_DIRECT and O_LARGEFILE options. I'm
> downloading a newer kernel (2.4.21) from suse website - hopefully that has these
> options enabled.
> 	My harddisk is setup as explained in my previous mail, 100 GB
> meta-data section, and 800 GB realtime section. More questions on realtime
> subvolume:
> 	When I open a file and try to place it in the realtime section with:
> 
> struct fsxattr myfsxattr;
> struct xfs_flock64 file;
> 
> fd = open("/data/kram/data/file1.dat",O_CREAT|O_RDWR);
> myfsxattr.fsx_xflags =  XFS_XFLAG_REALTIME;
> ioctl(fd,XFS_IOC_FSSETXATTR,&myfsxattr);
> file.l_whence = 0;
> file.l_start = 0;
> file.l_len=2*1024*1024;
> ioctl(fd,XFS_IOC_RESVSP,&file);
> close(fd);
> 
> 	And then followed it up with, ioctl(fd,XFS_IOC_FSGETXATTR,&myfsxattr);
> This call returns fsx_xflags as 0x00 (meaning no realtime, and not
> preallocated). Is it because of the absence of O_DIRECT in the open call?

For starters, your file cannot have had any non-realtime I/O to it
before you try to set it as REALTIME.  Is this the case?

I do something like this:

        if (ioctl(ofd, XFS_IOC_FSGETXATTR, &fsxinfo) == -1) {
                perror("fsgetxattr failed");
        } else {
                fsxinfo.fsx_xflags = XFS_XFLAG_REALTIME;
                if (ioctl(ofd, XFS_IOC_FSSETXATTR, &fsxinfo) == -1) {
                        perror("setxattr ioctl failed");
                }
        }

Without digging around too much - I don't remember offhand - I'd suggest
opening the file O_DIRECT as well, yes.

> What are the alignment constraints? The man page of xfsctl explains that
> "d_mem is the memory alignment requirement of the user's data buffer.
> d_miniosz specifies block size, minimum I/O request size, and I/O alignment.The
> size of all I/O requests must be a multiple of this amount and the value of the
> seek pointer at the time of the I/O request must also be an integer multiple of
> this amount. d_maxiosz is the maximum I/O request size which can be performed on
> the file descriptor." Does this mean a write of "char array[1024*1024]" is
> aligned with 512-byte boundary for the I/O to succeed? Or in other words, is
> array[1024*1024] d_mem aligned, or do I have to do something special to get it
> aligned?

While I do think that xfsctl should return proper alignment info, it
really should be just the same as normal O_DIRECT requirements on Linux,
I think.  i.e. page-aligned, multiple of page-sized.  You can use
memalign to get aligned memory.

See http://oss.sgi.com/~sandeen/odirect-rt.c for an example of something
which can do O_DIRECT / Realtime I/O - note I didn't write most of it
but hacked in some realtime goodness for testing.  It worked last time I
tried it, I think.  :)

-Eric

-- 
Eric Sandeen      [C]XFS for Linux   http://oss.sgi.com/projects/xfs
sandeen@sgi.com   SGI, Inc.          651-683-3102