Date: Sat, 10 May 2003 06:30:46 +0200
From: Andi Kleen <ak@suse.de>
To: Rick Smith <rgsmith72@hotmail.com>
Cc: linux-xfs@oss.sgi.com
Subject: Re: O_DIRECT address restrictions
Message-ID: <20030510043046.GA4250@Wotan.suse.de>
References: <Sea2-F34ZXSI4d5Xr0N0000751d@hotmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Sea2-F34ZXSI4d5Xr0N0000751d@hotmail.com>
Sender: linux-xfs-bounce@oss.sgi.com
Errors-to: linux-xfs-bounce@oss.sgi.com
Precedence: bulk
Content-Length: 1117
Lines: 23

On Fri, May 09, 2003 at 09:11:30PM -0700, Rick Smith wrote:
> It turns out that my problem is not specifically with XFS, but with any 
> filesystem that supports O_DIRECT. Is it possible to perform direct I/O to 
> addresses 0xF0000000 and above that are mmaped into user space? I am able 
> to write to this memory with normal I/O, so how does this differ from 
> O_DIRECT in this respect?

I don't think that will work. The direct IO path works with struct page 
structures in the kernel mem_map array, but these hardware mappings don't have
an entry in mem_map and no struct page.

In theory you could hack your kernel to include them in the mem_map, but
it would need some surgery in early bootup and could potentially need a 
lot of memory. The mem_map array does not allow holes in a non NUMA kernel 
so if you try to map 0xF0000000 you'll always need an mem_map that covers 
the full 4GB of low memory. All these struct pages add up and cost quite 
a lot of memory.

It will likely work when you use an machine with more than 4GB of memory
installed.  It already has mem_map entries for the PCI holes.

-Andi