[Top] [All Lists]

DMAPI questions...

To: linux-xfs@xxxxxxxxxxx
Subject: DMAPI questions...
From: Josh Helmer <joshhelmer@xxxxxxx>
Date: Wed, 29 Jan 2003 13:03:39 -0700
Sender: linux-xfs-bounce@xxxxxxxxxxx
User-agent: KMail/1.4.3
Hey folks...

Perhaps this is not the appropriate place to ask this... If not, please 
forgive me, but this looks like just about the best place to get the feedback 
I am looking for.

I am in the early design stages of some backup/HSM software.   At this point I 
am just researching my options and I was hoping someone could help me out 
with some questions I have about DMAPI in general (since XFS seems to be the 
only place where anyone is working in this area under linux  - at least since 
the OpenXDSM project died) and about the XFS implementation of DMAPI.

One of our requirements is near-real-time backup.  Basically what we need to 
be able to do is duplicate changes to secondary storage as soon as possible, 
and then after a file has aged we will go back and poke holes as necessary.   
When a file is accessed we would fill in the holes again.   Unfortunately our 
secondary storage is SIGNIFICANTLY slower than a hard drive (writes at 
between 1.0 and 1.5Mb/s).  Rather than trying to duplicate every write, we 
were thinking about just adding files to a backup queue when the file is 
closed.  Unfortunately, the easist way that I can see for handling this is to 
handle the debut event (initialize the event list for a file) and the close 
event(schedule the file for backup).  We understand that this method will not 
ensure EVERY change to a file is backed up, but for our application, files 
will not change often enough to make this a big concern.

Of course, this will only work if XFS adds support for the debut and close 
events.  Are there any plans to implement these events in XFS?  If so, is 
there an estimate on when this will be available?  If not, is there any other 
way of handling this that I have not considered yet.

I have also considered creating a separate write queue.  In this 
implementation I would basically try to consolidate writes (a non-trivial 
operation with questionable advantages - may just forget about trying that) 
and then after no write events have appeared after a pre-determined amount of 
time, flush all writes to the secondary storage.  As I stated earlier I am 
still in the VERY early stages of design and I have probably overlooked some 
issues (locking and file consistency... ugh...  LOTS of work to do there...) 
with this approach too.  Anyhow, any input would be appreciated.

Maybe I would be better off just going with the standard NFS-server solution?


<Prev in Thread] Current Thread [Next in Thread>