On Thu, Sep 25, 2008 at 06:43:43PM +1000, Peter Leckie wrote:
Well I added some tracing code to the __wake_up_common, however it never
which made me think "are we even being woken up from the wait queue", or
directly waking us up from the task struct. So I had a look and found
Still, don't check it in until we understand whether sv_t's are
completely broken or not...
mp->m_ail.xa_target = threshold_lsn;
Which is indirectly called from xlog_grant_push_ail, which is called
from various other
Ok, so that one will only wake up the xfsaild, which does not flush
pinned items - it will never end up in an unpin wait for any type
of item, so we can rule that one out.
In fact this bug is not restricted to the aild the xfssyncd also hit
this issue a number of times
during todays testing where it was woken while waiting on sv_wait for
the pincount to drop
Ok, so there is the fundamental issue. This one is problematic
because xfssyncd calls into xfs_sync() -> xfs_qm_sync(). It does
so with the flag SYNC_BDFLUSH set, which means:
1014 * We won't block unless we are asked to.
1016 nowait = (boolean_t)(flags & SYNC_BDFLUSH || (flags & SYNC_WAIT)
We should not be blocking when flushing dquots. IOWs, we should not
be waiting on pinned quots in xfs_qm_sync() when it calls
xfs_dqflush(). i.e. it should behave exactly like the inode flush
i.e. the reason why we are seeing this is that xfs_dqflush is not
obeying the non-blocking semantics of the sync that it is being
asked to run. If we enter xfs_sync() from anywhere else, then we
won't have task wakeups occurring to interrupt a pin wait on a
It also is woken up from a number of functions in xfs_super.c including
xfs_syncd_queue_work(), xfs_sync_worker(), xfs_fs_sync_super()
Yeah, when different work needs doing.
The change that introduced the wake_up on the aild was introduced from
Move AIL pushing into it's own thread
However xfssyncd has had a long history of the task being woken up from
so it looks like it's simply not safe for either the aild or xfssyncd to
sleep on a queue assuming that
no one else will wake the processes up.
Given that both xfsaild and xfssyncd are supposed to be doing
non-blocking flushes, neither of them should ever be waiting on a
pinned item, therefore fixing that problem in xfs_qm_dqflush()
should make this problem go away. It will also substantially
reduce tehnumber of log forces being triggered by dquot writeback
which will have positive impact on performance, too.
So I would say the fix I proposed is a good solution for this issue.
but it doesn't fix the underlying problem that was causing the
spurious wakeups, which is the fact that xfs_qm_dqflush() is not
obeying non-blocking flush directions.
The patch below should fix
that. Can you please test it before you add your patch?
However there are other functions that use sv_wait and should also be
fixed in a similar way so I'll
look into the other callers and prepare a patch tomorrow.
The log force and write sv_t's are already in loops that would catch
spurious wakeups, so I don't think there's a problem there....