Search String: Display: Description: Sort:

Results:

References: [ +subject:/^(?:^\s*(re|sv|fwd|fw)[\[\]\d]*[:>-]+\s*)*Still\s+seeing\s+hangs\s+in\s+xlog_grant_log_space\s*$/: 57 ]

Total 57 documents matching your query.

1. Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Juerg Haefliger <juergh@xxxxxxxxx>
Date: Mon, 23 Apr 2012 14:09:53 +0200
Hi, I have a test system that I'm using to try to force an XFS filesystem hang since we're encountering that problem sporadically in production running a 2.6.38-8 Natty kernel. The original idea was
/archives/xfs/2012-04/msg00951.html (16,011 bytes)

2. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Dave Chinner <david@xxxxxxxxxxxxx>
Date: Tue, 24 Apr 2012 00:38:43 +1000
If you kill the workload, does the file system recover normally? What you see is not necessarily a hang. It may just be that you've caused your IO subsystem to have so much IO queued up it's complete
/archives/xfs/2012-04/msg00952.html (10,079 bytes)

3. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Juerg Haefliger <juergh@xxxxxxxxx>
Date: Mon, 23 Apr 2012 17:33:40 +0200
Hi Dave, The workload can't be killed. When it hangs, there are zero IOs going to the disk. The machine has 100GB of RAM. experienced hangs with that configuration so I gave it a shot. 4 2TB SAS 6Gb
/archives/xfs/2012-04/msg00953.html (12,724 bytes)

4. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Dave Chinner <david@xxxxxxxxxxxxx>
Date: Tue, 24 Apr 2012 09:58:40 +1000
OK. Can you get an event trace across the period where the hang occurs? .... Wonderful. Allocation groups. Ok, that's fair enough - it's handy to tell us that up front, though. ;) Alright, then I nee
/archives/xfs/2012-04/msg00965.html (12,035 bytes)

5. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Juerg Haefliger <juergh@xxxxxxxxx>
Date: Tue, 24 Apr 2012 10:55:22 +0200
Ah sorry for not being clear enough. I thought my intentions could be deduced from the information that I provided :-) Very good. Will do. What kernel do you want me to run? I would prefer our curren
/archives/xfs/2012-04/msg00978.html (14,013 bytes)

6. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Dave Chinner <david@xxxxxxxxxxxxx>
Date: Tue, 24 Apr 2012 22:07:31 +1000
If you can reproduce it on a current kernel - 3.4-rc4 if possible, if not a 3.3.x stable kernel would be best. 2.6.38 is simply too old to be useful for debugging these sorts of problems... Cheers, D
/archives/xfs/2012-04/msg00979.html (14,669 bytes)

7. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Juerg Haefliger <juergh@xxxxxxxxx>
Date: Tue, 24 Apr 2012 20:26:04 +0200
OK, I reproduced a hang running 3.4-rc4. The data is here but it's a whopping 2GB (yes it's compressed): https://region-a.geo-1.objects.hpcloudsvc.com:443/v1.0/AUTH_9630ead2-6194-40df-afd3-7395448d45
/archives/xfs/2012-04/msg00986.html (16,667 bytes)

8. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Dave Chinner <david@xxxxxxxxxxxxx>
Date: Thu, 26 Apr 2012 08:38:45 +1000
That's a bit big to be useful, and far bigger than I'm willing to download given that I'm on the end of a wet piece of string, not a big fat intarwebby pipe. I'm assuming it is the event trace that i
/archives/xfs/2012-04/msg01009.html (11,529 bytes)

9. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Juerg Haefliger <juergh@xxxxxxxxx>
Date: Thu, 26 Apr 2012 14:37:50 +0200
Fair enough. Can I shorten the existing trace.dat? I stopped the trace automatically 10 secs after the the xlog_... trace showed up in syslog so effectively some 130+ secs after the hang occured. ...
/archives/xfs/2012-04/msg01016.html (13,039 bytes)

10. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Dave Chinner <david@xxxxxxxxxxxxx>
Date: Fri, 27 Apr 2012 08:44:12 +1000
No idea, but that's likely the problem - I don't want the binary trace.dat file. I want the text output of the report command generated from the binary trace.dat file... Extract the text report from
/archives/xfs/2012-04/msg01020.html (14,164 bytes)

11. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Juerg Haefliger <juergh@xxxxxxxxx>
Date: Fri, 27 Apr 2012 01:00:08 +0200
Well yes. I did RTFM :-) trace.dat is 15GB. Compressed trace_report.txt is 2GB. Sorry, haven't had the time today to look into this. I'll cut the size down somehow. ...Juerg
/archives/xfs/2012-04/msg01021.html (15,860 bytes)

12. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Dave Chinner <david@xxxxxxxxxxxxx>
Date: Fri, 27 Apr 2012 09:07:38 +1000
OK, that's a lot larger than I expected for a hung filesystem.... Can you look at the last timestamp in the report file, and trim off anything from the start that is older than, say, 180s before that
/archives/xfs/2012-04/msg01022.html (14,217 bytes)

13. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Juerg Haefliger <juergh@xxxxxxxxx>
Date: Fri, 27 Apr 2012 11:04:33 +0200
Cut the trace down to 180 secs which brought the filesize down to 93MB: https://region-a.geo-1.objects.hpcloudsvc.com:443/v1.0/AUTH_9630ead2-6194-40df-afd3-7395448d4536/xfs-hang/report-2012-04-24-180
/archives/xfs/2012-04/msg01023.html (16,523 bytes)

14. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Dave Chinner <david@xxxxxxxxxxxxx>
Date: Fri, 27 Apr 2012 21:09:22 +1000
Looks like you can - the "trace-cmd split" option. I see the problem - the trace.dat file is hosted on an XFS filesystem, so all the writes to the trace.dat file are causing events to be logged, whic
/archives/xfs/2012-04/msg01030.html (15,764 bytes)

15. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Juerg Haefliger <juergh@xxxxxxxxx>
Date: Fri, 27 Apr 2012 15:07:45 +0200
Third try: https://region-a.geo-1.objects.hpcloudsvc.com:443/v1.0/AUTH_9630ead2-6194-40df-afd3-7395448d4536/xfs-hang/report-2012-04-27-180secs.tgz Filtered by device, trace events go to a different f
/archives/xfs/2012-04/msg01031.html (18,745 bytes)

16. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Juerg Haefliger <juergh@xxxxxxxxx>
Date: Sat, 5 May 2012 09:44:35 +0200
Did anybody have a chance to look at the data? Thanks ...Juerg
/archives/xfs/2012-05/msg00041.html (17,579 bytes)

17. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Ben Myers <bpm@xxxxxxx>
Date: Mon, 7 May 2012 12:19:08 -0500
Hey Juerg, https://bugs.launchpad.net/ubuntu/+source/linux/+bug/979498 Here you indicate that you have created a reproducer. Can you post it to the list? Thanks, Ben
/archives/xfs/2012-05/msg00073.html (9,236 bytes)

18. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Dave Chinner <david@xxxxxxxxxxxxx>
Date: Tue, 8 May 2012 08:59:44 +1000
I've had a quick look, but I need to write scripts to visualise it (i.e. graph it) to determine if there's any pattern to the issue. Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx
/archives/xfs/2012-05/msg00083.html (10,718 bytes)

19. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Dave Chinner <david@xxxxxxxxxxxxx>
Date: Wed, 9 May 2012 17:35:07 +1000
And, as expected, something unexpected popped out. Judicious use of awk on the log space grant events shows an interesting pattern occuring from time to time: Transaction Wait queues Grant head Write
/archives/xfs/2012-05/msg00112.html (13,913 bytes)

20. Re: Still seeing hangs in xlog_grant_log_space (score: 1)
Author: Juerg Haefliger <juergh@xxxxxxxxx>
Date: Wed, 9 May 2012 09:54:08 +0200
Ben, Canonical attached them to the bug report that they filed yesterday: http://oss.sgi.com/bugzilla/show_bug.cgi?id=922 ...Juerg
/archives/xfs/2012-05/msg00115.html (11,115 bytes)


This search system is powered by Namazu