login
Header Space

 
 

Re: Xen paravirt frontend block hang

Previous thread: 'git gc' always run, on old kernel repo? by Jeff Garzik on Thursday, February 28, 2008 - 3:57 pm. (6 messages)

Next thread: Re: Xen paravirt frontend block hang by Jeremy Fitzhardinge on Thursday, February 28, 2008 - 4:03 pm. (1 message)
To: Christopher S. Aker <caker@...>
Cc: <virtualization@...>, Linux Kernel Mailing List <linux-kernel@...>, Xen-devel <xen-devel@...>
Date: Thursday, February 28, 2008 - 4:00 pm

I've been running this all night without seeing any problem.  I'm using 
current x86.git#testing with a few local patches, but nothing especially 
relevent-looking.

Could you try the attached patch to see if it makes any difference?
To: Jeremy Fitzhardinge <jeremy@...>
Cc: <virtualization@...>, Linux Kernel Mailing List <linux-kernel@...>, Xen-devel <xen-devel@...>
Date: Saturday, March 1, 2008 - 8:43 pm

Meh .. what backend are you using?  We're using LVM volumes exported 
directly into the domUs like so:


Unfortunately we're still in the same place... pv_ops kernels are still 
hanging after heavy disk IO:

works - 2.6.18.x (from xen-unstable)
hangs - 2.6.25-rc3-git3
hangs - 2.6.25-rc3-git3 + your patch

Any other suggestions or debugging I can provide that would be useful to 
squash this?

-Chris

--
To: Christopher S. Aker <caker@...>
Cc: <virtualization@...>, Linux Kernel Mailing List <linux-kernel@...>, Xen-devel <xen-devel@...>
Date: Sunday, March 2, 2008 - 11:35 am

Are you running an SMP or UP domain?  I found I could get hangs very 
easily with UP (but I need confirm it isn't a result of some other very 
experimental patches).

    J
--
To: Jeremy Fitzhardinge <jeremy@...>
Cc: <virtualization@...>, Linux Kernel Mailing List <linux-kernel@...>, Xen-devel <xen-devel@...>
Date: Sunday, March 2, 2008 - 12:03 pm

The hang occurs with both SMP and UP compiled pv_ops kernels.  SMP 
kernels are still slightly responsive after the hang occurs, which makes 
me think only one proc gets stuck at a time, not the entire kernel.

-Chris

--
To: Christopher S. Aker <caker@...>
Cc: Xen-devel <xen-devel@...>, Linux Kernel Mailing List <linux-kernel@...>, <virtualization@...>, xming <xmingske@...>
Date: Tuesday, March 18, 2008 - 12:01 pm

The patch I posted yesterday - "xen: fix RMW when unmasking events" - 
should definitively fix the hanging-under-load bugs (I hope).  It 
problem came from returning to userspace with pending events, which 
would leave them hanging around on the vcpu unprocessed, and eventually 
everything would deadlock.  This was caused by using an unlocked 
read-modify-write operation on the event pending flag - which can be set 
by another (real) cpu - meaning that the pending event wasn't noticed 
until too late.  It would only be a problem on an SMP host.

The patch should back-apply to 2.6.24.

    J
--
To: Jeremy Fitzhardinge <jeremy@...>
Cc: Xen-devel <xen-devel@...>, Linux Kernel Mailing List <linux-kernel@...>, <virtualization@...>, xming <xmingske@...>
Date: Monday, March 24, 2008 - 9:37 pm

Confirmed-by: caker@theshore.net

Nice work!

-Chris

--
Previous thread: 'git gc' always run, on old kernel repo? by Jeff Garzik on Thursday, February 28, 2008 - 3:57 pm. (6 messages)

Next thread: Re: Xen paravirt frontend block hang by Jeremy Fitzhardinge on Thursday, February 28, 2008 - 4:03 pm. (1 message)
speck-geostationary