login
Search
Rushing in where angels fear to tread... I had been quite happily ignoring the suspend-blockers controversy. However, now that I have signed up for the Linaro project that involves embedded battery-powered devices, I find that ignorance is no longer bliss. I have therefore reviewed much of the suspend-blocker/wakelock material, but have not yet seen a clear exposition of the requirements that suspend blockers are supposed to meet. This email is a attempt to present the requirements, based on my interpretation of the LKML discussions. Please note that I am not proposing a solution that meets these requirements, nor am I attempting to judge the various proposed solutions. In fact, I am not even trying to judge whether the requirements are optimal, or even whether or not they make sense at all. My only goal at the moment is to improve my understanding of what the Android folks' requirements are. That said, I do include example mechanisms as needed to clarify the meaning of the requirements. This should not be interpreted as a preference for any given example mechanism. But first I am going to look at nomenclature, as it appears to me that at least some of the flamage was due to conflicting definitions. Following that, the requirements, nice-to-haves, apparent non-requirements, an example power-optimized applications, and finally a brief look at other applications. Donning the asbestos suit, the one with the tungsten pinstripes... Thanx, Paul ------------------------------------------------------------------------ DEFINITIONS o "Ill-behaved application" AKA "untrusted application" AKA "crappy application". The Android guys seem to be thinking in terms of applications that are well-designed and well-implemented in general, but which do not take power consumption or battery life into account. Examples include applications designed for AC-powered PCs. Many other people seemed to instead be thinking in terms of an ill-conceived or useless ...
On Sat, 31 Jul 2010 10:58:42 -0700 I don't like the term "Power aware application". An application is well AC power is not relevant in discussions around power: Applications MUST behave well, AC or DC both. Just ask any data center operator on how much they run on DC and if he cares about power consumption. Conversely, most mobile usages (both phone, tablet or netbook) are "DC I don't buy this argument as is. I do buy that there are many sloppy applications; mostly written quickly for <appstore of the month>. But most if not all of these are written for the device in question (at least that is true for Apple and Android) --
Applications are often complex enough to be aware of some things, naive about others, well behaved in some ways, and ill-behaved in others. This has been the case for some decades now, so it should not come as a surprise. I am of course open to suggestions for alternatives to the term "power aware application", but most definitely not to obfuscating the difference between power awareness (or whatever name one wishes to call it) and the overall quality of the application, whatever "quality" might mean Alan Stern suggested "external power" in place of "AC", which makes sense to me. His suggestion does have the virtue of rendering irrelevant the difference between an AC adapter and a DC adapter for use in automobiles. And in my experience, people working with mobile devices care -much- more Making an application power-aware or power-naive is a choice, similar to the choice between making an application SMP-aware or single-threaded. This sort of choice is completely orthogonal to ill-behavior, sloppiness, or whatever the pejorative of the day might be. The choice between power-aware and power-naive will depend on who is available to do the programming and how valuable power-awareness is for the application in question. Given that people who program PC-class applications are much more common than are people who program with energy efficiency in mind, the power-naive choice will be attractive in many cases. Or do you have some other interpretation of what the Android guys are looking for? Thanx, Paul --
On Sat, 31 Jul 2010 22:48:16 -0700 I do not like the term "aware". At all. It implies "awareness", and implies it does things based on the power circumstances. It's about *behaving well*. (not polling, not having activity when there is nothing to do etc etc). Not about being aware that power matters. I can write a very shitty application that polls all the time and otherwise keeps the CPU and system busy, but it'll be aware that power I'm not sure I buy that. 4 years ago.. yes. Today.. with PowerTOP and co out there for a long time? I don't believe that anymore. Most of our open source PC apps are actually very well behaving in the power sense. Yes an occasional bug slips in, and then gets fixed quickly. (speaking as someone who does this sort of work for a Linux distribution... yes bugs happen on a whole distro level, but they're also not all that common, and get fixed quickly) -- Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org --
I understand that you would prefer that we group applications into "good" and "bad" categories, but then again, I suspect that most of us understood that before you posted this message. Given your significant and well-appreciated contributions to power efficiency over the past several years, I must confess to be quite disappointed that you failed to do more than to simply restate your preference. However, your words below are much more to the point, so I will respond I agree that much progress has been made over the past four years. My laptop's battery life has increased substantially, with roughly half of the increase due to software changes. Taken over all the laptops and PCs out there, this indeed adds up to substantial and valuable savings. So, yes, you have done quite well. However, your reliance on application-by-application fixes, while good up to a point, is worrisome longer term. The reason for my concern is that computing history has not been kind to those who fail to vigorously automate. The Android guys have offered up one way of automating power efficiency. There might be better ways, but their approach does at the very least deserve a fair hearing -- and no one who read the recent LKML discussion of their approach could possibly mistake it for anything resembling a fair hearing. And yes, I do understand and appreciate your contributions in the form of things like timer aggregation, which does automate the otherwise-tricky task of coordinating power usage across unrelated applications, at least in some important cases. But power efficiency will likely require multiple approaches, especially if the Linux stack is to approach the power efficiencies provided by dedicated devices. Thanx, Paul --
Paul, I very much agree with what you stated later, with respect to doubting whether the whack-a-mole approach to application fixups is workable. Given how many applications screw up using fsync() correctly, to the extent that XFS, ext4, and btrfs all had to agree on a common hueristics to deal with the fact that application programmers are aggressively ignorant, and outnumber the file system developers, I too doubt the general strategy of relying only on application programmers to do the right thing. That's not to say that we shouldn't give up on trying to influence application programmers --- but relying on that as the only strategy seems to depart from the path of wisdom. There is however a much bigger point, which is that it's unfortunately black and white to talk about applications being "good" and "bad". In fact, it's a continuing point of concern I have with the whole qos approach to power management. In fact, power is often something that needs to trade off against performance. For example, an application could aggressively prefetch e-mail messages or web pages that a user _might_ view --- or it could aggressively pre-resolve DNS queries, etc, which might make perfect sense when the device is hooked up to AC mains, but which I might not want to do on when I only have 800mWh worth of battery --- however, if I'm using a laptop with 94,000mWh, maybe I'd be happy if the application was a bit more profligate. So for Arjan to claim that all applications will be held to the same standard, whether they are hooked up to the AC mains, or are limited to 800mWh of battery, or 94,000 mWh worth of power, is vastly oversimplifying the problem. Of *course* if I'm writing an application that will be running in a cloud data center, I'm going to care about power. But I may have different tradeoffs about what might considered acceptible when considering the qualify of user experience I'm delivering to the end-user when I'm connected to AC mains, versus a cell phone battery, ...
I'll agree with this. this is extremely situational.. not only where things run, but also, which part of the application you're talking about. If your application has 3 features (a $0.99 app store app) it's not too burdensome to make all 3 features work well for "power on a phone". If your app has 5000 features... what do you do when 4500 are behaving this is a very tricky area; because prefetching may well be good for power as well.... but "it depends"... getting DNS ahead of time when you're doing other network stuff anyway? probably a good idea for power, at least when done in moderation. It might save a modem wakeup two seconds down the road. excessively prefetching DNS all the time? probably not a good idea for yes if we allowed apps to do this, we absolutely need QOS also should be a request.. and depend on who's asking. If you run as root with realtime privileges, the kernel should probably just give you what you asked for. If you run as UID 500.... there needs to be either a static rlimit kind of thing, or an intelligent dynamic "clip" to what you ask versus what the system administrator (or his delegate in the form of some policy daemon) wants you to do. -- Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org --
I agree with that. Generally speaking, you can divide applications into the ones that will be allowed to influence ths system's behavior with respect to power management and the others that won't be allowed to do it. The latter may be forcibly "frozen" (this way or another) when the "trused" ones collectively decide it's a good idea to enter a deeper "energy saving" state. However, it is not a given that specific applications will always be in the same group. They may be "trusted" on some systems and they may not be "trusted" on some other system, depending on the configuration etc. That even may change over time on the same system. Thanks, Rafael --
On Mon, 2 Aug 2010 01:02:24 +0200 it might even be a sliding scale; the voting rights don't have to be "0" and "1", but could also be "0.2" and such.... --
Um, so this behaviour is isomorphic to the suspend block case for the applications. I think everyone agrees that suspend block isn't optimal, but we were prepared to use it as a starting point given the lack of enthusiasm in android for the more innovative approaches that have been That's hardly a fair criticism: this is easily fixable but the people looking into it have other calls on their time. At least you got some of the basic problems (like init from atomic context and sort efficiency) fixed this time around. If some large organisation actually cared That's why you present the user with choices and report on the outcomes. At the end of the day the choice becomes binary: if the mobile optimised browser burns you battery on the power meter, users will either uninstall and move on to the next browser or deny the current browser Right, but this comes back to the axes of control. They have to be presented to the user in a simple but meaningful manner. James --
Um, yes, that was deliberate. For some people I think people have gotten hypersensitive about suspend blockers, to the point that I think sometimes minds get closed automatically to say that suspend blockers or their requirements are evil/not really a requirement, so I decided to use another example other than scheduler control in the hopes of pointing out (a) there is a more general problem, and (b) application knows best is not enough, and (c) Arjan's claim that we don't need to do anything because all applications should be written to be "power optimized" and it shouldn't matter whether they are Or they may decide to drop the device which has much worse battery lifetime in favor of the hardware that seems to do a much better job of controlling power overall... which is why if the Nokia folks want to claim that suspend blockers are no good, it's probably not going to change what ships with Android, and may be better power management In the general case, at least for today, I think it's useful if applications are told, "you are on AC mains", "you are on cell phone battery", "you are on a laptop battery". And from a user's perspective, I suspect if you are wanting something simple but meaningful, it's probably a single linear scale ranging between "performance optimized" and "power optimized", with maybe 1-3 points in between. Advanced users will want a much finer level of control, though --- I can imagine having a number of different sliders that control the tradeoff between power vs. performance on a number of different scales: networking, GPS performance, scheduler control, schreen brightness, etc. Android phones have a very simplified version of this widget, which allows you to toggle GPS on/off, bluetooth on/off, etc. And the GPS toggle is a good example of what I'm talking about. If you disable the GPS, then even if the application wants to use GPS, tough luck; it will have to settle for the less precise cell tower triangulation method. Again, it's the ...
Actually, I don't necessarily agree with this. In fact, I think it's a dangerous proposition. The reason is that the only class of applications I've seen usefully made power aware are some of the system housekeeping ones (as in don't index my man pages, prelink my binaries etc. when I'm on battery power). Perhaps I could see things that have to poll (like imap without push support on the server) do a "poll when woken else every X minutes", but the potential for annoying the user by Yes and no; agree about the user not the application but ... I actually find the current five android "power" toggles to be largely useless. If I don't want GPS killing my battery, I don't start GPS using applications, for instance rather than disabling GPS. I think this represents the "axis" problem again ... I like to manage my power in a particular way, and others want to manage it in different ways. James --
Sure, but that's why you need to let the user choose. They can decide between, "behave as if you were on AC", and "try to save power as much as possible", and maybe one or two points in between, and let them choose how much performance/battery life they are willing to give up. If they are on a long flight to Europe/Australia with no power, they might choose a very different tradeoff point than if they know they are only going to be in the meeting room for an hour before they can recharge their battery. The point is you can annoy users by burning power to improve their "user experience" just as much as you can annoy users by trying to sip power as delicately as possible. It's true that many applications don't do a lot in this space, but that's mainly becaue of application laziness. Which is why I really don't buy Arjan's whack-a-mole approach to solving the problem. I *know* I can save tons of power by doing "killall -STOP firefox" when I don't need to use the browser. I've don't the power management when I've been carefully nursing my battery while at a conference. So I have no problem if the system does that automatically for me when I switch to a different virtual console --- in fact, I'd prefer it! Similarly, having tools which forcibly choose different pm_qos settings, even if it's not what the applications want, is things that *can* very much make a difference. So yes, maybe applications won't do much with that. But that just reinforces the argument that it's the framework or the kernel that needs to do these sorts of things, because the application programers won't. - Ted --
Unless someone can reliably automate whacking the moles, history is not on the side of the mole-whackers. I won't say that such automation is impossible, but only because of all the "impossible" things in my lifetime that have turned out to be possible. And depending on how badly you need the application to run. "Find me the nearest source of AC power!" "Sorry, can't do that because that app is too poorly behaved to run when the battery is nearly exhausted." ;-) Agreed! Thanx, Paul --
On Sun, 1 Aug 2010 20:03:04 -0700 we have a few things going for us here though 1) It's easy to programatically detect the problems 1a) so programmers can detect issues before they ship software, unlike the fsync() thing Ted mentions. History is showing this is at least relatively successful in open source, but not with Adobe's proprietary software such as Flash 2) Users can be made aware of what the bad guys are 3) The business models of many of the mobile apps gives programmers a strong incentive to ship well behaving. The moment your first and second review of your app read "this app was identified as reducing battery life a lot"... your revenues will not go beyond the $1.98.. ... assuming these first two guys didn't refund their app. Since we can detect who the bad guys are, we can also automatically quarantine them for the common cases..... which is good news. I'm a little worried that this whole "I need to block suspend" is temporary. Yes today there is silicon from ARM and Intel where suspend is a heavy operation, yet at the same time it's not all THAT heavy anymore.... at least on the Intel side it's good enough to use pretty much all the time (when the screen is off for now, but that's a memory controller issue more than anything else). I'm pretty sure the ARM guys will not be far behind. --
remember that this 'block suspend' is really 'block overriding the fact that there are still runable processes and suspending anyway" having it labeled as 'suspend blocker' or even 'wakelock' makes it sound as if it blocks any attempt to suspend, and I'm not sure that's what's really intended. Itsounds like the normal syspend process would continue to work, just this 'ignore if these other apps are busy' mode of operation would not work. which makes me wonder, would it be possible to tell the normal idle detection mechanism to ignore specific processes when deciding if it should suspend or not? how about only considering processes in one cgroup when deciding to suspend and ignoring all others? David Lang --
On Sun, 1 Aug 2010 22:06:34 -0700 (PDT) We then get again to the "runnable tasks" problem that was discussed earlier... the system get's "deadlock-prone" if a subset of tasks is not run. Interprocess dependencies are not so easy to get right in general. Cheers, Flo --
I'm not suggesting that you don't run the 'untrusted' tasks, just that you don't consider them when deciding if the system can suspend or not. if the system is awake, everything runs, if the system is idle (except for the activity of the 'untrusted' tasks) you suspend normally. David Lang --
On Sun, 1 Aug 2010 23:06:08 -0700 (PDT) Ah, yes. Sorry. It's pretty early in the morning over here, I don't seem to have my eyes fully opened yet... A "ignore-these-processes" cgroup could probably work... It would have the advantage of not having to maintain a special purpose API.... Cheers, Flo --
On Mon, 2 Aug 2010 08:40:03 +0200 Thinking about it.. I don't know much about cgroups, but I think a process can only be in one cgroup at a time. So you'd need to provide a) a way to race free migrate them to the "suspend block" cgroup (or dropping them out of the "ignore" cgroup) b) you can't use cgroup for other purposes anymore. I.e. if you want to have 2 groups that each only have half of the memory available, how would you then integrate the cgroup-ignore-for-idle-approach with this? hmmm.. Cheers, Flo --
why does it need to be race free? being in transition is going to be logically the same as being in the other group. it's not like applications will be moving back and forth between the two two answers to this 1. does this matter? do you really need to combine this 'suspend, even if there are processes trying to run' with other cgroup limitations? 2. who says that this must be limited to one cgroup? a cgroup can have several different flags/limits set on it, so why can't one of them be this 'ignore for suspend' flag? these seem like simple issues, what I don't know is if it's possible for the process that controlls suspend to follow such a flag without major surgury on it's innards (if it can, this seems like a easy win, but I can imagine internal designs where the software just knows that _something_ is trying to run and would have a very hard time figuring out what) David Lang --
On Mon, 2 Aug 2010 00:02:04 -0700 (PDT) hmm... i envisioned a mechanism where applications would be able to switch the groups freely depending on what context the device is in. I didn't even thought about a static grouping. But maybe that is possible also and maybe even sufficient... Well, i fear it becomes some sort of parallel-tree structure... If you want a cgroups-partitioning for one kind of attribute you now need 2 containers for every possible stamping of that attribute... one being flagged with 'ignore-for-suspend-decision' and one without that flag. Do you see what I'm getting at, or do I need more coffee and it is irelevant to this concept? Cheers, Flo --
yes, it could mean a doubleing in the number of cgroups that you need on a system. and if there are other features like this you can end up in a geometric explosion in the number of cgroups. in practice I question if there is likely to be a need for this sort of thing on a system that's complex enough to use cgroups for other purposes. in particular, in this case the 'ignore for syspend' flag is not going to be set for programs that are trusted to be well behaved. Such programs are unlikely to be placed under other restraints (because they _are_ trusted to be well behaved) David Lang --
No, it would be additive - you can mount different subsystems on separate hierarchies. So if you had X divisions for memory, Y divisions for CPU and Z divisions for suspend-blocking (where Z=2, probably?) you could mount three separate hierarchies and have X+Y+Z complexity, not X*Y*Z. (Not that I have a strong opinion on whether cgroups is an appropriate mechanism for solving this problem - just that the problem you forsee shouldn't occur in practice). Paul --
On Mon, 2 Aug 2010 21:41:17 -0700 Ah yes, mea culpa. I've got this wrong. Cheers, Flo --
A thread can only be in one cgroup in each hierarchy at one time. You can mount multiple cgroups hierarchies, with different resource You could mount the subsystem that provides the "ignore-for-idle" support on one hierarchy, and partition the trusted/untrusted processes that way, and the memory controller subsystem on a different hierarchy, with whatever split you wanted for memory controls. Paul --
Hi! On Mon, 2 Aug 2010 21:38:12 -0700 Thank you for the clarification. That renders my original objections more or less void. I've still got some doubts about the flexibility of this approach (think an open system with arbitrary software components). But with a userspace manager that sorts processes into the groups this may be a possible solution. But we should probably concentrate first on the requirements now. If we have a set of requirements everyone can agree too, we may be on our way to get a solution. Cheers, Flo --
Why not flesh this out and compare it to the draft requirements? (I expect to be sending another version by end of day Pacific Time.) The biggest issue I see right off-hand is that a straightforward implementation of your idea would require moving processes from one cgroup to another when acquiring or releasing a suspend blocker, which from what I understand would be way to heavyweight. On the other hand, if acquiring and releasing a suspend blocker does not move the process from one cgroup to another, then you need something very like the suspend-blocker mechanism to handle those processes that are permitted to acquire suspend blockers, and which are thus not a member of the cgroup in question. That said, I did see some hint from the Android guys that it -might- be possible to leverage cgroups in the way that you suggest might help save power during times when suspend was blocked but (for example) the screen was turned off. The idea would be to freeze the cgroup whenever the screen blanked, even if suspend was blocked. The biggest issue here is that any process that can hold a suspend blocker must never to an unconditional wait on any process in this cgroup. Seems to me that this should be possible in theory, but the devil would be in the details. If I am misunderstanding your proposal, please enlighten me! Thanx, Paul --
you are close, but I think what I'm proposing is actually simpler (assuming that the scheduler can be configured to generate the appropriate stats) my thought was not to move applications between cgroups as they aquire/release the suspend-block lock, bur rather to say that any application that you would trust to get the suspend-block lock should be in cgroup A while all other applications are in cgroup B when you are deciding if the system shoudl go to sleep because it is idle, ignore the activity of all applications in cgroup B if cgroup A applications are busy, the system is not idle and should not suspend. this requires that the applications in cgroup A actually go idle as opposed to simply releaseing the suspend-block lock, but it would mean that there are no application changes required for to move a system from the status "even if it's busy, go ahead ans suspend" to "this application is important, don't suspend if it's got work to do", it would just be classifying the application in one cgroup or the other. This assumes that an application that you would trust to hold the suspend-block lock is going to be well behaved (if it isn't, how can you trust it to not grab the lock inappropriatly?) David Lang --
Triggering suspend from idle has been suggested before. However, idle is not a signal that it is safe to suspend since timers stop in suspend (or the code could temporarily be waiting on a non-wakeup interrupt). If you add suspend blockers or wakelocks to prevent suspend while events you care about are pending, then it does not make -- Arve Hjønnevåg --
isn't this a matter of making the suspend decision look at what timers have been set to expire in the near future and/or tweaking how long the system needs to be idle before going to sleep? to properly do a good job at suspending hyperagressivly you need to look at when you need to wake back up (after all, if you are only going to sleep for 1 second, it makes no sense to try and enter a sleep state that takes .5 seconds to get into and .6 seconds to get out of, you need to pick a lighter power saving mode) if you really want to have the application be able to say 'I am ready for you to go to sleep NOW and I don't want any chance of waking back up until the system is ready for me to do so' it may be possible to have a special dev file that when a program attempts to read from it the program is considered to have been idle forever (expiring any 'delay since last activity' timers that are running for that process) and when the device wakes back up (or after a little bit of time if the device decides not to go back to sleep), allow the return from the blocking call to allow the app to continue running. but I am not sure that something like that is needed. I think just checking for timers that are due to expire 'soon' and tweaking how long the system must be 'idle' before it decides to go to sleep should be good enough. David Lang --
You are describing low power idle modes, not suspend. Most timers stop in suspend, so a timer set 10 seconds from now when entering suspend will go off 10 seconds after resume so it should have no impact on how -- Arve Hjønnevåg --
so what is the fundamental difference between deciding to go into low-power idle modes to wake up back up on a given point in the future and deciding that you are going to be idle for so long that you may as well suspend until there is user input? David Lang
Low power idle modes are supposed to be transparent. Suspend stops the monotonic clock, ignores ready threads and switches over to a separate set of wakeup events/interrupts. We don't suspend until there is user input, we suspend until there is a wakeup event (user-input, incoming network data/phone-calls, alarms etc..). -- Arve Hjønnevåg --
s/user input/wakeup event/ and my question still stands. low power modes are not transparent to the user in all cases (if the screen backlight dimms/shuts off a user reading something will notice, if the system switches to a lower clock speed it can impact user response time, etc) The system is making it's best guess as to how to best srve the user by sacraficing some capibilities to save power now so that the power can be available later. as I see it, suspending until a wakeup event (button press, incoming call, alarm, etc) is just another datapoint along the same path. If the system could not wake itself up to respond to user input, phone call, alarm, etc and needed the power button pressed to wake up (or shut down to the point where the battery could be removed and reinstalled a long time later), I would see things moving into a different category, but as long as the system has the ability to wake itself up later (and is still consuming power) I see the suspend as being in the same category as the other low-power modes (it's just more expensive to go in and out of) why should the suspend be put into a different category from the other low-power states? David Lang
OK, I'll bite... From an Android perspective, the differences are as follows: 1. Deep idle states are entered only if there are no runnable tasks. In contrast, opportunistic suspend can happen even when there are tasks that are ready, willing, and able to run. 2. There can be a set of input events that do not bring the system out of suspend, but which would bring the system out of a deep idle state. For example, I believe that it was stated that one of the Android-based smartphones ignores touchscreen input while suspended, but pays attention to it while in deep idle states. 3. The system comes out of a deep idle state when a timer expires. In contrast, timers cannot expire while the system is suspended. (This one is debatable: some people argue that timers are subject to jitter, and the suspend case for timers is the same as that for deep idle states, but with unbounded timer jitter. Others disagree. The resulting discussions have produced much heat, but little light. Such is life.) There may well be others. Whether these distinctions are a good thing or a bad thing is one of the topics of this discussion. But the distinctions themselves are certainly very real, from what I can see. Or am I missing your point? Thanx, Paul --
Ok, this is a complication to what I'm proposing (and seems a little odd, but I can see how it can work), but not neccessarily a major problem. it depends on exactly how the decision is made to go into low power states and/or suspend. If this is done by an application that is able to look at either all activity or ignore one cgroup of processes at different times I see this as simply being a matter of what devices are still enabled at the different power savings levels. At one level the touchscreen is still powered, while at another level it isn't, and at yet another level you have to hit the power soft-button. This isn't fundamentally different from powering off a USB peripheral that the system decides is idle (and then if you have the ability to wake for an alarm, you have the ability to wake for a timer (if from no other method than to set the alarm to when the these big distinction that I see as significant seem to be in the decision of when to go into the different states, and the difference between the states themselves seem to be less significant (and either very close to, or within the variation that already exists for power saving modes) If I'm right bout this, then it would seem to simplify the concept and change it from some really foreign android-only thing into a special case variation of existing core concepts. you have many different power saving modes, the daemon (or kernel code) that is determining which mode to go into would need different logic (including, but not limited to the ability to be able to ignore one or more cgroups of processes). different power saving modes have different trade-offs, and some of them power down different peripherals (which is always a platform specific, if not system specific set of trade-offs) This all depends on the ability for the code that decides to switch power modes (including to trigger suspend) to be able to see things in sufficient detail to be able to do different things depending on ...
The touchscreen on android devices is powered down long before we suspend, so that is not a good example. There is still a significant difference between suspend and idle though. In idle all interrupts work, in suspend only interrupts that the driver has called If you just program the alarm you will wake up see that the monotonic clock has not advanced and set the alarm another n seconds into the future. Or are proposing that suspend should be changed to keep the monotonic clock running? If you are, why? We can enter the same hardware states from idle, and modifying suspend to wake up more often would increase the average power consumption in suspend, not improve it for idle. In other words, if suspend wakes up as often as idle, why Suspend is not an android only concept. The android extensions just allow us to aggressively use suspend without loosing (or delaying) wakeup events. On the hardware that we shipped we can enter the same power mode from idle as we do in suspend, but we still use suspend primarily because it stops the monotonic clock and all the timers that use it. Changing suspend to behave more like an idle mode, which seems The hardware specific idle hook can (and does) decide to go into any -- Arve Hjønnevåg --
are you talking about Android here or are you talking genricly across all no, I was thinking that you set the alarm to go off, and when it goes off and wakes you up, you correct other local clocks before doing anything else. even if they wake up at the same time, you would use suspend instead of idle if it saved more power (allowing for the power to get in and out of suspend vs the power to get in and out of idle) in this case, another reason you would consider using suspend over idle is that you can suspend until the next timer that your privilaged applications have set, and skip timers set by the non-privilaged Ok, If I am understanding you correctly I think this is an important point. What Android calls suspend is not what other linux distros call suspend, it's just a low-power mode with different wakeup rules. Is this correct? If this is the case it seems even more so that the android suspend should be addressed by tieing into the power management/idle stuff rather than the suspend/hibernate side of things is the reason you want to stop the onotonic clock and the timers because the applications need to be fooled into thinking no time has passed? or is it to prevent extranious wakeups? This I know is an Andoid specific thing. On other platforms power states >> not without major surgery)
Hi, On Tue, 3 Aug 2010 21:47:49 -0700 (PDT) I think my laptop (x86-64) uses the same notion of suspend as Android. I am confused now. Android 'suspend' is equivalent to 'echo "mem" > /sys/power/state' Which distros call it "low-power mode with different wakeup rules"? Gentoo doesnt. In KDE/Gnome it's also called suspend or suspend-to-ram iirc. Cheers, Flo --
This appears to be the current Linux driver model. Old platform code Suspend and idle use the same power state on the devices we shipped. Without wakelock or suspend blockers this can still break since a privileged application may be waiting on a resource held by a Yes, but this is not an Android extension, it is part of the normal No (assuming you are asking about the clock), the actual hardware clock (on msm) stops even in idle but it is resynchronized on wakeup -- Arve Hjønnevåg --
not quite the question I was trying (and failing) to ask, but from the answer it sounds like setting suspend to wake up on all interrupts is the same as idle. so here suspend is looking like just a variation of low-power idle Ok, so if you set the alarm to the next time the timer for a privilaged task would run and wake up then to be completely transparent to the privilaged task, or you could decide to disable the timers as well since the non-privilaged application is never frozen when a privilaged application is running, I'm not sure how you would get this to happen that wouldn't also be a problem with wakelocks. if you want to have a privilaged application keep the system awake while it waits for a non-privilaged application, all it would need to do is to set a timer near enough in the future that it's considered 'awake' all the time. this will cost a context switch every timeout period (for the application to check that it's still waiting and set the next timer), but that's pretty cheap. one thing this would destroy is the stats currently built around the wakelock, but since they would be replaced by stats of exactly the type of thing that powertop was written to deal with, I don't see this as a problem (powertop would just have to learn a mode where it ignores the In Linux, a full suspend normally takes the system down to a mode that can't be reached by the normal idle mechnism (including powering down peripherals). It also takes significantly different actions to wake up. no, as noted by others in this thread, normal linux suspend/resume modifies the clock to show that time has passed since the suspend you are stating that this must be suspend because low-power idle must be transparent to the user. I am saying that on Linux, low-power idle commonly is not transparent to the user, so the requirement for it to be transparent (therefor putting the suspend into a different category) is an Android only requirement. David Lang
With wakelocks we annotate that we have important work to do, until that work is accomplished we do not suspend. If you modify the idle code ignore some processes' timers the system may get stuck in idle We use the normal suspend sequence. The shipping hardware we have just The monotonic clock always stops. There are a few different ways to If you ignore the timers and don't stop the clock they are based on, you break a lot of apps when you resume (watchdogs and timeouts trigger). If you don't ignore timers, you can't sleep very long with I'm am not talking about minor latencies. If you have a platform that for instance turns off you screen dma when entering idle, it is broken whether is running Android or not. If it does the same in suspend it is not a problem. -- Arve Hjønnevåg --
On Wed, 4 Aug 2010 00:10:21 -0700 Perhaps transparent is badly worded. Both suspend and idle are in some way "transparent". Idle doesnt happen when an application is runnable. Suspend is transparent in that the application can not easily detect if it happened. Cheers, Flo --
if you don't have a wakelock the system could go into suspend under the exact same conditions. If the privilaged program wants to be sure of preventing this, all it needs to do is to set of timer to take action you are only doing this every several seconds to prevent the system from suspending. it's possible that I'm making false assumptions about how quickly you want to go into full suspend mode. if a user is doing nothing that would warrent wakelocks, but has an unprivilaged application running (a dancing cows game), and is doing nothing other than occasionally hitting a button, how short is the timeout that you would set that would have the system go into suspend? (i.e. how frequently must the user do something to keep the system awake) or let's use a better example, the user has an unprivilaged book-reader application, how quickly must they change pages to prevent the system from suspending? I'm figuring that these times are in the 1-5 minute range. therefor the timeout period I am mentioning above could easily be one wakeup every 40-50 seconds. that is not going to kill your idle power. if you can stop the clocks they are based on, you could also shift them into the future. but in any case, significant changes in time can cause problems for suspended apps (if nothing else, broken network connections This isn't sounding quite right to me. I've seen too many discussions about things like idle and USB devices/hubs/drives/etc getting powered down for power savings modes to make me readily accept that everything must be as transparent as you imply. Just the case of drive spin-down shows that it's possible to do things that would be considered destructive, but you have to have a flag and wake-up path to recover within a 'reasonable' amount of time (I guess that this could be 'transparent' if that only implies that things must work eventually, which isn't what I read into the statement) David Lang
It would be much better if you gave specific examples. Because, for instance, Well, consider a single character device and suppose there is an application talking to the driver using read(), write(), ioctl(), whatever. Now suppose you want to put the device into a low-power state such that the device can't do the I/O in that state. You need to ensure that the app won't be able to reach the device while in that state and you can (1) arrange things so that the device is put into the full-power state whenever the app tries to access it and (2) "freeze" the app so that it won't try to access the device being in the low-power state. Generally speaking (1) is what idle (and any other form of runtime PM) does and (2) is what suspend does with respect to the whole system. In the suspend case, when you have frozen all applications, you can sequentially disable all interrupts except for a few selected ("wakeup") ones in a safe way. By disabling them, you ensure that the CPU will only be "revived" by a limited set of events and that allows the system to stay low-power for extended time intervals. To achieve the same result in the "idle" case, you'll need to have a mechanism to disable interrupts (except for the "wakeup" ones) avoiding synchronization problems (eg. situations in which task A, blocked on a "suspended" device access, holds a mutex waited for by task B that needs to run so that we can "suspend" another device). That, however, is a difficult problem. Thanks, Rafael --
I gave examples in other messages that included dimming/turning off the screen and spinning down the disk among others. the definition of 'transparently' was then clarified to not mean transparent to the user, but transparent (except for possibly delays) to the applications (they may have to wait for the disk to spin up for the benifit of this will depend on what wakeups you are able to avoid by putting the hardware to sleep. Depending on the hardware, this may be not matter that much. In addition, there are input devices like a touchscreen that could be disabled even short of a full suspend, as long as there is some way to get I would say that the difficulty of the problem depends on the hardware and how much userspace interaction is needed. If the kernel can disable the hardware in the driver (so that it can wake it up again when accessed by an application) it would seem that this isn't a problem. David Lang --
That's correct, but evidently it does make a difference with the hardware That's correct again, but it doesn't matter too much as far as the difference between "deep idle" and suspend is concerned. These devices are generally not Well, I guess it's useful to consider the sequence of events in the "idle" case. We first detect that a CPU is idle, so it can be put into a C-state (or equivalent). It's easy to put that CPU into such a state, but suppose we want to do more and put all devices into low-power states at that point. We have to ensure that there are no devices in the middle of DMA and that the other CPUs are idle (that's kind of easy, but still). Next, we have to figure out the right ordering in which to put devices into low-power states (that is not trivial, because the tasks that use those devices may depend on each other in various ways). If we decide to shut down clock source interrupts, we must ensure that the timekeeping will be correct after that and so on. Moreover, things get ugly if there are shared interrupts. Suspend kind of works around these difficulties by taking the entire user space out of the picture with one big sledgehammer called the freezer, but these problems would have to be actually _solved_ in the "idle" case. Thanks, Rafael --
Ok, but is there a way to put some of this to sleep without involving a I'm not opposed to doing a suspend if the system is idle enough, I'm just questioning if suspend is really that different, or if it can be viewed as yet another low power mode (one that's significantly more expensive to transition in and out of) David Lang --
Technically, maybe, but we have no generic infrastructure in the kernel for that. Suspend is really different. First, the way it is entered is different (ie. it starts from the freezing of user space and sequentially powers down things going from leaf devices towards the "core" of the system). Second, it shuts down interrupt sources that cannot be easily shut down from idle (at least in general). Finally, on some platforms (eg. ACPI) it requires extra hardware black magic that cannot be triggered from the cpuidle context. The deactivation of more interrupt sources appears to be the most significant difference from the energy saving point of view, but it is achieved because of the way suspend is entered. It may or may not be achieved in a different way, but (1) no one has implemented anything close to a working alternative solution yet and (2) it is not certain that's worth doing. Thanks, Rafael --
well, I know that we have specific cases of this (drive spin-down, cpu speed, display backlight for a few examples), is it worth trying to define a generic way to do this sort of thing? or should it be left as a per-device thing (with per-device knobs to control it) I thought I had seen discussion on how to define such a generic power management interface, and I thought the results had been acceptable. David Lang --
The ability to put specific devices into low-power states in certain well-defined situations is clearly not sufficient. For one example, usually you can easily put an Ethernet adapter into a low-power state when the network cable is detached from it. It is not clear, however, what criteria should be used for deciding to put that adapter into the low-power state when the cable is attached to it (and open() has been called). To mimic suspend you'll have to be able to put _all_ devices into low-power states and shut down the interrupts that allow the monotonic clock to advance. If you have a pointer to that discussion, I'm interested. :-) Thanks, Rafael --
true for the generic case, my thought was that for specially built hardware it may be possible to select hardware that lets you get very sorry, it was several months ago. David Lang --
You're supposed to fix the clock after you wake up. That's part of the cost of going suspend. OG. --
I'm not sure what you are referring to. The generic Linux timekeeping code makes sure the monotonic clock stops while the system is suspended regardless of what values the (hardware specific) clocksource returns. -- Arve Hjønnevåg --
That just means the android-like suspend should not act in that specific area the same as the generic suspend to ram. Which is not entirely surprising. In any case, it's not "keep the monotonic clock running", which would cost power. It's fix it up afterwards, using the same means you already do but perhaps at a higher level currently. People don't want to see the wrong time of the day just because the phone went to suspend. Laptop STR is different in perception because it shuts down things people don't want to see shutdown just by being idle, namely wifi connectivity. If your next polling timer is in half an hour and the screen/touchpad are off, you want to go as low in power as possible until then, and that's the deepest level of idle you can find that responds to alarms which may happen to be called suspend. But from a user point of view, it's just another idle level. And from a power management code/policy decisions point of view, it should be the same. OG. --
The monotonic clock is not the same as the realtime clock which is Calling the deepest idle state suspend is just confusing. Linux suspend has a different api than idle. In suspend every driver that has a suspend hook is notified. Also, once cpu can be idle while -- Arve Hjønnevåg --
Hmmm... The bit about the monotonic clock not advancing could help explain at least some of the heartburn from the scheduler and real-time folks. ;-) My guess is that this is not a problem for Android workloads, which probably do not contain aggressive real-time components. (With the possible exception of interactions with the cellphone network, which I believe are handled by a separate core with separate OS.) However, pulling this into the Linux kernel would require that interactions with aggressive real-time workloads be handled, one way or another. I can see a couple possible resolutions: 1. Make OPPORTUNISTIC_SUSPEND depend on !PREEMPT_RT, so that opportunistic suspend simply doesn't happen on systems that support aggressive real-time workloads. 2. Allow OPPORTUNISTIC_SUSPEND and PREEMPT_RT, but suppress opportunistic suspend when there is a user-created real-time process. One way to handle this would be with a variation on a tongue-in-cheek suggestion from Peter Zijlstra, namely to have every real-time process hold a wakelock. Note that such a wakelock would need to be held even if the real-time process in question was not runnable, in order to meet possible real-time deadlines when the real-time process was awakened. 3. Your proposal here. ;-) Thoughts? --
I think that indeed is the case, although they haven't expressed that directly The case when there's a real-time process that's not using its time slices (because it doesn't have anything to do) seems to be hard. You'd probably want to suspend in that case, but then meeting the real-time deadlines would be kind of unrealistic ... Thanks, Rafael --
On Tue, 3 Aug 2010 17:10:15 -0700 for "system suspend", this is an absolutely valid statement. for "use suspend as idle state", it's not so clearly valid. (but this is sort of a separate problem, basically the "when do we freeze the tasks that we don't like for power reasons" problem, which in first order is independent on what kind of idle power state I would argue that this is both a hardware specific issue, but also a policy issue. From the user point of view, screen off with idle and screen off with suspend aren't all that different (if my phone would decide to idle rather than suspend because some app blocks suspend... I wouldn't expect a difference in behavior when I touch the screen). "Screen off -> don't honor touch after a bit" is almost an independent, but very real, policy problem (and a forced one in suspend, I'll grant you that). I could even argue that the policy decision "we don't care about the touch screen input" is a pre-condition for entering suspend (or in android speak, caring for touch screen input/having the touch I'll debate it even harder in that it's platform specific whether timers can get the system out of suspend or not. Clearly on the Android platform in question that's not the case, but for some of the Intel phone silicon for example, timers CAN be wake sources to get you out of suspend just fine. It just depend on which exact hw you talk about. Generally, even if the fast timers aren't wake up sources, there'll be some sort of alarm thing that you can pre-wake.. but yes you are right in saying that's rather lame. Either way, it's not a general property of suspend, but a property of suspend on the specific platform in question. -- Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org --
note that what I'm speculating about would never freeze some of the tasks, it would run everything if anything is run, but it would not consider the actions of some of the programs when deciding if it can shutdown. so if you have all your privilaged applications in long sleeps, but still have your bouncing cows running, peggng the CPU, making noise, and updating the screen, the system would decide the system is 'idle' and go into the 'suspend' low power state until there is a wake activity. but if you have a privilaged application doing other stuff (say you are talking on the phone, have a GPS mapping program running and giving you directions, etc), the bouncing cows would continue to run and there would never be an attempt to freeze them while leaving the other stuff active. David Lang --
On Tue, 3 Aug 2010 21:55:07 -0700 (PDT) I think the only difference between your proposition and the current android practice is that in your scheme the partition is along the process/task boundary (i.e. good apps vs bad apps) whereas the android looks at actions or events or ... let's call it "stuff" and blocks suspend whenever "stuff" needs to be done. The main thing is that even an process that is allowed to block suspend doesn't do so all the time. Only when it is critical that "stuff" get's done right away. One way to make an application work the same way with your scheme (as I understand it) on android could be to split the application into an "blocker" part, that is not part of the "ignore-cgroup" (the "trusted" part) and another one that is in the "ignore-cgroup" (the "untrusted" part). Whenever the untrusted part needs to block suspend it notices the "trusted" part via ipc and the trusted part goes into some sort of "activity loop". This looks kind of stupid. But I bet it is what application developers are going to do if there is no provision for a flexible runtime enable/disable "suspend blocker". hmmm... Cheers, Flo --
with the caviot that only privilaged applications are allowed to define effectivly, only when it's important that "stuff" gets done at all. With current smartphone battery capacities it's not uncommon that the phone could be in suspend until the battery dies, since suspend is not zero remember that only "trusted" applications were allowed to set the suspend-blocker/wakelock flag in the first place. my thinking is that since "trusted" applications are assumed to be well written (or they can just raise the suspend-blocker/wakelock and keep the phone from ever sleeping) they should be able to be trusted to sleep appropriately so that if they aren't actually doing anything important the system can go ahead and go to sleep. Since powertop was designed to find ill-bahaved applications, you would still be able to point the finger at the offenders. yes, you could trivially have a 'keep me awake' daemon that other apps can talk to to request the system stay awake (the same way you could with a suspend-blocker/wakelock). The "activity loop" for such a daemon can be pretty trivial, set a timer just a little shorter than the idle-go-to-suspend timeout, sleep that long and reset the timer. I could be mistaken here, but my assumption is that since normal applications running would never get trigger the wakelock except when there is user input, the timeout here is on the order of 10s of seconds, if not a few minutes. having a privilaged application wake up once every several seconds is not going to be significant to anything other than things looking for the system being idle for X amount of time.. Even if this is done, having a userspace 'wakelock' daemon that works this way is significantly better in many people's eyes than having the kernel implement this policy. It's much easier to put application/platform specific logic in the userspace daemon than into the kernel (say, things like 'any app can request a delay for up to X amount of time, but after ...
I disagree. On the msm platform the low level timer that brings us out of the low power state is the same for idle and suspend. The difference is where which kernel api the request comes from. In idle, the next event on the clockevents device is usually the first event. In suspend the generic kernel timekeeping code cancels this event and the rtc wakeup event remains. -- Arve Hjønnevåg --
OK, good point. This choice of kernel APIs governs what I was calling "policy". Thanx, Paul --
From what I can see, the Android folks are are using "suspend" in the "system suspend" sense. I agree that the proposals for freezing subsets of the tasks in the system are independent of whether idle or suspend is being used. Instead, such freezing depends on (for example) whether or not the display is active. That said, freezing subsets of tasks is a nice-to-have rather than a hard requirement for Android. Though I suspect that the appearance of a reliable way of freezing subsets of tasks just might promote I agree that the subset of input events that do not bring the system out Good point, I do need to emphasize the fact that whether or not timers pull the system out of suspend also depends both on hardware and on policy. So I will change my statement to say something like "The system comes out of a deep idle state when a timer expires. In contrast, timers do not necessarily expire while the system is suspended, depending on both hardware support and platform/application policy." --
That's correct IMO. Thanks, Rafael --
Hi, On Sun, 1 Aug 2010 21:05:48 -0700 I think that the only thing that really matters (longterm) with suspend is that processes don't get scheduled anymore when the system is suspended. Cheers, Flo --
Part of the problem here may also be a naming problem. There are multiple reasons why you might want to block suspend, and it goes far beyond what the CPU can do. Let's give another example, from the laptop world. If you close the the MacBook, it "suspends". That is, all processes stop, and the CPU enters a sleep state. Sounds just like Linux's suspend-to-ram, right? Except for the fact that if you insert a USB cable connected to a iPod/iPad/iPhone, the CPU wakes up, and iTunes will do a sync, and then the machine goes back to sleep. And if the Mail application is in the middle of manipulating the IMAP mailbox, it will be allowed to finish what it is doing, and then the system goes to sleep. So in the case of both the MacBook, where the user has closed the laptop, and in the case of Cell Phone, where the user has pushed the button saying he/she's done working with the application, the _normal_ thing to do force all processes to go to sleep, and then let the CPU go to sleep. But, you may have applications that are specially privileged so they can ignore the user command to suspend the machine. Maybe it's an iTunes-like situation, where in response to some hardware interrupt, it's allowed to wake up, do its thing, and then go back to sleep, allowing the hardware to go back to sleep. Maybe it's a mail application that wants to wakeup every 15 minutes, suck down any new mail, and then go back to sleep. The suspend blocker is a way to achieve this --- and this has nothing to do with chip technology, but with a specific use case. - Ted --
I _think_ you can implement that with the current kernel's interfaces and an suitable power manager in user space. At least you should be. The question really boils down to where the "power manager" activity should be located. Should it be in the kernel or in user space? Or is it better to have a "collective power manager", like in the wakelocks case? Moreover, what should the power manager be responsible for and what the kernel's role should I frankly don't think it's legitimate to overrule the user. If the user tells us "suspend" and specifies a set of devices that can be used for wakeup, we should follow, possibly monitoring the wakeup devices for events. However, that doesn't apply 100% to the cell phone case, because on cell phones the software is actually owned by the device vendor and/or the network operator and they may want to have a mechanism to overrule the person using the phone. Assuming that your "power manager activity" will be a collective one, along the Android lines. Still, in principle, you can obtain a functionally equivalent behavior with the help of a power manager in user space. Thanks, Rafael --
Unless of course the app reducing battery life a lot does something Heh! One could make the "might be temporary" argument against any new feature, including the ones that you are proposing. For but one example, I very well remember being told by many people in many communities that SMP was temporary -- Moore's-Law increases in clock frequency would soon mean that even the biggest systems would be uniprocessors. If by "memory controller" you are referring to the need to save state for cache SRAMs and perhaps also DRAM, then yes, I agree that some of the most difficult low-power-state tradeoffs involve these devices. Thanx, Paul --
On Sun, 1 Aug 2010 12:12:28 -0700 power indeed needs multiple approaches. For example the timer work you mention. What the timer work allows is controlling the behavior of applications that are not well behaved. This is an important way of driving progress, one the android guys sort of swept under the table. Is the flash plugin (which seems to be the most popular and also worst behaving software in these type of discussions) waking up 100 times per second? <small adjustment by a policy manager> ... now it's only waking up once every 10 seconds, or when something else causes system activity anyway. That capability is there in Linux today, and has been there for a long time now. It's highly underused unfortunately... at least outside the MeeGo world of Linux distributions. Another one: freezing whole cgroups..... we have that today. it actually works quite well.... of course the hard part is the decision what to put in which cgroup, and at what frequency and duration you let cgroups run. on the suspend blockers for drivers; the linux device runtime PM is effectively doing the same things; it allows drivers to suspend/resume individually (with a very nice API/programming model I should say) based on usage. And it works on a tree level, so that it's relatively easy to do things like "I want to go to <this magic deep idle state>, but only if <this set of devices is suspended already>". This is obviously an important functionality for all low power devices, ARM or x86. Suspend blockers had this functionality as part of what it did (they do more obviously) but I'd wager that the current Linux infrastructure is outright nicer. It's what I want to be using for the Intel phone silicon (which isn't all that different in power behavior/requirements than ARM based silicon), and I need to thank Qualcomm here for providing some very good patches to the kernel to allow this model to work for this scenario. As for the aware-versus-good-behaving point; applications can be ...
I agree that timer control is important and will continue to be. It doesn't seem to me to be in conflict with the Android guys' approach, in fact, it should help reduce power consumption while the device Indeed, the Android guys seemed to be quite excited by cgroup freezing until they thought about the application-classification problem. Seems like it should be easy for some types of applications, but I do This is what Rafael has been working on? Of course, the Android guys also want to pay attention to which apps I must have missed those, and I am not seeing any obvious commits If I understand you correctly, a key point of agreement between you and the Android guys is that both the system and the user have some say over how applications are treated by the system in terms of how seriously the system takes a given application's requests. The Android guys also want the user to have some say about what applications are permitted to have some control over "I want to go to <this magic deep idle state>" requests. Does that seem reasonable to you? Thanx, Paul --
On Sun, 1 Aug 2010 18:10:06 -0700 I personally think it's one of those things where... well we can get a LONG way automatically (by just observing things); asking the users is very very often just caving in rather than solving the problem. Asking the user should only be done for things the user 1) Can give an intelligent answer to and 2) Are something the user WANTS to be involved in. (rather than 'stupid thing, why don't just do the right thing'.. think the Windows Vista security questions) -- Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org --
Combining this with your previous email, I believe that you are saying that it is necessary for the user to be able to exert some control, but that the UI design had better be sufficiently intelligent with sufficiently good defaults that the user almost never actually -needs- to exert such control. If this is indeed what you are saying, I certainly agree. Thanx, Paul --
This isn't more difficult than deciding which applications will be allowed to use wakelocks (in the wakelocks world). It actually seems to be pretty much In fact the runtime PM framework is also important to Android, because it can be used in there, for example, to implement the "early suspend" thing I referred to in one of my previous messages in this thread. Thanks, Rafael --
If I understand correctly, the problem they were concerned about was instead "given that a certain set of apps are permitted to use wakelocks, which of the other apps can safely be frozen when the display blanks Now we just need to convince the Android guys of that. ;-) Thanx, Paul --
I _think_ the problem should be reformulated as "which of the other apps can be safely frozen without causing the wakelocks-using ones to have problems" instead (the particular scenario is that one of the wakelocks-using apps may need one of the other apps to process something and therefore the other app cannot be frozen; however, that may be resolved by thawing all of I believe there's no need for that, as we were talking about that a few months ago. Thanks, Rafael --
I agree that your statement is equivalent to mine. From what I can see, the current Android code resolves this by not freezing any app while a wakelock is held. Just out of curiosity, how are you detecting the situation in order to Here is hoping... Thanx, Paul --
Well, in fact I would only be able to talk about that theoretically, as I'm currently not involved in any project using cgroups for power management. I have considered that, but I haven't tried to implement it. Thanks, Rafael --
On Sun, Aug 1, 2010 at 6:10 PM, Paul E. McKenney The dependencies is what made this solution uninteresting to us. For instance, we currently use cgroup scheduling to reduce the impact of some background tasks, but we occasionally saw a watchdog restart of the system process were critical services were waiting on a kernel mutex owned by a background task for more than 20 seconds. If we froze a cgroup instead, we would not hit this particular problem since tasks cannot be frozen while executing kernel code the same way they can be preempted, but nothing prevents a task from being frozen while holding a user-space resource. -- Arve Hjønnevåg --
Excellent point -- I had completely missed this failure mode!!! Thanx, Paul --
On Mon, 2 Aug 2010 21:56:10 -0700 one of the solutions we're looking at to solve this is to unfreeze the cgroup on a regular basis (say, every 15 to 30 seconds) briefly to avoid this kind of deadlock... -- Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org --
Cute!!! ;-) If this doesn't work for the Android folks for whatever reason, another approach would be to do the freeze in user code, which could track whether any user-level resources (pthread mutexes, SysV semas, whatever) where held, and do the freeze on a thread-by-thread basis within each "victim" application as the threads reach safe points. I must of course defer to the Android folks as to whether or not this would actually work for Android. But if your approach works for them, why worry? Thanx, Paul --
The main problem I see with the cgroups solution is that it doesn't seem to do anything to handle avoiding loss of wakeup events. -- Matthew Garrett | mjg59@srcf.ucam.org --
there are two cgroup solutions in discussion at this point. weeks ago there was the suggestion to put non-privilaged tasks in a cgroup so that they could be frozen as a group. I thought the problem there was that there can be user-space dpendancies that would prevent privilaged tasks from working. a couple days ago I made the suggestion to put non-privilaged tasks in a cgroup so that the idle/suspend decision code could ignore acitivity caused by this cgroup. in the second version wakeup events would be 'activity' that would be counted and therefor the system would not be idle. As for the race with suspending and new things happening, wouldn't that be handled the same way that it is in a normal linux box? David Lang --
No! And that's precisely the issue. Android's existing behaviour could be entirely implemented in the form of binary that manually triggers suspend when (a) the screen is off and (b) no userspace applications have indicated that the system shouldn't sleep, except for the wakeup event race. Imagine the following: 1) The policy timeout is about to expire. No applications are holding wakelocks. The system will suspend providing nothing takes a wakelock. 2) A network packet arrives indicating an incoming SIP call 3) The VOIP application takes a wakelock and prevents the phone from suspending while the call is in progress What stops the system going to sleep between (2) and (3)? cgroups don't, because the voip app is an otherwise untrusted application that you've just told the scheduler to ignore. -- Matthew Garrett | mjg59@srcf.ucam.org --
Even in the current implementation (wakelocks), Since the VOIP application isn't allowed to take a wakelock, wouldn't the system go to sleep immediatly anyway, even if the application gets the packet and starts the call? What would ever raise the wakelock to keep the phone from sleeping in the middle of the call? David Lang --
There's two parts of that. The first is that the voip application is allowed to take a wakelock - but that doesn't mean that you trust it the rest of the time.The second is that the incoming network packet causes the kernel to take a wakelock that will be released once userspace has processed the network packet. This ensures that at least one wakelock is held for the entire relevant period of time. -- Matthew Garrett | mjg59@srcf.ucam.org --
why would you trust it to take a wakelock, but not trust it the rest of the time? in my proposal I'm saying that if you would trust the application to take a wakelock, you instead trust it to be sane in the rest of it's power activity (avoiding polling, etc) and so you consider it for sleep how do you determine that userspace has processed the network packet so that the kernel can release the wakelock (or is this one of the cases where there is a timer related to the wakelock) two things here, on the dirty networks that I see as common, refusing to sleep if network packets are arriving will mean that you never go to sleep. secondly, nothing stops the code doing the idle/suspend decision from considering network activity. I would be surprised if there weren't already options to support this today. David Lang --
Because I trust that when the application author says "I explicitly need the machine to stay awake" that they mean it, whereas I don't trust the application author to write an application that avoids consuming When we say "trust", we're not using the same meaning as we do with security. Yes, it's possible that an application that can block suspend will do so at inopportune times. But given that blocking suspend is an explicit act it's much more likely that the developer will only use it in reasonable ways, while it's still entirely plausible that the application will generate unnecessary wakeups. Pretending otherwise is unrealistic. I recently had to fix the fact that the kernel IPMI layer would generate a constant 1000 wakeups a second even if it had an The current implementation uses a timer, but Rafael's implementation If you proxy every potential wakeup event through some central server then this becomes much easier, but it's also a performance hit. The alternative is that you poll for network activity, but that's a power hit. -- Matthew Garrett | mjg59@srcf.ucam.org --
on the other hand, making an application avoid consuming inappropriate background resources helps everywhere. an explicit "don't let the machine sleep" only works if you are trusted by a system that implements this flag. yes, it is painful to make the change, but the end result is better (and I'm not sure I buy the distinction, but I'm not a maintainer so it's others you have to convince. it's better to have one type of problem with one set of tools that document why the machine isn't sleeping, than to have orthoginal ways of influencing power management. but if you have an application in the mid-level trust situation, go ahead and have it talk to a 'keepalive' daemon that is in the 'trusted' set and let the rest of the app run untrusted. As I noted elsewhere, the keepalive daemon would need very little in the way of resources and can implement much more complex policies than anyone is going to be willing to put in similar to the current implementation, the arrival of a packet could be but don't most new smartphones also connect up to wifi networks? those are I'm not suggesting running all events through some central server (unless you count the kernel as that server), I'm saying that the decision that the system is idle and therefor can be stopped should be able to take this infomation into account, and if there's a race here, it should be a race that exists everywhere else, so there should be a general solution, not something specific to one use-case. (and definantly not something that requires all software to be modified and trusted to implement) David Lang --
Nobody's denying that it's better, but people are saying they want to design a platform that's capable of handling the software we have rather How? You're either polling in order to know whether there's network activity, or you're having every network packet wake up the policy daemon, or you've got something that looks like the kernel side of Wifi radio typically dwarfs your other power consumption anyway. The You're right. It's a race that exists everywhere else, and it's inherent unless you have some handshaking between userspace and the kernel when entering suspend. Which is what wakelocks provide. Nobody has proposed a solution that works without modifying existing code. -- Matthew Garrett | mjg59@srcf.ucam.org --
infrequent polling (on the order of seconds or 10s of seconds) is not a on some devices (my kindle for example) the cell network dwarfs my other I think the issue is confusion between the different purposes of the wakelock. as you say, I don't think anyone is opposed to the kernel being modified to eliminate this race. The disagreement is far more about the interface to userspace and userspace use of wakelocks. the kernel implementation of this doesn't have to set a single flag, and it doesn't have to notify userspace directly. The kernel already produces many stats today and it may be appropriate for the kernel to produce some more. for example, if you want to abort the suspend because there is network activity, you can check the packet count of your network interface, decide to go to sleep, setup the network interface to raise a 'wake me up' interrupt (because you have decided in a userspace policy that you want this), and then check to see if the packet count has changed. If it has, abort the suspend, if not continue the suspend and once you are suspended if the 'wake me up' interrupt is set you will wake back up. there are probably cleaner/better ways of doing this than the simple logic that I'm listing, but why wouldn't the simple logic provide the correct result? David Lang --
If your network interrupt happens before the network driver's suspend() hook is called this works -- the check in the suspend hook observes the change and returns an abort status. If your network interrupt happens after the suspend() hook is called this does not work -- the event comes after your opportunity to abort suspend has happened, your interrupt handler processed it, set the flag, but the system proceeds to suspend anyway, missing the event. The wakelock/suspendblock mechanism avoids races like the above. Brian --
The word "trust" does not appear to be helping here. ;-) The VOIP application acquires a suspend blocker when it needs to prevent the system from suspending, and releases that suspend blocker when it can tolerate the system suspending. It is important to note that while the VOIP application holds the suspend blocker, the system won't suspend even if it is completely idle (for example, if the VOIP application uses blocking system calls, during the time that the VOIP application is There are two cases: 1. The application is permitted to acquire suspend blockers. In this case, the application would acquire a suspend blocker before reading the input. It would then read the input (at which point the kernel releases its suspend blocker), do any needed processing, and finally release the suspend blocker. So in this case, the system knows that the application is done processing the input when that application releases its suspend blocker. 2. The application is prohibited from acquiring suspend blockers. In this case, the system might well be suspended before the application has a chance to do more than read the input. But the application will get a chance to process the input I don't know about the general networking case for Android, but the example of downloading was discussed some time back. The application doing the download acquires a suspend blocker, which it releases once the download is complete (or once a timeout expires, if I remember correctly). In this particular case, the network packets were not bringing the device out of suspend. There might well be other cases where networking packets -do- bring the system out of suspend, but I must leave this to someone who knows more about Android than do I. Thanx, Paul --
In the terminology I have been using, the VOIP sofware is then trusted to take the wakelock appropriately, and I'm then saying it would be in the in my proposal, the application is trusted to take the wakelock, so it would be trusted to not use the CPU wildly inappropriatly and so it In this case the system would go ahead and suspend, but the next time the sustem wakes up for any reason, this application would continue to run and it would seem reasonable to say that if a packet arrives for an existing connection (which the kernel does know) it is considered activity for purposes of sleeping. I don't know if you would care enough to try and say that packets for untrusted apps network connections don't keep the system awake, or just allow them to (after all, keypresses going to untrusted apps do keep the this would be the normal wake-on-lan type of functionality that exists without Android. The primary thing that I was getting at was the other things above. David Lang --
... here you seem to be assuming that "trusted to properly use a wakelock" implies "coded to optimize power usage when not holding a wakelock." That would be up to the people creating the system in question. In some cases, they might (as you say) want every packet arriving to wake up the system, in other cases they might not. We should not Again, this is up to the people creating the system in question. On some Android systems, there is a particular button you have to press Although this wake-on-LAN functionality applies only to special wake-up packets, not to normal packets, right? --
I am saying that 'trusted to use a wakelock' does imply 'trusted to not waste power' I am sure that there are apps that do not manage power effectivly, but I don't expect that giving those same developers wakelock power will make them do much better. I expect that any time they discover the system going to sleep on them, they will just add another wakelock to keep it awake. If they were the type to carefully consider what was really important and what wasn't, I would expect them to write fairly power efficiant code in well, if you have the kernel take a wakelock when the packet arrives, you note that here I am talking about thigns that will keep the system awake, that's not my understanding. my understanding (which could be flawed) is that wake-on-lan programs your IP into the NIC and if the NIC sees traffic for you it will wake you up. I've never had a reason to use it, so I could easily be mistaken. David Lang --
I _think_ you can use the just-merged /sys/power/wakeup_count mechanism to avoid the race (if pm_wakeup_event() is called at 2)). Thanks, Rafael --
Yes, I think that solves the problem. The only question then is whether it's preferable to use cgroups or suspend fully, which is pretty much up to the implementation. In other words, is there a reason we're still having this conversation? :) It'd be good to have some feedback from Google as to whether this satisfies their functional requirements. -- Matthew Garrett | mjg59@srcf.ucam.org --
The issue with cgroup freezer as currently defined is that it can freeze processes that hold user-level resources (pthread mutexes, SysV semas, ...). If some non-frozen process attempts to acquire that resource, you get a hang. There might be some ways to work around this, for example, Arjan suggests momentarily unfreezing periodically, and I suggested doing the freeze in user-space code, but we don't know if either of these will really do what is required. Also, I believe that Android's use of cgroups would be in addition to suspending fully rather than instead of. Freezing a subset of the applications allows cutting power drain from output-only apps when the screen blanks but where some app such as a download needs to keep the system active. They still would need to suspend once the download completes. But we do need to hear from the Android guys on these points. Thanx, Paul --
One can argue that once the download has been completed, the cpuidle framework should make the system reduce its energy consumption to the level achievable by using suspend. However, the problem is the cpuidle framework only deals with CPUs right now (at least generally) and there's a problem of the interrupt sources that allow the monotonic clock to advance and are deactivated during suspend (which allows more energy to be saved, because periodic timers are then effectively disabled). So, it seems, system suspend is necessary to maximize energy savings as long as the cpuidle framework cannot take care of I/O devices and interrupt sources in general. Thanks, Rafael --
How? By passing a timeout to pm_wakeup_event when the network driver gets the packet or by passing 0. If you pass a timeout it is the same as using a wakelock with a timeout and should work (assuming the timeout you picked is long enough). If you don't pass a timeout it I have seen no proposed way to use cgroups that will work. If you leave some processes running while other processes are frozen you run into problems when a frozen process holds a resource that a running That is "this"? The merged code? If so, no it does not satisfy our requirements. The in kernel api, while offering similar functionality to the wakelock interface, does not use any handles which makes it impossible to get reasonable stats (You don't know which pm_stay_awake request pm_relax is reverting). The proposed in user-space interface of calling into every process that receives wakeup events before every suspend call is also not compatible with existing apps. -- Arve Hjønnevåg --
Which makes the driver and/or network stack changes identical to using Not having stats or not knowing what pm_relax is undoing? We need stats to be able to debug the system. If the system does not suspend at all or is awake for too long, the wakelock stats tells us which component is at fault. Since pm_stay_awake and pm_relax does not operate on a handle, you cannot determine how long it prevented Only if the driver blocks suspend until user-space has read the event. This means that for android to work we need to block suspend when input events are not processed, but a system using your scheme needs a pm_wakeup_event call when the input event is queued. How to you switch between them? Do we add separate ioctls in the input device to enable each scheme? If someone has a single threaded user space power manager that also reads input event it will deadlock if you block suspend until it reads the input events since you block when reading the wake -- Arve Hjønnevåg --
Arve, you say that like it is a bad thing. ;-) Seriously, the hope is that Rafael's implementation is useful to other projects in addition to Android. And, all else being equal, the more people who need a given facility, the more likely that facility is to make it to mainline, right? And yes, I see you call out some additional things that Android needs, but hopefully this gap can be closed one way or another. --
I think we're resigned to the fact that we need to indicate wakeup events in a manner that's pretty equivalent to wakelocks. The only real issue is what the API looks like. Anyone who's still talking about cgroups seems to be trying to solve a different problem. -- Matthew Garrett | mjg59@srcf.ucam.org --
Ok, it is now sounding to me like there are two different (but somewhat related) purposes that wakelocks are being used for 1. deciding if the system should go to sleep now or not (what most of the discussion has been about) 2. narrowing the race between going to sleep and wakeup events. I'm not sure it's possible to completely eliminate the race, even with wakelocks there is some time between the time you last check if the wakelock is set and when the hardware finishes responding to your commands to go to sleep (Unless you can set a level-based interrupt that will wake you up as soon as you finish going to sleep) David Lang --
The transition into sleep is race-free in the wakelock model -- either the wakeup event happens during the kernel suspend (and grabs a wakelock), in which case suspend is aborted (and not attempted again until there are once again no more wakelocks held), or the system fully suspends to its lowest power mode until a wakeup event brings it back out again. Entry to lowest power mode must abort if a wakeup event/interrupt occurs while it's in process -- exactly how the handoff happens here is hardware dependent but in practice this is doable on just about any modern system. Brian --
It's typically the case that you can - we've even seen that on x86 with ACPI GPEs. -- Matthew Garrett | mjg59@srcf.ucam.org --
You have the stats in struct device and they are available via sysfs. Well, if you need that, you can add a counter of "completed events" into struct dev_pm_info and a function similar to pm_relax() that Well, until someone actually tries to implement a power manager in user space it's a bit vague. Thanks, Rafael --
Our wakelock stats currently have (name,)count,expire_count,wake_count,active_since,total_time,sleep_time,max_time and last_change. Not all of these are equally important (total_time is most important followed by active_since), but you only have count. Also as discussed before, many wakelocks/suspendblockers are not Not having clear rules for what the drivers should do is a problem. The comments in your code seem to advocate using timeouts instead of overlapping pm_stay_awake/pm_relax sections. I find this recommendation strange given all the opposition to wakelock/suspendblocker timeouts. But more importantly, calling pm_wakeup_event with a timeout of 0 is incompatible with the android user space code, and I would prefer that the kernel interfaces would encourage drivers to block suspend until user space has consumed the event, which works for the android user space, instead of just long enough to work with a hypothetical user space power manager. -- Arve Hjønnevåg --
OK How much of that is used in practice and what for exactly? Well, that are your personal preferences, which I respect. I also have some personal preferences that are not necessarily followed by the kernel code. Thanks, Rafael --
Debugging power related issues is pretty critical to building competitive mobile devices. Throughout the several months of this discussion I have been continually scratching my head at this "debugability considered harmful" attitude that I see in reaction to our interest in having the ability (gated behind a config option even -- really, that'd be fine, not everyone need enable it) to gather useful stats and examine the state of the system. At this point it sounds like there's no interest in the solution we have, which works and has worked for a few years, and has been revised 10+ times based on feedback here, and no interest in providing a solution that accomplishes similar functionality, so perhaps it's time for us to cut our losses and just go back to maintaining our patches instead of having the same arguments over and over again. Brian --
The problem is what kind of stats would be actually sufficient and we haven't seriously discussed that. You said you'd need statistics and we took that for Please remember that I also have spent considerable amount of time trying to push your solution upstream and defending it. I might have spent that time on other things instead, so please don't tell me about "losses". If you want to stay in your ivory tower, that's perfectly fine by me, but I guess that won't really benefit anyone in the long run. Thanks, Rafael --
In my case, it has not been "debuggability considered harmful", but rather my lack of understanding of the kinds of bugs that can arise and what information is most helpful in tracking them down. Arve's post listing the meanings of the stats helped me quite a bit, although I am And this brings up another aspect of Android requirements. Mainlining Android features is not necessarily an all-or-nothing proposition. Here is a prototype list of degrees of mainlining, along with at least a few of the benefits and difficulties of each level: o Mainline that provides exactly what Android needs. This is of course the best case, but, as you may have noticed, might not be easy to achieve. The plain fact is that Linux must support a wide variety of workloads, so some give-and-take is usually required. o Mainline that exactly matches the ideal API, but which fails to implement some feature or another. This is still not bad. You have to carry a specific patch to provide the feature(s), but all the calling sites are carried upstream in mainline. o Mainline provides something that is close to the ideal API, but some additional constant arguments are required. This is not so good, but at least the drivers can be upstreamed and the changes required to get to an Android-ready kernel are fairly simple. o Mainline provides something, but Android requires changes to the supplied API which require additional data to be passed from call site to call site. Again, at least drivers can be upstreamed, but the changes required to get to an Android-ready kernel are a bit more involved. o Mainline provides a stubbed-out API. Once again, the drivers can at least be upstreamed, but Android is the only project validating the placement of calls to the API. This means that changes to the drivers are likely to mess up the placement of the call sites. Therefore, getting to an Android-ready kernel requires careful inspection of the drivers to adjust for bugs ...
count, tells you how many times the wakelock was activated. If a wakelock prevented suspend for a long time a large count tells you it handled a lot of events while a small count tells you it took a long time to process the events, or the wakelock was not released properly. expire_count, tells you how many times the timeout expired. For the input event wakelock in the android kernel (which has a timeout) an expire count that matches the count tells you that someone opened an input device but is not reading from it (this has happened several times). wake_count, tells you that this is the first wakelock that was acquired in the resume path. This is currently less useful than I would like on the Nexus One since it is usually "SMD_RPCCALL" which does not tell me a lot. active_since, tells you how long a a still active wakelock has been active. If someone activated a wakelock and never released it, it will be obvious here. total_time, total time the wake lock has been active. This one should be obvious. sleep_time, total time the wake lock has been active when the screen was off. max_time, longest time the wakelock was active uninterrupted. This used less often, but the battery on a device was draining fast, but the problem went away before looking at the stats this will show if a Wake locks in drivers sometimes need to be debugged. If the api has no Why not? I think allowing drivers to modify a global reference count I'm referring to this paragraph: * Second, a wakeup event may be detected by one functional unit and processed * by another one. In that case the unit that has detected it cannot really * "close" the "no suspend" period associated with it, unless it knows in * advance what's going to happen to the event during processing. This * knowledge, however, may not be available to it, so it can simply specify time * to wait before the system can be suspended and pass it as the second -- Arve Hjønnevåg --
This one seems a little odd, and won't make sense on some devices like those with E Ink displays or those which don't have screens at all. I guess it's really focused on metering when the user is not actively interacting with the device? --
Ah, I hadn't considered that an power-oblivious application could take this approach to waste power. I now better understand the purpose of Hmmm... The statistics for apps are collected in userspace, correct? If so, this would be collecting the total time that suspend blockers that were acquired by some driver have been held while the screen was powered off. So if things were working properly, sleep_time should be quite small -- with the exception of the suspend blocker that the userspace daemon was using as a proxy for all of the userspace suspend blockers. Got it! Thank you for this additional info, it really shines some light on what you are trying to do. One question: are these statistics enabled in production devices, are So there is a separate set of suspend-blocker statistics collected in userspace, correct? (The kernel probably doesn't need to care about this, and I don't believe that I need anything more than a "yes" or "no" answer, The reason you want the handle is to allow the calling code to indicate which calling points are dealing with the same suspend-blocker entity? --
I should have asked this earlier... What exactly are the apps' compatibility constraints? Source-level APIs? Byte-code class-library invocations? C/C++ dynamic linking? C/C++ static linking (in other words, syscall)? Thanx, Paul --
For Java/Dalvik apps, the wakelock API is pertty high level -- it talks to a service via RPC (Binder) that actually interacts with the kernel. Changing the basic kernel<->userspace interface (within reason) is not unthinkable. For example, Arve's suspend_blocker patch provides a device interface rather than the proc interface the older wakelock patches use. We'd have to make some userspace changes to support that but they're pretty low level and minor. In the current model, only a few processes need to specifically interact with the kernel (the power management service in the system_server, possibly the media_server and the radio interface glue). A model where every process needs to have a bunch of instrumentation is not very desirable from our point of view. We definitely do need reasonable statistics in order to enable debugging and to enable reporting to endusers (through the Battery Usage UI) what's keeping the device awake. Brian --
Thank you for the info! Thanx, Paul --
the proposal that I nade was not to use cgroups to freeze some processes and not others, but to use cgroups to decide to ignore some processes when deciding if the system is idle, stop everything or nothing. cgroups are just a way of easily grouping processes (and their children) into different groups. David Lang --
That does not avoid the dependency problem. A process may be waiting on a resource that a process you ignore owns. I you ignore the process that owns the resource and enter idle when it is ready to run (or waiting on a timer), you are still effectively blocking the other process. -- Arve Hjønnevåg --
and if you don't have a wakelock the same thing will happen. If you expect the process to take a while you can set a timeout to wake up every 30 seconds or so and wait again, this would be enough to prevent you from going to sleep (or am I misunderstanding how long before you go into suspend without a wakelock set, see my other e-mail for the full question) David Lang --
The difference between the Android scheme and your proposal is that the Android scheme freezes -all- the processes, not just a subset of them. Therefore, in the Android scheme, the case of one process attempting to acquire a resource held by a frozen process. In contrast, any scheme that attempts to freeze only a subset of the processes must somehow either avoid or properly handle the situation where a frozen process is holding a resource that a running process is trying to acquire. Thanx, Paul --
One suggestion: Get an Android Phone, test your ideas on it, then comment. --
I actually have played with an Android phone. ;-) Thanx, Paul --
My proposal would never freeze a subset of processes. what my proposal: only consider the activity of a subset of processes when deciding if we should suspend or not. If the decision is to suspend, freeze everything. you (and many other people) are confusing what I've proposed (use cgroups to indicate what processes to care about and what ones to not care about when deciding to suspend/go to idle) with the prior cgroup proposal (use cgroups to freeze a subset of tasks while leaving others runnable) David Lang --
That alone doesn't allow you to handle the race Matthew was referring to (ie. wakeup event happening right after you've decided to suspend). A mechanism of making a decision alone is not sufficient, you also need a mechanism to avoid races between wakeup events and suspend process. Thanks, Rafael --
I thought you just posted that there was a new feature that would be able to abort the suspend and so that race was closed. David Lang --
Yes, you can use that for this purpose, but then you'd need a user space power manager who would decide whether or not to suspend. Then, however, the problem boils down to setting up appropriate communication between the power manager and the other applications in user space (ie. the kernel doesn't need to be involved in that at all). Thanks, Rafael --
This race sounds like it's generic across all platforms and not an Android specific problem. Android is just more sensitive to it as they do a suspend more frequently. one thought on this, as a generic solution to the problem would it be possible for the suspend controller (whatever it is) to do the following 1. decide to suspend 2. setup monitors for the wake events 3. double check if it still wants to suspend this way you don't pay the overhead of the wake monitors while you are running normally, but if one hits while you are suspending you wake up again as quickly as you can (which could involve aborting the suspend and backing out, or going fully into suspend and waking up immediatly, depending on which is better/easier at the time you get the wakeup) David Lang --
The decision on whether or not to go to sleep isn't the difficult bit of this problem space. -- Matthew Garrett | mjg59@srcf.ucam.org --
but isn't that all that wakelocks do? affect the decision on whether or not to go to sleep. David Lang --
You could think of them that way, but it's not the useful aspect of them - that much could be implemented entirely in userspace. Wakelocks provide a mechanism for userspace to ensure that it's handled all received events before a system suspend takes place. -- Matthew Garrett | mjg59@srcf.ucam.org --
For userspace or the kernel -- some events may not require userspace intervention, but do require the kernel to stay awake long enough to finish chewing on them. Say perhaps a wifi irq comes in, the wifi driver/stack needs to process some beacon packets or whatnot. Brian --
On Wed, 4 Aug 2010 16:10:03 -0700 No no. In the David-Lang-CGroup-Scheme[1](tm?) suspend-from-idle is used. For idle decision a certain subset of tasks is ignored. Where suspend is prevented by the trusted process in android-world taking a wakelock, here it would just prevent the system from going idle by arming timers. This would be pretty equivalent to the suspend-blocker scheme and not introduce new userspace api. But the downside is, as Arve pointed out, that now one can not get full-idle-power-leverage while suspend is blocked. [1] http://permalink.gmane.org/gmane.linux.kernel/1018452 and following Cheers, Flo --
On Thu, 5 Aug 2010 07:33:59 +0200 There are a few downsides that got mentioned already in reponse.. I got a little lagged behind. There are upsides to this approach like not having a special purpose userspace api, conceptually integrating suspend into the idle mechanism .. Short summary of the cons that got mentioned: - applications need to resort to polling to keep the system out of idle (-> system will never be fully idle) - the race between deciding to suspend and becoming active again is not handled - no special statistics available - the timers of the ignored applications will behave unexpected (as the monotonic clock is not stopped)... while applications have already to cope with network-loss, other side effects of suspend without monotonic clock stopped are to be expected... Cheers, Flo --
true, although the power difference between a system that is completelyidle (but with a wavelock held) and a system where one this one I will admit to not fully understanding, it sounds like there are or no special statistics needed. Powertop is already avaible to analyse a system and report what processes are keeping it awake. Why would this not I'm not sure this is true. there's nothing stopping the suspend (when it finally does happen) from stopping the monotonic clock. The core of my proposal is tweaking how we decide to suspend. Other than the possibility that we decide to suspend between timers (and set an alarm to wake up when a timer would go off) I'm not suggesting that the suspend itself change once it's finally triggered. David Lang --
Not the same thing. If you don't hold a wakelock the entire system will suspend and when it wakes up it continues where it left off. I don't think polling is an acceptable solution to this problem. You user space code know needs to know what "idle" timeout you have selected so it can choose a faster poll rate. When is it safe to stop We suspend as soon as no wakelocks are held. There is no delay. -- Arve Hjønnevåg --
in what I'm proposing, if the 'privilaged/trusted" processes are idle long enough the entire system will suspend, and when it wakes up everything I think the timeouts are of such an order of magnatude that the polling can be infrequent enough to not be a significant amount of load, but be So, if I have a bookreader app that is not allowed to get the wakelock, and nothing else is running, the system will suspend immediatly after I click a button to go to the next page? it will not stay awake to give me a chance to read the page at all? how can any application run without wakelock privilages? David Lang
Isn't a wakelock held as long as the display is lit, so that the system would continue running as long as the page was visible? Thanx, Paul --
what holds this wakelock, and what sort of timeout does it have? (and why could that same timeout be used in other ways instead) how many apps really need to keep running after the screen blanks? there are a few (audio output apps, including music player and Navigation directions), but I don't have see a problem with them being marked as the 'trusted' apps to pay attention instead. if the backlight being on holds the wakelock, it would seem that almost every other use of the wakelock could (and probably should) be replaced by something that tickles the display to stay on longer. David Lang --
I defer to the Android guys on what exactly holds the display's Downloading is another. The music player is an interesting example. It would be idle most of the time, given that audio output doesn't consume very much CPU. So you would not want to suspend the system just because there were no runnable processes. In contrast, allowing the music player to hold a wake lock lets the system know that it would not be appropriate to suspend. The problem with this approach is that the display consumes quite a bit of power, so you don't want to leave it on unnecessarily. So if the system is doing something (for example, playing music) that does not require the display, you really want the display to be off. Thanx, Paul --
this is definantly an interesting case, do you want an active network connection to keep the machine awake? if so do you want it for all network the system would need to be idle for 'long enough' (configurable) before deciding to suspend, so as long as 'long enough' is longer than the music what percentage (and types) of apps are really useful with the display off. I think that there are relativly few apps that you really want to keep running if the display is off. David Lang --
The Android approach is that everything is permitted to run when the From a user standpoint, having the music player tell the system when it is OK to suspend (e.g., when the user has paused playback) seems The length of time those apps are running is the governing factor for battery life, and not the number of such apps, right? Thanx, Paul --
I wasn't suggesting freezing some network connections while letting others run, I was thinking in terms of 'if the system patcher daemon is downloading, don't go to sleep, but if it's dancing cows (or mail client to an IMAP server) go ahead and sleep, even if there is some data passing every system that I have seen has a configurable "sleep if it's idle for this long" knob. On the iphone (work issue, I didn't want it) that I am currently using it can be configured from 1 min to 5 min. this is the sort of timeout I am talking about. with something in the multi-minute range for the 'do a full suspend' doing correct, but the number of such apps indicates the scope of the problem. From another e-mail tonight it sounds like almost everything already talks to a userspace daemon, so if "(the power management service in the system_server, possibly the media_server and the radio interface glue)" (plus possibly some kernel activity) are the only things looked at when considering if it's safe to sleep or not, all of these can (or already do) do 'something' every few seconds, making this problem sound significantly smaller than it sounded like before. Android could even keep it's user-space API between the system power daemon and the rest of userspace the same if they want to. over time, additional apps could be considered 'trusted' (or flagged that way by the user) and not have to interact with the power daemon to keep things alive. as for intramentation, the key tool to use to see why a system isn't going to sleep would be powertop, just like on other linux systems. David Lang --
Ah, I was assuming -much- shorter "do full suspend" timeouts. My (possibly incorrect) assumption is based on the complaint that led to my implementing RCU_FAST_NO_HZ. A (non-Android) embedded person was quite annoyed (to put it mildly) at the earlier version of RCU because it prevented the system from entering the power-saving dyntick-idle mode, not for minutes, or even for seconds, but for a handful of -milliseconds-. This was my first hint that "energy efficiency" means something completely different in embedded systems than it does in the servers that I am used to. But I must defer to the Android guys on this -- who knows, perhaps The number of such apps certainly indicates the amount of effort required Hmmm... Isn't it the "trusted" (AKA PM-driving) apps that interact with Powertop is indeed an extremely valuable tool, but I am not certain that it really provides the information that the Android guys need. If I understand Arve's and Brian's posts, here is the scenario that they are trying to detect: o Some PM-driving application has a bug in which it fails to release a wakelock, thus blocking suspend indefinitely. o This PM-driving application, otherwise being a good citizen, blocks. o There are numerous power-oblivious apps running, consuming significant CPU. What the Android developers need to know is that the trusted application is wrongly holding a wakelock. Won't powertop instead tell them about all the power-oblivious apps? Thanx, Paul --
if the system was looking at all applications I would agree that the timeout should be much shorter. I have a couple devices that are able to have the display usable, even if the CPU is asleep (the OLPC and the Kindle, two different display technologies). With these devices I would like to see the suspend happen so fast that it can suspend between keystrokes. however, in the case of Android I think the timeouts have to end up being _much_ longer. Otherwise you have the problem of loading an untrusted book reader app on the device and the device suspends while you are reading the page. currently Android works around this by having a wakelock held whenever the display is on. This seems backwards to me, the display should be on because the system is not suspended, not the system is prevented from suspending because the display is on. Rather than having the display be on causing a wavelock to be held (with the code that is controls the display having a timeout for how long it leaves the display on), I would invert this and have the timeout be based on system activity, and when it decides the system is not active, turn off I was looking at it from a kernel point of view, "trusted" (AKA PM-driving) apps are ones that have permission to grab the wakelock. Any app/daemon that is so trusted can communicate with anything else in userspace as part of making it's decision on whento take the wakelock, but in my proposal (without a wakelock), powertop would tell you what applications are running and setting timers. If we can modify the kernel/suspend decision code to only look at processes in one cgroup when deciding if the system should go to sleep, a similar modification to poewrtop should let you only show stats on the "trusted" applications. If you have a userspace power management daemon that accepts requests from untrusted programs and does something to keep the system from sleeping (either taking a wakelock or setting a 'short' timer), it needs ...
From what I can see, the decision between these two approaches comes down to their energy efficiencies, and thus their battery lifetimes. Are you So you are saying that PM-driving apps can check up on power-oblivious apps as part of their decision process. Sounds reasonable to me, though such checking increases the dependencies among apps, which might Mark Brown suggests adding suspend-blocker information to powertop, which might well be a very good way to handle this. This sounds plausible to me, but then again, I have never chased down wakelock bugs on Android systems. A key point is that it is not enough to focus on PM-driving apps, you instead need to focus on only those PM-driving apps that currently hold According to Brian Swetland, Android does in fact have such a power-management daemon. I would guess that this daemon tracks which apps it is holding suspend blockers on behalf of. This approach might well make it harder to bring powertop to bear, as powertop would need to communicate with Android's power-management daemon. But perhaps something could be arranged. Thanx, Paul --
no, I'm not in a position to benchmark them against each other. I am claiming that (in this area) the two are equivalent (or at lease very close to equivalent given the approximate magnatude of the timeouts that are involved) 1. a daemon controls the screen, monitors input and holds a wakelock until a configurable timeout after input, after which the system sleeps 2. a daemon monitors input, it wakes up every few seconds (to 10s of seconds) when there is no input for a configurable timeout the daemon sleep without waking up periodically. when the daemon sleeps for longer than a configurable period the system is considered idle and goes to sleep. in both cases the system will be awake as long as there is input within the initial timeout period. the second case could eata smidge power as there can be an additional lag before the system sleeps (the second timeout), that depends on exactly how more to the point, I'm saying that that's how it works currently with wavelocks. the unprivilaged processes can ask the privilaged process to if you have wavelocks, then you just need info on the use of wavelocks with my proposal staying awake is based on process activity, which is exactly what powertop is monitoring now, it just needs to be able to My expectation is that instead of modifying apps to talk to the power management daemon, the apps would be classified as 'trusted' or 'untrusted' more appropriatly. if user input is activity that keeps the system alive, then only applications that will run for extended period with no user input need to be trusted. David Lang --
IIRC, this was a major point of their (Android's) power management policy. User input of any kind would reset the "display active" timeout, which is the primary thing keeping random untrusted user-facing programs from being suspended while in use. They seemed to consider this to be a special case in their policy, but from the kernel's point of view it is just another suspend blocker being held. I'm not sure this is the best use case to look at though, because since it is user-facing, the timeout durations are on a different scale than the ones they are really worried about. I think another category of use case that they are worried about is: (in suspend) -> wakeup due to network -> process network activity -> suspend or an example that has been mentioned previously: (in suspend) -> wakeup due to alarm for audio processing -> process batch of audio -> suspend In both of these cases, the display may never power on (phone might beep to indicate txt message or email, audio just keeps playing), so the magnitude of the "timeout" for suspending again should be very small. Specifically, they don't want there to be a timeout at all, so as little time as possible time is spent out of suspend in addition to --
The display being on should not prevent suspend unless the particular Yeah, we much prefer the wakelock model where the moment the resource (system-needing-to-not-be-suspended) is released we return to the lowest power state, rather than waiting for a timer to expire (and/or burning cycles polling for "can we suspend yet?" when we're definitely not ready to suspend). Brian --
if that is the case, I'll go back to my question about the untrusted reader app. what is it that will prevent the system from going to sleep while I am at the micro level it is definantly better for power, but at the macro level I don't think it needs to make much difference in the overall battery life of the device. To borrow a phrase, I think you may be into premature optimization of power :-) David Lang
when you suspend the audio will shut off, so it's sleep ->wake -> sleep, it really depnds on the frequency of the wakeups. if you get a network packet once every 5 min and need to wake to process it, staying awake for 20 seconds after finishing procesing is FAR more significant than if you get a network packet once every hour. It's not just the factor of 20 that simple math would indicate because the time in suspend eats power as well. I don't know real numbers, so these are made up for this example if suspend (with the cell live to receive packets) is 10ma average current and full power is 500ma average current packets every 5 min with .1 sec wake time will eat ~13maH per hour packets every 5 min with 10 second wake time will eat ~37maH per hour packets every hour with .1 sec wake time will eat ~10maH per hour packets every hour with 10 sec wake time will eat ~11maH per hour so if you have frequent wakeups, staying awake 100 times as long will cut your battery life to 1/3 what it was before. if your wakeups are rare, it's about a 10% hit to stay awake 100 times as long. there is a lot of room for tuning the timeouts here. David Lang --
Especially given different scenarios, for example, audio playback when the device is in airplane mode. ;-) Thanx, Paul --
hmm, I've been thinking and talking in terms of two classes of cgroups, trusted and untrusted. I wonder if it would be possible to set timeouts for each cgroup instead) the system would go to sleep IFF all cgroups have been idle longer than the idle time (with -1 idle time being 'ignore this cgroup') if this could be done you could set longer times for things designed for user-interaction than you do for other purposes. you could set media to 0 idle time (so that as soon as it finishes processing the system can sleep until the next timer) to do this, the code making the decision would have to be able to find out the following fairly cheaply. 1. for this cgroup, what was the last time something ran 2. for this cgroup, what is the next timer set it would be nice to get network traffic/connection stats. so two questions. first, what else would you need to get accumulated for the cgroup second, is there a fairly easy way to have these stats available? for the 'last time it ran' stat, this seems like you could have a per-cpu variable per cgroup that's fairly cheap to update, but you would need to take a global lock to read accuratly (the lock may be expensive enough that it's worth trying to read the variables from the other cpu without a lock, just to see if it's remotely possible to sleep/suspend) with timers, is it possible to have multiple timer wheels (one per cgroup)? David Lang --
I apologize in advance for what I am about to write, but... If you continue in this vein, you are likely to make suspend blockers look very simple and natural. ;-) Thanx, Paul --
if that's the case then they should be implemented :-) on the other hand, this may be something that's desirable for idle-low-power as well. David Lang --
It would be good to get some sort of range for the "timeout". In the audio-output case, my understanding that the spacing between bursts of audio-processing activity is measured in some hundreds of milliseconds, in which case one would want the delays until suspend to be on the millisecond scale. But does Android really suspend between bursts of audio processing while playing music? Very cool if so! ;-) --
On Thu, Aug 5, 2010 at 3:31 PM, Paul E. McKenney Oops, yea that's actually a really bad example, that's probably something that would be handled by low-power states. I think the incoming text message example is a good one though. There seemed to be a focus on user-interaction scale time scales, and I wanted to point out that there are also very short duration time scales to consider as well. *back to lurking* --
good point, but I do think the short time scales are less common than people think. I'd love to get good examples of them on my iphone when a text message arrives the phone displays an alert for user-interaction times (it even lights the display to show who the message is from, and optionally a preview of the message) so what would wake a phone up from suspend where the phone should go back to sleep in under a second? David Lang
Here are some real-world examples from shipped android devices: - battery gauging happens every 10 minutes, need to wake long enough to chatter with the 1w interface and make sure the battery is not exploding - always on mail/im/calendar/etc sync often has network events that happen every 5-10 minutes which cause devices to briefly wake up and return to sleep - gps tracker app might wake every couple minutes or every n gps events to log location - low power audio subsystems can wake you up every 1-4 seconds (pcm) or 1-4 minutes (mp3) to fetch more data Brian --
Interesting! So for an mp3 playback, does an Android suspend between data fetches? Thanx, Paul --
On Thu, Aug 5, 2010 at 4:03 PM, Paul E. McKenney It can if the latency is long enough (which is why I point out low power audio which is usually high latency). For low latency (system sounds, etc) 10-25ms between buffers it's not practical to fully suspend but we will go to the lowest power state in idle if possible. Brian --
the playback is able to continue even with all the clocks stopped? that surprises me. I would hav expected it to be able to sleep while playing audio, but not do a full suspend. David Lang --
Obviously not all clocks are stopped (the DSP and codec are powered and clocked, for example), but yeah we can clock gate and power gate the cpu and most other peripherals while audio is playing on a number of ARM SoC designs available today (and the past few years). Brian --
does this then mean that you have multiple variations of suspend? for example, one where the audio stuff is left powered, and one where it isn't? David Lang
While the cpu (and the bulk of the system) is suspended, it's not uncommon for some peripherals to continue to operate -- for example a cellular radio, gps, low power audio playback, etc. Details will vary depending on the SoC and board design. It's not so much a different suspend mode (the system is still suspended), just a matter of whether a peripheral can operate independently (and if it is lower power for it to do so). Brian --
this helps, but isn't quite what I was trying to ask. on a given piece of hardware, does suspend always leave the same peripherals on, or do you sometimes power more things down than other times when suspending? David Lang
Different bits of hardware get powered down depending on current system state. In the audio case (which is so far as I know the only case for this sort of stuff that currently does anything in mainline) we'll keep alive any active paths (that is, paths carrying live audio) between endpoints in the audio subsystem which have been explicitly marked as staying alive during suspend. Other audio paths will be powered down when the system suspends. During normal run time only paths that are active will be powered up. --
This was the core of the issue I was raising in the last thread about this (the one following the rename to suspend blockers). Essentially what happens in a mainline context is that some subsystems can with varying degress of optionality ignore some or all of the instruction to suspend and keep bits of the system alive during suspend. Those that stay alive will either have per subsystem handling or will be outside the direct control of the kernel entirely (the modem is a good example of the latter case in many systems - in terms of the software it's essentially a parallel computer that's sitting in the system rather than a perhiperal of the AP). --
This underscores a basic difference between servers and these embedded devices. When you suspend a server, it is doing nothing, because servers rely very heavily on the CPUs. In contrast, many embedded devices can perform useful work even when the CPUs are completely powered down. Thanx, Paul --
Well, not really from the Linux point of view. It's not massively different to something like keeping an ethernet controller sufficiently alive to allow it to provide wake on LAN functionality while the system is suspended in terms of what Linux has to do, and quite a few servers have lights out management systems which aren't a million miles away from the modem on a phone in terms of their relationship with the host computer. --
The wake-on-LAN and the lights-out management systems are indeed interesting examples, and actually pretty good ones. The reason I excluded them is that they don't do any application processing -- their only purpose is the care and feeding of the system itself. In contrast, the embedded processors are able to do significant applications processing (e.g., play back music) while any CPUs are completely shut down and most of the memory is powered down as well. Thanx, Paul --
one other significant issue is that on the PC, things like wake-on-LAN, lights out management cards, etc require nothing from the main system other than power. If they do something, they are sending the signal to the chipset, which then wakes the system up. they don't interact with the main processor/memory/etc at all. So as I see it, we need to do one of two things. 1. change the suspend definition to allow for some things to not be suspended or 2. change the sleep/low-power mode definition to have a more standardized way of turning things off, and extend it to allow clocks to be turned off as well (today we have things able to be turned off, drive spin-down for example, but per comments in this thread it's all one-off methods) to me #2 seems the better thing to do from a design/concept point of view David Lang --
Guys, please try to cut unneeded text from the quotes - it makes it much This isn't a particularly meaningful distinction, things like the LoM systems on servers are generally at least as capable as things like the DSPs doing tasks like offloaded MP3 decode and often provide useful services in themselves (like system monitoring). It's really just semantics to treat them differently to something like a cellular modem - at a high level they're both just independant processors ticking away I don't see that it makes much difference what gets kept alive - at the end of the day the point is that we're making a decision to keep bits of Currently things like clock trees are frequently managed orthogonaly to the system power state to at least some extent anyway - for example, perfectly normal wake events like button presses will often require clocks for things like debouncing. --
I agree that a smartphone's cellular modem can be argued to be very similar to wake-on-LAN. The smartphone applications that seem to me to be very different from wake-on-LAN are things like audio playback, where the system is providing service to the user during the time that The distinction is whether or not the system is perceived to be actively doing something useful while it is suspended. Yes, this is subjective, The time-of-day clock is certainly a case in point here. ;-) --
The cellular modem case includes not just hanging off the network but also being on a call - the voice path for a phone call doesn't need the CPU to do anything. It's probably best to view a phone as a bunch of interconnected systems that happen to sit in the same box, and there's various design decisions that can be taken about which systems own the shared components. --
OK, apologies, I thought you were talking about the wait-for-a-call case. If there actually is a call ongoing, then the user perceives the system as doing something, so this is similar to audio playback and quite different from wake-on-LAN or system monitoring. Thanx, Paul --
I recognise that #1 is essentially what Android is already doing. I'm asking the question, "Is this what Linux should be doing? Personally, I think that suspend should be treated much more like a low-power state and much less like hibernation than it currently is (I believe that Linus has also voiced this opinion). And I think that the situation with Android suspending while audio is playing between busts of CPU activity is a perfect example. for the moment, forget the problem of other apps that may be running, and consider a system that's just running a media player. the media player needs bursts of CPU to decode the media so that the output device can access it (via DMA or something like that) the media player needs bursts of I/O to read the encoded program source from storage. What we want to have happen in an ideal world is when the storage isn't needed (between reads) the storage should shutdown to as low a power state as possible. when the CPU isn't needed (between decoding bursts) the CPU and as much of the system as possible (potentially including some banks of RAM) should shutdown to as low a power state as possible. today there are two ways of this happening, via the idle approach (on everything except Android), or via suspend (on Android) Given that many platforms cannot go to into suspend while still playing audio, the idle approach is not going to be able to be eliminated (and in fact will be the most common approach to be used/deugged in terms of the types of platforms), it seems to me that there may be a significant amount of value in seeing if there is a way to change Android to use this approach as well instead of having two different systems competing to do the same job. David Lang --
Most other devices use a lot more power at idle; in some cases it's because the hardware just isn't as power optimized (why bother, when you have 94,000 mWh of power at your disposal with a 6 cell laptop battery, as opposed to the 800-1000 mWh that you might have on a cell phone battery). In other cases, it's because the kernel and the low-level software stack (never mind the applications) are waking up the CPU too darned often --- in other words, idle simply isn't idle enough. So you may want to consider whether part of the problem is that general purpose Linux systems need a radical redesign to get power utilization down to those sorts of levels --- where the CPU might only be waking up once every half-hour or so, and then only do actual useful work. Can you get there by assuming that every single application is competently written? In an idle approach, you have to. That way lies Maemo, where installing just one bad application will cut your battery life time by a factor of 2-3. You could try stopping processes by using kill -STOP, but this at that point, you've moved into Android strategy of "suspend". And the only question is what is the most efficient way to allow the system to run when there is true work that needs to be done, and how to avoid deadlocks by stopping processes that might be holding user space locks --- and to administer when and how to suspend the processes. Sure, you could do someting amazing complicated using cgroups, and user space processes that have to wake up the CPU every 30 seconds to see if it's safe to suspend the system --- but why not just use the system which is being used by 200,000 new phones every day? It's simple and it works. And unlike Maemo, untrustworthy applications don't end up chewing up your battery lifetime. - Ted --
Exactly the same happens on Android. Install one bad application, a request for PM permissions is made, you No. You are assuming that PM permissions will be magically set appropriately; that's not the case. 1) Install a bad application that requests PM permissions and is granted those In this case you've gained nothing with user-space suspend blockers. 2) Install a good application that is not requesting PM permissions (or is denied them) Imagine this application requires expensive initialization, but afterwards it only needs to send a small packet each minute to a server. If this application forgets to request PM permission (or is denied them), then it might miss the 1 minute mark, and would have to re-initialize when a trusted app starts some work. In this case suspend blockers cause more battery drain (and the application to behave poorly). There's only one case when suspend blockers might actually help: a) The application is badly written (most probably a UI app drawing when it shouldn't, which incidentally is not a problem in Android since it's not multi-tasking) b) The application is either not requesting, or denied PM permissions c) The application is not affected too badly if it's not running all the time (piggy-backing on trusted apps is ok) For that it's much better to have a different strategy: all applications are trusted, bad applications must be manually tagged. Also, instead of having each and every user figure out which are the bad apps, such information is more pertinent in the app store, where users could vote when an app is bad PM-use or not; crowd-sourcing the problem. But guess what; once you have an app store system that can detect when an application is badly written, why would you expose the users to such bad apps? In Maemo, 3rd party applications have to go trough a community review (maemo.org extras testing) before they go into the main repository, and this way bad applications (functionality or PM-wise) don't make it to the ...
It's clearly possible for a pathological Android application to destroy the power management policy. But to do that, the author would have to explicitly take a wakelock. That's difficult to do by accident. The various failure modes that exist in a non-wakelock world can be triggered in a wide variety of ways by accident. A sufficiently reductionist viewpoint will equate the two situations, but in the real world they're clearly different. -- Matthew Garrett | mjg59@srcf.ucam.org --
The writer can take a wakelock the whole time the application is running (isn't that the typical case?), because perhaps the author realizes that way the application works correctly, or he copy-pasted it from somewhere else. -- Felipe Contreras --
No, that's not the typical case. -- Matthew Garrett | mjg59@srcf.ucam.org --
That would be exceptionally unusual. A more common case is that the application will take a wakelock while performing some specific long running task which needs no user intervention such as downloading a file or displaying constantly update status that the user is not expected to respond to. There's no need for applications to take wakelocks while the user is directly interacting with them since the system will be kept awake as a result of the user interaction, the wakelocks are used to override the default suspend that occurs when the user is not interacting with the device.--
On Sun, Aug 8, 2010 at 9:34 PM, Mark Brown Fair enough, but if that the case, if suspend blockers are to be used in desktop software, everything would need extensive modifications just to work. I remember somebody said that was not the case, I thought it was because the lock could be held the whole time the application is running. -- Felipe Contreras --
Unfortunately, the criteria for "not being needed" are not really There is a fundamental obstacle to that, though. Namely, the Android developers say that the idle-based approach doesn't lead to sufficient energy savings due to periodic timers and "polling applications". Technically that boils down to the interrupt sources that remain active in the idle-based case and that are shut down during suspend. If you found a way to deactivate all of them from the idle context in a non-racy fashion, that would probably satisfy the Android's needs too. Thanks, Rafael --
if you can ignore the activity caused by the other "unimportant" processes in the system, why is this much different then just the one process polling applications can be solved by deciding that they aren't going to be allowed to affect the power management decision (don't consider their CPU useage when deciding to go to sleep, don't consider their timers when well, we already have similar capibility for other peripherals (I keep pointing to drive spin down as an example), the key to avoiding the races seems to be in the drivers supporting this. the fact that Android is making it possible for suspend to selectivly avoid disabling them makes me think that a lot of the work needed to make this happen has probably been done. look at what would happen in a suspend if it decided to leave everything else on and just disable the one thing, that should e the same thing that happens if you are just disabling that one thing for idle sleep. David Lang --
But isn't the whole point of wakelocks to permit developers to easily and efficiently identify which processes are "unimportant" at a given point in time, thereby allowing them to be ignored? I understand your position -- you believe that PM-driving applications should be written to remain idle any time that they aren't doing something "important". This is a reasonable position to take, but it is also reasonable to justify your position. Exactly -why- is this better? Here is my evaluation: o You might not need suspend blockers. This is not totally clear, and won't be until you actually build a system based on your design. o You will be requiring that developers of PM-driving applications deal with more code that must be very carefully coded and validated. This requirement forces the expenditure of lots of people time to save a very small amount of very inexpensive memory (that occupied by the suspend-blocker code). Keep in mind that there was a similar decision in the -rt kernel. One choice was similar to your proposal: all code paths must call schedule() sufficiently frequently. The other choice was to allow almost all code paths to be preempted, which resembles suspend blockers (preempt_disable() being analogous to acquiring a suspend blocker, and preempt_enable() being analogous to releasing a suspend blocker). Then as now, there was much debate. The choice then was preemption. One big reason was that the choice of preemption reduced the amount of real-time-aware code from the entire kernel to only that part of the kernel that disabled preemption, which turned out to greatly simplify the job of meeting aggressive scheduling-latency goals. This experience does add some serious precedent against your position. So, what do you Agreed, and the focus is on how one decides which applications need to be considered. After all, the activity of a highly optimized audio-playback application looks exactly like that of a stupid polling application -- they both ...
the issue isn't avoiding the memory useage, the issue is avoiding the special API requirement that make the userspace code no longer be portable. note that there are a lot of battery powered embedded devices out there that work just fine without wakelocks. They are able to use the existing idle/sleep and suspend options to get good battery life. The key difference is that Android allows other programs to be loaded on the system, and the current idle/sleep/suspend triggers can't tell the for one thing, there was never any thought that any code that would have to have preempt written would ever run anywhere else other than inside the linux kernel. If you had proposed that userspace be allowed to do preempt_enable/disable calls, it would have been a very different discussion. In the case of real-time applications, we require that things that are given real-time priority be carefully coded to behave well, and that if they depend on things that are not given real-time priority they may not behave as expected. Priority Inheritance is a way to avoid complete system lockup in many cases, but it would still be possible for a badly written real-time app to kill the system if it does something like go into a busy-loop waiting for a file to be created by a non-real-time process. wakelocks are like implementing real-time by allowing userspace to issue preempt_disable() calls to tell the scheduler not to take the CPU away from them until they make a preempt_enable() call. In addition wakelocks cannot replace the need to write efficient code. all that wakelocks do is to prevent the system from doing a suspend, you still want to have the code written to not do unneccessary wakeups that would prevent you from using the low-power modes other than suspend. On the other hand, it _is_ possible for the idle/sleep states to be extended to I want the kernel to be explicitly told that this application is important (or alternativly that these other applications ...
There certainly are such devices, but their power-optimized software is highly non-portable, so I fail to see how this example can possibly support your position. In addition, there are quite a few non-portable Linux extensions to the user-mode API, so your point would not carry in But the suspend blockers can, to the Android guys' point. And something needs to tell the difference. It is not helpful for you to try to hide Portability is for the common-case power-oblivious applications. Even on Android, the power-oblivious applications do not need to use suspend blockers. Suspend blockers are instead used by PM-driving and power-optimized applications. And as you yourself pointed out in an earlier email, the PM-driving and power-optimized applications are in the minority. Furthermore, the suspend-blocker approach allows the bulk of the code in a PM-driving application to be written in a power-oblivious manner, greatly easing its implementation while still providing power-efficient operation. There have been proposals that userspace be allowed to disable preemption, and the proposals indeed did not get far. However, the Linux user-kernel API -was- extended to accommodate the underlying need, namely with futexes. And futexes provide an excellent example of a non-portable extension to the Linux user-kernel API. And therefore real-world applications often are designed to minimize the amount of code that needs real-time privileges, exactly because real-time code is harder to develop than is non-realtime code. In addition, a number of Linux-specific facilities have been used to mitigate the effects of bad behavior by real-time applications. A similar effect is making itself felt in the power-efficiency arena -- apps minimize the amount of code that must obey the PM-driving rules. Android suspend blockers are one mechanism to carry this minimization further. Easing development of code is something you would do well to take more Just as it is possible for a ...
I really don't know the answer myself, so I was really asking the question rather than trying to catch you out. --
Right, and this isn't just information for developers - Android handsets expose this information to end users (so they can indentify any badly behaved applications they have installed or otherwise modify their handset usage if they are disappointed by their battery life). That said, powertop and similar applications could always be extended to also include data from wakelocks. --
Good point!!! Of course, powertop would need to interact with Android's user-level daemon for this to work, but perhaps this could be arranged. (There is a user-level daemon in Android that acquires kernel-level suspend blockers on behalf of applications, so a naive mod to powertop would just finger the user-level daemon.) Thanx, Paul --
If you are triggering a system suspend from idle (I assume all cpus idle), you also have to consider when to resume. You cannot abort suspend just because a cpu is not idle anymore, since suspend itself How do you ever enter suspend in this system? Currently timers in the kernel and trusted user space code causes a significant power draw and A wakelock is active when the screen is on. -- Arve Hjønnevåg --
since the premise is that we only consider the activity of some processes when we are deciding if the system is idle, it's very possible that not all CPUs will be idle when we decide to suspend. will suspend wake up the application threads? or will it create it's own no, I want recent activity from a privilaged/trusted process to prevent suspend. I'm saying that such a process can wake up infrequently enough that the wakeups themselves are not a significant amount of processing if it wants to keep the system awake so that other things can keep running. you say below that the system will never suspend while the backlight is on, well the process that monitors/controlls the backlight would be one of so what controls when the screen is on? what still needs to run (and keep the system awake) if the screen is off? David Lang
In different message, Arve said they are actually using low-power idle to emulate suspend on Android. This came like a bit of a shock to me ("why do they make it so complex then"), but... it also means that as soon as you are able to stop "unwanted" processing, you can just leave normal cpuidle mechanisms to deal with the rest... (Of course, you'll also have to fix kernel timers not to beat unneccessarily often; still that's better solution that just stoping them all and then sprinkling wakelocks all over the kernel to deal with obvious bugs it introduces...) Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html --
How do you differentiate between "unwanted" and "wanted" processing in the same task in a race-free manner? -- Matthew Garrett | mjg59@srcf.ucam.org --
how much do you care? (not from a theoretical point of view, but from a practical point of view, how much difference does it make) how much "unwanted" processing would you allow a process to do before shutting down? how likely are apps that you would trust to just assert "I know better than the system or the user how important I am, don't you dare sleep now" that do this properly, but then do other processing that you really don't want to have happen? (I expect that if the authors of those apps ever ran into this case, they would just add another wakelock to prevent it) If the only things you have running are apps that are trusted to take the wakelock, are you really going to sleep any more frequently with the wakelock infrastructure than you would with just idle detection? (this can be tested today by just stubbing out the wakelock checks so they have no effect on sleeping) David Lang --
Hello, Pavel, Could you please point me at this message? --
AFAICT, this tells us that idle and suspend is the same hardware state on current Android hardware: Pavel Message-ID: <AANLkTinjH0C0bSK=Y2wKASnbJFsR2BN303xBXkaHbmRC@mail.gmail.com> Arve said: If you just program the alarm you will wake up see that the monotonic clock has not advanced and set the alarm another n seconds into the future. Or are proposing that suspend should be changed to keep the monotonic clock running? If you are, why? We can enter the same hardware states from idle, and modifying suspend to wake up more often would increase the average power consumption in suspend, not improve it for idle. In other words, if suspend wakes up as often as idle, why use suspend? -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html --
On Wed, 4 Aug 2010 23:15:13 +0200 They always told us from the beginning, that on the msm platform they reach the same powerlevel from suspend and idle. They still get gains from using opportunistic suspend. Cheers, Flo --
Yes, you will always get gains if you shutdown while there is still work to do. the question I am raising is. If, instead of doing opportunistic suspend (with wakelocks to keep things awake, requiring explicit application code changes to implement), how close would you be able to get if instead you just were able to tell the system to ignore some processes when considering if the system can sleep or not? With badly written 'trusted' apps you will have poor power useage, but with badly written apps grabbing wakelocks inappropriately you will have poor power useage. David Lang --
This is all massively system dependant. On some systems when the system is in the same idle state in the lowest power idle mode as in suspend however as a result of not doing the suspend (which causes Linux to quiesce most of the hardware) many more things will be able to generate wake events. On other systems you will achieve a lower power state by using suspend for various reasons, some fixable and some not. I rather suspect Arve was talking about the former case. --
cgroups alone don't but there is a solution which doesn't require routing
all event data through a single server -- use SIGIO.
Suppose we've got two cgroups of tasks -- those in the initial freezer
cgroup and those in a freezer cgroup meant for power-naive apps. Let's
call the second cgroup the naive cgroup.
One task -- let's call it the "waker" -- is in the initial cgroup is normaly
asleep waiting for SIGIO. Note it's not an "app" -- it's been trusted/blessed
to be a non-power-naive task. It will be signaled via SIGIO by the
applications which want to be unfrozen when an event comes in.
When the power-naive app in the naive cgroup opens a file descriptor it's
going through the Android interpretter to make the syscall. The interpretter
can do fcntl() on the fd to cause SIGIO to be delivered to the waker task.
When the waker gets SIGIO it unfreezes the naive cgroup and thus wakes the
power-naive app. When the power-naive app wakes it will
poll/return-from-poll/read/return-from-read and thus retrieve the event.
Then it's just a matter of choosing when to freeze the naive cgroup. That
requires a userspace implementation of the suspend blockers API plus
opportunistic suspend but does not require any other kernel pieces. Then you
can use sigprocmask to prevent the freeze/wake-event race. You would
probably even merge the waker with the daemon which implements
opportunistic suspend.
Cheers,
-Matt Helsley
--
The Android execution model includes native code in addition to the dalvik VM, and in the future could include other runtimes -- there are many standard libraries that directly make posix file IO calls, and apps can bundle native libraries which can also directly make syscalls. It's not practical to instrument every piece of userspace code that opens a fd somehow to make additional fcntl calls in various places. Brian --
Perhaps using an LD_PRELOAD will work. Cheers, -Matt Helsley --
I own a Nokia N900. Some applications are ported straight from regular Linux and they're definitely power-naive, as they have little or no optimization for mobile. I agree that this is not "sloppy" or "bad", it's just that mobile (low power) wasn't the intended target of the application when it was written, and this commonly shows. Just like some people will burn CPU cycles by writing their application in a high-level language because it requires fewer man-hours and that you get thousands of cpu-hours for the cost of a man-hour programming the thing (often by externalising the true cost of power which makes power even less of a problem), not caring much about power is rational behaviour when creating an application for pc. Just look at flash ads, they consume huge amounts of power and they commonly never stop. I imagine that the power used by PCs around the world displaying web pages with advertising with the monitor in power-save mode (powered down) can be counted in gigawatts. When I need my battery to last on my laptop I make sure I don't have any unneccessary tabs enabled because it seriously affects my battery time. I think laptop users would benefit from more power optimized applications as well, so if the OS could do the same for laptop/desktop use as well (spending less time on programs updating display when the display is off), I think there are real world benefits for every desktop/laptop/mobile user. If nothing else, it's environmentally friendly. -- Mikael Abrahamsson email: swmike@swm.pp.se --
I have another aspect I just thought about. I work for a telephony company. We provide Internet connectivity throught various means, DSL, Ethernet to the Home, mobile etc. For ETTH and DSL, network usage is pretty straight forward, you send packets, they get delivered pretty quickly with low marginal cost per packet. For mobile, this is not quite so simple. Mobile networks are designed for terminal/UE (user equipment) to use low power, so they go down in low power state after a while. Let's take the case of 3G/HSPA: After a short while (second) of idleness (no packets being sent), the mobile network negotiates away the high speed resources (the one that enables multimegabit/s transfers) and tries to give it to someone else. After approximately 30 seconds, the terminal goes to "idle", meaning it has no network resources at all. Next time it wants to send something (or the network wants to deliver something to it), network resources need to be negotiated again. This can take 1-2 seconds and uses battery power of course. It also consumes resources in the operator network (because mobility control units need to talk to base stations, tunnels need to be re-negotiated etc). Anyhow, my point is that not only is there a benefit in having multiple applications wake up at the same time for power reasons within the device, there is also a point in having coordination of their network access. If a device is running 3 IM programs at the same time, it'd be beneficial if they were coordinated in their communication with their Internet servers. Same goes for the "check for new email" application. If they all were optimized to only wake up the network connectivity once every 180 seconds instead of doing it when the individual application felt like it, power and other resources would be saved by all involved parties. -- Mikael Abrahamsson email: swmike@swm.pp.se --
This is a good point. Within some limits, the timer-aggregation changes that have gone into Linux can handle this, but I am not sure whether or not 180 seconds is within the reasonable boundaries for timer jitter. Of course, the timers might be synchronized upon wakeup after a sufficiently long suspension, but they would not necessarily stay synchronized without the help of some other mechanism, such as the afore-mentioned timer-aggregation changes. Thanx, Paul --
On Sun, 1 Aug 2010 12:27:08 -0700 this is why operating systems for mobile devices offer heartbeat services... where apps subscribe to and do background work like checking email at "convenient" times. I'm not sure if the OS you use on your desktop has one, but MeeGo and Maemo and I'm pretty sure Android and most other mobile Linux OSes have one. It's a higher level activity alignment layer, well above the kernel. -- Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org --
Thank you, good to know! Thanx, Paul --
So the reason everyone's having trouble with this definition is that it actually conflates two separate axes of power management. There are good and bad applications in the power sense ... burns less vs burns more. And there are user mandated vs user optional processes ... necessary/wanted vs unnecessary/unwanted. What android actually does is reward well written applications because they "just work" and they don't have to be power aware at all ... they're usually event driven and split into the android provider/consumer model. Badly written applications that are not suspend block aware get shut down (by system suspend) when the screen turns off, so they're also power/suspend unaware. Applications that want to present the user with a choice in android are power/suspend aware because the only way they get to present the choice is via suspend blockers. The "power problem" always devolves to resolving a set of choices around a given set of control axes. The problem is that the set of control axes isn't unique and doesn't have a well agreed upon selection. This makes it hard to make definitive terminology because you have to pick the set of axes (implicitly or explicitly) before defining terms ... James --
That does seem to be about the size of it... :-/ Thanx, Paul --
On Sat, Jul 31, 2010 at 10:58 AM, Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote: The problem with using open and close to prevent an allow suspend is not that it is too slow but that it interferes with collecting stats. The wakelock code has a sysfs interface that allow you to use a open/write/close sequence to block or unblock suspend. There is no limit to the amount of kernel memory that a process can consume with this interface, so the suspend blocker patchset uses a /dev interface with ioctls to block or unblock suspend and it destroys the kernel We don't key the stats off the program name, but having useful statistics is critical too us. The current code in linux-next does not appear to allow this (I'm referring to pm_stay_awake here, etc not wake-lock/suspend-blocker timeouts have nothing to do with the timeout used by applications when waiting for a response from a less trusted I would say it should suspend even if power aware applications are It must be power-efficient. Repeated attempts to suspend will kill the Unconditional initialization makes it easier to add suspend blockers to existing kernel code since you don't have to add new failure exit -- Arve Hjønnevåg --
Ah, I missed this point. What I am doing to adjust is to strike the above requirement, and to add verbiage to the "statistics" requirement about using ioctl() to implement suspend-blocker operations, so that the statistics can be tracked based on the device being open throughout the OK, maybe I was confused earlier. So you do not track statistics via the device being open throughout the application's lifetime? OK, I moved this to a new "SUGGESTED USAGE" section and removed the The point being that a power-aware application does not block suspend -unless- it holds a suspend blocker, correct? Good point! I changed "Transition to low-power state must be efficient" Ah, that makes more sense! I moved this to a new "NICE-TO-HAVES" section. I also changed the last parenthesized sentence to read "Such unconditional initialization reduces the intrusiveness of the Thank you again for looking this over and for your comments!!! Thanx, Paul --
The suspend blocker patchset does track statistics while the device is open, but it it not keyed of the program name. The name is passed from user-space and a single process can have the device open several times. The wakelock interface that we currently use just creates a new object every time it sees a new name and never frees it. No. Sure. -- Arve Hjønnevåg --
Please elaborate on this. I expect the pm-qos stats interface will collect stats across user open/close because that's how it currently This is an implementation detail only. The pm-qos objects are long lived, so their stats would be too. I would guess that explicit stat clearing might be a useful option. James --
The pm-qos interface creates the request object in open and destroys it in release just like the suspend blocker interface. We need stats for each client which is lost if you free the object every time you unblock suspend. Or are you talking about user space opening and closing the stats There is no way fix it without changing the user space visible behavior of the API. The kernel does not know when it is safe to free Which pm-qos objects are you referring to? The struct pm_qos_object that backs each pm-qos class is long lived (I don't know why this is named pm_qos_object), but we need stats in struct pm_qos_request_list. -- Arve Hjønnevåg --
? right at the moment it doesn't do stats. I don't see why adding a per They're freed on destruction of the long lived kernel object or on user Actually, why not two separate lists? one for the request and one for the stats? OK, so I'm tired and I've had a long flight to get to where I am, so I may be a bit jaded, but this isn't fucking rocket science the question is how do we implement what you want on what we have ... there looks to be multiple useful solutions ... we just have to pick one and agree on it (that's standard open source). James --
Then I don't know what you are asking. This specific requirement was about the userspace interface to block and unblock suspend. The existing pm-qos interface to update pm-qos requests is similar to the user space suspend blocker interface (it has the same object What is a user-space clear request? Clearing all stats. Deleting a suspend blocker? Unblocking suspend? The android wakelock interface to userspace creates new wakelocks every time it sees a new name. This was rejected on this list because it does not put any limit on the amount of kernel memory used as a Why do you want to do another lookup? Is there a reason you don't want I don't think adding stats to pm-qos is hard. It is not as easy as the wakelock/suspend blocker interface since the number of possible request values is unknown, but we could for instance have stats on default vs non-default request values. However, it is not at all clear that switching our code to the pm-qos interface would make it acceptable for mainline inclusion. The main objections to the last suspend blocker patchset seemed to be that we should not use suspend at all. There are also some new api in linux-next that try to solve the same problem in an non android compatible way. Are drivers supposed to use both this interface and pm-qos to prevent suspend? -- Arve Hjønnevåg --
Continuing to rush in where angels fear to tread... This is an updated version of my list posted a couple of days ago at http://lkml.org/lkml/2010/7/31/73. Again, this email is an attempt to present the Android guys' requirements, based on my interpretation of LKML discussions. Please note that I am not proposing a solution that meets these requirements, nor am I attempting to judge the various proposed solutions. In fact, I am not even trying to judge whether the requirements are optimal, or even whether or not they make sense at all. My only goal at the moment is to improve my understanding of what the Android folks' requirements are. That said, I do discuss example mechanisms where needed to clarify the meaning of the requirements. This should not be interpreted as a preference for any given example mechanism. But first I am going to look at nomenclature, as it appears to me that at least some of the flamage was due to conflicting definitions. Ducking into the nearest bunker to avoid the hailstorm of frozen fish... Thanx, Paul ------------------------------------------------------------------------ DEFINITIONS These have been updated based on LKML and linux-pm discussions. The names are probably still sub-optimal, but incremental progress is nevertheless a very good thing. I have also added a section entitled "CATEGORIES OF APPLICATION BEHAVIOR" based on a suggestion from James Bottomley. o "Ill-behaved application" AKA "untrusted application" AKA "crappy application". The Android guys seem to be thinking in terms of applications that are well-designed and well-implemented in general, but which do not take power consumption or battery life into account. Examples include applications designed for externally powered PCs. Many other people seemed to instead be thinking in terms of an ill-conceived or useless application, perhaps exemplified by "bouncing cows". Assuming I have correctly guessed what the Android guys were thinking of, ...
One thing that I think it's important to document here is theinformation that Brian provided in response to your question about how many (or actually how few) applications fall into this catefory David Lang For Java/Dalvik apps, the wakelock API is pertty high level -- it talks to a service via RPC (Binder) that actually interacts with the kernel. Changing the basic kernel<->userspace interface (within reason) is not unthinkable. For example, Arve's suspend_blocker patch provides a device interface rather than the proc interface the older wakelock patches use. We'd have to make some userspace changes to support that but they're pretty low level and minor. In the current model, only a few processes need to specifically interact with the kernel (the power management service in the system_server, possibly the media_server and the radio interface glue). A model where every process needs to have a bunch of instrumentation is not very desirable from our point of view. We definitely do need reasonable statistics in order to enable debugging and to enable reporting to endusers (through the Battery Usage UI) what's keeping the device awake. Brian --
I think I need to clarify here. When I say "app" in the context of Android, I mean "an application running under the Android app model -- sandboxed under a per-app or app-group uid", not "a process". The vast majority of processes on an Android device are "apps" in this sense, but some (usually low level services or daemons) are not. Also I use "wakelock" as a place holder for "suspend blocker" or whatever exact API we're trying to hash out here, because it's shorter and I'm lazy. Any app may obtain a wakelock through the standard Android APIs, provided it has permission to do so. In the current implementation, apps obtain wakelocks via making a binder RPC call to the power manager service which tracks high level wakelocks (for apps!) and backs them by a single kernel wakelock. Access control is at the RPC level. This implementation could be changed to have the app API simply open /dev/suspendblock or whatnot, with access control enforced by unix permissions (the framework would arrange for apps with the android "can block sleep" permission to be in a unix group that has access to the device). For native services (native daemons that run "underneath" the android app framework -- for example the media service, the radio interface, etc), the kernel interface is used directly (ok, usually via a very thin C convenience wrapper). --
Thank you for the added detail on Android user-space operation!!! --
Agreed!!! I have added this, and it will appear in the next version. --
Strange. Arve claimed that open/close is too slow, and few microseconds faster ioctl is needed, and now we learn it actually uses RPC. -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html --
For the high level Java API, yes. For lower level userspace code, like the code that processes keypresses, the kernel interface is used directly. I think an open/close per keypress would be a bit excessive, for example. In any case, that ignores the fact that it's useful to have statistics, which are tricky to maintain meaningfully if you destroy the handle after use every time (by closing the fd). Brian --
Final report from this particular angel-free zone for the time being... This is the third and final version of my Android requirements list (last version available at http://lkml.org/lkml/2010/8/4/409). Again, this email is an attempt to present the Android guys' requirements, based on my interpretation of LKML discussions. This past week's discussion was quite productive, and I thank everyone who took part. Please note that I am not proposing a solution that meets these requirements, nor am I attempting to judge the various proposed solutions. In fact, I am not even trying to judge whether the requirements are optimal, or even whether or not they make sense at all. My only goal at the moment is to improve our collective understanding of what the Android folks' requirements are. That said, I do discuss example mechanisms where needed to clarify the meaning of the requirements. This should not be interpreted as a preference for any given example mechanism. Thanx, Paul ------------------------------------------------------------------------ CONTENTS o DEFINITIONS o CATEGORIES OF APPLICATION BEHAVIOR o REQUIREMENTS o NICE-TO-HAVES o APPARENT NON-REQUIREMENTS o SUGGESTED USAGE o POWER-OPTIMIZED APPLICATIONS o OTHER EXAMPLE APPLICATIONS o ACKNOWLEDGMENTS DEFINITIONS These have been updated based on LKML and linux-pm discussions. The names are probably still sub-optimal, but incremental progress is nevertheless a very good thing. o "Ill-behaved application" AKA "untrusted application" AKA "crappy application". The Android guys seem to be thinking in terms of applications that are well-designed and well-implemented in general, but which do not take power consumption or battery life into account. Examples include applications designed for externally powered PCs. Many other people seemed to instead be thinking in terms of an ill-conceived or useless application, perhaps exemplified by "bouncing cows". This document uses ...
one other nice-to-have (or conflicting requirement, depending on your point of view), and I think one of the big things causing people to dislike wavelocks, is the desire to not have to modify applications to have them work with the infrastructure. you sort of touch on this when you say that power oblivious applications need to be able to be intergrated, but it goes beyond what that statement implies. with wavelocka, even power optimized applications need to be modified, or the system may halt them at any time. one thing that has been very clear over the years is that if an API only exists on Linux, no matter how good it is, most application developers won't use it. In this case we are in an even worse situation, it's not only specific to Linux, it's specific to a subset of Linux systems, and not using it will cause no problems most of the time. now, android is betting that the apps are all developed specifically for the android from scratch, so having a different API is acceptable, even if it cuts them off from the rest of the *nix applications. For a phone this is not neccessarily an unreasonable stance, but as Android moves into the spaces where normal applications are in use (netbooks and tablets), this becomes a much shakier stance to take. David Lang --
Yes, I believe that Android would require most power-optimized application be modified to use wakelocks. But power-optimized applications require so much tweaking that the addition of suspend blockers (or whatever other power-control mechanism) is pretty much a non-issue by comparison. And the number of power-optimized applications should be small, which, Many application developers do indeed value portability. But this in turn means that most application developers will not be writing power-optimized applications, because the process of power-optimization significantly degrades portability. Just as does the process of performance tuning, beyond a certain point. Of course, the reason that application developers value portability is that this is one way to gain large unit volumes. Another way to gain large unit volumes is to code for a very popular platform, which explains the large number of apps that run only on iPhone, Android, and Windows. And in my experience, developers who have decided to commit to a single platform are usually not at all shy about exploiting special facilities It seems to me that PM-driving and power-optimized applications are going to be highly platform specific, whether that platform be Linux or some There certainly does seem to be a large and growing number of Android apps, so I might be reluctant to bet against them. And the Android guys appear to be making another bet as well -- that almost all applications will be power-oblivious. Their design handles this rather well, given that such applications need not worry about special power-control features. --
except, power optimized applications aren't just written for android, they are also written for many other battery powered devices, many of which don't use wavelocks (and in fact, have no real reason to as they don't also have the untrusted apps running) As such the pool of power optmized software will grow faster than the pool just like performance tuning, power optimization only degrades portability beyond a certin point. A lot of power optimization is like a lot of performance tuning, find algorithms that you are using (usually frequent polling in the case of power optimizations) and change them. These changes are very frequently very portable. In this case we are talking about changes that not only aren't portable across different Operating systems, they would not even be portable across and they are not shy about ignoring other variations of a platform either (for example, Adobe has flash for i386 linux, but not amd64 linux, arm linux, powerpc linux, etc) there is also a growing population of nook applications, kindle applications (ok, in this category there are only two released, both in the last week, but the SDK has not hit public release yet ;-), maemo applications, OLPC applications, tom-tom applications, ..... unless all of these platforms are going to start using wavelocks there are going to be power optimized applications out there that don't use wavelocks. From this discussion, it looks to me like Android wants two key features that they don't see a way to get today 1. the ability to decide to suspend while there are still some 'unimportant' apps running. 2. changes to idle/suspend so that they can get into a state of lower power consumption thatn any existing idle state (by being able to disable clocks), but still have some parts of the system powered (so that they are more awake than suspend) If these two features were available, I think that the rest of what they are looking for could be built up without ...
"Normal" apps work reasonably well -- they get halted when the screen turns off, just like they do when my laptop suspends. Wakelocks are useful for mobile-centric apps that you want to keep running in the background, wake up and do work when the device is "asleep", etc. Brian --
that description sounds far more like normal sleep power management that suspending. especially since they want to set timers to wake the system up and the defining characteristic of suspend (according to this thread) is that timers don't fire while suspended. as I am seeing it, there are two reasons why this don't "just work" 1. sleeping can't currently save as much power as suspending 2. the current logic for deciding to sleep can't ignore the other apps on the system. David Lang --
No, I don't think that's the case at all. The key thing here is that *most* applications don't need to be modified to use suspend locks, because even though they might be in an event loop, when the user user turns off the display, the user generally doesn't want it doing things on their behalf. Again, take for example the Mac Book, since Apple has gotten this right for most users' use cases. When you close the lid, you even if the application is under the misguided belief that it should be checking every five seconds to see whether or not the web page has reloaded --- actually, that's not what you want. You probably want the application to be forcibly put to sleep. So the whole point of the suspend blocker design is that you don't have to modify most applications; they just simply get put to sleep when you close the MacBook lid, or, in the case of the Android device, you push the button that turns off the screen. So the reason why this doesn't work is that power management for small mobile devices *is* different from power management for laptops and data center servers, and if you want a rich application ecosystem, it's best if you don't require them to be specially tuned to use the absolute minimum power. (And that means waking up every 30 seconds might be too much; as Brian and Arve have pointed out, with the G1 in airplane mode, the CPU might be waking up once every half hour or more --- and at that rate, powertop will be waking up the CPU more than Android system would be doing so.) So the real key here is to take most applications, which may be written using techniques that might be considered well written from a laptop point of view, but not for a cell phone, and not require modifications. Even though the application writer might think it's doing well by waking up every 15 seconds, if the laptop lid is down, or if the screen is off, for **most** applications, it should be forcibly put to sleep. It's only the rare applications that should really be allowed ...
But in principle that need not mean suspending the entire system. To get applications out of the way, you need to freeze user space. However, that's not sufficient, because in addition to that you need to prevent deactivate the majority of interrupt sources to avoid waking up the CPU (from C-states) too often. Thanks, Rafael --
s/prevent deactivate/deactivate/ Rafael --
True, but again, consider the MacBook. If you plug in an iPod, the machine will wake up for *just* long enough to let the iTunes sync the iPod, but once its done, the machine goes back to sleep again immediately. I doubt MacOS has something called a "suspend blocker" which prevents the machine from sleeping until iTunes finished, which when released, allows the machine to suspend again immediately. But neither did I see any evidence that it took 30 seconds for some kludgy polling process to decide that iTunes was done, and to allow the MacBook to go back to sleep. Clearly, the MacBook allows some interrupts through, and some USB insert events through, but clearly not all. (Inserting a USB drive doesn't wake up the laptop; at least, not for long.) Can we do something as smooth with a Linux desktop? And if not, why not? (Oh yeah, and wasn't this supposed to be the year of the Linux Desktop? :-) -- Ted --
nobody has proposed a polling process to decide the system can suspend, what I proposed was to have idle detection decide the system can suspend, which would mean that if you don't want the system to suspend you have to do something to keep it from being idle. At the time I didn't know that Android always disabled suspend for the entire time the display was on, so I made the assumption that timouts had to be long enough for input events to keep the system awake, which would put them on the order of 30 seconds or longer. If all that is wanted is to disable the suspend for the entire time the backlight is on you don't need wakelocks, and you don't need a privilaged thread waking up, all you need is to tell the power management system not to suspend through the existing mechanism. you could create a new mechanism (wakelock) to pass the same information, but what is the advantage of doing so? David Lang --
On the contrary, I suspect that it's precisely equivalent to userspace suspend blockers. There's no way to conditionalise USB wakeups - the system comes up when you plug or unplug any USB device. The system is then fully awake and I'd *guess* that ipods are magically exempted in some way, with itunes sending a signal when it's complete in order to gnome-power-manager supports applications inhibiting suspend, but right now I suspect that it'll never deal with the case where you resume with the lid closed. It's a simple matter of coding, though. -- Matthew Garrett | mjg59@srcf.ucam.org --
this doesn't require wakelocks or anything else new. all it takes is setting the policy that you don't want anything to run when the lid is closed and a switch to detect the lid being closed. Laptops have been note that nothing that I have proposed would wake up a sleeping system. the 'every several seconds' thing that I proposed was that on a system that's fully awake, busy and doesn't want to sleep, there would be a context switch periodically by a privilaged process so that the system would not end up deciding it was idle and halt everything. Now that I think about this more, it's not needed if you want to override this to keep everything running for a significant amount of time, just change the power saving mode from "sleep if a privilaged task isn't running" to "disable suspend". This can be done today by changing the right sysfs value. normal privilaged processes would never need to do this, only gatekeeper daemons that want to let unprivilaged processes run even if no privilaged processes want to run would need to do this (i.e. in the current system, whatever process controls the screen would probably be the question is what it takes to make an application privilaged like this. what I proposed was to make it possible for the user/admin to select what applications are allowed to keep the system awake. wakelocks require that the application developer decide that they want to keep the system awake as well as the user/admin take your example of a mail client waking up every 15 min. with Android it needs to be privilaged to grab the wakelock while fetching the mail, it also needs to use a privilaged API to set the wakeups to wake it up at those times. with what I proposed all you need to do is to tag the application as power privilaged and then if the application sleeps for 15 min between doing thing the system will wake up every 15 min, work for a short time, then go back to sleep. if you want to put everything to sleep when the screen ...
Please re-read Ted's paragraph above, but carefully this time. You will then see that the desired policy is -not- "you don't want anything to Whereas you require that the application developer redesign/rewrite applications to decide when to keep the system awake, e.g., by carefully determining when to idle themselves. The difference is that the Android developer need only release a suspend blocker. In contrast, you are requiring that the developer rewrite all the code that follows the point where the Android developer would release a suspend blocker. Your way seems to require that the developer do more work for the same result. Why? Now, I do agree that a safe way to freeze the power-oblivious applications could be valuable, as it might reduce power consumption while the screen was blanked but while a PM-driving application was holding a suspend blocker. However, as we have discussed, you have to be very Which is in fact a critical requirement that you appear to have been But your suggestion seems to require quite a bit more coding effort for little gain. The Android developer can just release a suspend blocker, and in contrast, a developer using your proposal needs to rewrite all the code following the point at which the Android developer released the And for good reason. Thanx, Paul --
portability, and the fact that it will save more power even when the I admit that the way I wrote this is confusing, but I don't think I am ignoring this requirement. the example above was how an application that the system is not wanting to put to sleep would run with the two no, the coder needs to make his code not do unnessasary things. Even with a wakelock this is needed for the system to be able to use lower power modes while the system is not going to do a suspend anyway. I am say that if the coder does that, the coder should not also have to code in wakelocks. I want it to be as easy as possible to port applications between mobile platforms. On most platforms wakelocks are not needed because standard idle/sleep/suspend mechanisms are good enough. Android allows other applications to run that confuse the idle/sleep/suspend mechanisms, so something needs to be done to cut through the confusion. Android does this by creating a new API to control this, I'm suggesting that there should be a way to tell the system to ignore the other applications so that the existing mechanisms can make good decisions again. David Lang --
You are overgeneralizing; there are many applications that run in the background, and you want to keep them running even when the display is off. You seen to be concentrating on UI-only applications, for those it's worth noting that Android provides separate mechanisms for power saving. Since Android doesn't have true multi-tasking, the applications must serialize their states so that the next time they are opened they seem to have not been closed. So, the current active UI application can be closed while turning off the display, and re-opened later. User-space suspend blockers are relevant for background services, and as it has been discussed before; suspend blockers (not activating them) might actually degrade power usage. -- Felipe Contreras --
Really? How many do you really expect will be running at the same time on a mobile platform with a 800-1000 mWh battery? And what percentage of the applications that might be in a Android or Moblin or Maemo app store will have things running in the background? 10%? Actually in practice, the process or processes which comprise current active UI application generally won't actually be killed when you turn off the display. It *might* happen, if one of the rare backround applications needs more memory than is available without closing some of the more recently open applications. In practice, the last 2-3 most recently used applications are still loaded in memory so you can switch back and forth between them simply and easily. It is true they have to be ready to be killed at any time in case their memory is needed for the currently active application, but that generally does't happen right away. The design is to keep this transparent to the user so the user does't have to keep track of how many apps currently running on the system. In any case, the key thing to keep in mind is that when you deal with extreme power savings, very often you end up making compromises that probably make sense for other reasons anyway (such as the small screen size). It's not at all clear that supporting generalized multi-tasking applications makes sense just from a screen real-estate Yes, absolutely. Note though that with the iPhone, Apple has decided that the only background services that will be allowed is audio, VOIP, location/navigation, and push notifications. iOS developers don't get access to anything else, and the argument is that nothing else is really *needed*. Android is more flexible in that it allows for non-Apple developers to create new background services, but it's not clear how many you really *need*. This goes back to your first assertion that there are *many* applications that need to run in the background. I just don't think that's true. There will be a few, and ...
At any given time most of the applications are running on the background, how much exactly depends on the platform. In my N900 I can see that at least 90% of the process are running on the background My last experience using an Android system was that switching applications wasn't done "simply and easily". But supposing it is, Android's UI is completely different to anything else; you cannot take a GNOME/KDE app and expect it to run on Android. Since UI applications are written for Android anyway, then they should My guess is you haven't used a truly multi-tasking device like the N900; now that I've got used to it, I consider that functionality *essential*. Multi-tasking and good PM is possible, and the N900 is a good example. Rather than giving up multi-tasking to see how much longer the battery can last, I would rather like to see how to improve batter life for The argument in favor of suspend blockers is that you could take applications that are not designed for embedded, and make them run on an embedded device without draining excessive battery life; those applications would have to be background services not conflicting with Android's design. I agree there probably would not be that many background apps, and probably even less ported background apps, but that is actually an argument against suspend blockers. The rest of the apps (UI apps), cannot be ported, but have to be written specifically for Android, and therefore should have PM in mind, and not require suspend blockers to have good power usage. -- Felipe Contreras --
On Sun, Aug 8, 2010 at 10:40 AM, Felipe Contreras I think there's some confusion here. The default operating mode for Android is to aggressively reach the lowest possible power state (typically full suspend). suspend blockers are used by applications to inform the system that they are currently doing work which requires immediate processing, so the system will avoid fully suspending until that work is done. You could think of it as setting your laptop power management settings to be "blank screen after 60 seconds, power down 5 seconds after blanking", except that there's also a mechanism for apps to hold off on that power down if they're doing important work, and a lot more ways to wakeup beyond opening the lid or pushing the power button. Straight-forward UI apps don't actually require much special handling -- it's preferable that they do busy-wait on things, but even when power is not an issue, wasting cpu spinning needlessly is frowned upon, so this isn't a special request. There's nothing stopping somebody from writing an Android app that acts as a bridge to X11 apps (embedding an X server, etc) and making it possible for native X11 apps to coexist with the Android composition model. I'm a little surprised nobody has done so, actually. I figured *somebody* would decide that they like their phone but absolutely need to run xclock or something locally as a first class app. The Android app model does say that if you're not in the foreground (from a UI standpoint) you should save your state, as you may be terminated if the system runs low on resources. The system only does this when it has to -- generally when we bottom out on memory. On devices with sufficient memory, it's quite common for many apps to remain resident for days-weeks -- I've seen as many as 45-50 apps running on my nexus one. I am a little puzzled by the assertion that Android lasks "true multitasking", but that's another discussion entirely. We certainly don't lack for many concurrent ...
I've used an N800, and I wasn't impressed; if anything, the fact that
I have to worry about manually killing off applications when memory
gets low, I actually thought it was incredibly sucky. It was a
miniature, badly done laptop.
Maybe the N900 is different, but the bigger question is what do you
mean by "multi-tasking"? Definitions are critical here, which is why
Paul was so careful in his definitions section of his document.
Do you mean:
* allowing multiple processes running at the same time?
* allowing some applications to be able to update mail, play audio,
upload location information so your friends know where you are?
* allowing arbitrary applications that users can interact with
simultaneously (which implies a window manager, the need to have
the concept of window focus for keyboard input), etc?
If you are using a GUI framework which is optimized for a single-
application-focus-at-a-time UI that isn't GNOME or KDE, then that will
require the applications to be written. However, that's not because
of suspend-blockers.
If you assume a GUI framework which is flexible enough --- maybe Qt
falls into this category, maybe not --- for the rest of the
applications, they don't need to *know* about suspend blockers, and
they certainly do't have to be rewritten or modified specifically for
suspend blockers. So if your argument is that applications that don't
need bacground services (which you've admitted comprises majority of
applicatios) need to be modified or written specifically to support
suspend blockers, that's simply not true. They don't need to be
modified at all.
As far as whether they *should* require suspend blockers to be in the
kernel to get power usage that is suitable for cell phone batteries, I
would agree that in the ideal world, it would be nice if you could
have applications that make the correct performance/battery
utilization tradeoff for devices running on 800 mWh batteries, 94,000
mWh batteries, and while running on the ...Likewise I thought the lack of proper multi-tasking apps in Android was comically bad. Nice device completely wrecked by the inability to flip between running apps each with their own screen - something that was old tech in 1990. It's no use looking at this from an "In my opinion" angle. A proper solution works for most if not all opinions except maybe around corner That's also untrue as anyone who has worked in industrial control or entertainment systems will tell you. You may well have fourteen apps Suspend blockers drive the system policy part way into the apps, that in turn makes the apps very vulnerable to change in their environment because you've specialised them. I am sure that in the Android world it's considered fine, and that the marketing and business people even like this binding together - but it doesn't generalise and will blow up in people's faces in the future. To consider your tide analogy - suspend blockers is like trying to program the waves individually. Show me a suspend blocker aware open office patch 8) --
On Mon, Aug 09, 2010 at 11:24:53AM +0100, Alan Cox wrote: But wouldn't an office suite run as a power-oblivious application on an Android device? After all, office applications do not need to run when the screen is turned off, so these the applications do not need to use suspend blockers. That said, I could easily imagine that significant work would be required to make OpenOffice run on Android, not due to suspend blockers, but rather due to Android's unusual user space. On devices that do not have suspend blockers, a normal application runs in a manner similar to a hypothetical Android application that acquires a suspend blocker when it starts and holds that suspend blocker until it exits. In contrast with Android, this situation requires that each and every application be carefully written to avoid battery drain, which I suspect is what Ted is getting at with his King Canute analogy. Thanx, Paul --
pick your application if you don't like the example. but also, which android system should the applicaton be written for? the phone with a 800maH battery or a larger device with a 94,000maH battery? well bahaved applications (not doing unnecessary wakeups, etc) are well bahaved, no matter what system they are on (explicitly setting allowable timer fuzz is linux specific, but will again help on any system) David Lang --
Yep. Your point being what exactly? That all applications should be required to be power-optimized, and that any technology that automates energy efficiency should be rejected out of hand? If so, please justify Setting aside the question of how timer fuzz will help on non-Linux systems if timer fuzz is specific to Linux... Thanx, Paul --
I was waiting for soemone to leap down the pit I dug Office suites have some quite important background activities. Consider the case of a power oblivious Open Office. You type a critical document, you suspend, your phone battery dies a bit later, you lost your document. Office suites do timed backing up as one simple obvious example. That could become a power You are tightly linking suspend blockers with Android. If they were a sensible general solution they would be generic not tied closely to Android Some of the other bad assumptions being made in this discussion: - That the phone is special. Todays Android phones are up with the PC's of some years back (but with better graphics and more faster storage), in a few more generations of the technology what will they look like ? I'm sure that within a few years there will be people playing Warcraft or similar on their phone in the train. - That Android will continue to tbe offering the same services in future as today. If it does it'll go the way of PalmOS and Symbian. Equally you can't just bust all the apps as it changes As devices get more complex and varied you cannot afford to put the detailed awareness of platform behaviour in the applications. It doesn't scale. Android developers are haivng enough fun coping with all the OS variants, customisations and new phones - and thats far less variety than PC hardware. Generally the PC app folks are not having the Which is flawed and not the case. The same argument could be made for multi-tasking DOS "Each application implements its own internal multitasking/polling if needed" Windows 3.x "Each application has an event loop and is built in a certain way" (the 'suspend blocker' mentality) Real OS "The scheduler operates in a manner which prevents CPU hogs breaking the system or abusing it sufficiently to threaten its functionality" The same applies to power. Only the OS has the big picture and can hide the hardware and general policy variety from ...
Doesn't the same problem exist on my linux laptop? I write out my manifesto in open office, close the lid of my laptop, the system suspends before my huge document finishes writing out, later my battery dies or I foolishly remove it or whatnot... the main difference seems to be that laptops, with their big 'ol batteries, are less aggressive about power management and the result is wider windows before and less frequent wakeups from suspend and thus better odds at missing the race condition. As Arve has pointed out previously, there are a number of uses for suspend blockers, even on plugged-into-the-wall systems -- take his example of wanting his mythtv backend to power down when not busy, but never power down when he happens to be using it on console, or issues with multiple services that want to wake up and keep the device awake while working. Brian --
this is a good example of why the office suite should be a privilaged application. If the system were to consider it privilaged, it could wake up when the timer is scheduled to fire to save the document. the office suite probably has a periodic timer that would fire every 10 min no matter what, but if there was a reason to change it, it could be modified to only setup the timer when the document is changed. If no changes can take place because the system is asleep, the timer never gets setup. David Lang --
It is a sad day on LKML when we are busy digging pits for each other. On the other hand, I have seen far sadder days, and the pit you dug happens to lead somewhere useful. As Brian noted in his reply, the situation you call out can occur with manual suspend-to-RAM already: the fact is that this is a design choice. You could indeed make a PM-driving office application if you wished, or run the same application as a power-oblivious application. There is a slight difference in the contract with the user. Of course, it is quite clear which one -you- Android is certainly where suspend blockers originated, and is to the best of my knowledge is still the only platform that uses them. But there is a first user of every new mechanism, and for some time that first user will by definition be the only user of that mechanism. So the fact that Android is most probably the only user of suspend blockers does Indeed, people have played games on their cellphones for some years. Not sure whether any of the games resembles World of Warcraft, but they probably soon will. If so, perhaps they will make use of significant hardware acceleration -- most other embedded systems do so. But that doesn't guarantee that solutions developed for PCs and laptops will be optimal (or even usable) on cellphones. Sufficient difference in scale can easily change the form of the optimal solution, so you have not yet made your case. (For that matter, you also haven't proven that adoption of suspend blockers or something like them depends on the I hope that no one is arguing that Android will remain unchanged, just as I hope no one would argue that Linux will remain unchanged. In fact, I am quite confident that both Linux and Android will continue to change. So exactly what point were you attempting to make here? As to busting all apps, lthough there have been situations where busting all the apps turned out to be the right thing to do, I agree that these situations are extremely rare. Such situations ...
Losing data is a design choice ? The application set a timer, the OS shouldn't be ignoring it in that situation. It might want to delay it, it might want to warn the user its hogging things it shouldnt (powertop, They don't appear to solve the constraints on power management that you have in other environments, nor do they happen to be conveniently backward compatible or available on other platforms - which makes code Your cellphone is to all intents a laptop from ten years ago, it even has a similar display resolution and internet availability. The underlying difference between the two is solely form factor - the laptop had a That anything which ties to a particular style of behaviour and set of current usage assumptions is broken. If you rewrite all the apps to Android 2.1 and design in a single tasking model then you'll have to unrewrite half of it again when Android grows up. Ditto suspend blockers - encode too much policy in your apps and you lose the ability to change the environment under them. See the mess Microsoft got into with Win16 on Like Microsoft Windows 16bit co-operative multi-tasking ? It's rarely right. It's merely that in certain cases the value in the market is large enough that it can be used as a big stick to beat people into doing lots What do you mean by "opportunistic suspend", lots of systems drop into lowest power states whenever they can. "Suspend is different" is a bit of Android religion that I am dubious has any basis in reality as seen from the application end of the universe. You may also wish to review the earlier parts of the discussion where it was explicitly stated by several developers that they were using "suspend" type modes as power states already and not using suspend blockers. So it's being done, today on ARM and your statement is directly contradicting the code. Modern ARM processors and x86 MID devices can suspend and resume extremely fast (fast enough that the fact Linux x86 rewriting all the SMP alternatives on ...
There's a clear and absolute difference between system suspend and entering the same hardware state from the idle loop. That difference is that processes aren't scheduled until an explicit wakeup event occurs. Android is entirely capable of entering the same low power state at idle (it's done with a hardcoded idle loop on Qualcomm, cpuidle on omap), but if you have more than 0 scheduling wakeups a second then your power draw is going to be greater. I agree that we should be targetting 0 wakeups per second. I don't agree that it's realistic to insist that a use model that assumes imperfect software is an invalid use model. -- Matthew Garrett | mjg59@srcf.ucam.org --
Explicit suspend is user triggered - in the laptop case its a bit different. I don't btw like the fact that a laptop explicit suspend doesn't end up as a dbus "save data" and openoffice save (or it didn't And nothing stops you also implementing a 'forced' suspend, although you can do that nicely by simply stopping the process group that contains the stuff you don't want to stop suspend and dropping into suspend when you No argument. The question is what applications should be expressing to the kernel which is not tied to assumptions like 'suspend mode' and which is generic. We don't have xfs stuff for example splattered all over userspace and in lots of drivers - we have generic interfaces. That way user community A doesn't have to care about user community B's choices, and JFFS flash fs people don't offend big data centre people and vice versa. Alan --
It's not inherently user triggered. Machines typically have an idle timeout that triggers suspend. This differs from the Android opportunistic suspend approach only in that the timeout is significantly We've already had the discussion about this resulting in potential deadlocks if there's any intercommunication between the trusted and untrusted apps, and if untrusted apps can be the consumers of wakeup events then you still end up with the wake event race condition. If we want to avoid the case where system suspend can cause wakeup events to be lost, we're pretty much required to implement something like suspend blocks (and, in fact, Rafael's implementation of this is already I wholeheartedly agree. But nobody has yet provided a generic approach to avoiding the loss of wakeup events, and that's an issue you can hit whenever you use full-system suspend - it's not a problem that's unique to Android. -- Matthew Garrett | mjg59@srcf.ucam.org --
Which reminds me. I forgot a very important way that user space signals the kernel as to its intents. The kernel can see whether code is sleeping by choice or sleeping because someone blocked it (eg a page fault). The kernel also knows a lot about the event being waited for (much more so in -rt than base). We don't capture all of that info but it seems it would advance progress in the right direction to do so better. Alan --
As one approach of many, this makes a great deal of sense. Thanx, Paul --
If the primary difference between sleep and suspend is not scheduling processes, instead of messing with oppurtinistic suspend/suspend blockers/wakelocks/etc, why not just 'temporarily' change the timer fuzz value to a very large value (say an hour). That would still let things like openoffice saves ahve a fair chance to trigger before the battery died completely, but would wake the system so infrequently that it will be effectivly the same as a full suspend. David Lang --
Because it only affects processes that sleep. It's a question of how much pathology you want to be able to tolerate. -- Matthew Garrett | mjg59@srcf.ucam.org --
Standard system stats will show you hogs like this. The Android people claim that wakelock stats will let the user identify processes that prevent the system from suspending properly and then remove them. If this is the case, a process that never sleeps will be even easier to identify and even more obvious an offender. If that isn't enough, then you can go back to the other idea I advanced, having some way to tell the system not to consider some processes when trying to decide if the system should sleep or not. David Lang --
Hmmm... Let's put the two approaches side by side so that we can compare them more easily: Opportunistic Suspend Idle + Timer Jitter Set timer Set timer Suspend OS delays timer Resume OS continues delaying timer Timer fires Timer fires These two cases look quite similar to me. But as you say, the battery can run out. So let's add that to the comparison: Opportunistic Suspend Idle + Timer Jitter Set timer Set timer Suspend OS delays timer Battery runs out Battery runs out Data loss Data loss The two cases still look quite similar. You might note, quite correctly, that the time between suspend and resume might be quite a bit longer than the typical time that the OS delays a timer. But the power consumption on some platforms is quite a bit lower in the suspend case than it is There does seem to be some debate (if you can call it that) over whether suspend blockers (or something like them) apply to non-Android environments, though I do understand that you believe that suspend blockers are relevant only to Android. And indeed, adding features to a particular platform can degrade portability, but the same is true of any number of There are similarities and differences. You have called out some of the similarities. Differences include the more-aggressive hardware power management on cellphones, the greater number and variety of hardware accelerators on cellphones, battery capacity, and, as you say, physical size. People currently use cellphones differently than they do laptops or desktops. The usage might converge, but we will see. There is as much reason to expect increasing specialization as there Yes, sooner or later, changes in environment and usage will break our assumptions. This includes your assumptions, my assumptions, the Android folks' assumptions, and everyone else's assumptions. We nevertheless build things anyway. A better comparison for Win16->Win32 would be UNIX's move from 16-bit Interesting choice ...
it has been stated that the android can hit the exact same power state either with sleep or suspend, and that the same clock can wake it up (it appears as a timer expiring for sleep, or an alarm for suspend, but it's the same clock firing the signal) so in at least some cases the hardware supports doing both with equal You are talking about Android as if it was a cell phone only thing, it's not. there are shipping tablets (and I believe netbooks, i.e. laptops) and wakelocks require all applications that can take a wakelock be well behaved. and applications that do nt take a wakelock directly cannot expect to run unless something else takes a wakelock on their behalf David Lang --
It indeed has been so stated. But in this section we were discussing I was talking about cellphones. But yes, Android (and thus suspend blockers) are used for tablets as well as cellphones, thank you for Almost. Suspend blockers require that only those portions of a PM-driving application that hold a suspend blocker be carefully written to avoid wasting power. Thanx, Paul --
you specifically stated that suspend would use less power. I am pointing out that ther is info posed in this thread to say that's not always the case. in either case it is possible for the system to wake up again later to let the timer fire and the application save it's work. It's arguably easier in the idle case as it doesn't require application modification for example Idle + Timer Jitter set timer OS sets timer jitter to 1 hour system sleeps for 1 hour with no wakeups timer fires, applications wake and process data system sleeps (for 1 hour or more with no wakeups) (repeat as needed until battery runs out) much less chance for data loss as there is _far_ more of a chance that the timer waking up once per hour for a short time is not going to make a noticable difference in your total battery life of a cell phone due to the significant idle power draw needed for the cell circuitry. On something like a e-reader with no radio link and a month-long idle time it could make a difference. note that this is with a badly written app running that is trying to wakeup repeatedly. If the app is well behaved and only schedule a timer when they will have work to do, they may not schedule a timer at all after the data is saved, and so would have the data safe and just as long a the fact that it's used there means that you can't argue that it's difference because cell phones are so different (unless you are saying that Android is really not appropriate for those uses) On the other hand, lots of things are used in places where it is inappropriate, Windows CE is used on phones, tablets, etc but I think most people would say that the use of windows there isn't appropriate. David Lang --
I specifically stated something different, and if you specifically look up a few paragraph, you will specifically see that I specifically used a specific qualifier, which you specifically seem to have lost track of. --
On Wed, Aug 11, 2010 at 3:42 AM, Paul E. McKenney And in N900 touching the screen doesn't bring the device out of idle, I guess because it's off. What devices do what on which circumstances on what platform is Only if you make these assumptions 1) All the applications use suspend-blockers only when they absolutely must 2) The user has given the right applications the right access If not, you'll see much worst energy efficiency. So in theory maybe, but in practice you can't say that. -- Felipe Contreras --
And if there are runnable tasks, then then don't behave the same. As long as touching the N900 screen doesn't bring the device out of Do you -really- want me to start listing counterexamples to that Ahem. This is a list of differences between idle and suspend, not a list of problems to be solved. But to answer your question, if a timer does not bring a given device out of suspend, then a range timer is not likely to, either. Don't get me wrong, I do believe that range timers have an important part to play in the energy-efficiency arena, You believe that these assumptions are unreasonable? Compared to the assumption that all applications are carefully written to conserve power? If so, on what grounds? It seems to me that the same social-engineering approaches work in Really? What makes you say that? Thanx, Paul --
On Thu, Aug 12, 2010 at 1:12 AM, Paul E. McKenney Unless they have blocked suspend. Anyway, Alan was picturing a hypothetical point in time when x86 can suspend/resume as fast as ARM, and thus the question of whether or not to enable suspend-blockers in a system that runs openoffice becomes relevant. If applications have been fixed by that time to not wake the system unnecessarily, as many of them have already been tanks to tools like powertop, then suspend-blockers would not make that much of a difference, therefore the effort required to implement suspend-blockers properly on all applications in the system, including I remember clearly Android people explaining that dynamic PM is orthogonal to suspend-blockers; if a suspend is blocked, you still want dynamic PM to reach the lower power state. Therefore the target You said "there can be a set of input events that do not bring the system out of suspend, but which would bring the system out of idle". There's no suspend (in the Android sense) in N900, only idle, and the events that bring N900 out of idle can be mapped to the set of events that bring Android out of suspend. IOW. Alan wasn't talking about idle vs suspend on the same device, he Certainly, but the context is the set of differences that would aid in the decision of whether or not to go for suspend-blockers in a general No, I think both (for opportunistic suspend and dynamic PM) are completely reasonable. But think again; if you have the assumptions met on both, then both work fine, if you don't meet them, then both don't work correctly. My point is that suspend-blockers don't magically reduce power usage, just like dynamic PM, it depends on what user-space actually does. You Yes, but if dynamic PM works as advertised, you don't need For starters an application might be holding the wakelock more than it should, also, an application might miss a timer due to not having PM permissions to hold the lock, and thus might need an expensive initialization ...
You are quite correct. I should have said something like "But if there are runnable tasks, none of which is holding a suspend blocker, then One more time, this time with feeling. ;-) Only PM-driving applications need concern themselves with suspend blockers, and experience thus far indicates that PM-driving And I have not seen anyone argue that suspend blockers are a replacement for dynamic power management. Many people instead seem to be advocating using them both -- dynamic power management for when at least one PM-driving application or kernel driver is keeping the system awake, and suspend blockers for when only power-oblivious applications are running. In contrast, you are advocating dynamic power management to the exclusion The most convincing comparisons will be of suspend vs. idle on the same device. If multiple devices are involved, then the most convincing experiments would compare suspend vs. idle separately on each device. Again, I am in no way arguing for suspend blockers to the exclusion of other approaches. Heck, I am mostly just trying to get a clear statement of the problem. In contrast, you do seem to be advocating for dynamic That is true of any artifact, software or otherwise: if you don't meet the assumptions inherent in its design and construction, the artifact might fail. Of course, you argument also applies to dynamic power management: and in fact it takes just one power-oblivious application to drain your battery -- especially if that application goes into an infinite loop. And if that application goes into an infinite loop only under some unusual set of conditions, it might have passed your test suite with flying colors, so that you might even have given it a power-management gold star, right? And yes, you can get the same effect with an infinite loop in a PM-driving application that happens to be holding suspend blockers at the time. However, in the suspend-blocker case, one need only inspect that code that holds suspend ...
On Thu, Aug 12, 2010 at 7:19 PM, Paul E. McKenney There's no experience on this. Have you tried to run a small debian system with suspend blockers enabled? I can image at least all cron daemons need modifications, probably dbus, links, lftp, wget, rsync and a bunch of applications that download data too. It's easy to speculate one way or the other, but the fact of the matter is that we don't really know how many changes are needed in order to have a functional system that actually benefits from suspend blockers. What we know is that if an application is not analyzed to see if it needs suspend blockers, and implement them if needed, it might get * if dynamic PM was perfect, spend-blockers would *not* be not needed * if suspend-blockers were perfect, dynamic *will* still be needed All I said in that sentence you are replying is that dynamic PM will The point you are trying to highlight is to which events the system reacts, that has nothing to do with dynamic PM vs opportunistic What you are doing is copy pasting a definition of what is opportunistic suspend and making it pass as an advantage. This particular point (3.) is not an advantage, over dynamic PM, it's I've cut most of your explanation. If I understand correctly what you are saying is that suspend blockers are harder to get wrong. I agree, but my point against 4. remains the same: suspend-blockers don't I don't understand this question. Dynamic PM is needed regardless. We are discussing your point 4 where you say suspend blockers inevitably Remove "but worse energy efficiency" and I think that point would be correct, albeit it's not really an argument in favor of opportunistic As it has been explained before, there's a sweet-spot of idleness: http://article.gmane.org/gmane.linux.kernel/995525 http://article.gmane.org/gmane.linux.ports.arm.omap/37982 Do you agree that there's such a thing, and if so, do you agree that the benefits of opportunistic suspend are much less once that point ...
Now this is much more like it!!! Thank you!!! You are quite correct, my last sentence does in fact assume that suspend will always have lower power consumption than does the deepest idle state. This of course sentence must be fixed. How about the following, which makes this assumption explicit, and adds words ("often", "might") that indicate that one's mileage might vary? 4. Suspend generally forces devices to go into their low-power states immediately. In contrast, idle generally leaves unused devices at full power, relying on timers to shut down these devices. Idle thus often has shorter average wakeup latencies, but on systems where suspend can use deeper sleep states than can idle, idle might It never was intended to be an argument of any kind. It was instead supposed to help clarify the difference between idle and suspend, in response to people stating that there is no difference between suspend and idle. If my mistake in the wording of the last sentence made you think otherwise, please accept my apologies. [ . . . ] (For now, anyway. I might respond to the rest later.) Thanx, Paul --
I have not. I would be surprised if anyone else did that work unless suspend blockers were in mainline. I would be very happy to be proven That would be your speculation. Perhaps you are correct, but it seems more likely that some large sets applications can be considered as a group OK, good to know! Please understand that your sentence "Therefore the target of not having unneeded runnable tasks is shared by Android folks" can be interpreted as meaning "the Android guys really don't need suspend That is certainly not the way I read the sentence I was replying to, No, I was copy pasting a definition of a difference between idle and Indeed you have! But you should be safe in doing so, as most people probably won't bother to do the web search that would lead them to http://lkml.org/lkml/2010/8/12/198 and then search that page for the I am glad you agree that suspend blockers might be harder to get wrong. As to point #4, you were correct that I badly worded the last But can dynamic power management do everything optimally? If not, some other approach may be needed in addition to dynamic power management. If you really are arguing that the Linux kernel should support no power-management mechanism other than dynamic power management, you I agree that there will be a sweet spot of idleness (though I would call it a "point of diminishing returns"), but only if all the applications are power-optimized. The advantage of opportunistic suspend is instead its tolerance of power-oblivious applications with minimal degradation And in the paragraph above, I proved to you that relying solely on dynamic power management might actually hurt. And I am not trying to prove that opportunistic suspend always leads to better results -- that is a strawman that you set up. And yes, you might have been led to set up that strawman because I messed up the wording of one of the suspend-vs.-idle definitions. Of course, had you called out that sentence to start ...
On Fri, Aug 13, 2010 at 6:14 PM, Paul E. McKenney So when it comes to practice you are relying solely on what Android people say. If it's true that it's easy to spot the "PM-driving applications", then it shouldn't be hard for a guy from the Android team to assemble a basic typical system (X.org, dbus, etc.) with suspend-blockers in a couple of days. -- Felipe Contreras --
So when it comes to practice with suspend blockers, only the Android people have any experience, so theirs is the only experience with I am happy to let the Android guys determine the best use for their time. Thanx, Paul --
sorry for the late response, the last week has been very hectic. I just wanted to note that there is already a tool in the kernel to deal with this, the timer jitter/fuzz control. This can be set by an application for itself, or it can be set by some other process for an application (I don't remember the details of all the ways this can be set) This could be used in a way similar to how userspace wakelocks are set today, if the power management process (that holds the wakelock and keep sthe screen lit today) thinks the system should be awake, let the jitter/fuzz be small, if that process thinks the system should probably be asleep, set the jitter/fuzz to a larger value. If other things are running anyway, the timers can fire and be serviced normally, otherwise the kernel is free to delay the timer going off even for badly written processes. David Lang
Indeed, Rafael and Alan Stern are working to make pm_qos do what the Android guys need, which could (very roughly) be thought of as extending the jitter/fuzz such that it meets Android's requirements. Thanx, Paul --
On Tue, Aug 10, 2010 at 7:45 AM, Paul E. McKenney No, it's the fact that *nobody* else has said: hey, that looks like a good idea, we should use that in our mobile platform (or any platform). -- Felipe Contreras --
I don't think lack of external adoption is a terribly useful data point either way at the minute. While the feature is controversial a lot of the OSs will probably hold off on it (because it's effort to handle out of tree stuff and folks are really busy) and there's not that many out there which support random externally written packages (which is the major push for exporting the feature to userspace) in the first place. --
On Mon, Aug 9, 2010 at 9:16 PM, Paul E. McKenney Ideally the system would be suspended even when the screen is on. If there are no "trusted" applications running at the same time, then openoffice wouldn't load at all. Right? -- Felipe Contreras --
My understanding is that Android systems in fact do not suspend when the screen is on, and that most (perhaps all) other systems do not opportunistically suspend at all. There has been some speculation about what a hypothetical Android having a non-volatile display might do, but as far as I know, this is just speculation. Thanx, Paul --
On Thu, Aug 12, 2010 at 1:28 AM, Paul E. McKenney I have a desktop system in mind. If opportunistic suspend is only triggered when the display is off, then it's no good for normal usage, and therefore dynamic PC needs to get its act together... specially for laptops. -- Felipe Contreras --
If I understand you correctly, you are saying that both opportunistic suspend and dynamic power control should be used together, with dynamic power control being used for short non-busy periods (as in between keystrokes) and opportunistic suspend being used for longer non-busy periods (as in while grabbing a coffee). That combination of usage sounds promising to me. That said, I don't know that anyone has really sat down and thought through how one might apply suspend blockers to a desktop system. I suspect that there are several ways to go about it. Thanx, Paul --
On Thu, Aug 12, 2010 at 4:06 AM, Paul E. McKenney Think in terms of an ARM laptop. What good is opportunistic suspend if it's not going to help when the laptop is being used? -- Felipe Contreras --
For when the laptop is not being used, presumably. Thanx, Paul --
On Thu, Aug 12, 2010 at 6:44 AM, Paul E. McKenney Right, but if you have to optimize for dynamic PM anyway for normal usage, how much would you gain by opportunistic suspend? As it has been explained before, there's a point of diminishing returns: http://article.gmane.org/gmane.linux.kernel/995525 http://article.gmane.org/gmane.linux.ports.arm.omap/37982 Now, how much would dynamic PM have progressed by the time we start thinking on opportunistic suspend on laptops (ARM, or fixed PM), 10 seconds idle? 1 minute idle? Would it make sense to rewrite *all* user-space in order to archive that little extra performance, *or* would it make more sense to keep investing on dynamic PM which we have to do anyway? All this has already been explained. BTW, the gain is even less if you consider that laptops already automatically go to suspend after a while, so the gains of opportunistic suspend would have to be measured only for a small period of time (like 30 min or so). -- Felipe Contreras --
Felipe, One thing I'm not clear on --- what's your goal? Is your goal to keep suspend-blockers out of the kernel? Is it to try to convince the android team suspend-blockers are a bad idea and to change Android to not use them? Is it to push some other agenda? Is it to discourage the Android team from trying to waste more time trying to get suspend-blockers (or equivalent functionality) from being added into the kernel? You seem to be arguing quite passionately, but I'm not sure what you're trying to do. -- Ted --
My goal is to shine light. I've heard many invalid arguments in favor of suspend blockers, I want to shut them down. In my mind it's crystal clear that independently of what opportunistic suspend is supposed to be fixing, the fact of the matter is that it's not a silver bullet as it's claimed to be. So far, nobody has refuted these: 1) opportunistic suspend needs a good behaved user-space to work properly 2) if suspend blockers are enabled in a system, *all* user-space must implement them to work correctly 3) implementing suspend blockers in user-space is not a straight-forward task 4) there's a point where sleeping (not doing work) has diminished returns So, as the length of this thread has shown, the benefits of opportunistic suspend are *dubious* at best, and more likely not worth the changes needed in user-space which eventually will get pretty close to what suspend blockers can achieve even in ideal circumstances by just doing dynamic PM. -- Felipe Contreras --
For this note that there's a fairly strong expectation that even in a phone type environment a sane userspace implementation will involve a very large portion of userspace just totally ignoring suspend blockers. This means that while it is true that userspace as a whole must have support for suspend blockers the changes required are substantially less invasive than you appear to expecting. --
On Thu, Aug 12, 2010 at 5:09 PM, Mark Brown Correct, but still a considerable amount of changes would need to be done, which _nobody_ has expressed any intention to do. Besides, IMO a good mobile platform would share as much as possible with desktop software. Say, the improvements Nokia has endorsed on the Telepathy IM framework can only help the people already using it on the desktop. However, personally, if I ever have to do './configure --enable-suspend-blockers', I would think that something that just doesn't belong has creped by to user-space. I don't see why there should something particularly different between mobile phones and laptops, and I think this has been already expressed over, and over. -- Felipe Contreras --
On Thu, Aug 12, 2010 at 9:57 AM, Felipe Contreras So, because you feel that phones should be little laptops you oppose providing (optional!) support for environments that take a different view to that? I'll echo Ted's question -- is this the opinion of the kernel community at large? If so, there's not much point in continuing to have discussions around suspend blockers. I think that we're still a ways away from a world where we can treat mobile devices the same as laptops and get reasonable user experiences. I think it's unfortunate if the attitude here is "wait and someday it won't matter", especially because I'm skeptical that we're likely to hit that "someday" any time soon. Brian --
Hi Brian, Look. I'll share with you a little experience. For years I've been working on a piece of software to access Microsoft's WLM IM service on linux. I have high quality standards for my software, so I optimized algorithms and bandwith, I used valgrind to find memory leaks, OProfile for performance bottlenecks, and so on. All this on a laptop. When I ported this code to the N900 it just worked, not more-or-less; perfectly. I could stay online the whole day without the battery running out, and I didn't have to make *any* change. Then at some point people started complaining about battery usage, but lo and behold, on the desktop people started reporting too much bandwith/cpu usage as well. I tried on my laptop, and was able to reproduce, and fix (bug related to daylight savings time). For years people have been trying to unify certain KDE/GNOME code, and freedesktop.org was born. Now the same technologies are used by Nokia on Maemo, and now MeeGo (DBus, Telepathy, GStreamer, tracker, etc.), so the optimizations done for mobile benefit everyone; desktop power usage also improves. So, yeah, I think the fact that my laptop and phone share the same Why do you keep ignoring the N900? It has a lot of components that come directly from the linux ecosystem and people are having more than Nobody is waiting for anything. There's hard work all over the place to improve power usage. If you think suspend blockers are useful for you, great, use them. I just don't see why they should be merged if nobody is going to use them but you. But maybe you are right, and maybe dynamic PM alone never reaches the same levers of performance, then people would ask for it to be merged, and it will. No biggie. In the meantime, it would be wise to remember that this is not Nokia/Intel vs Google. Nokia has invested a great deal in dynamic PM, and those changes have benefited Android products directly (I even remember somebody from Google directly thanking Nokia for that), and that ...
Aren't there many drivers in the kernel sources which are only used by very few users? Android is used by million users, albeit on one platform, but I don't see this to be a valid reason to keep this functionality out of the kernel. Best, Dominik --
The question is not whether suspend blockers are a silver bullet (in my opinion there are no silver bullets), but rather whether or not suspend Really? From what I can see, only PM-driving applications need to use Fortunately, experience thus far has shown that only a small fraction of applications need to use suspend blockers. (Or perhaps you are instead saying that the implementation of the suspend-blocker infrastructure The length of this thread (and the ones preceding it) is mostly due to people talking past each other. For example, the Android folks seem to believe that it is important that relatively unskilled people be able to write simple apps, and that the system nevertheless be able to run these apps in a relatively energy efficient manner. Your proposals do not address this issue. This might be because you are not aware of this desire, because you are not aware of the computing history that argues in favor of this requirement, or because you simply don't like this requirement. Whatever the reason, until you face this requirement head on, either addressing it or proving that it need not be addressed, you will continue to be talking past the Android folks. Thanx, Paul --
On Thu, Aug 12, 2010 at 8:43 PM, Paul E. McKenney "PM-driving applications" is a new invention, so how do you know if an application belongs to this category or not? Some application might be non-PM-driving most of the time, but suddenly become PM-driving. Well, you have to analyze *all* of them. Think about this... is bash a PM driving application? No, but what if you run: 'sleep 3600 && alarm.sh'. Perhaps I should rewrite that as: Wrong. We don't have any experience on that at all on typical linux Perhaps half of the thread, or even one quarter of the thread can be attributed to that, but still the rest I think it's because people This "requirement" is specific to Android's user-space, isn't it? Not Ubuntu, not Fedora, not MeeGo, not anyone with a typical user-space seems to be having this problem. I can argue to you that this problem can be solved in easier ways, but instead I will argue that perhaps we should wait for somebody besides Android to complain about it before providing a "solution". Because after all, what good is a "solution" provided by the kernel, if the user-space is not going to use it, ever. -- Felipe Contreras --
On Thu, Aug 12, 2010 at 12:34 PM, Felipe Contreras I'm curious, when does Android count as a user of the kernel? I gather that volume of sales or users doesn't count. Do we have to include some percentage of "desktop" Linux? If we're an undesirable second-class citizen, why do people care that "android is forking the kernel"? I guess I don't understand, Brian --
I guess I don't, either -- there are some parts of the kernel used by only a handful of users... and here we speak about million users... Best, Dominik --
On Thu, Aug 12, 2010 at 10:52 PM, Dominik Brodowski A driver that sits in a corner and doesn't show in the 'menuconfig' of most people and is used by small fraction of user-space applications (perhaps even 0) is just fine. But we are talking of something centric, that can be enabled on all systems, and that affects all user-space. -- Felipe Contreras --
Because in general the bits for a small user base don't leak out into other bits of the kernel. It's a much higher barrier if you want to affect other people's code because it is asking everyone else to help maintain stuff they don't need. That's not to say sometimes its not the right choice, but I still find it remarkable that nobody else but Android seems to want it,. Alan --
> Why do people care that "android is forking the kernel"? A lot of us don't. That's what the GPL is about. It's a big pain if you start splitting it off into two entire things long term - because your maintenance grows harder exponentially but parallel forks are common - SuSE and RHEL Enterprise type editions are forks for example. As far as I can tell it all comes from one or two people with a big PR presence and almost no code in the kernel saying silly things years ago which the IT press see as a good way to create a 'we need a column, bash someone' story. "What the junk press think" is a common but broken way to run national policy in some countries, and its just as broken for the kernel. Alan --
You are *one* user of the kernel. Let's suppose Android wasn't using suspend-blockers, and there was another equally successful linux mobile platform, Foobingo, and they were using them, and they were the only ones interested on implementing them. What does that change? Nothing. They still need to convince the community that what they are proposing to be merged is actually useful, and somebody will use it. If not, they can just keep the patch Nobody has expressed anything remotely like that (that you are a second-class citizen). Why makes you think so? Lots of people get patches denied. Like Nokia's u_char driver, which is *way* less controversial than this one. -- Felipe Contreras --
So the current users of the Linux kernel are the following? o GNU/Linux o Android Do any other distributions or devices with unusual user-space layouts qualify? --
QTembedded distros are somehow similar to android (one big binary). Then there's minor stuff, like webOS and Motorola A1200... Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html --
It does sounds like there is some variety, then. Thank you for the info! Thanx, Paul --
Glad we agree that both dynamic power management and opportunistic power management need applications to be written carefully. Of course, dynamic power management requires all of the code in those applications to be carefully written, while experience indicates that opportunistic A PM-driving application is one that exerts control over the system's power state. In the case of Android, a PM-driving application is one That is an excellent example, as it applies equally to dynamic power Android's experience might not apply exactly to typical Linux ecosystems, but we really can learn quite a bit from it -- at least if we can bring Fair enough. I wasn't differentiating between people mistakenly talking past each other and intentionally talking past each other, but if you At this point in the discussion, I am quite prepared to believe that you will avoid using suspend blockers, and that you will further do everything in your power to prevent anyone else from using suspend blockers. ;-) Thanx, Paul --
On Fri, Aug 13, 2010 at 6:22 PM, Paul E. McKenney I already mentioned that a "PM-driving application" might not have suspend-blockers; because the user denied them, or because the Huh? Certainly not days, which if Android guys are right, might be how Not really. Say you have 100 packages in your system, how do you know which ones would be PM-driving? Can you grep for something, or see if they open certain file? No, you have to analyze them one by one; they *all* are affected, although not all might require modifications. But assuming I'm wrong, that's precisely the reason why a practical I'm not tying anybody's hands. How are people using real-time linux if it's not on mainline? Well, duuh, you apply the patches. If say Fedora was interested on it, they could apply the patches, and see for themselves. People do that all the time, with the mm tree, with Con Koliva's patches, etc. Once people are happy with the results, things get merged. Why should this be any different? -- Felipe Contreras --
Because millions of users are happy -- with Android, including suspend blockers. Best, Dominik --
I'd guess there are on the order of 20 users that have built and deployed a kernel with Android suspend blockers support. The total deployed systems is relatively meaningless. --
Well, it wasn't meaningless wrt noveau: "I've heard all the excuses. If it isn't ready, they shouldn't ship it to millions of people. And if it's ready, they should work on merging it. No excuses. Linus" So let's get the suspend blockers to Linus. Best Dominik --
On Fri, Aug 13, 2010 at 6:57 PM, Dominik Brodowski I explicitly said somebody besides Android, specifically, somebody with a typical linux ecosystem. You are not addressing the argument at hand, that nobody else wants to tackle the issue this way, thus only making the discussion more difficult. -- Felipe Contreras --
Can we stop arguing about the pointless?
The facts are that suspend blockers identifies a race within our suspend
to ram system that permeates from top to bottom (that's from server to
mobile). The problem is that resume events are racy with respect to
suspend and vice versa. This manifests itself most annoyingly on my
laptop in the "double suspend" case: where I suspend with a pending
suspend event, my laptop will resume and then immediately re-suspend
(leading me to kick myself and remind myself to check it stayed up
before pushing unsuspend and walking away). The other annoying case is
that if I accidentally close the lid before presenting, I have to wait
until the system is fully down before pressing resume.
In a Data Centre controlling power, if you sent a suspend then a wake on
lan, there's a window where the machine will still be down (because the
wol got ignored).
There are easy fixes to all the above ... I should wait to verify
suspend and resume in my laptop and I have to accept the wait time
between the two. In the data centre, you just repeat your power control
commands a few times with about 5s between them and so on.
The simple hacky work arounds mean that a user space invasive solution
like suspend blockers is a bit of a non starter as a solution to the
general case. However, it has shown that we do have a problem and
furthermore it's a problem encountered by more than android.
The technical problem with suspend blockers is that they're a solution
to a general problem that only works for a specific case. What we're
searching for is a general solution that can also be used in the android
specific case.
So far, we have three possibilities:
1. Stubs with deprecation - this has been rejected by android, so
looks like a non starter.
2. update pm_qos so that the suspend blocks become qos constraints.
This may or may not be coupled with a user space suspend
manager, but in the latter case it's essentially full ...This is all true, but it's also only one aspect of the problem. I agree with you that this is the part of the problem which affects Linux at all scales, from Cloud servers in a data center that want to suspend themselves when there's no work to do (and then fail to respond to the WOL packet) to mobile platforms that are suspending much more frequently. However, it doesn't follow that this is the _only_ problem that the Android folks might be interested in solving. Opportunistic suspend is a different part of the problem space, which is generally believed by the Android developers as being far more efficient than a user-space suspend manager. Rafael has stated his complete unwillingness to deal with this part of the problem. OK, so that probably means that for Android, it will have to be an out-of-tree kernel patch. The question, then, is whether a solution which addresses the only part of the problem which Rafael is interested in dealing with at this point, is sufficient such that (a) the kernel-level opportunistic suspend can be done as an out-of-tree patch, while simultaneously (b) allowing device drivers for Android devices can utilize Rafael's interfaces to solve the race design bug currently found in our suspend subsystem, while (c) requiring minimal changes to the Android userspace, and (d) providing all of the statistics and debugging functionality required by the Android userspace. If we can engineer a solution which meets (a), (b), (c), and (d) above, then everyone will be happy. - Ted --
Arve's suspend blockers patch stack actually separates the core functionality (ability for drivers to delay suspend while doing work suspend would interfere with), from the ability to hold suspend blockers from userspace (a separate, smaller patch building on the core functionality). Brian --
OK, so I tried desperately to avoid the question of whether opportunistic suspend is a good way of managing power. However, it seems to me that it is in use by several systems (android, olpc, etc). I'll defer the question of whether it's better in user space or kernel space to Rafael's investigations ... but I will point out that the kernel space patch, once the suspend blockers issue is taken care of looks like a single patch to one file, so should be locally containable That's my goal. James --
In fact, we (which means basically Alan Stern and me at this point) are working with Arve on this right now. Thanks, Rafael --
Hi, At least for (d) we - Nokia - would be also interested as well to contribute, see [1]. In general it should help with keeping the solution more generic. cheers, igor [1] https://lists.linux-foundation.org/pipermail/linux-pm/2010-August/028248.html --
Or in time between keystrokes for most of the platform (backlight excepted). The Intel MID x86 devices are at the point that suspend/resume time on x86 is being hurt by the kernel rewriting smp alternatives as we go from 2 processors live to 1 and back. Alan --
Given that you are talking about going from 2 processors to 1 and back, I would guess that you are not actually talking about suspend/resume, which is a system-wide thing rather than a CPU-by-CPU thing. I am not sure whether you are using CPU hotplug or invoking SMP alternatives once all but one CPU is idle. Can't say that I can tell exactly what pit you are digging for me here. ;-) Thanx, Paul --
When entering system suspend, we disable non-boot-CPUs to simplify locking. We reenable them when going out of suspend. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html --
Thank you for the info, Pavel! So once you are down to one CPU, the last CPU shuts the system off, itself included? Or does the last CPU "run" in a deep idle state throughout suspend? (My guess is the former, and I am also curious whether the cache SRAMs are powered off, etc. But figured I should ask rather than guessing.) Thanx, Paul --
On Sat, 14 Aug 2010 08:10:48 -0700 they tend to go "off". however I think you're making an assumption that there is a real difference between a deep idle state and "off".... For modern x86 hardware, that assumption isn't really valid. (other than a very very small sram that stores register content in the idle case) -- Arjan van de Ven Intel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org --
I am and have been taking you at your word that some systems can reach power levels while idle that rival suspended/off. The differences between idle and suspend are instead semantic, have been posted here more than once, and make themselves felt when the non-suspended system is non-idle, even for systems whose deep-idle power approximates that of suspend/off. Thanx, Paul --
Really? I thought the TSC stopped in suspend. Does having a package go into C6 mean that the RAM goes into self-refresh? -- Matthew Garrett | mjg59@srcf.ucam.org --
This is more of CPU statement than a chipset or laptop/device-wide statement, correct? If "hardware" applies to the entire laptop, how "modern" is "modern"? - Ted --
Rather beside the point for this thread and it's perhaps not so intuitive as it ought to be but holding down the home button should do what you're looking for here. --
Yes, the N900 is drastically different, for starters, it has an actual window manager. By multi-tasking I mean me (the user) being able to perform multiple tasks at the same time. For example: writing an email, while browsing the web, while having IM conversations. Obviously not exactly at the same time; start writing an email, go browse for some url, copy, answer a pending IM message, Let's concentrate; Android is the only mobile platform that has expressed interested in suspend blockers. UI applications in Android are written specifically for Android. Period. That's not my argument at all. I was talking about a counter-argument to "suspend blockers make porting desktop apps easer". Android's UI applications are unimportant here because they have not been ported from the desktop realm; they were designed specifically for Android, including all its PM capabilities. The only relevant applications are the ones designed for the desktop that are ported to Android, and thus might make assumptions that hurt battery life. These are background services. If you are saying that there are few background apps, then that's an argument against suspend blockers: 1) few applications can be ported 2) the few applications that can be ported, being background services, might miss timers and behave worst than _without_ suspend Power is just like any other resource, why are desktop applications not using 100% CPU, or 100% of memory? Because if they did nobody would be using them. There's a social pressure to use less resources, or at least less resources than the competence. If an application uses too much battery time nobody would use it, unless perhaps if the functionality is too good and there are no alternatives. I believe social forces already deal with this problem, all we need to do is provide better tools, not patronize user-space applications. -- Felipe Contreras --
I think you're wrong that this will be sufficient, but I've already stated my position before, namely that what you do when you only have 800mWh has very different performance/battery tradeoffs than when you have 94,000mWh batteries. One of the reasons why my mail archive for this discussion has over 1800 messages and is close to 10MB is that everyone keeps saying the same thing over and over again. So I'm not going to say it again. I was thinking that the only way we can tell is for us to go away and come back in two years, and we can see whether or not Meego is getting the same 200,000 activations per day that Android is, and maybe *then* people will understand. However, earlier today, I had a very productive conversation with an engineer from Nokia who has had many years of experience of doing power management for cell phones, and who is now trying to make Meego power efficient enough for cell phones, and he seemed to think that problem which suspend blockers or wakelocks are trying to solve was valid. I now believe that part of the problem is that is that many Meego folks only have an experience with Netbooks or Laptops, and don't appreciate that sometimes, when you make such a radical change in battery capacity to a mobile handset, things become qualitatively different, and not just quantitatively different. Put another way, a laptop has six hours of battery lifetime with 94,000 mWh worth of battery; a cell phone needs to be able to be usable over a 20-24 hour period of despite having a 800-1200 mWh battery --- and what you need do for these two are different. This is very similar to how trying to make a kernel scale to 512 NUMA nodes is quite different to trying make a kernel work with 4-8 SMP cpu's. Until you've actually tried doing it, you might think that all you have to do is just throw in a bunch of spinlocks and rw spinlocks. But it turns out it's a lot harder than that --- but it's very hard to convince someone who hasn't had that experience to ...
Applications need to take smart decisions to save power based on a number of things: is the screen on? is the ear close to the speaker? is there a network available? is it a cellular connection? has the user decided to automatically disconnect from the network after certain time, or at night? etc. I think that, and range timers / ip heartbeat is enough. But that, as you say has been discussed before. Now, only Android has decided to use suspend blockers, that's a *fact*, and I wanted to narrow the discussion to Android in order to make it easier to understand that Android doesn't need suspend blockers, once we have agreed that, then I'd gladly discuss it's Are you kidding? Even if MeeGo fails in being as popular as Android, that doesn't saying anything about whether suspend-blockers are sensible or not. Besides, there is also the possibility that Android So? I'm sure there's some people in Google that believe that Right. Nokia people only have experience with laptops. Surely my N900 I argued to you that suspend-blockers are not required in Android, and suddenly you decide we should agree to disagree without arguing back? Well, suit yourself. I still maintain that suspend-blockers is just an expensive workaround, and in some cases actually degrades power consumption; the right solution is much more sophisticated. -- Felipe Contreras --
On Wed, Aug 11, 2010 at 2:25 PM, Felipe Contreras On behalf of the Android folks, we don't agree with this. If you're going to wait until we suddenly change our minds, I think you're going Once "the right solution" exists and solves our problems, we'll certainly look into switching over to it. I've yet to see a proposal in all this arguing that appears to me to be an improvement over what we have today with suspend blockers. I find the "don't do what you're doing because someday, somebody will do it better" to be an uncompelling argument. Given your opinion that Android lacks multitasking (what? really?) and various other strange statements about the platform, I'm likely to be taking your suggestions with generous helping of skepticism. Brian --
I'm sure as a team that's the case, but you can't know what's in the That was not an argument, it was an opinion. If you want an argument go back to read this one: This is what I'm talking about when I say multi-tasking, Android certainly doesn't have anything remotely like that: http://www.youtube.com/watch?v=7emvUBpEkbU -- Felipe Contreras --
On Wed, Aug 11, 2010 at 3:03 PM, Felipe Contreras Home + pick app -> switch to the app in whatever state it was in. Home (longpress) -> recent app list + pick app from popup -> switch to recent app in whatever state it was in. pull down alert windowshade + pick app from alerts list -> switch to that app I routinely flip back and forth between email / browser / IM / musicplayer / etc I have no problem with you perhaps liking the visuals of the N900 better, but to say Android lacks multitasking (as you explain it As far as implementing app switching with a deck of cards metaphor, I've seen that done by some OEMs with Android, though it's not the default system UI. Brian --
If the app stops running, that's *fake* multi-tasking, and doesn't match my experience; I can't remember the details, but I tested it on a Nexus One, and the item I was looking at was gone, I had to scroll again. Also, I cannot go to last.fm on the browser, play some music, and do other stuff at the same time, can I? Anyway, what you do in Android is your problem. The point is that in Linux we need good PM *with* multi-tasking (not really an argument against opportunistic suspend, I think, but a clarification from Ted's comment) -- Felipe Contreras --
On Wed, Aug 11, 2010 at 5:46 PM, Felipe Contreras That'd be surprise. Last I checked I was able to launch 40-50 apps on N1 before we even started getting low on memory and the system won't kill apps off unless we completely bottom out on memory. It's possible that some app might do the wrong thing when going to the I'm not sure off the top of my head if the browser will keep music/flash playing on pages when it's not the foreground app, but you can certainly run something like Pandora (or a number of other streaming music apps) in the background and play streaming music while you do other stuff. There's no reason the browser *couldn't* do this Well I'm glad we agree that multitasking and PM are important. Brian --
On Sat, Aug 7, 2010 at 1:54 AM, Paul E. McKenney These definitions are wrong. 1) There are trusted applications that misbehave (the user clicks Yes when asked about PM permissions) 2) There are untrusted applications that are power optimized (The user clicks No) The proponents of suspend blockers in user-space have tried to ignore this fact, but the truth is that PM permissions and power optimization are orthogonal to each other. -- Felipe Contreras --
These would be buggy PM-driving applications. Of course, any type of I did indeed exclude this category by saying "power-optimized applications are those PM-driving applications that have been aggressively tuned to reduce power consumption." The reason I excluded this case is that that there are a number of cases where removing the PM-driving attribute could destroy the power optimization. If you have an example power-optimized application that retains its power-optimized property despite lacking PM-driving privileges, please put it forward. In doing so, please keep two things in mind: 1. The definition of power-optimized is more aggressive than many people are used to -- look at http://lkml.org/lkml/2010/8/4/409 and search for "POWER-OPTIMIZED APPLICATIONS". 2. In a number of the implementations that do not use suspend blockers, -all- applications are in effect PM-driving applications. So it is quite possible that different people are using subtly different flavors of the "PM-driving applications" definition. For example, an Android person might consider a application to be PM-driving only if it is permitted to use suspend blockers, while a Maemo person might consider an application to be PM-driving if it had permission to invoke pm_qos functions. If these two hypothetical people each try to apply their definition of PM-driving to each others' platforms, they will likely have severe problems communicating with each other, right? ;-) But if you do have a good example, perhaps I will need to change my In fact, you might have noticed that the proponents of each platform have been quite persistent in shouting past each other from the warmth and security of their own particular viewpoints. ;-) Thanx, Paul --
