| === henrix_ is now known as henrix | ||
| apw | rtg, i am pushing bits for the i386 cloud tools, so expect unreleased changes in -meta for you next upload | 09:53 |
|---|---|---|
| rtg | apw, ack | 09:53 |
| === ming is now known as Guest82927 | ||
| shadeslayer | jsalisbury: apw remember our discussion from http://irclogs.ubuntu.com/2014/06/10/%23ubuntu-kernel.html#t16:51 | 12:56 |
| shadeslayer | jsalisbury: apw we were wondering if we can change to CFQ in 14.04 as well | 12:56 |
| shadeslayer | https://bugs.launchpad.net/ubuntu/utopic/+source/kubuntu-settings/+bug/1378789 | 12:56 |
| ubot5 | Ubuntu bug 1378789 in kubuntu-settings (Ubuntu Trusty) "[SRU] Set the default IO scheduler to CFQ in Kubuntu Trusty" [Undecided,New] | 12:56 |
| shadeslayer | jsalisbury: apw would be awesome if you could comment there about your thoughts | 12:56 |
| apw | there is little chance of us changing default IO scheduler in a released OS | 12:57 |
| shadeslayer | apw: this would be for Kubuntu | 12:58 |
| apw | shadeslayer, iirc you are changing it via udev rules right ? | 12:58 |
| shadeslayer | and more specifically, it'd be a udev rule, which is loaded post boot ( I think ) | 12:58 |
| shadeslayer | apw: yes | 12:58 |
| apw | ok, so what i am i commenting on :) | 12:58 |
| shadeslayer | apw: ScottK wanted to know if this was alright with the kernel team | 12:59 |
| shadeslayer | apw: let me get him in here :p | 12:59 |
| shadeslayer | ScottK: so if I understood it correctly, you just wanted to ask the kernel team if this was alright for 14.04? | 13:00 |
| ScottK | I'd like to get an appreciation of the potential risks associated with the change to assess if it's SRU worthy. | 13:00 |
| shadeslayer | apw: fwiw we've already changed in 14.10, so far no issues for the last month that we know of | 13:00 |
| ScottK | Right, but the level of risk we generally accept in the development series is different than for an SRU. | 13:01 |
| shadeslayer | ScottK: right, just bringing apw up to speed with the current situation in Kubuntu :) | 13:02 |
| * apw concurs on that | 13:02 | |
| apw | we moved away from cfq because it was creating performance instabilities on interactive workloads | 13:02 |
| apw | but you are relying on a cfq specific feature, so you are somewhat hosed | 13:02 |
| cking | ..I guess what ever choice is made there is always a particular usecase that can show up performance issues with specific usecases | 13:03 |
| ScottK | apw: How long ago did we switch from cfq? | 13:04 |
| * shadeslayer thinks that we're circling back to the original conclusion of : Benchmarks are outdated | 13:04 | |
| * ScottK wonders if it might be better now. | 13:04 | |
| apw | ScottK, it was somewhere arond precise i think, a long time ago | 13:05 |
| shadeslayer | ScottK: something that was bought up in the original discussion, no one has up-to-date benchmarks ( this was 3 months ago ) | 13:05 |
| * cking has a peek at the latest data to remind himself of the current position | 13:05 | |
| apw | ScottK, yeah it might be better, i cann't recall if we have done anything since | 13:05 |
| apw | cking, is stats master | 13:05 |
| shadeslayer | also, benchmarking depends highly on the kind of workload | 13:05 |
| * ScottK starts to imagine a trip to the tech board on this one (which is not terribly unusual for "we think we should SRU, but it has risks." | 13:07 | |
| cking | shadeslayer, exactly, and it is kind of tricky to select a set of tests that cover every application's useage patterns. it's like 10 pin bowling, you kind of aim for the best all round choice, but you know there may be some pins on the side that you may miss | 13:07 |
| shadeslayer | ScottK: *groan* | 13:07 |
| shadeslayer | cking: right | 13:07 |
| ScottK | shadeslayer: You weren't around when we released an X update for Dapper that broke X ~everywhere. | 13:08 |
| ScottK | It kind of sucked. We have the rules in place for a reason. | 13:08 |
| cking | http://kernel.ubuntu.com/~cking/fs-tests/sept/test/kernel-trends/iometer-emu-light/index.html | 13:08 |
| Riddell | this is only changing a setting back to the upstream default is it not? | 13:09 |
| ScottK | shadeslayer: My alternate thought was wait for a month after 12.10 is released to see if issues come up. | 13:09 |
| cking | oops, I mean: http://kernel.ubuntu.com/~cking/fs-tests/sept/test/kernel-trends/ | 13:09 |
| apachelogger | Riddell: yes | 13:09 |
| shadeslayer | ScottK: right, but IO Schedulers are supposed to be drop in replacement ... | 13:09 |
| shadeslayer | unless there are very intrusive patches | 13:09 |
| shadeslayer | that screw up things | 13:09 |
| shadeslayer | which I would imagin there are not | 13:09 |
| shadeslayer | but then, not a kernel dev | 13:09 |
| ScottK | If IO schedulers are drop in for each other, then logically there's no need to change. | 13:09 |
| ScottK | Not the flaw in that logic. | 13:10 |
| cking | you should be OK to drop in the change to cfq, but like all things, there may be issues with that which cause other performance drops compared to deadline | 13:10 |
| apachelogger | yeah | 13:10 |
| ScottK | Not/Note | 13:10 |
| apachelogger | that logic is so flawed it hurts | 13:10 |
| ScottK | Actually the logic is correct, it's the premise that's broken. | 13:10 |
| apachelogger | deadline does not implement a priority system because by design deadline deadlines, regardless of the priority | 13:10 |
| apachelogger | it's kinda the point of deadline | 13:10 |
| ScottK | Right. By definition these things are not just drop in replacements for each other. | 13:11 |
| apachelogger | of course addressing requests as they deadline instead of as they make sense hdd-sector-wise or as they are prioritized as it were is kind of bad in spinning media world | 13:11 |
| Riddell | it seems fairly obvious that the major io user in kubuntu is baloo, baloo just needs upstream defaults, it does seem like a bug that ubuntu changes those and the whole kde desktop suffers | 13:13 |
| apachelogger | ScottK: they are drop in replacements in that they both will schedule your IO and your IO is going to happen at some point $soon, the meaning of soon is divergent | 13:15 |
| ScottK | Right. It would have been much easier to address pre-release. | 13:15 |
| apachelogger | which is also why one shouldn't make assumptions about the soonyness of things | 13:15 |
| === sr105|aw` is now known as sr105 | ||
| cking | it would also be interesting to see what kind of i/o patterns baloo is doing as well | 13:17 |
| ScottK | Is there something like bootchart for capturing data like that? | 13:18 |
| cking | blktrace perhaps, http://smackerelofopinion.blogspot.co.uk/2009/10/block-io-layer-tracing-using-blktrace.html | 13:19 |
| Riddell | we know baloo needs cfq and suffers with deadline, vishesh has done lots of tests of that | 13:19 |
| cking | Riddell, any references on that analysis? | 13:20 |
| cking | ScottK, http://smackerelofopinion.blogspot.co.uk/2009/10/blktrace-baded-tools-seekwatcher.html | 13:21 |
| shadeslayer | btw, I think that argument was made primarily because Baloo uses ioniceness | 13:21 |
| shadeslayer | which was provided only by CFQ | 13:21 |
| apachelogger | yep | 13:21 |
| apachelogger | we know baloo won't suck with CFQ because CFQ takes ioniceness into account ^^ | 13:21 |
| cking | ah, so it's kinda niche i/o tuning | 13:21 |
| shadeslayer | so baloo itself doesn't suffer performance degradation when using deadline, other apps on the desktop do | 13:22 |
| apachelogger | well, technically everything suffers | 13:23 |
| apachelogger | as the problem is that since baloo does loads of read requests it will constantly have requests hitting the deadline once it reached the upper ceiling of the HDD's capabilities | 13:23 |
| apachelogger | after that point deadline will basically be performing in the worst case scenario where it constantly addresses deadlined requests + one batch of queued requests | 13:24 |
| apachelogger | and that in turn causes pointless HDD seeking as it does no longer access things in a way that makes sense sector-wise, so more stuff is going to deadline and the entire system is slowing down | 13:25 |
| cking | with this in mind, it's worth revisiting this for U+1 for sure | 13:33 |
| * cking wonders what baloo did before ioniceness was added to the kernel | 13:36 | |
| apachelogger | cking: didn't exist then I think | 13:37 |
| apachelogger | and the previous thing was broken by design and implementation, so most people didn't actually use it ^^ | 13:37 |
| cking | I think that feature landed in 2.6.26 ish | 13:37 |
| apachelogger | yeah, baloo only got invented like 2 years ago | 13:38 |
| cking | ah, it's interesting that one is now kind of locked into just using cfq because of the behaviour required by baloo. I wonder if that is totally acceptable | 13:39 |
| apw | whether its totally reasonable, indeed | 13:46 |
| apachelogger | it is somewhat peculiar, but I for one lack the knowledge to argue any better approach to the problem | 13:47 |
| * apw idly wonders how the other indexers we use on the desktop work without hammering the machine to death | 13:48 | |
| * ScottK thinks "not very well" | 13:53 | |
| === amitk_ is now known as amitk | ||
| hallyn | apw: someone reported that in 3.15 overlayfs switched from 'upperdir=' to 'workdir='. that doesn't seem to e the case in utopic though. d oyou know anything about it? | 14:09 |
| hallyn | in particular an easy way for lxc to tell which it needs to use? (or whether it'll be reverted upstream) | 14:09 |
| hallyn | i don't see how chnaging that could possible meet the 'don't mess with userspace' mandate | 14:09 |
| hallyn | (https://git.kernel.org/cgit/linux/kernel/git/mszeredi/vfs.git/tree/Documentation/filesystems/overlayfs.txt?h=overlayfs.v24) | 14:10 |
| apw | hallyn, well its outside the kernel so ... it doesn't get held to any kind of compatibility standard | 14:10 |
| hallyn | oh. right | 14:10 |
| hallyn | it'd be nice if either upperdir was always supported or there was a clean way to tell which to use from a script though :) | 14:10 |
| apw | hallyn, we haven't switched to the latest version because it has another incompatible change, which i need to think about before we could | 14:10 |
| hallyn | ok | 14:10 |
| apw | hallyn, indeed .. sigh ... | 14:11 |
| hallyn | for now i'll just say "tough noogies" :) | 14:11 |
| * apw will look into it | 14:11 | |
| apw | hallyn, that doc says you need a new option workdir which would need to point to a private space, lost+found stylee for use with the new atomic rename swap stuff they are using | 14:12 |
| hallyn | oh so they still take upperdir? | 14:13 |
| apw | hallyn, that is my reading of it yes | 14:19 |
| apw | mount -t overlayfs overlayfs -olowerdir=/lower,upperdir=/upper,\ | 14:19 |
| apw | workdir=/work /merged | 14:19 |
| apw | has all three in it | 14:19 |
| hallyn | apw: ok, thanks | 14:26 |
| * hallyn checks whether tha tis optional | 14:26 | |
| hallyn | doesn't look so | 14:26 |
| apw | hallyn, well we won't be taking that version for a bit yet, don't know if it is for us you are worrying of course | 14:27 |
| hallyn | apw: well that too, but also whoever reported it, who presumably has a bug as a result :) | 14:28 |
| hallyn | i'll ask neal about it | 14:28 |
| apw | you might be able to always supply it ... | 14:28 |
| apw | even though it is ignored | 14:28 |
| hallyn | nope | 14:31 |
| apw | so attempt ot mount with, fail and attempt to mount without, and hope | 14:32 |
| hallyn | classy | 14:32 |
| apw | i know, very kernel | 14:33 |
| hallyn | thx apw | 14:33 |
| * apw feels dirty | 14:34 | |
| hallyn | i'm having a very scatterbrained week | 14:34 |
| davmor2 | apw: stop plating on windows then | 14:35 |
| slangasek | cking: hi, so I replied to your mail because I didn't understand your response | 17:32 |
| slangasek | maybe we can discuss here | 17:32 |
| slangasek | apw: "other indexers on the desktop" - don't you remember? they were all horrible and we either got rid of them, or stopped noticing the impact at the same time we all switched to SSDs | 17:36 |
| slangasek | ok, caught up on the discussion here wrt baloo | 17:37 |
| slangasek | that's some useful context | 17:37 |
| cking | slangasek, sorry, I was having some food | 17:37 |
| cking | CFQ is the only one that handles ioniceness | 17:37 |
| slangasek | cking, apw: what I would like to understand is, do we actually have benchmarks measuring cfq vs. deadline for rotational vs. ssd? Because it seems there's quite a relevant difference in behavior; are we sure deadline is actually the right default on rotational? | 17:37 |
| slangasek | (i.e., is there a reason that we shouldn't consider the udev rule that kubuntu-settings has implemented to be correct across the board?) | 17:38 |
| slangasek | maybe we still wouldn't want to SRU such a change into the kernel even if we thought it was correct | 17:38 |
| cking | slangasek, I need to look up my historical data on this, which may take me a while to do | 17:38 |
| slangasek | but I'd like to have a clearer understanding of what we do think is correct | 17:38 |
| slangasek | cking: ok; no hurry for my part, but I'd appreciate having that info | 17:38 |
| cking | slangasek, all choices are correct and also wrong depending on the use case :-/ | 17:39 |
| apw | i'd have to defer to cking's numbers, i know we thought not previously | 17:39 |
| === bjf[afk] is now known as bjf | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!