| === Logan_ is now known as Guest83048 | ||
| === Guest83048 is now known as Logan_ | ||
| === GridCube is now known as GridNet | ||
| pitti | Good morning | 04:25 |
|---|---|---|
| pitti | smoser: infinity reconfigured the VMs to use 4 GB of RAM, to avoid over-committing (that wreaked havoc, as VMs encountered kernel paging errors) | 04:26 |
| pitti | smoser: three VM qemus still need to be restarted; it's not urgent any more as the total RAM is now sufficient, but I understood he wanted to do it at some point | 04:26 |
| pitti | smoser: we couldn't do it on Tuesday with a build queue of 500 items or so (glibc + qt), but now things are quiet | 04:27 |
| === doko_ is now known as doko | ||
| doko | jamespage, maven main alarm | 07:19 |
| === dbarth_ is now known as dbarth | ||
| === dupondje_ is now known as dupondje | ||
| === Spads_ is now known as Spads | ||
| LocutusOfBorg1 | pitti, simple question: will vivid have systemd by default? I don't see online anything related ATM | 09:55 |
| seb128 | LocutusOfBorg1, http://summit.ubuntu.com/uos-1411/meeting/22401/systemd-transition/ | 10:00 |
| seb128 | UOS session today | 10:00 |
| LocutusOfBorg1 | wow thanks! | 10:02 |
| LocutusOfBorg1 | seb128, quick question | 10:03 |
| LocutusOfBorg1 | vivid universe package universe/net/boinc-client requires systemd migration (sysv=True, upstart=False, systemd=False) | 10:04 |
| LocutusOfBorg1 | if you sync it from experimental you fix the issue :) | 10:04 |
| seb128 | LocutusOfBorg1, depending strictly on systemd before it's default seems buggy | 10:05 |
| seb128 | or do you mean the experimental version resolves that? | 10:05 |
| seb128 | what's the question exactly? ;-) | 10:06 |
| LocutusOfBorg1 | yep, I added the service file in the experimental upload | 10:07 |
| LocutusOfBorg1 | I was lookit at the blueprint link See: http://people.canonical.com/~jhunt/systemd/packages-to-convert/ (updated daily). | 10:08 |
| LocutusOfBorg1 | since I do not want to spare anybody's time I already did the file and uploaded on experimental | 10:08 |
| seb128 | LocutusOfBorg1, why not unstable? | 10:09 |
| LocutusOfBorg1 | freeze :/ | 10:09 |
| seb128 | isn't 'make it work with the default init system' a value reason to get a freeze exception? | 10:10 |
| LocutusOfBorg1 | don't know :) | 10:10 |
| LocutusOfBorg1 | asking now on release channel | 10:11 |
| seb128 | thanks | 10:12 |
| LocutusOfBorg1 | <jcristau> it very much isn't. | 10:13 |
| seb128 | k | 10:13 |
| LocutusOfBorg1 | I can upload anyway, and revert if an RC steps up | 10:13 |
| LocutusOfBorg1 | it is not a major release | 10:14 |
| pitti | LocutusOfBorg1: right, aiming for it, but it depends on getting more hands on porting upstart-only packages (still ~ 200 to go ) | 10:23 |
| LocutusOfBorg1 | pitti, I understand, systemd handles the systemV scripts, but not the upstart ones? | 10:27 |
| LocutusOfBorg1 | what about patching systemd then? :) | 10:28 |
| pitti | LocutusOfBorg1: to do what? :-) | 10:29 |
| LocutusOfBorg1 | to avoid porting creating 200 service files prior to vivid release :) | 10:34 |
| LocutusOfBorg1 | it will give us a "smoother" transition | 10:34 |
| LocutusOfBorg1 | but maybe most of that 200 packages just needs to be sync'd over from debian I guess | 10:34 |
| pitti | LocutusOfBorg1: well, there was an alternative proposal to run upstart alongside systemd | 10:34 |
| pitti | but nobody looked at that | 10:34 |
| LocutusOfBorg1 | I guess you will avoid that ;) | 10:35 |
| pitti | LocutusOfBorg1: nope; those 200 are mostly the ones where we have heavy ubuntu development/deltas; mostly in the cloud/server/touch space | 10:35 |
| LocutusOfBorg1 | oh... ok | 10:35 |
| pitti | LocutusOfBorg1: conceptually upstart and systemd are so far apart that it's pretty much impossible to "just run" upstart jobs by systemd | 10:35 |
| LocutusOfBorg1 | anyway creating a .service file is something easy for a person with enough knowledge I guess, I never saw a service file more than 20 lines | 10:36 |
| LocutusOfBorg1 | long | 10:36 |
| pitti | right, every individual one isn't hard, it's just cumulatively hard due to sheer number | 10:36 |
| LocutusOfBorg1 | anyway, I guess you know what to do, who am I? :) | 10:36 |
| LocutusOfBorg1 | yes of course, 200 is an high number | 10:36 |
| LocutusOfBorg1 | I guess I'll try to help a little bit, but I'm really not a systemd person (at least not now) | 10:37 |
| pitti | so if you want to help out with submitting missing systemd units (or even just init.d scripts), that'd be greatly appreciated | 10:37 |
| LocutusOfBorg1 | you mean vivid/main packages right? I see 191 left | 10:38 |
| LocutusOfBorg1 | grub-common seems a false-positive, hostname might just need a sync/revert, ifupdown needs a little merge? | 10:43 |
| LocutusOfBorg1 | oh... many of them seems to be sysv=True, so grep "sysv=False" |wc -l returns just 106 entries | 10:45 |
| LocutusOfBorg1 | I hope I'm on the right list http://people.canonical.com/~jhunt/systemd/packages-to-convert/2014-11-13.txt | 10:45 |
| pitti | right | 10:53 |
| pitti | jodh_: ^ I think we should really start with that -- if a package has an init.d script, it's fine for now | 10:54 |
| pitti | then it doesn't block the switch and thus shoudln't be part of the transition | 10:54 |
| LocutusOfBorg1 | yep, 106 seems a better number :) | 10:54 |
| LocutusOfBorg1 | and of the 106 many are still false positive, like "upstart*" | 10:55 |
| LocutusOfBorg1 | or "systemd*" | 10:56 |
| LocutusOfBorg1 | I guess you need to start with sysv=False && systemd=False | 10:56 |
| LocutusOfBorg1 | grep "sysv=False" |grep "systemd=False" |wc -l | 10:57 |
| LocutusOfBorg1 | 91 | 10:57 |
| LocutusOfBorg1 | even better | 10:57 |
| LocutusOfBorg1 | Laney, how do you feel about this patch for the transparency problem of gnome.-terminal? https://bug695371.bugzilla-attachments.gnome.org/attachment.cgi?id=274727 | 11:41 |
| LocutusOfBorg1 | bug 1292282 | 11:41 |
| ubottu | bug 1292282 in gnome-terminal (Ubuntu) "background transparency is not working on gnome terminal" [Low,Confirmed] https://launchpad.net/bugs/1292282 | 11:41 |
| Laney | LocutusOfBorg1: what transparency problem? | 11:41 |
| Laney | -> #ubuntu-desktop maybe | 11:41 |
| LocutusOfBorg1 | well yes | 11:42 |
| cjwatson | LocutusOfBorg1: grub-common isn't entirely a false positive, I just haven't yet worked out how to do that "run as late as possible to approximate the system being fully booted" thing in systemd | 11:46 |
| cjwatson | It's not urgent though, as the sysv file should do the job for the time being | 11:47 |
| LocutusOfBorg1 | cjwatson, I meant "also a debian issue" :) | 11:47 |
| cjwatson | Sure | 11:48 |
| === MacSlow is now known as MacSlow|lunch | ||
| cjwatson | bdmurray,ev,pitti: ubuntu-rtm has Contents now, would be good to confirm that retraces can now work | 12:50 |
| pitti | cjwatson: hooray, thanks! | 12:56 |
| === MacSlow|lunch is now known as MacSlow | ||
| pitti | barry: FYI, autopkgtest 3.8 is now in vivid, that also contains the funny apt fix | 13:02 |
| jamespage | doko, ack | 14:02 |
| jamespage | doko, not much time today but will look as soon as I can | 14:02 |
| === alai` is now known as alai | ||
| barry | pitti: awesome, thanks | 14:22 |
| === davidcalle_ is now known as davidcalle | ||
| smoser | pitti, stgraber i've rebooted host on wolfe | 14:40 |
| smoser | and restarted your instances there. | 14:40 |
| pitti | smoser: ack, thanks | 14:42 |
| stgraber | smoser: thanks | 14:45 |
| stgraber | smoser: is it just my imagination or did we get a memory downgrade in those VMs? | 14:48 |
| pitti | might be that infinity reconfigured your's as well | 14:48 |
| pitti | the intention (from my POV) was to just reduce mem from 8 to 4 GB for wolfe-02 to -09 (the autopkgtest ones) | 14:49 |
| stgraber | yeah, looks like we got downgraded from 8GB to 4GB, which may be a slight problem for us since we run all our builds in tmpfs as I/Os were horribly slow on there (10min on tmpfs vs 1h30 on disk last time I tested) | 14:49 |
| pitti | stgraber: I think the wolfe host now has enough RAM left to run your's with 8 | 14:51 |
| pitti | rebooting the autopkgtset ones free'ed 12 GB, and we were under the "64 GB total" line before | 14:52 |
| infinity | stgraber: What are you building that has such awful I/O characteristics? | 14:55 |
| stgraber | infinity: tons of debootstraps | 14:55 |
| infinity | stgraber: In theory, wolfe should be no different than denneed, fisher, and postal, and they perform quite well. | 14:55 |
| stgraber | usually 3-4 of those in parallel | 14:56 |
| infinity | stgraber: Ask smoser to start your guests with cache=unsafe | 14:56 |
| infinity | stgraber: Then watch 'em fly. | 14:56 |
| stgraber | infinity: yeah, cache=unsafe on the data volume would probably help quite a bit | 14:57 |
| pitti | infinity: do they all need to have the same size? | 14:57 |
| pitti | infinity: because we certainly do have the RAM now | 14:58 |
| infinity | pitti: No, they don't have to be. He can certainly have bigger ones. | 14:58 |
| infinity | pitti: But "I like building in tmpfs" is also a lousy reason to eat all the RAM. ;) | 14:58 |
| pitti | well, who doesn't :) | 14:58 |
| infinity | Exactly. | 14:58 |
| * pitti builds nowhere else at home | 14:58 | |
| pitti | also, if we can get the same cache=unsafe hack on "my" wolfes, I wouldn't complain loudly either (and it might even be easier as all VMs use the same config) | 14:59 |
| stgraber | infinity: well, my concern was that our testsuite and image building scripts do a ton of IOPs so tmpfs obviously make things faster for us and also means we don't hammer the disks for everyone else on the box | 14:59 |
| stgraber | but cache=unsafe may very well do that too just at a different layer :) | 14:59 |
| pitti | can you just swap the stickers on the motherboard, so that we have 1 TB RAM and 64 GB disk? should be enough to fit the VM images :) | 15:00 |
| pitti | (TGIF!) | 15:00 |
| infinity | As long as no one cares about losing the occasional syslog or whatever, cache=unsafe isn't really all that unsafe on a VM where the base system doesn't change much. | 15:00 |
| pitti | for my VMs, I don't care at all; if they break, I run my magic script to set up a fresh one | 15:01 |
| infinity | (Okay, it's horribly unsafe, but if you don't like your data...) | 15:01 |
| * pitti likes "fast" better than "safe" in this case | 15:01 | |
| stgraber | infinity: well, that's why we've got two disks on those VMs right? I sure don't want to loose data on the main volume, but the data volume I don't care about | 15:05 |
| stgraber | so setting cache=unsafe for /dev/vdb would be perfectly fine | 15:05 |
| infinity | stgraber: Do you have a host I can kill and restart to test this? | 15:13 |
| infinity | s/host/guest/ | 15:13 |
| infinity | stgraber: I hacked up smoser's guest-start script to start the ephemeral image with cache=unsafe, want to test that I did it right. :P | 15:16 |
| caribou | arges: here are the debs : http://people.canonical.com/~lbouchard/makedumpfile-1.5.7-3/ | 15:16 |
| infinity | pitti: Or if you have one I can kill. | 15:17 |
| infinity | pitti: (And are you using the same root/data split?) | 15:17 |
| infinity | Looks like you probably are. | 15:17 |
| pitti | infinity: kill 07 or 08 (or both) | 15:17 |
| pitti | infinity: yes, I mount /dev/sdb1 as /data | 15:18 |
| pitti | which has the containers and logs | 15:18 |
| pitti | but as I said, nothing on these machines is precious | 15:18 |
| stgraber | infinity: you can kill 01. Just make sure that only sdb is affected, sda pretty much never changes and is a pain to setup so I'd like that one to be properly saved to disk :) | 15:21 |
| infinity | That looks like it worked (for wolfe-08) | 15:22 |
| infinity | stgraber: Want to halt 01 for me, or shall I just kill it? | 15:22 |
| infinity | pitti: Check wolfe-08 seems happy, that's the one I twiddled. | 15:23 |
| stgraber | infinity: shutting down now | 15:23 |
| pitti | infinity: yep, already at it | 15:23 |
| infinity | If this hack makes you guys happy, it's a simple diff to scott's guest-start that just applies cache=unsafe to all eph* images. | 15:24 |
| infinity | stgraber: 01 should be back. | 15:25 |
| infinity | smoser: http://paste.ubuntu.com/9007931/ | 15:26 |
| infinity | smoser: That's what I applied to wolfe. | 15:27 |
| infinity | stgraber: If that's not responsive enough for you, we can give you more RAM too, but I suspect that should be pretty decent. | 15:27 |
| pitti | infinity: doesn't really feel faster to me (doing another lxc-create), but at least it seems to work | 15:27 |
| infinity | Well, I suppose it's also possible cache=unsafe is a noop for non-virtio... | 15:28 |
| infinity | I have no idea. | 15:28 |
| pitti | eek, this isn't virtio? | 15:28 |
| infinity | qemu and I, we're not best buddies. | 15:28 |
| infinity | pitti: No. virtio is buggy as heck on !x86. | 15:29 |
| infinity | pitti: This is ibm-vscsi, which is pretty performant, in my testing. | 15:29 |
| stgraber | root@wolfe-01:/var/lib/lxc# dd if=/dev/zero of=blah.img conv=fsync bs=4M count=1000 | 15:29 |
| stgraber | 1000+0 records in | 15:29 |
| infinity | But a bunch of VMs all contending for the same disk will hurt any I/O subsystem. | 15:29 |
| stgraber | 1000+0 records out | 15:29 |
| stgraber | now to test with a real load | 15:29 |
| stgraber | 4194304000 bytes (4.2 GB) copied, 3.09727 s, 1.4 GB/s | 15:29 |
| infinity | stgraber: Well, the real test is if unsafe is doing as advertised, and not passing syncs down to the host. | 15:30 |
| infinity | stgraber: Since that's the slow down for debootstrap/dpkg. | 15:30 |
| stgraber | infinity: still get 1.5GB/s with conv=sync so it looks like it's ignoring syncs as expected | 15:31 |
| stgraber | the fs is also mounted with data=writeback for good measure too :) | 15:32 |
| pitti | I'm running lxc-create under eatmydata, add force-unsafe-io to dpkg, but it still sucks :/ (there is a looong time during debootstrap when eatmydata isn't active) | 15:32 |
| stgraber | pitti: you could use the download template, that'd at least make that part faster | 15:35 |
| pitti | stgraber: I tried that a while ago, and there was some reason why it wasn't practical, but I forgot; probably firewalled or so | 15:35 |
| stgraber | root@wolfe-01:/var/lib/lxc# grep proxy /etc/environment | 15:36 |
| stgraber | http_proxy=http://squid.internal:3128 | 15:36 |
| stgraber | https_proxy=http://squid.internal:3128 | 15:36 |
| stgraber | then it works fine | 15:36 |
| stgraber | it may also have been that back then there were no ppc64el images | 15:36 |
| pitti | ah | 15:36 |
| pitti | stgraber: or that | 15:36 |
| pitti | hm, jenkins-slave fails to start now, WTH | 15:38 |
| stgraber | infinity: test build was about as fast as on tmpfs, so I'm fine with sticking to that | 16:13 |
| infinity | stgraber: \o/ | 16:13 |
| infinity | stgraber: Did you have two guests that needed this treatment, or just that one? | 16:13 |
| stgraber | I just have the one | 16:14 |
| infinity | Kay, cool. | 16:14 |
| * infinity logs out of wolfe and wanders off. | 16:14 | |
| stgraber | 16min vs 14min and since that's still about 4x as faster than our armhf builder, we're good :) | 16:15 |
| stgraber | with the previous I/O performances I was getting on wolfe, it was slower than my armhf devboard (which arguably does SATA-3 and uses a SSD) which seemed a bit ridiculous :) | 16:15 |
| infinity | stgraber: Yeah, I imagine that was just insane I/O contention with every guest on there syncing like crazy. | 16:16 |
| infinity | stgraber: Since the machine has 9 guests, all of whom pretty much just run dpkg a whole lot. | 16:16 |
| infinity | stgraber: And qemu/kvm performs very poorly in that scenario with default settings. | 16:16 |
| stgraber | clearly | 16:16 |
| infinity | stgraber: The only thing to watch out for with cache=unsafe is that it really is very unsafe. A poorly-timed crash could mean that filesystem needs reformatting on reboot. | 16:19 |
| stgraber | which is fine because that's what I do at every reboot anyway :) | 16:20 |
| infinity | Handy. | 16:20 |
| stgraber | I never want crap sticking around in /var/lib/lxc | 16:20 |
| === MacSlow is now known as MacSlow|errand | ||
| dpm | willcooke, slangasek, shadeslayer, who is doing the developer track summary? | 17:06 |
| dpm | i.e. the presentation at the end | 17:06 |
| willcooke | I guess me | 17:07 |
| dpm | cool, thanks willcooke | 17:08 |
| dpm | gaughen, and I guess you're presenting the summary for the cloud track? | 17:10 |
| gaughen | dpm, well actually I have a conflict. trying to see if i can re-arrange it. | 17:10 |
| gaughen | it's with a partner | 17:10 |
| dpm | gaughen, thanks. Otherwise, perhaps another cloud track lead can do it? Just let us know | 17:11 |
| === JanC_ is now known as JanC | ||
| === MacSlow|errand is now known as MacSlow | ||
| pitti | jodh_: ah, thanks for updating the systemd conversion lists! | 17:59 |
| jodh_ | pitti: np - scripts are here fwiw: http://people.canonical.com/~jhunt/systemd/scripts/ | 18:00 |
| === Elbrus_try_again is now known as Elbrus | ||
| pitti | wgrant: can we move the langpack schedule to produce RTM langpack exports on Tuesday evening or Wednesday early morning? | 19:02 |
| pitti | ogra_: ^ so when exactly do you want the new packs in RTM? then we can calculate back from that | 19:02 |
| pitti | about 1.5 h to prepare the sources, upload them, have them build, and propagate through -proposed, and about 6 hours for the export, plus two hours safety margin | 19:03 |
| ogra_ | pitti, as per https://wiki.ubuntu.com/LandingTeam/MilestoneSchedule we are freezing wed. morning (EU time) now ... | 19:04 |
| pitti | ogra_: does the freeze include the langpacks, or you want to freeze, then export/upload langpacks, to catch the last string changes before freeze? | 19:05 |
| ogra_ | pitti, well, we freeze to be able to do image rebuilds for single fixes ... would eb good if everything langpack realted would be in place when the freeze starts so it doesnt taint tested images that get re-rolled | 19:06 |
| pitti | sure | 19:07 |
| pitti | wgrant: so could we kick off RTM exports around Tue 21:00 UTC? then I'd build packs around Wed 05:00 UTC, and we would be ready to build images around 07:00 UTC | 19:07 |
| pitti | ogra_: ^ does that fit? | 19:07 |
| ogra_ | that sounds good, yeah | 19:09 |
| pitti | ack; I'll adjust the cronjobs as soon as I get wgrant's ok | 19:09 |
| jodh_ | pitti, slangasek: just fixed and re-run http://people.canonical.com/~jhunt/systemd/scripts/ - outstanding main packages to convert figure is now 77. | 19:10 |
| slangasek | jodh_: neato, thanks | 19:10 |
| pitti | wgrant: if that collides with some other exports, I'm happy to move the other stuff around too | 19:10 |
| pitti | jodh_: yay; can you fix it some more? :-) | 19:10 |
| slangasek | jodh_: btw, do we know if any of these are packages that just require no-change rebuilds with current debhelper? | 19:10 |
| pitti | jodh_: ah, and there's things like "upstart" and "mountall" which don't need conversion | 19:11 |
| slangasek | you mean I don't get to write a mountall systemd unit? | 19:11 |
| pitti | jodh_: could this grow a "known false positive" blacklist? | 19:11 |
| jodh_ | slangasek: no - it's a very crude script atm :) | 19:11 |
| jodh_ | slangasek: ... like it even shows that upstart and upstart-bin needs conversion so atleast a few false-positives there :) | 19:12 |
| slangasek | sure | 19:12 |
| pitti | or ureadahead | 19:12 |
| slangasek | jodh_: oh, and there's no separation of user session jobs vs. system jobs | 19:13 |
| pitti | or hostname | 19:13 |
| slangasek | (indicator-*) | 19:13 |
| jodh_ | pitti: yes, it need to. | 19:13 |
| jodh_ | slangasek: right. still some whittling to do. | 19:13 |
| slangasek | jodh_: thanks for making this script! Let us tell you everything that's wrong with it | 19:13 |
| pitti | oh right, all the indicator bits are session upstart | 19:13 |
| slangasek | ;) | 19:13 |
| jodh_ | slangasek: yeah, I should be asking you for patches :) | 19:13 |
| pitti | which is kind of fair, as we need to vet them for signals | 19:13 |
| pitti | jodh_: is that based on xnox's scripts? ISTR that they spat out quite a bit more | 19:14 |
| * pitti waves good night | 19:15 | |
| pitti | /qui/quit | 19:15 |
| pitti | ok, let's try that again :) | 19:15 |
| jodh_ | pitti: no - it's a rewrite in python that uses a local mirror for speed. It's now here: https://code.launchpad.net/~ubuntu-foundations-team/+junk/migration-to-systemd | 19:37 |
| dannf | i'm looking at a vnc4 upload, rebasing on debian to pull in arm64 support. i noticed that, though it has been rebased since, it is retaining some old ubuntu changelog (<= hardy): http://paste.ubuntu.com/9014774/ | 23:29 |
| dannf | should i follow suit and keep the recent ubuntu entries, or just add a Merge, Remaining changes: entry to the top? | 23:29 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!