[00:56] <adie> hi
[00:56]  * adie huggles TingPing 
[00:58] <adie> hi #ubuntu dev people, I have a problem in unity launcher API stuff, and I was wondering if you can help me out :O
[00:59] <adie> I am trying to set a msg count number in hexchat with python launcher.set_property ('count', count), and it works the first time, but if I reload the script is breaks like this: http://pastebin.com/raw.php?i=eudXgAcG
[01:00] <adie> I was wondering if there was something I can do to clear out whatever is preventing the script from working upon second launch
[01:09] <sarnold> adie: you didn't miss any responses while gone..
[01:09] <adie> okay, ty :|
[01:24] <adie> :<
[01:25] <sarnold> adie: still nothing while you were gone. perhaps try in twelve-sixteen hours, more people will be around
[01:26] <adie> :D maybe
[01:26] <adie> sorry for quit/join
[01:26] <adie> have to restart the client to make the script work :D
[01:26] <sarnold> ah :) hehe
[01:26] <TingPing> adie, if only it were possible to have two clients open at the same time.. hmm
[01:31] <adie> TingPing, I wouldn't be surprised if this script would interfere with two client instances
[01:33] <adie> targeting the same external resource
[01:33] <TingPing> adie, well you would be wrong
[01:33] <adie> Isn't that the rule?
[01:33] <adie> "Adie is always wrong"
[01:34] <TingPing> the error has something to do with loading the glib library from the the same python library twice i guess
[02:12] <micahg> adie: #ubuntu-unity might be of some help
[02:13] <adie> okay
[05:07] <pitti> Snow-Man: yes, unfortunately there is no way to ship a logrotate file which works on both < 3.8 and >= 3.8, so during package build we have to create a file for a particular version
[05:07] <pitti> Snow-Man: FYI, unstable will switch to 9.3 as soon as it gets released (beta2 just came out)
[05:07] <pitti> Good morning
[05:08] <adie> mornin
[07:42] <mwhudson> hi
[07:42] <mwhudson> apache2-mpm-worker-dbgsym seems to be uninstallable
[07:43] <dholbach> good morning
[08:10] <mwhudson> https://bugs.launchpad.net/ubuntu/+source/apache2/+bug/1196440
[11:23] <maxoiaojun> cjwatson: ping
[11:24] <cjwatson> maxoiaojun: just leave your message directly, please
[11:25] <maxoiaojun> cjwatson: I believe pad.lv/1035136 is not fixed even on saucy, please try "/usr/lib/lsb/install_initd" if you have saucy box
[11:36] <xnox> bug #1035136
[11:38] <cjwatson> maxoiaojun: I've followed up in the bug.
[11:41] <maxoiaojun> thanks, i'm firing a saucy vm
[11:42] <cjwatson> maxoiaojun: A chroot should be enough.
[12:08] <jibel> hallyn, stgraber with kernel 3.10 I get the following error when I create an LXC container
[12:08] <jibel>       lxc-start 1372673552.852 ERROR    lxc_cgroup - Invalid argument - write /sys/fs/cgroup/devices/lxc/saucy-i386-20130701-1012/devices.deny : Invalid argument
[12:09] <jibel> hallyn, stgraber does this kernel requires a config change of the containers?
[12:10] <jibel> I don't see this error with 3.9.0-7
[12:10] <maxoiaojun> a bit confused now
[12:10] <maxoiaojun> https://bazaar.launchpad.net/~ubuntu-branches/ubuntu/saucy/lsb/saucy/files and what installed on saucy box is different?
[12:12] <xnox> maxoiaojun: the new lsb is not in release pocket yet, above imports from release pocket with a propagation delay.
[12:13] <hallyn> jibel: no, nothing like that should have changed.
[12:14] <maxoiaojun> xnox: but the bzr branch is behind what is actually installed
[12:14] <jibel> hallyn, okay, I'll report a bug
[12:15] <cjwatson> I ignored the branch
[12:15] <cjwatson> Also, in any case, the files you're probably talking about are modified in debian/rules
[12:15] <cjwatson> (Looks like the branch is up to date otherwise)
[12:16] <cjwatson> maxoiaojun: https://bazaar.launchpad.net/~ubuntu-branches/ubuntu/saucy/lsb/saucy/view/head:/debian/rules
[12:16] <cjwatson> $(twotothree) -n -w debian/lsb-core/usr/lib/lsb/*
[12:17] <maxoiaojun> i see
[12:18] <cjwatson> Not the way I prefer to do things myself, but whatever
[12:20] <maxoiaojun> how about raring and quantal then, I propose that we just change rules so that /usr/lib/usb will continue to use python 2
[12:21] <cjwatson> I'd rather we backported the fixes
[12:23] <maxoiaojun> because it's all rules magic
[12:24] <maxoiaojun> i guess just remove the python => python3 magic for /usr/lib/lsb/ is enough
[12:25] <maxoiaojun> https://bazaar.launchpad.net/~ubuntu-branches/ubuntu/raring/lsb/raring-proposed/view/head:/debian/rules#L104
[12:25] <cjwatson> Well, whatever.  I'm not going to do the work here.  But I think it'd be wiser to bring quantal/raring more into line with saucy rather than going backwards, as a general principle of SRUing
[12:25] <maxoiaojun> https://bazaar.launchpad.net/~ubuntu-branches/ubuntu/quantal/lsb/quantal-proposed/view/head:/debian/rules#L104
[12:26] <cjwatson> It's already running 2to3 over things there so it's not that big a change to expand that
[12:26] <cjwatson> And it would be a more direct backport
[12:26] <cjwatson> See bug 798192
[12:27] <cjwatson> In fact this bug is a dup of that one, I think
[12:30] <maxoiaojun> indeed, but 2to3 is not enough for the scripts in quantal and raring I guess
[12:32] <maxoiaojun> as someone mentioned "sudo sed -i "s/python3/python/" /usr/lib/lsb/install_initd"  as a workaround
[12:32] <cjwatson> maxoiaojun: The bug fixed in 4.1+Debian9ubuntu1 was that 2to3 wasn't run over the proper files.  Backport that
[12:32] <maxoiaojun> i see using tested python 2 code better than using untested 2to3 generated python 3 code
[12:34] <cjwatson> I'm concerned about pulling python (2) back into environments where it wasn't needed before; and in general I prefer straight backports of what's in the development release for SRUs, where at all possible
[12:35] <ogra_> also testing happens in saucy anyway, if there are bugs, fices can be backported easily
[12:35] <ogra_> *fixes
[12:35] <jibel> hallyn, I filed bug 1196518
[12:35] <cjwatson> But, as I say, I'm not the one who's going to be doing the work here, so you don't have to persuade me
[12:35] <jibel> apw, ^
[12:35] <xnox> diverging further increases maintenance cost as the next fix will have to be written twice....
[12:36] <ogra_> xnox, ++
[12:37] <maxoiaojun> but the python 2 code is well tested by debian i guess
[12:37] <apw> jibel, oh goodie
[12:38] <ogra_> maxiaojun, that doesnt make maintaining two patchesets less work
[12:38] <ogra_> -e
[12:39] <maxiaojun> but the lsb package is already in different versions anyway
[12:39] <maxiaojun> you can check the rules file, saucy one is very different from quantal/raring one
[12:40] <maxiaojun> you know, saucy contains lsb4.1-blah while older releases all lsb4.0
[12:42] <hallyn> jibel: thanks
[12:44] <ScottK> barry: I see that socket.error is now deprecated: http://www.python.org/dev/peps/pep-3151/ - Is it deprecated, but won't be dropped for a long time because it's widely used or deprecated and I better make sure I don't use it because it's going away soon?
[12:45] <barry> ScottK: the former, most likely.  i wouldn't expect it to go away before 3.5 at the earliest, if ever
[12:48] <ScottK> barry: Thanks.
[12:55] <stgraber> jibel: LXC appears to work fine here, can you pastebin your container config?
[13:22] <stgraber> jibel: nevermind, just saw the bug report now
[13:32] <barry> xnox: ping
[13:32] <xnox> barry: hola =)
[13:33] <barry> xnox: hi!  there's a comment in emacs24 24.3+1-1ubuntu1: - debian/emacsVER.desktop: Also set StartupWMClass for bamf and gnome-shell.  have you by any chance seen the icon break for tab completion after this?
[13:33] <barry>  
[13:35] <maxiaojun> https://bugs.launchpad.net/ubuntu/+source/lsb/+bug/1035136 branches linked
[13:35] <xnox> barry: define break? after this patch the icon has become a beautiful svg emacs bubble with a fountain pen; before this patch it was an ugly pixalated xmp icon instead.
[13:35] <xnox> barry: what DE do you use?
[13:35] <barry> xnox: unity
[13:36] <xnox> same
[13:36] <barry> xnox: i get a nice big question mark on a beautiful gray background
[13:36] <jibel> stgraber, sorry was otp. The setup is a bit special because it uses 2 loop-mounted devices + an overlay
[13:36] <xnox> barry: i can quickly test in a clean / new environment.
[13:36] <barry> xnox: great, thanks.  i see it on two saucy desktops now
[13:37] <jibel> stgraber, but downgrading to 3.9.0-7 clearly fixed the problem and we could dozens of test without any issue.
[13:50] <xnox> barry: all works correctly in a saucy VM. Do you have custom .desktop files? are you launching emac24 or emacs which happens to be update-alternatives launching emacs24? are you starting emacs via upstart or some-such?
[13:50] <ogra_> hallyn, i need to link a dir across from the containers /dev to the hosts /dev (there are sockets the binary android blobs create) ... i used to use a link from /dev/socket into /proc/$lxc_pid/root/dev/socket but seems a normal user cant access that, any idea how to do that more elegant and with permission for normal users ?
[13:51] <barry> xnox: launching emacs via gnome do.  i wonder if that would matter
[13:51] <barry> xnox: nope
[13:52] <xnox> barry: emacs (symlink) or emacs24 ? can you type "emacs24" in gnome-do?
[13:52] <barry> xnox: so i see the right icon in both the hud when i use the launcher to start it, and in gnome do when i use that to start it
[13:52] <barry> xnox: but either of those, and starting from the shell, all give me broken icons
[13:53] <xnox> weird. no idea.
[13:53] <barry> dang
[13:53] <barry> not that it's a huge deal of course
[14:02] <jibel> xnox, jodh there is a regression in upstart 1.8-0ubuntu1.1 in Raring and bug exists in Saucy (bug 1124384 and bug 1195955)
[14:03] <hallyn> ogra_: well you should be able to get around the permission issue by doing a bind mount instead of a link
[14:03] <hallyn> ogra_: probably the more robust way would be to use mounts propagation to cause the contaienr's / or /dev to be mounted someplace where the host can see it
[14:04] <hallyn> ogra_: is the container's /dev a static dir, or a tmpfs?
[14:05] <jodh> jibel: yeah, looks like a typo from the last uploader
[14:05] <ogra_> hallyn, android uses a tmpfs for /dev
[14:05] <ogra_> and we hold our boot until thats propagated
[14:06] <hallyn> ogra_: what do you mean by propagated?
[14:06] <hallyn> populated, or are you actually propagating the mount already?
[14:06] <stgraber> hallyn: IIRC you can't bind-mount /proc/<pid>/root/<something> (or at least it wouldn't let me do that the last time I tried)
[14:07] <xnox> jibel: that's mine.
[14:07] <ogra_> hallyn, like ubuntu has udev, android uses ueventd to propagate the tmpfs /dev
[14:07] <ogra_> stgraber, right, i had the samme issue, thus the link
[14:08] <ogra_> which worked fine until rsalveti came around :P
[14:08] <ogra_> (trying to access it as normal user)
[14:08] <stgraber> ogra_: btw, you mean "populate" not "propagate" :)
[14:08] <ogra_> errr, yeah
[14:09] <hallyn> ogra_: ok, i think you should make the container's rootfs --make-rshared
[14:09] <ogra_> what does that chare and how ?
[14:09] <ogra_> *share
[14:09] <hallyn> set lxc.rootfs.mount to /container/rootfs in the container config,
[14:10] <stgraber> hallyn: won't help much as the final setup (once we're fully flipped and loop-mounted) uses a tmpfs rootfs mounted from the container ns
[14:10] <ogra_> we want the container to stay haf way safe ...
[14:10] <hallyn> and on the host do mount --bind /container/rootfs /container/rootfs; mount --make-rshared /container/rootfs
[14:10] <stgraber> hallyn: so the rootfs isn't visible outside of the container mountns
[14:10] <hallyn> stgraber: I think I might need a pic :)
[14:11] <stgraber> hallyn: basically /var/lib/lxc/android/rootfs is empty (or in ogra's case doesn't even exist), at boot time we have a pre-start which mounts a tmpfs on top, then unpack a cpio initrd, then does a bunch of stuff and finally boots that as rootfs
[14:11] <hallyn> stgraber: that shouldn't matter
[14:12] <hallyn> stgraber: so long as you make lxc.rootfs.mount = /container/rootfs, then the actualy rootfs that you pivot-root into will be /container/rootfs
[14:12] <stgraber> hallyn: but as all of this happens in pre-start, we're already in the container mountns so the rootfs isn't visible from the host, which means that if we were to use rshared, we still wouldn't be able to see it
[14:12] <hallyn> and if you make that rshared, then the mounts from the child container should be reflected on the host
[14:12] <hallyn> stgraber: the rootfs you're building isn't visibile in the host,
[14:12] <hallyn> but when you mount that onto /container/rootfs to pivot-root to it, you should see that on the host
[14:13]  * ogra_ thinks he slowly gets it
[14:13] <hallyn> i coudl be wrong, as this is being held together by duct tape, but that's what I"d expect :)
[14:13] <stgraber> ah, let me try that
[14:13] <hallyn> (duct tape == bind mounts :)
[14:13] <ogra_> :)
[14:13] <hallyn> cool.  meanwhile i've got someone telling me they can't provide apport info bc the bug is happening on fedora.
[14:21] <ogra_> hallyn, so adding "lxc.rootfs.mount = /var/lib/lxc/android/rootfs" .... makes the container sigsev
[14:22] <ogra_> nothing further in the logs
[14:25] <ogra_> (or did you mmean literally /container/rootfs ?)
[14:25] <hallyn> ogra_: yeah :)
[14:25] <ogra_> doesnt work either
[14:25] <hallyn> does that directory exist?
[14:26] <ogra_> do i need to create /container soemwhere ?
[14:26] <ogra_> heh, snap :)
[14:26] <hallyn> mind you i wouldn't expect /var/lib/lxc/android/rootfs to segv...  it should just bind mount that onto itself then pivot to it.
[14:26] <hallyn> ogra_: and after you create /container/rootfs, remember to make it rshared
[14:27] <hallyn> so mkdir -p /container/rootfs; mount --bind /container/rootfs /container/rootfs; mount --make-rshared /container/rootfs
[14:27] <apachelogger> heya, any kernel guys around?
[14:28] <ogra_> hallyn, well, the container doesnt start
[14:30] <hallyn> i can't reproduce quite your setup, but let me try a simpler case here
[14:30] <hallyn> ogra_: which release is this on, and which lxc package?
[14:30] <ogra_> apt-get source lxc-android-config ...
[14:31]  * stgraber fights with make-rshared being a bit stupid
[14:31] <ogra_> hallyn, saucy latest lxc package indeed
[14:31] <hallyn> k
[14:31] <ogra_> stgraber, does your container start at all ?
[14:32] <stgraber> ogra_: yes, mine is fine because I'm using my loop-mounted images which have a saner lxc setup
[14:32] <ogra_> why didnt you pull  that into the non loop one :P
[14:32]  * ogra_ wants sanity too !
[14:32] <stgraber> no point we'll get rid of those eventually and I have no way of testing them :)
[14:32] <ogra_> phablet-flash --flipped
[14:32] <ogra_> :P
[14:33] <ogra_> easy to test
[14:34] <hallyn> jibel: so just to be clear, you created that container completely by hand, writing your own config?  If not, can you add steps including lxc-create command to the bug?
[14:35] <ogra_> stgraber, point is that we need a fix for flipped atm, it doesnt help if it works for you in the loop case
[14:35] <ogra_> (we want to swtich to flipped by default today, that one is a blocker)
[14:36] <stgraber> ogra_: well, my container works with a lxc.rootfs set, rshared still doesn't so getting you past the lxc.rootfs won't help much
[14:36] <bjf> bug 1196363
[14:37] <bjf> ^ this is saucy, 100% reproduceable
[14:37] <ogra_> stgraber, hmm, k
[14:37]  * ogra_ would at least like to know what keeps his container from starting at all :(
[14:38] <hallyn> ogra_: can you do lxc-start -l info -o outout and pastebinit outout?
[14:38] <hallyn> i'm still waiting for my testbed to install saucy...
[14:39] <ogra_> root@ubuntu-phablet:/# cat outout
[14:39] <ogra_>       lxc-start  947224102.538 INFO     lxc_start_ui - using rcfile /var/lib/lxc/android/config
[14:40] <hallyn> uh
[14:40] <hallyn> dat not good
[14:40] <hallyn> ogra_: in that case can you strace it?  (strace -f -ooutout2 lxc-star ...)
[14:40] <ogra_> we start from an upstart job btw ...
[14:40] <ogra_> (on which other boot bits depend)
[14:41] <ogra_> bah, sigh, cant install strace without NM running
[14:42] <stgraber> ogra_: wget http://ports.ubuntu.com/ubuntu-ports/pool/main/s/strace/strace_4.5.20-2.3ubuntu2_armhf.deb + adb push strace_4.5.20-2.3ubuntu2_armhf.deb /tmp/ + dpkg -i /tmp/strace_4.5.20-2.3ubuntu2_armhf.deb
[14:42] <ogra_> stgraber, :P i know, i'm lazy
[14:43]  * ogra_ just rebooted with reverted container config
[14:43] <stgraber> hallyn: ok, finally got /var/lib/lxc/android/rootfs mounted rshared here, pivot_root fails
[14:44] <ogra_> hallyn, http://paste.ubuntu.com/5816923/
[14:45] <ogra_> not really informative, it seems to die when reading the fist line of the config
[14:46] <stgraber> slangasek: hey, with all the /run talks a while back, do you ever remember someone mentioning /etc/mtab? just noticed that one being an obvious candidate for /run and pretty annoying when /etc is read-only
[14:47] <ogra_> stgraber, didnt we always follow the "link to /proc/mounts" directive for readonly rootfs ?
[14:48] <hallyn> stgraber: no, /var/lib/lxc/android/rotofs is not to be rshared,
[14:48] <hallyn> stgraber: /container/rootfs is
[14:48] <hallyn> stgraber: in other words, /var/l/ib/lxc/android/rootfs is the *source* of the mount, but you want the target to be rshared - the dir you end up pivot_rooting into
[14:49] <stgraber> ogra_: yeah and that's what I'm doing here, though /run may make more sense (so that things like mount -n work)
[14:50] <ogra_> yeah. /proc/moounts definiely isnt 100% equal
[14:50] <stgraber> hallyn: hmm, right, that'll be even trickier because of mount's silly restriction of having the target be in /etc/mtab...
[14:50] <jibel> hallyn, I don't use the ubuntu template but scripts from https://code.launchpad.net/otto if that's what you mean by "by hand". The default config is a modified version of an install on Saucy months ago.
[14:51] <jibel> hallyn, the major difference being the hooks, and lxc.cgroup.devices.allow on snd, dri, input and tty
[14:52] <jibel> and the memory limits
[14:52] <hallyn> holy schnickities
[14:52] <jibel> hallyn, I also noticed +lxc.hook.mount = /usr/share/lxc/hooks/mountcgroups which is not there with latest template
[14:52] <jibel> I'll add steps to reproduce
[14:53] <hallyn> jibel: thanks
[14:54] <hallyn> stgraber: holy schnickities, bug in saucy's lxc with lxc.rootfs.mount set.  oh!  i think i knew about that one
[14:54] <stgraber> hallyn: so I added:
[14:54] <stgraber> mount --bind $LXC_ROOTFS_PATH $LXC_ROOTFS_PATH > /tmp/debug 2>&1
[14:54] <stgraber> mount --make-rshared $LXC_ROOTFS_PATH > /tmp/debug 2>&1
[14:54] <stgraber> hallyn: to the pre-start, was that what you were saying?
[14:54] <hallyn> stgraber: yeah
[14:54] <stgraber> hallyn: because if it's, then pivot_root still fails
[14:55] <hallyn> d'oh
[14:55] <hallyn> right, kernel won't allow that will it
[14:55] <stgraber> yeah, same thing we had with UFA I think, kernel won't let us do it
[14:56] <hallyn> ok, new plan :)
[14:56] <hallyn> mkdir /junk;  mount --bind /junk /junk;  mount --make-rshared /junk
[14:56] <hallyn> start the container as per usual
[14:56] <hallyn> oh no, I guess bind-mount /junk into the container with lxc.mount.entry
[14:57] <hallyn> then, mkdir /junk/console; lxc-attach $container mount /dev/console /junk/console
[14:57] <stgraber> hallyn: we're on old kernels, can't rely on lxc-attach :)
[14:57] <stgraber> hallyn: but I have an idea for our specific case
[14:58] <stgraber> ogra_: how about we create /dev/sockets on the host (ubuntu), then bind-mount that to /sockets in the container and then have init.rc mount --move it to /dev/sockets right after /dev is mounted in android?
[14:58] <hallyn> stgraber: then just ssh $container mount /dev/console /junk/cnosole :)
[14:58] <stgraber> hallyn: no ssh or getty either :)
[14:58] <ogra_> stgraber, not sure how the blobs will cope with that ...
[14:59] <ogra_> we try to not change anything on the android side that could affect the binary blob functionallity (we already have way to many intrusive changes in the android setup)
[14:59] <hallyn> stgraber: club it over the head and shout?
[14:59]  * hallyn will wait to hear what stgraber is trying and whether it worked :)
[15:00] <stgraber> ogra_: they won't have to, it's just an extra line in the init.rc to make /dev/sockets a mount instead of a standard dir
[15:00] <stgraber> ogra_: unless the blobs start checksuming /proc/mounts or something, they won't notice and if they're looking at /proc/mounts, they'd already fail today :)
[15:01] <ogra_> stgraber, ok, lets try it
[15:02] <stgraber> ogra_: ok. I'm stuck in meetings for the next 2 hours at least but will try to do some tests in parallel
[15:02]  * ogra_ is in a call now as well
[15:05] <xnox> jibel: uploaded a fix into raring-proposed unapproved queue and pinged about it on #ubuntu-release. Also committed a fix for the next upstart upload into saucy.
[15:07] <ev> mpt: I don't think using the rate of errors is going to work for determining whether the architecture is significant.
[15:07] <ev> As an example, if we have 1,000 errors from 1,000 am64 machines in a day, the error rate is 1. If we have 1 error from 1 machine in that same day, the error rate is also 1.
[15:10] <xnox> ev: but if on adm64 for a particular crash error rate is 0.98, whilst i386 error rate is 0. it does imply an arch specific bug. And a single error from a single i386 machine instantly makes the bug report "not arch specific, yet triggered easier on amd64"
[15:10] <ev> xnox: the error rate would be 1 on both cases
[15:12] <ev> or am I missing something?
[15:12] <xnox> ev: i see what you mean, as we only collect crashes.
[15:12] <xnox> ev: i guess what matters is proportion of architectures within a given crash report, and whether it's statistically significant disproportion.
[15:12] <ev> xnox: as opposed to machines that were out there running a particular version, package, and architecture?
[15:13] <xnox> that's overly specific, but yeah =)
[15:14] <ev> xnox: yeah, proportion seems to be more helpful at least for architecture differences
[15:14] <xnox> if foo is 50/50 amd64-i386 yet one crash of foo is 90/10 amd64-i386 it does hint arch-specific bug or "... easier to trigger on amd64"
[15:15] <ev> I've been using >= 75% in my tests
[15:15] <xnox> ev: but even comparing with overall architecture split of all machines reporting any errors would be fine. As anything within 20% of a split can be accounted to random.
[15:16] <xnox> ev: 75 sounds good, but what's the current split between amd64/i386 reporters? 50/50? 60/40? 70/30?
[15:16] <xnox> (across all crashes ever uploaded)
[15:16] <ev> that's a good question, and an interesting thing to start monitoring
[15:17] <xnox> it's not true indication of install base, but for the purpose of hinting arch-specific bugs that's the only baseline we can go by.
[15:17] <xnox> ev: don't forget about multi-arch though, just because my machine is :amd64 a crash in :i386 package should still count towards i386.....
[15:18] <ev> oh ew.
[15:18] <ev> I had not considered that.
[15:18] <ev> I've been going off the Architecture field in the report.
[15:18] <xnox> ev: all skype crashes are i386 binaries & libraries & dependancies. As there is no amd64 skype.
[15:18] <ev> yeah
[15:20] <xnox> ev: if the Architecture field from the report comes from the dpkg Architecture filed, all is fine. If it comes from machine/kernel then it's not so useful for this exercise.
[15:20] <xnox> ev: of course there are kernel bugs that only matching i386 kernel + i386 userspace binary will trigger, but those are rare.
[15:20] <xnox> and given enough reporters should still generate significant arch bias.
[15:21] <mpt> ev, if we have 1000 errors from 1000 amd64 machines in a day, that would be a fluke unless the error is clock-based. Nonetheless, the calculated error rate wouldn't be 1000/1000, it would be 1000/(weighted count of AMD64 machines).
[15:22] <mpt> Which, at the moment, is 1000/(number of AMD64 machines that reported any error in the past 90 days).
[15:23] <mpt> But would become more sophisticated once bug 1077122 is fixed.
[15:23] <ev> why would 1,000 errors from 1,000 amd64 machines be a fluke?
[15:24] <mpt> It would be a fluke that every single machine that had the error had it only once in that 24 hours -- that none of them had it twice or thrice.
[15:25] <mpt> (And thank you for giving me an opportunity to use the word "thrice". I was beginning to lose hope of using it at all in 2013.)
[15:25] <ev> you could start listening to the band. That would give you plenty of opportunity
[15:27] <ev> I'm not convinced this solves the problem though. The frequency an error occurs at is not what makes the architecture interesting. The percentage of instances that occurred on said architecture is though.
[15:28] <ev> mpt: You may have an error that, on average, happens twice a day for i386 and twice a day for amd64, but is significantly more prevalent on amd64.
[15:28] <stgraber> ogra_: early-init is pretty much the first thing to be called right?
[15:28] <ev> even when factoring in the population of systems seen in the past 90 days
[15:28] <stgraber> ogra_: (talking of init.rc targets)
[15:29] <ogra_> stgraber, hmm, yeah, i think so
[15:29] <stgraber> ogra_: ok, I'll try to dump my extra mount line as the first thing it does, that seems to be the easiest to do with sed and should avoid any potential race
[15:34] <slangasek> stgraber: current util-linux supports /run/mount/utab; however, the implementation is only half-finished despite having been rolled out in Debian, Debian bug #702935
[15:37] <stgraber> slangasek: ok, good to hear someone's working on that. I indeed noticed /run/mount/utab in the strace output but the file ended up empty so wasn't of much use (which appears to be exactly the bug you mentioned)
[15:38] <mpt> ev, if you're controlling for the proportion of machines that have that architecture, you'll end up with exactly the same result starting from the proportion of errors coming from an architecture as if you start from the error rate for the architecture. They're just two routes to the same formula: (errors(amd64)*count(all arches))/(count(amd64)*errors(all arches)).
[15:41] <slangasek> stgraber: right; it's meant to be supplemental to /proc/mounts, so it's assumed that /etc/mtab is a symlink there first of all - but it still fails in various scenarios
[15:55] <doko> Daviey, zul: can I pass you the  system-config-cluster merge?
[15:56] <Daviey> doko: I think that one is better handled by either roaksoax or ivoks TBH
[15:56] <doko> roaksoax, ivoks: ^^^. thanks for the pointer
[15:58] <ivoks> sigh
[15:58] <ivoks> that thing is still alive?
[15:58] <ev> mpt: so we don't actually keep a count of the number of unique machines over a 90 day period per architecture, yet.
[15:59] <ev> mpt: we never did return to the denominator question. Is this "count of machines (eventually weighted)" for the bucket in question, or for all problems seen that day?
[15:59] <mpt> ev, remember it's not just per-architecture: That's just one of the most interesting variables.
[15:59] <mpt> Package versions being another, locale settings, upgrade vs. fresh install, etc.
[16:00] <ev> sure, I was just doing architecture first as it seemed easiest
[16:00] <ev> "c(amd64)" implies an architecture-specific count of machines
[16:00] <mpt> Okay, just as long as you don't fix that problem in a way that works for architecture and nothing else
[16:01] <ev> I can't envision a way in which I would, but I'll keep that in the back of my mind
[16:01] <mpt> ev, re. denominator: Neither. It's the count of machines that are using Ubuntu.
[16:01] <roaksoax> doko: i can take care of it
[16:05] <doko> roaksoax, thanks
[16:09] <ev> mpt: yeah, we don't keep a count of the number of unique systems in the past 90 days for a given architecture. I'll work on a branch to add this to both the current approach and the weighting code
[16:45] <stgraber> ogra_: ok, I have an updated pre-start.sh that fixes /dev/socket (triple testing it now)
[16:45] <ogra_> yay
[16:46] <stgraber> ogra_: had to read android's init source code to figure out exactly how those commands work... they look like shell but they're not at all...
[16:47] <ogra_> heh, no, its its own language
[16:47] <stgraber> after reading builtins.c it's much clearer what's allowed and what's not :)
[16:49] <stgraber> ogra_: I guess I should probably fix cpuctl too while I'am at it?
[16:49] <ogra_> i'm not even sure we access it
[16:50] <ogra_> but yeah, all subdirs in /dev should eb available in the hosts /dev preferably
[16:50] <ogra_> we can drop them still if we find we dont need them
[16:50] <stgraber> ogra_: ok. I'll do the right thing for it which is to bind-mount /sys/fs/cgroup/cpu to /dev/cpuact (we already have it in the host, just not at the same place as Android)
[16:51] <ogra_> stgraber, err, with the same permissions ?
[16:51] <stgraber> or rather, have it symlinked when booting the current flipped images and have it bind-mounted when booting the loop-mounted images (much simpler to implement)
[16:52] <ogra_> i took the /dev nodes because i know everything is already processed there and listening processes are in place proper
[16:52] <stgraber> ogra_: yep, the permissions are identical as the cgroupfs isn't namespace aware and so any change from android or from ubuntu will show up on the other side
[16:53] <ogra_> ah, good
[16:59] <ogra_> cjwatson, just FYI, maguro and grouper images work fine from the new build
[17:03] <stgraber> ogra_: please test http://paste.ubuntu.com/5817323/
[17:04] <stgraber> ogra_: tested here and I get /dev/socket and /dev/cpuctl working as expected (and no more /proc/<pid> black magic)
[17:07] <ogra_> stgraber, i think the cpuctl stuff should become a udev rule instead
[17:08]  * ogra_ reboots with the changes
[17:10] <stgraber> ogra_: that'd trigger on what? you need to wait until cgroup-lite is started before it can be bind-mounted so it can't be a reaction to a uevent
[17:10] <cjwatson> ogra_: Brilliant
[17:12] <ogra_> stgraber, hmm, i was thinking to just match on sysfs node creation ... but that might indeed not work then
[17:12] <ogra_> stgraber, the change doesnt work
[17:12] <ogra_> Jul  1 17:12:07 ubuntu-phablet ofonod[868]: Excluding AT modem driver
[17:12] <ogra_> Jul  1 17:12:07 ubuntu-phablet ofonod[868]: create_ril: can't connect to RILD: Connection refused (111)
[17:12] <ogra_> Jul  1 17:12:07 ubuntu-phablet ofonod[868]: Exiting...
[17:13] <stgraber> ogra_: what do you have in /dev/socket on Ubuntu?
[17:13] <ogra_> the right files
[17:14] <ogra_> but apparently the connnection between rild and the socket on the android side isnt working
[17:14] <stgraber> Jul  1 17:03:35 ubuntu-phablet ofonod[1253]: [UNSOL]< UNSOL_RIL_CONNECTED
[17:14] <stgraber> Jul  1 17:03:35 ubuntu-phablet ofonod[1253]: No SIM card present.
[17:14] <stgraber> I have that on mine
[17:14] <ogra_> weird
[17:14] <ogra_> i have ofono just adly respawning on maguro
[17:14] <ogra_> *madly
[17:16] <ogra_> stgraber, out of ten reboots one worked
[17:16] <ogra_> stgraber, so the race got a lot worse than without the change
[17:17] <ogra_> (it was one out of 20 or so that failed before)
[17:17] <ogra_> and the boot seems to have gotten slower
[17:17] <ogra_> but thats a subjective feeling
[17:17] <ogra_> (at with an 1.5min boot probably even not a real issue :P)
[17:18] <stgraber> ogra_: not sure what would make boot slower, we're doing less stuff now :)
[17:18] <stgraber> ogra_: can you get: lsof -n -p <pid of rild> 2>/dev/null
[17:18] <ogra_> yeah
[17:23] <stgraber> ogra_: to avoid a respawn loop we could have a start on starting upstart-file-bridge job create /dev/socket, then have ofono.override use FILE=/dev/socket/rild
[17:24] <ogra_> doesnt work
[17:24] <ogra_> :)
[17:24]  * ogra_ spent days on that bit already ... just to find that inotify doesnt seem to operate on sockets 
[17:24] <ogra_> so the file bridge doesnt work
[17:25] <ogra_> stgraber, i think having it respawn properly is the solution... just add that upstart fix so ofono doesnt die on respawns :)
[17:26] <stgraber> ogra_: well, upstart as a respawn limit, so on fast phones we may hit it, then ofono won't start...
[17:26] <ogra_> oh, we actually want the limit being used
[17:26] <ogra_> but we dont want it to die on the second one already :)
[17:27] <stgraber> actually we should just have a wait loop in pre-start, that'd be much simpler :)
[17:27] <stgraber> oh, just like we do, nevermind
[17:28] <stgraber> oh, 8s sleep is a bit excessive isn't it? it's basically the time the phone takes to do a full reboot here
[17:29] <ogra_> lol
[17:29] <ogra_> yeah, mako isnt our target arch
[17:29] <ogra_> a reboot on maguro takes  about 2-3min
[17:29] <ogra_> the boot lies between 50sec and 1.5min alone
[17:30]  * ogra_ would really like to get rid of the shutdown process ... having the shell thell the apps to save the UI realted data, calling sync and reboot -f would be so much more effective)
[17:30] <stgraber> ogra_: anyway, got that lsof output?
[17:31] <ogra_> stgraber, the 8s was already in ... i think telepathy required it in unflipped
[17:32] <ogra_> stgraber, stop sending me copy paste lines with 2>/dev/null ... then i can easier find out that lsof isnt installed :P
[17:33] <stgraber> ogra_: ah yeah, should have told you you'd need to install it first ;)
[17:33] <ogra_> heh, i thought it was in -minimal
[17:33] <stgraber> ogra_: the 2>/dev/null will be useful once you have it though (otherwise it spits out errors about unknown uids)
[17:34] <ogra_> http://paste.ubuntu.com/5817408/
[17:34] <ogra_> looks ok
[17:34] <ogra_> this boot also has found the socket in time
[17:35] <ogra_> stgraber, so your idea is to loop over lsof until it connected ?
[17:39] <stgraber> ogra_: can you get me the same thing from a boot where the socket doesn't work?
[17:40] <stgraber> ogra_: well, I'm not sure why it doesn't connect... to me it seems like if the socket exists it should work, so I'd like to see exactly what rild is listening on when it doesn't work
[17:40] <ogra_> if i only could get one now
[17:40] <ogra_> the first ten were all broken ... the next ten werent
[17:41] <stgraber> I first thought it was a race in my init.rc change but I triple-checked and early-init runs before everything else including ueventd so there's no way that rild can be spawned before that and listen to the wrong /dev/socket (the one under my bind mount)
[17:41] <slangasek> stgraber: race between the bind() and listen() calls?
[17:41] <slangasek> well, not a race /between/ those calls; but a race because bind+listen is not atomic
[17:41] <ogra_> stgraber, http://paste.ubuntu.com/5817423/
[17:42] <ogra_> stgraber, nah, the race is oldm nothing you caused ... but the timing of the startup indeed changed a bit
[17:42] <ogra_> ofono used to come up later before
[17:43] <ogra_> slangasek, well, likely something inside rild ...
[17:43] <ogra_> not much we can do about it
[17:43] <cjwatson> Can anyone unpick https://jenkins.qa.ubuntu.com/job/saucy-adt-apt-clone/7/ARCH=amd64,label=adt/artifact/results/log ?  It doesn't happen for me locally, and I don't think acpi-support has changed.  Although I suppose it relies on archive state so maybe it's inherently chancy ...
[17:43] <stgraber> ogra_: that second lsof shows that it's listening twice
[17:43] <ogra_> ah
[17:44] <slangasek> ogra_: yes, bind+listen are the calls made by rild
[17:44] <ogra_> stgraber, hw do you see that ?
[17:44] <ogra_> i only see one entry
[17:45] <slangasek> listen-before-bind would be race-free, but I'm not sure if that's actually allowed... it's not the conventional order
[17:45] <ogra_> stgraber, you mean the first one shows it is listening twice, right ?
[17:46] <ogra_> oh, how did my browser tabs flip their places
[17:46] <cjwatson> We can't have rild fork/daemonise/whatever after it listens?
[17:46] <ogra_> stgraber, ignore me ...
[17:46] <cjwatson> Ah, I guess it isn't supervised by upstart
[17:46] <ogra_> cjwatson, rild is a highly confidential binary blob
[17:46] <cjwatson> i.e. broken :)
[17:47] <ogra_> living in android ... and each vendor seemss to have its pown rild implementation
[17:47] <ogra_> *own
[17:47] <cjwatson> Can we ptrace it?
[17:47] <slangasek> currently it's launched by android init
[17:47] <ogra_> they are not even 100% protocol compatible
[17:47] <cjwatson> i.e. ptrace until it calls listen, detach, emit upstart event
[17:48] <slangasek> we could unpick all of that, but currently we're trying to avoid having to make deep changes there
[17:48] <slangasek> I suppose a ptracing wrapper could work
[17:48] <slangasek> cjwatson: do you know ptrace well enough to cook something like that up quickly?
[17:48] <ogra_> for kernels where we have ptrace :)
[17:49] <cjwatson> Do we not have ptrace on the relevant kernels?
[17:49] <stgraber> that should be all of them, otherwise upstart would have some difficulties
[17:49]  * ogra_ doesnt think it is enabled on ports 
[17:49] <slangasek> ogra_: er, that seems unlikely
[17:49] <ogra_> cjwatson, in our nexus kernels it definitely is
[17:49] <slangasek> the ports kernels need to have ptrace enabled, period
[17:50] <cjwatson> slangasek: It'd be ab initio, but I can read docs quickly :)  However I'm at EOD for today
[17:50] <slangasek> if they aren't currently enabling it, that's a bug that they'll need to fix
[17:50] <slangasek> cjwatson: mmk, never mind then :)
[17:50] <ogra_> slangasek, yeah, its a one line change to the porting docs
[17:50] <ogra_> just saying i dont think they have it now
[17:50] <cjwatson> If they don't, as stgraber says, Ubuntu won't work
[17:50] <stgraber> ogra_: so, just to confirm the theory, what happens if you add let's say a 5s sleep after you find the socket and before starting ofono?
[17:50] <ogra_> oh, k
[17:51] <ogra_> stgraber, to the existing 8s sleep you mean ?
[17:51] <stgraber> ogra_: if we indeed have a race between bind/listen, that should be way enough to get it past that point and so it should be reliable (wouldn't recommend it as a solution, but good enough to test)
[17:52] <stgraber> ogra_: right, the current loop won't be executed (no sleep) if the socket is found, the 8s is only if it's not there
[17:52] <stgraber> ogra_: so just add a "sleep 5" after the while loop
[17:52] <ogra_> oh, right, let me add 5s above the while
[17:52] <stgraber> below please
[17:52] <ogra_> bah, to late ...
[17:52] <stgraber> otherwise we may still get the race
[17:53] <ogra_> even tough ... heh ... another hanging reboot
[17:53] <stgraber> (if the socket happens to show up right after the sleep 5, we'll skip the loop and start running ofono before the bind/listen is done)
[17:53] <ogra_> yeah
[17:53] <stgraber> ogra_: can you retry with the sleep after the while loop, as the last thing in pre-start?
[17:54] <ogra_> yes, waiting for the boot ro finish
[17:54] <ogra_> (as i said, takes minutes)
[17:54] <ogra_> this one worked, let me do a few more
[17:58] <ogra_> stgraber, still there ... but more like it was before the shole change
[17:58] <ogra_> *whole
[17:58] <ogra_> 1 out of 10 fails
[17:59] <stgraber> not very reassuring
[17:59] <ogra_> well, as long as the respawn works all is fine
[18:00] <stgraber> right and I gave a workaround to rsalveti for that earlier
[18:00] <ogra_> adding the 5s sleep and having respawn work should get us going until we can do something proper
[18:00] <ogra_> so its just that upstart fix :)
[18:01] <ogra_> stgraber, ok with you if i add all the chnages to lxc-android-config now ?
[18:01] <ogra_> or do you have a package branch ready
[18:01] <stgraber> ogra_: I'll upload in a couple of minutes
[18:01] <ogra_> ok
[18:02] <ogra_> rsalveti, readable /dev/socket for you ^^^
[18:02] <stgraber> ogra_: I won't push the ugly while loop (instead of respawn) though as I just thought of a couple of problems that'd cause (pid tracking), so I'll just try to fix the upstart bug asap (once I'm done with lunch, uploading that stuff and a couple of meetings)
[18:03] <ogra_> stgraber, right, the 5s sleep should be fine for now
[18:03] <ogra_> as long as we arent more broken than before i'm fine for now
[18:04] <ogra_> the respawning usually works at least once ... its the subsequent ones that fail
[18:04] <rsalveti> ogra_: thanks, patch seems fine
[18:04] <rsalveti> stgraber: thanks :-)
[18:07] <ogra_> stgraber, so i used the UI for the first time with the change in place ... *something* changed ... it is a lot less choppy !
[18:08] <ogra_> i can scroll through G+ without it stuttering now ... never worked on the flipped images
[18:10] <stgraber> ogra_: uploaded
[18:11] <ogra_> thx
[18:37] <ogra_> stgraber, ugh, copying the two files over to the grouper breaks completely :(
[18:40] <stgraber> ogra_: hmm, that's weird, can't think of why the change wouldn't work on grouper... let me update mine quickly (for some 30min definition of quickly... that's why I don't use it)
[18:41] <ogra_> stgraber, lxc_conf - File exists - failed to symlink '/dev/pts/ptmx'->'/dev/ptmx'
[18:41] <ogra_> weirdo
[18:41] <stgraber> ogra_: weird indeed, can't remember changing anything that'd affect that...
[18:42] <ogra_> yeah
[18:43] <ogra_> they are also the exact same image both installed at the same time
[18:43] <ogra_> the only thing that differs is that there is indeed no rild on the N7
[18:45] <ogra_> stgraber, aha, the init.rc mangling doesnt happen
[18:45] <stgraber> ogra_: ah? missing early-init target?
[18:46] <ogra_> nope, it is there
[18:46] <ogra_> but the mount line isnt
[18:46] <stgraber> ogra_: what mount line? my new sed only requires "on early-init" to be present, then it adds a mkdir and a mount line below it
[18:46] <ogra_> right, these two arent there on grouper
[18:47] <ogra_> i see them on maguro
[18:48] <ogra_> hmm, no red herring
[18:48] <ogra_> seems it didnt even unpack the initrd
[18:52] <ogra_> stgraber, so it looks like we dont get into pre-start.sh at sll
[18:52] <ogra_> *all
[19:13] <stgraber> ogra_: container started fine here (using loop mount)
[19:14] <ogra_> very weird
[19:14] <stgraber> ogra_: ah, wait a sec, I'm not on the new lxc-android-config ;)
[19:14] <ogra_> i really only replaced the upstart job and pre-start.sh with the ones from maguro
[19:14] <ogra_> on a brandnew image
[19:17] <stgraber> ogra_: ok, rebooted with the new version of lxc-android-config and the container still starts /dev/socket and /dev/cpuctl look good and init.rc contains the two extra lines
[19:18] <ogra_> hmm weird, i'll wait for the package
[20:02] <tedg> ogra_, So when /sbin/initctrl is move to /sbin/initctrl.REAL is /sbin/init moved as well?
[20:03] <ogra_> nope
[20:03] <tedg> K
[20:03] <tedg> ogra_, Thanks
[20:05] <ogra_> tedg, debian-installer-utils has a good reference script (chroot-setup.sh), similar code is used all over the place when chroots are used and daemon startups need to be prevented
[20:06] <tedg> ogra_, Wow, that's a very evil script :-)
[20:07] <ogra_> well, thats what you need to do in an image build chroot or if you want to avouid to taint a fresh install while putting stuff into place
[20:09] <tedg> Sure, but that's usually a little bit lower than where I play.
[20:10] <ogra_> yeah, plumbing ...
[20:30] <luist> anyone familiar with multistrap?
[20:36] <ogra_> stgraber, lol ... so i transferred pre-start.sh via adb from one devices to the other ... guess what my issue was :) .... adb removes the executable bits from files
[20:37] <ogra_> stgraber, so flase alarm, all is fine with the scripts ...
[20:37] <stgraber> ogra_: oh, that'd explain it ;)
[20:37] <stgraber> good
[20:38] <ogra_> wow
[20:39] <ogra_> and even the nx7 all of a sudden feels snappier
[20:42] <luist> can anyone help me with multistrap? I'm trying to add my custom repository/package , but its not working. This is the config file: http://pastie.org/8101130
[20:49] <roaksoax> doko: makes sense? http://paste.ubuntu.com/5834245/
[21:00] <doko> roaksoax, did the fix for the desktop file exist before? if yes, then maybe write in the changelog "resync ..., remaining changes"- but that's a minor nit.  and use -v to include the changes from the debian updates in the changes file
[21:06] <roaksoax> doko: cool thanks1
[21:12] <hallyn> jibel: ok, sorry this took so long - had some other issues.  but all i'm seeing is that memsw.limit_in_bytes appears to be misbehaving
[21:20] <bdmurray> dobey: are you working with software-center now?
[21:21] <dobey> bdmurray: it's in maintenance mode, so sort of. my team owns it now though, yes.
[21:24] <bdmurray> dobey: there is a reference in aptd.py to ERROR_PACKAGE_MANAGER_FAILED errors (from aptdaemon) and those being reported by dpkg and this isn't really happening and trying to sort out why
[21:26] <bdmurray> aptdcon creates package install failure crash files and so does apt itself, so I think software-center is stopping them some how
[21:27] <dobey> bdmurray: unfortunately i don't really know much about the core of the code. maybe send an e-mail to glatzor/mvo about it?
[21:27] <bdmurray> dobey: okay, thanks
[21:27] <dobey> sure
[21:56] <hallyn> jibel: when i remove that line from the config, i can start the otto container - no complaints about devices cgroup
[21:57] <hallyn> oh, d'oh.  but that wasn't kernel 3.10.  sigh.