=== skrewler_ is now known as skrewler === Hussain is now known as Guest62667 === dendro-afk is now known as dendrobates === daker_ is now known as daker === hggdh_ is now known as hggdh [16:41] smoser: ping [16:45] adam_g, here... [16:51] smoser: https://bugs.launchpad.net/software-properties/+bug/829109 breaks the apt CC module because cloud-init/cfg are running with a tty attached. any thoughts? im looking at it now [16:51] Launchpad bug 829109 in software-properties "add-apt-repository - New confirmation dialog breaks existing scripts" [Medium,Fix released] [16:52] adam_g, will look at it in 15 minutes [16:53] smoser: if you're jammed up don't worry about it [16:54] no, just patch piloting for another 15 minute sor so [16:54] k [17:20] smoser: /win 5 [17:21] smoser: errr, fix incoming === chuck_ is now known as zul === dendrobates is now known as dendro-afk === dendro-afk is now known as dendrobates [18:22] hallyn, any progress? [18:23] adam_g, so do you ahve a redirect to /dev/null or '-y' add for cloud-init ? or you want me to do that [18:24] i guess i wonder why there is a console there anyway.. maybe we should generally kill that in cloud-init [18:24] as there is no reason that there should be a console [18:26] smoser: not sure yet how/why, but i think the root cause is that /var/lib/nova is a symlink to /mnt/var-dirs/nova [18:26] ah, yes [18:26] i getr it [18:27] it's a symlink. that symlink doesn't correctly dereference while we're pivot-rooting [18:27] or at least, not to /.oldroot/... [18:28] maybe just using the right path in xml will work? lemme try [18:30] really? [18:30] holly carp [18:30] so that pivot root issue would be actually kernel ? [18:30] no, [18:31] issue is that libvirt_lxc is trying to mount --move from /.oldroot/var/lib/nova/.../dev/pts, but a part of that is a symlink which does not dereference [18:32] biam [18:39] hallyn, so why did that not fail with reasonable debug output ? [18:39] virReportSystemError(errno, "%s", _("Failed to mount /dev/pts in container")); [18:39] well in the end (IF i'm right) that's pretty reasonable output :) [18:42] its *very* reasonable output [18:42] but i did not see it in any log [18:42] smoser: sorry, back [18:42] oh [18:43] yeah [18:43] does it segfault or something? [18:43] adam_g, i see your merge proposal [18:43] anyway, it's definately a bug, but we'd have to (and this will take time) come up with a reproducible case and work with upstream for a good fix [18:43] but, i'm still not 100% working [18:43] adam_g: sent a merge proposal that just redirects input to /dev/null otherwise it'll have a tty. thats the better way to do it, as '-y' would only work with the recent software-properties versions [18:43] adam_g, do you happen to have an idea on how we would say "stdout is not a terminal" globally in cloud-init ? [18:45] hallyn, that is easy to do if you're right. (the test case) [18:45] if thats the only issue. [18:45] smoser: we want to do that for stdin, and its handled per subprocess via the subp() function in cloudinit/util.py [18:45] and i'm *really* sorry that we wasted time on this... i never would have expected that symlink i was doing to cuase such a pita [18:46] right... stdin, not stdout. my bad. [18:46] adam_g: im here :) [18:48] smoser: well, the thing i'm hanging on now (after not using the symlink) is a failed (-EBUSY) umount of .oldroot. which should NOT be hanging. it's not busy [18:48] we may need to make it do lazy umount if umount fails. not sure how danpb will feel about that [18:48] smoser: isn't there a way (in user-data) to tell add-apt-repository to use -y? [18:48] or, if it's just masking over real problems [18:49] oh wait, maybe bc i didn't change console file [18:49] RoAkSoAx, i could think of a hack [18:50] smoser: what do you have in mind? [18:51] boothook that did: f=/usr/bin/add-apt-repository; [ -f "$f.real" ] || mv $f $f.real && echo '#!/bin/sh\nexec $0.real -y "$@"\n' && chmod 755 $f [18:55] smoser: yes! i'd messed things up enough that i needed a reboot, but in the end that fixes it [18:55] smoser: cool thanks [18:56] hallyn, right. reboot fixes it. [18:56] well, sort of. === dendrobates is now known as dendro-afk [18:56] it would then break the nbd mount [18:57] which will break that directory from being an lxc container [18:57] smoser: well id' really been mucking with my mount tables [18:57] now it works, so long as i edit the xml to use the non-symlink path [18:57] smoser: you're using an older lxcguest in that rootfs? it has console.conf... [18:57] guess you don't really care [18:58] hallyn, well, where'd you et the image from ? [18:58] smoser: can you proceed this way (just sed -i 's./var/lib./mnt/nova-dirs.' on any xmls you create) [18:58] smoser: i follwed your instructions to the letter. (without paying much attention :) [18:59] yeah [18:59] that will get the most recent daily [18:59] hallyn, which is built either either yesterday or today depending on when you got it [19:00] (the right serial number should be in 'ls ~/images') [19:00] oh. hmm. actually console.conf has the right contents. So wtf is it running a getty? [19:02] all right ignore me i guess i was on crack. it works fine now [19:29] what works fine, hallyn? [19:36] smoser: starting those containers [19:36] smoser: once you sed -i 's./var/lib./mnt/nova-dirs.' on the xml [19:38] hm.. [19:38] so what do you thikn is failing ? [19:39] and why don't we get a debug message that says what is failing? [19:39] is it segfaulting somewhere ? [19:40] you have a long path /a/b/c/d/e/f/g, and c along the way is a symlink which is not valid when looking at /.oldroot/a/b/c/d/e/f/g [19:40] so i'm not sure whether upstream will say it is a bug in libvirt, or just abuse [19:40] the latter doesn't really seem reasonable, but i'm not sure what they can do to cleanly fix it [19:41] anyway we just need to report it upstream, and you need to just specify paths not containing symlinks in the meantime in .xml for libvirt-lxc [19:45] hallyn, ok. i'll try a work around in nova [19:45] that should be reasonable there. [19:45] basically we need a os.path.realpath() in libvirt-lxc [19:46] no that's not enough [19:46] oh, really? [19:46] oh, you mean for your workaround? [19:46] yes, that'll do :) [19:46] i think [19:47] i can easily do the os.path.realpath() in nova [19:47] ok [19:47] i'm less C capable, but if that is sufficient, doing it in libvirt-lxc would be more beneficial [19:47] if you do that in a python shell right now on /var/lib/libvirt/nova, what do you get? [19:49] yeah that works [19:49] it just has to be done while writing the xml [19:50] well, it doesn' thave to be done then. it could be done while consuming the xml. [19:56] true === dendro-afk is now known as dendrobates === uvirtbot` is now known as uvirtbot [21:22] smoser: oddly, when i try to reproduce the symlink failure with a busybox lxc container, it works just fine [21:23] ah, i think i see [21:34] nope. [21:35] smoser: apart from an hour or two tonight and a bit of checking over the week, i'll be out this week. if you manage to reproduce it with a trivial busybox container pls shoot me an email. i have, so far, failed [21:35] bbl [22:17] Is it normal for your nodes to not reconnect to the controller after a reboot? [22:52] Is there anything stored in eucalyptus's conf files that is mac-address specific? === m_3 is now known as mmm === mmm is now known as m_3 [23:03] Kyle__: if you mean about the NC host, then no [23:03] obino: OK. And on the CC? [23:04] Eucalyptus do not uses MAC. The only are involved when using STATIC mode [23:05] obino: OK. Thanks. Switched NICs out, having a bit of weirdness now [23:11] NC not reporting? === dendrobates is now known as dendro-afk === dendro-afk is now known as dendrobates