| === skrewler_ is now known as skrewler | ||
| === Hussain is now known as Guest62667 | ||
| === dendro-afk is now known as dendrobates | ||
| === daker_ is now known as daker | ||
| === hggdh_ is now known as hggdh | ||
| adam_g | smoser: ping | 16:41 |
|---|---|---|
| smoser | adam_g, here... | 16:45 |
| adam_g | smoser: https://bugs.launchpad.net/software-properties/+bug/829109 breaks the apt CC module because cloud-init/cfg are running with a tty attached. any thoughts? im looking at it now | 16:51 |
| uvirtbot` | Launchpad bug 829109 in software-properties "add-apt-repository - New confirmation dialog breaks existing scripts" [Medium,Fix released] | 16:51 |
| smoser | adam_g, will look at it in 15 minutes | 16:52 |
| adam_g | smoser: if you're jammed up don't worry about it | 16:53 |
| smoser | no, just patch piloting for another 15 minute sor so | 16:54 |
| adam_g | k | 16:54 |
| adam_g | smoser: /win 5 | 17:20 |
| adam_g | smoser: errr, fix incoming | 17:21 |
| === chuck_ is now known as zul | ||
| === dendrobates is now known as dendro-afk | ||
| === dendro-afk is now known as dendrobates | ||
| smoser | hallyn, any progress? | 18:22 |
| smoser | adam_g, so do you ahve a redirect to /dev/null or '-y' add for cloud-init ? or you want me to do that | 18:23 |
| smoser | i guess i wonder why there is a console there anyway.. maybe we should generally kill that in cloud-init | 18:24 |
| smoser | as there is no reason that there should be a console | 18:24 |
| hallyn | smoser: not sure yet how/why, but i think the root cause is that /var/lib/nova is a symlink to /mnt/var-dirs/nova | 18:26 |
| hallyn | ah, yes | 18:26 |
| hallyn | i getr it | 18:26 |
| hallyn | it's a symlink. that symlink doesn't correctly dereference while we're pivot-rooting | 18:27 |
| hallyn | or at least, not to /.oldroot/... | 18:27 |
| hallyn | maybe just using the right path in xml will work? lemme try | 18:28 |
| smoser | really? | 18:30 |
| smoser | holly carp | 18:30 |
| smoser | so that pivot root issue would be actually kernel ? | 18:30 |
| hallyn | no, | 18:30 |
| hallyn | issue is that libvirt_lxc is trying to mount --move from /.oldroot/var/lib/nova/.../dev/pts, but a part of that is a symlink which does not dereference | 18:31 |
| hallyn | biam | 18:32 |
| smoser | hallyn, so why did that not fail with reasonable debug output ? | 18:39 |
| smoser | virReportSystemError(errno, "%s", _("Failed to mount /dev/pts in container")); | 18:39 |
| hallyn | well in the end (IF i'm right) that's pretty reasonable output :) | 18:39 |
| smoser | its *very* reasonable output | 18:42 |
| smoser | but i did not see it in any log | 18:42 |
| adam_g | smoser: sorry, back | 18:42 |
| hallyn | oh | 18:42 |
| hallyn | yeah | 18:43 |
| smoser | does it segfault or something? | 18:43 |
| smoser | adam_g, i see your merge proposal | 18:43 |
| hallyn | anyway, it's definately a bug, but we'd have to (and this will take time) come up with a reproducible case and work with upstream for a good fix | 18:43 |
| hallyn | but, i'm still not 100% working | 18:43 |
| adam_g | adam_g: sent a merge proposal that just redirects input to /dev/null otherwise it'll have a tty. thats the better way to do it, as '-y' would only work with the recent software-properties versions | 18:43 |
| smoser | adam_g, do you happen to have an idea on how we would say "stdout is not a terminal" globally in cloud-init ? | 18:43 |
| smoser | hallyn, that is easy to do if you're right. (the test case) | 18:45 |
| smoser | if thats the only issue. | 18:45 |
| adam_g | smoser: we want to do that for stdin, and its handled per subprocess via the subp() function in cloudinit/util.py | 18:45 |
| smoser | and i'm *really* sorry that we wasted time on this... i never would have expected that symlink i was doing to cuase such a pita | 18:45 |
| smoser | right... stdin, not stdout. my bad. | 18:46 |
| RoAkSoAx | adam_g: im here :) | 18:46 |
| hallyn | smoser: well, the thing i'm hanging on now (after not using the symlink) is a failed (-EBUSY) umount of .oldroot. which should NOT be hanging. it's not busy | 18:48 |
| hallyn | we may need to make it do lazy umount if umount fails. not sure how danpb will feel about that | 18:48 |
| RoAkSoAx | smoser: isn't there a way (in user-data) to tell add-apt-repository to use -y? | 18:48 |
| hallyn | or, if it's just masking over real problems | 18:48 |
| hallyn | oh wait, maybe bc i didn't change console file | 18:49 |
| smoser | RoAkSoAx, i could think of a hack | 18:49 |
| RoAkSoAx | smoser: what do you have in mind? | 18:50 |
| smoser | boothook that did: f=/usr/bin/add-apt-repository; [ -f "$f.real" ] || mv $f $f.real && echo '#!/bin/sh\nexec $0.real -y "$@"\n' && chmod 755 $f | 18:51 |
| hallyn | smoser: yes! i'd messed things up enough that i needed a reboot, but in the end that fixes it | 18:55 |
| RoAkSoAx | smoser: cool thanks | 18:55 |
| smoser | hallyn, right. reboot fixes it. | 18:56 |
| smoser | well, sort of. | 18:56 |
| === dendrobates is now known as dendro-afk | ||
| smoser | it would then break the nbd mount | 18:56 |
| smoser | which will break that directory from being an lxc container | 18:57 |
| hallyn | smoser: well id' really been mucking with my mount tables | 18:57 |
| hallyn | now it works, so long as i edit the xml to use the non-symlink path | 18:57 |
| hallyn | smoser: you're using an older lxcguest in that rootfs? it has console.conf... | 18:57 |
| hallyn | guess you don't really care | 18:57 |
| smoser | hallyn, well, where'd you et the image from ? | 18:58 |
| hallyn | smoser: can you proceed this way (just sed -i 's./var/lib./mnt/nova-dirs.' on any xmls you create) | 18:58 |
| hallyn | smoser: i follwed your instructions to the letter. (without paying much attention :) | 18:58 |
| smoser | yeah | 18:59 |
| smoser | that will get the most recent daily | 18:59 |
| smoser | hallyn, which is built either either yesterday or today depending on when you got it | 18:59 |
| smoser | (the right serial number should be in 'ls ~/images') | 19:00 |
| hallyn | oh. hmm. actually console.conf has the right contents. So wtf is it running a getty? | 19:00 |
| hallyn | all right ignore me i guess i was on crack. it works fine now | 19:02 |
| smoser | what works fine, hallyn? | 19:29 |
| hallyn | smoser: starting those containers | 19:36 |
| hallyn | smoser: once you sed -i 's./var/lib./mnt/nova-dirs.' on the xml | 19:36 |
| smoser | hm.. | 19:38 |
| smoser | so what do you thikn is failing ? | 19:38 |
| smoser | and why don't we get a debug message that says what is failing? | 19:39 |
| smoser | is it segfaulting somewhere ? | 19:39 |
| hallyn | you have a long path /a/b/c/d/e/f/g, and c along the way is a symlink which is not valid when looking at /.oldroot/a/b/c/d/e/f/g | 19:40 |
| hallyn | so i'm not sure whether upstream will say it is a bug in libvirt, or just abuse | 19:40 |
| hallyn | the latter doesn't really seem reasonable, but i'm not sure what they can do to cleanly fix it | 19:40 |
| hallyn | anyway we just need to report it upstream, and you need to just specify paths not containing symlinks in the meantime in .xml for libvirt-lxc | 19:41 |
| smoser | hallyn, ok. i'll try a work around in nova | 19:45 |
| smoser | that should be reasonable there. | 19:45 |
| smoser | basically we need a os.path.realpath() in libvirt-lxc | 19:45 |
| hallyn | no that's not enough | 19:46 |
| smoser | oh, really? | 19:46 |
| hallyn | oh, you mean for your workaround? | 19:46 |
| hallyn | yes, that'll do :) | 19:46 |
| hallyn | i think | 19:46 |
| smoser | i can easily do the os.path.realpath() in nova | 19:47 |
| hallyn | ok | 19:47 |
| smoser | i'm less C capable, but if that is sufficient, doing it in libvirt-lxc would be more beneficial | 19:47 |
| hallyn | if you do that in a python shell right now on /var/lib/libvirt/nova, what do you get? | 19:47 |
| hallyn | yeah that works | 19:49 |
| hallyn | it just has to be done while writing the xml | 19:49 |
| smoser | well, it doesn' thave to be done then. it could be done while consuming the xml. | 19:50 |
| hallyn | true | 19:56 |
| === dendro-afk is now known as dendrobates | ||
| === uvirtbot` is now known as uvirtbot | ||
| hallyn | smoser: oddly, when i try to reproduce the symlink failure with a busybox lxc container, it works just fine | 21:22 |
| hallyn | ah, i think i see | 21:23 |
| hallyn | nope. | 21:34 |
| hallyn | smoser: apart from an hour or two tonight and a bit of checking over the week, i'll be out this week. if you manage to reproduce it with a trivial busybox container pls shoot me an email. i have, so far, failed | 21:35 |
| hallyn | bbl | 21:35 |
| Kyle__ | Is it normal for your nodes to not reconnect to the controller after a reboot? | 22:17 |
| Kyle__ | Is there anything stored in eucalyptus's conf files that is mac-address specific? | 22:52 |
| === m_3 is now known as mmm | ||
| === mmm is now known as m_3 | ||
| obino | Kyle__: if you mean about the NC host, then no | 23:03 |
| Kyle__ | obino: OK. And on the CC? | 23:03 |
| obino | Eucalyptus do not uses MAC. The only are involved when using STATIC mode | 23:04 |
| Kyle__ | obino: OK. Thanks. Switched NICs out, having a bit of weirdness now | 23:05 |
| obino | NC not reporting? | 23:11 |
| === dendrobates is now known as dendro-afk | ||
| === dendro-afk is now known as dendrobates | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!