[01:10] cliffw, that's on our roadmap to accomplish [01:11] the goal would be that you'd not have to do anything and it'd all just come up like magic. [01:35] smoser: thanks for merge! [01:35] kvm is looking better, I'll go over it one more time tomorrow and do some more testing: https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/327646 [03:57] smoser: rharper, for tomorrow morning, I've queued up all changes in https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330384. just looking for an official approve before I push/merge. [16:39] tot doesn't build... [16:42] blackboxsw: something is up with https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330712 [16:42] tox passes fine [16:43] but when I try building now I get: https://paste.ubuntu.com/25534973/ [16:43] biulding deb or rpm? [16:44] deb [16:44] checking [16:44] deb [16:48] what I don't understand is why our CI didn't find these [16:48] agreed, same here [16:49] are you able to reproduce locally? [16:51] powersj: nope. not with make deb [16:51] lemme try a direct call to packages/bddeb [16:52] I use ./packages/bddeb [16:52] well make deb failed for me as well :\ [16:53] hrm. lemme look at that stack trace again [16:53] I'm thinking stale pyc files maybe, not sure [16:53] I have a fresh checkout though. hmm [16:53] two unmocked errors [16:54] same... [16:54] what's your version of httppretty [16:54] I'll check mine [16:54] git rev-parse HEAD [16:54] f761f2b5f58c8cf13cfee63619f32046216cf66a [16:54] ? [16:54] accd83c97fa3b3b928af2a7955febad410471fc2 for me [16:55] hold up... lemme check master [16:55] accd isn't master [16:55] I think I was in the chef branch specificall [16:55] yeah [16:56] sorry was on a branch. checking now [16:57] hmm make deb and ./packages/bddeb works for me there too [16:57] csmith@uptown:~/src/cloud-init (master)$ git rev-parse HEAD [16:57] f761f2b5f58c8cf13cfee63619f32046216cf66a [16:58] hmm [16:58] Removing python3-httpretty (0.8.6-2) # I'm dropping my xenial system packages to see if something changes [17:00] ohh wait unmet build dependency if I don't have that system package [17:01] powersj: dpkg-query --show python3-httpretty [17:01] python3-httpretty 0.8.6-2 [17:01] I'm wondering if we are hitting an incompatibility in our build envs [17:01] dpkg-query --show python3-httpretty [17:01] python3-httpretty 0.8.14-1 [17:01] I'm on xenial here, checking zesty [17:14] hmm zesty fails for a different reason [17:14] http://pastebin.ubuntu.com/25535133/ [17:14] xenial lxc fails with resizefs test issues [17:15] yeah that ^ [17:15] I run artful, so was trying xenial in a lxc [17:15] ok I'll work master for those two branches on zesty and artful and see what we can sort [17:16] I think we are dealing w/ either differences in nose or differences in httpretty [17:16] for the system packages === smatzek is now known as smatzek_ === smatzek_ is now known as smatzek [17:26] blackboxsw, you have a handle on that reesizefs test issues ? [17:26] (sorry for pulling that :-( [17:26] smoser I'm looking at that specifically right now from within an lxc. [17:26] it's good, it passed CI too [17:27] but a system package in the lxc on zesty++ is probably behaving differently. I can reproduce w/ osetests3 /root/cloud-init/tests/unittests/test_handler/test_handler_resizefs.py cloudinit/config/cc_resizefs.py on a fresh zesty container [17:32] smoser: KVM branch ready for another review https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/327646 [17:33] tests completed in description and updated commit msg [17:47] whoa smoser powersj am I crazy? [17:47] .. don't answer that yet: http://pastebin.ubuntu.com/25535264/ [17:47] hey smoser: https://code.launchpad.net/~dustymabe/cloud-init/+git/cloud-init/+merge/330701/comments/866238 [17:47] os.access in a zesty container is returning True for W_OK on a read-only file [17:48] I need my eyes/work checked on that pastebin [17:48] blackboxsw: i've seen odd behaviors with containers, what's the backing store? [17:49] dustymabe: thanks for following up! Now smoser can get that one in [17:49] nacc: ext4 [17:49] blackboxsw: sorry, i meant the lxd storage pool [17:49] oh it's on the ext4 disk? [17:49] powersj: np [17:49] :) [17:49] btw do you guys have any idea when the next release of cloud-init will be? [17:50] Next week :) [17:50] dustymabe: we are trying to cut branches to get them in by tomorrow for 0.7.10. yeah as powers says ;) [17:50] need a week of testing/validation [17:50] sweet [17:50] and fixing the bugs I introduced :/ [17:50] :) [17:52] https://lists.launchpad.net/cloud-init/msg00100.html [17:52] There is the mailing list announcement [17:53] nacc: I'm sorry I'm dense on the lxc front. lxc info shows storage type dir for me [17:53] storage: dir [17:53] storageversion: "" [17:53] dustymabe, powersj thanks. i'll pull. [17:54] nacc: but I don't think that's what you are asking. I'm trying to recall if I provided any unique storage pool setup on this install [17:54] blackboxsw: ok [17:54] dustymabe, and thanks again. you did a really good job with that. 6 character change... but loads of good sluething and doc. thanks! [17:55] blackboxsw: and on a xenial container, does the read/write work? [17:55] blackboxsw: ah i wonder if the uid mappings mess with it too? [17:56] blackboxsw: i guess it shouldn't, from within the container [17:56] dustymabe, you would consider that this "fixes" https://bugzilla.redhat.com/show_bug.cgi?id=1490505 [17:56] bugzilla.redhat.com bug 1490505 in cloud-init "cloud-init fails when calling `xfs_growfs /dev/mapper/atomicos-root` on Fedora Atomic Host" [Unspecified,New] [17:56] right ? [17:57] i'm going to c hange the commit message to include [17:57] rhbz: https://bugzilla.redhat.com/show_bug.cgi?id=1490505 [17:57] as larsks had done in a commit of his [17:59] +1 smoser good to know on that. I was wondering whether we duplicate bugzilla bugs in LP or how that works [18:00] blackboxsw: you can add a bugwatch in launchpad if you ever need to [18:00] smoser: thanks! [18:01] smoser: yes once we pull the patches in and rebuild i'll confirm [18:01] but I tested it out in a locally built version and it worked fine [18:03] dustymabe, right. i was just clarifying that you consider it to *fix* the bug rather than just be related to the bug. [18:03] thanks. its 'tox'ing here and pushing [18:04] +1 [18:04] blackboxsw, i dont really have a strong feeling on what to do [18:05] i just knew that larsks had committed some things with a reference, and wanted to keep that going in case he was using anything to read such messages. [18:06] smoser: "default to /srv/citest/kvm-nocloud or something." what about "/srv/images"? [18:08] because that's what I already updated it to ;) [18:11] powersj, well, then what happens when you run on the same system as curtin ? [18:12] well I am pushing to /srv/images/server [18:13] ok so powersj smoser the 'make deb' 'packages/bddeb' error I ran into with resizefs was because I was running as root. [18:14] that used to work though [18:14] so our os.access(, W_OK) always returns True regardless of file perms [18:14] ah [18:14] I'll tweak the test to mock os.access return val [18:14] so we don't care if root runs it versus unpriv user [18:15] * blackboxsw will also fix the OMNIBUS/chef bug if I can in this branch [18:15] smoser: ok so I'll go with mirror_dir: '/srv/citest/kvm-nocloud' [18:15] test bug [18:17] smoser: as far as renaming the class you suggested class KVMNoCloudInstance right? 1) do this to the images, platform, and snapshot files as well? 2) do I need to rename the files as well? [18:17] blackboxsw, so why did it used to work ? [18:18] powersj, i think for now rename them all. [18:19] one could argue that the images that you have could be generic for all kvm [18:19] but then one could argue that they were generic for even more than kvm. [18:19] right [18:19] file name stay kvm.py? [18:19] i say we rename to KVMNoCloud and then at a later date we can genericize them to just kvm. [18:20] what do you think about the file ? [18:20] i think at some point we're going to say "this should get renamed" [18:21] part of me says be verbose now, and make filename as well kvmnocloud [18:24] i think that makes sense. [18:24] ok I'll update the image dir, filenames and class names and resubmit [18:25] smoser: one more, on the CLI when we invoke which backend, should it also be kvmnocloud or remain kvm? [18:27] bikesheds are fun [18:27] so, now i'm thinking... sorry. [18:27] lol [18:27] NoCloudKVM [18:27] as i think it makes sense to have the platform first [18:27] as the backend is somewhat arbitrary [18:27] * powersj will do pretty much whatever smoser says at this point [18:28] do whatever* [18:28] ie, we could potentially have an AWSInstanceStore [18:28] and AWSEbs [18:29] smoser: ok and the platform name for the cli? nocloudkvm as well? [18:30] or just kvm [18:30] I'm thinking nocloudkvm [18:30] i think 'nocloud' there [18:30] yeah [18:30] or nocloud-kvm [18:30] nocloud-kvm it is (I like the best) [18:41] smoser: creating a dir in /srv requires root? [18:48] I guess my past runs have worked using /srv/images since it was already there and setup. [18:48] but if someone runs this the first time it will fail [18:54] powersj yeah. [18:54] we can make it ~/citest [18:54] default to that. [18:54] curtin defaulted to the /srv path [18:54] yeah i did somethingn similar on my home machinne [18:55] as we'd like them generally shared [18:55] for the purpose of testing as my user [18:55] but make it configurable [18:55] (environment is "configurable" enough for me) [18:55] that is what curtin uses [18:55] smoser: it is configurable in the yaml file, do you want more than that? [18:55] ah. thats fine enough. [18:55] ok [18:55] but /tmp not ok. [18:55] pushed renames and that fix to https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/327646 [18:56] powersj, ok. i'm reviwing 'collect-logs' from blackboxsw now. then to you [18:56] ok thanks [18:57] powersj: man, I'm on an artful container ./packages/bddeb still succeeds for me [18:58] no chef unit test failures [18:58] I had run 'make ci-ubuntu-deps' first though [18:59] https://www.irccloud.com/pastebin/1tJ4ifK7/ [19:02] blackboxsw: trying a artufl container now [19:06] blackboxsw: agreed, so something on my local system is causing it [19:06] how's it fail on your system ? [19:06] https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330774 here's the fix for leaking os.access as root user [19:07] smoser: https://paste.ubuntu.com/25535666/ [19:08] Using python3-httpretty 0.8.14-1 [19:08] hrm [19:11] blackboxsw, question at https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330774 [19:12] eww. right-o [19:12] smoser: I think you are right. we'd always see it as writeable in cloud-init [19:14] so, I need to fix that logic [19:19] fix or drop... hmm /me attempts to specify resize user-data to lxc to watch it break [19:20] blackboxsw, well, it will say it can't tehre [19:20] in a container [19:21] hm.. [19:21] well, it doesnt seem to do that well (looking at /var/log/cloud-init from a container) [19:21] smoser: I'm not seeing the check that would trigger that rejection [19:21] ? [19:21] if not os.access(devpath, os.W_OK): # will never be true right? [19:22] because we are root [19:23] Device '/dev/mapper/ubuntu--vg-root' did not exist in container. cannot resize: dev=/dev/mapper/ubuntu--vg-root mnt_point=/ path=/ [19:23] ahh [19:23] http://paste.ubuntu.com/25535747/ [19:23] yeah. ends up getting there somewhere. [19:23] so we don't get down that far [19:24] we bail in the ENOENT handling [19:24] ok so that said, container logic below line 197 is probably unused paths right? [19:25] only thing non-container read-only devpath, which likely can't be triggered as root user [19:25] well, it'd mostly be bogus path. [19:39] blackboxsw, in an iscsi root [19:39] with overlayroot [19:39] (like maas) [19:39] we see [19:39] 2017-09-14 19:33:42,855 - cc_resizefs.py[WARNING]: Device 'overlayroot' did not exist. cannot resize: dev=overlayroot mnt_point=/ path=/ [19:39] http://paste.ubuntu.com/25535835/ [19:40] oh. hmm.. but that is 'overlayroot' that it finds as its dev. [19:41] oh. hmm.. but that is 'overlayroot' that it finds as its dev. [19:46] strange, I'm not quite sure what we should do here https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330774/comments/866258 [19:53] blackboxsw, ok. i verified that the check for writable is pointless [19:53] as root at least [19:53] in an overlay root on read-only iscsi [19:54] i just hacked and changed devpath to be /dev/disk/by-label/cloudimg-rootfs [19:55] in that case what was the dev path perm? [19:56] 400? [19:56] s/perm/mode [19:57] http://paste.ubuntu.com/25535925/ [19:57] so yeah, that check is pointless [19:57] in the overlayroot path, it norlaly gets 'overlayroot' as its root device [19:57] from /proc/1/mounts [19:58] and that doesnt exist [19:58] so it warns [19:58] hack that so it used /dev/sda1 (the actual device on the filesystem) [19:58] and it sees its a overlayroot filesystem type [19:58] which it doesnt know how to resize [19:58] hack *that* so it gets ext4 [19:58] and in this particular case, resize2fs realizes there wasnt a resize to be done :) [19:59] so it just exited [19:59] gotcha ok, so we can drop the os.access check then right? [20:00] that whole conditional [20:05] pushed the update dropping that check [20:06] --force pushed [20:09] smoser: is that description valid? https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330774 [21:24] blackboxsw, i have to run. [21:24] but i had a ahah [21:24] see ya smoser have a good one [21:24] i remembered one other time i tried to test if something was writable [21:24] and the filesystem actually was lying to me [21:25] if you want to test if something is writable , you can say: [21:25] hey filesystem can i write to that file? [21:25] (thats what we're doing now) [21:25] or you can open it for writing [21:25] and not write anything [21:25] the second path gives you more reliable data :) [21:26] i think probably safe if you do that notrunc and then dont write anything. [21:36] sorry 1:1 just ended [21:38] "i think probably safe if you do that notrunc and then dont write anything." I parsed that safe, if you do 'that' (perform the os.access call?) on trunk and don't write anything