[01:10] <smoser> cliffw, that's on our roadmap to accomplish
[01:11] <smoser> the goal would be that you'd not have to do anything and it'd all just come up like magic.
[01:35] <powersj> smoser: thanks for merge!
[01:35] <powersj> kvm is looking better, I'll go over it one more time tomorrow and do some more testing: https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/327646
[03:57] <blackboxsw> smoser: rharper, for tomorrow morning, I've queued up all changes in https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330384. just looking for an official approve before I push/merge.
[16:39] <powersj> tot doesn't build...
[16:42] <powersj> blackboxsw: something is up with https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330712
[16:42] <powersj> tox passes fine
[16:43] <powersj> but when I try building now I get: https://paste.ubuntu.com/25534973/
[16:43] <blackboxsw> biulding deb or rpm?
[16:44] <blackboxsw> deb
[16:44] <blackboxsw> checking
[16:44] <powersj> deb
[16:48] <powersj> what I don't understand is why our CI didn't find these
[16:48] <blackboxsw> agreed, same here
[16:49] <powersj> are you able to reproduce locally?
[16:51] <blackboxsw> powersj: nope. not with make deb
[16:51] <blackboxsw> lemme try a direct call to packages/bddeb
[16:52] <powersj> I use ./packages/bddeb
[16:52] <powersj> well make deb failed for me as well :\
[16:53] <blackboxsw> hrm. lemme look at that stack trace again
[16:53] <blackboxsw> I'm thinking stale pyc files maybe, not sure
[16:53] <blackboxsw> I have a fresh checkout though. hmm
[16:53] <powersj> two unmocked errors
[16:54] <powersj> same...
[16:54] <blackboxsw> what's your version of httppretty
[16:54] <blackboxsw> I'll check mine
[16:54] <powersj>  git rev-parse HEAD
[16:54] <powersj> f761f2b5f58c8cf13cfee63619f32046216cf66a
[16:54] <powersj> ?
[16:54] <blackboxsw> accd83c97fa3b3b928af2a7955febad410471fc2 for me
[16:55] <blackboxsw> hold up... lemme check master
[16:55] <powersj> accd isn't master
[16:55] <blackboxsw> I think I was in the chef branch specificall
[16:55] <blackboxsw> yeah
[16:56] <blackboxsw> sorry was on a branch. checking now
[16:57] <blackboxsw> hmm make deb  and ./packages/bddeb works for me there too
[16:57] <blackboxsw> csmith@uptown:~/src/cloud-init (master)$ git rev-parse HEAD
[16:57] <blackboxsw> f761f2b5f58c8cf13cfee63619f32046216cf66a
[16:58] <powersj> hmm
[16:58] <blackboxsw> Removing python3-httpretty (0.8.6-2)    # I'm dropping my xenial system packages to see if something changes
[17:00] <blackboxsw> ohh wait unmet build dependency if I don't have that system package
[17:01] <blackboxsw> powersj: dpkg-query --show python3-httpretty
[17:01] <blackboxsw> python3-httpretty	0.8.6-2
[17:01] <blackboxsw> I'm wondering if we are hitting an incompatibility in our build envs
[17:01] <powersj> dpkg-query --show python3-httpretty
[17:01] <powersj> python3-httpretty	0.8.14-1
[17:01] <blackboxsw> I'm on xenial here, checking zesty
[17:14] <blackboxsw> hmm zesty fails for a different reason
[17:14] <blackboxsw> http://pastebin.ubuntu.com/25535133/
[17:14] <powersj> xenial lxc fails with resizefs test issues
[17:15] <powersj> yeah that ^
[17:15] <powersj> I run artful, so was trying xenial in a lxc
[17:15] <blackboxsw> ok I'll work master for those two branches on zesty and artful and see what we can sort
[17:16] <blackboxsw> I think we are dealing w/ either differences in nose or differences in httpretty
[17:16] <blackboxsw> for the system packages
[17:26] <smoser> blackboxsw, you have a handle on that reesizefs test issues ?
[17:26] <smoser> (sorry for pulling that :-(
[17:26] <blackboxsw> smoser I'm looking at that specifically right now from within an lxc.
[17:26] <blackboxsw> it's good, it passed CI too
[17:27] <blackboxsw> but a system package in the lxc on zesty++ is probably behaving differently. I can reproduce w/ osetests3 /root/cloud-init/tests/unittests/test_handler/test_handler_resizefs.py cloudinit/config/cc_resizefs.py on a fresh zesty container
[17:32] <powersj> smoser: KVM branch ready for another review https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/327646
[17:33] <powersj> tests completed in description and updated commit msg
[17:47] <blackboxsw> whoa smoser powersj  am I crazy?
[17:47] <blackboxsw> .. don't answer that yet:    http://pastebin.ubuntu.com/25535264/
[17:47] <dustymabe> hey smoser: https://code.launchpad.net/~dustymabe/cloud-init/+git/cloud-init/+merge/330701/comments/866238
[17:47] <blackboxsw> os.access in a zesty container is returning True for W_OK on a read-only file
[17:48] <blackboxsw> I need my eyes/work checked on that pastebin
[17:48] <nacc> blackboxsw: i've seen odd behaviors with containers, what's the backing store?
[17:49] <powersj> dustymabe: thanks for following up! Now smoser can get that one in
[17:49] <blackboxsw> nacc: ext4
[17:49] <nacc> blackboxsw: sorry, i meant the lxd storage pool
[17:49] <nacc> oh it's on the ext4 disk?
[17:49] <dustymabe> powersj: np
[17:49] <dustymabe> :)
[17:49] <dustymabe> btw do you guys have any idea when the next release of cloud-init will be?
[17:50] <powersj> Next week :)
[17:50] <blackboxsw> dustymabe: we are trying to cut branches to get them in by tomorrow for 0.7.10. yeah as powers says ;)
[17:50] <blackboxsw> need a week of testing/validation
[17:50] <dustymabe> sweet
[17:50] <blackboxsw> and fixing the bugs I introduced :/
[17:50] <dustymabe> :)
[17:52] <powersj> https://lists.launchpad.net/cloud-init/msg00100.html
[17:52] <powersj> There is the mailing list announcement
[17:53] <blackboxsw> nacc: I'm sorry I'm dense on the lxc front. lxc info shows storage type dir for me
[17:53] <blackboxsw>   storage: dir
[17:53] <blackboxsw>   storageversion: ""
[17:53] <smoser> dustymabe, powersj thanks. i'll pull.
[17:54] <blackboxsw> nacc: but I don't think that's what you are asking. I'm trying to recall if I provided any unique storage pool setup on this install
[17:54] <nacc> blackboxsw: ok
[17:54] <smoser> dustymabe, and thanks again.  you did a really good job with that.  6 character change... but loads of good sluething and doc. thanks!
[17:55] <nacc> blackboxsw: and on a xenial container, does the read/write work?
[17:55] <nacc> blackboxsw: ah i wonder if the uid mappings mess with it too?
[17:56] <nacc> blackboxsw: i guess it shouldn't, from within the container
[17:56] <smoser> dustymabe, you would consider that this "fixes" https://bugzilla.redhat.com/show_bug.cgi?id=1490505
[17:56] <smoser> right ?
[17:57] <smoser> i'm going to c hange the commit message to include
[17:57] <smoser> rhbz: https://bugzilla.redhat.com/show_bug.cgi?id=1490505
[17:57] <smoser> as larsks had done in a commit of his
[17:59] <blackboxsw> +1 smoser good to know on that. I was wondering whether we duplicate bugzilla bugs  in LP or how that works
[18:00] <dpb1> blackboxsw: you can add a bugwatch in launchpad if you ever need to
[18:00] <dustymabe> smoser: thanks!
[18:01] <dustymabe> smoser: yes once we pull the patches in and rebuild i'll confirm
[18:01] <dustymabe> but I tested it out in a locally built version and it worked fine
[18:03] <smoser> dustymabe, right. i was just clarifying that you consider it to *fix* the bug rather than just be related to the bug.
[18:03] <smoser> thanks. its 'tox'ing here and pushing
[18:04] <dustymabe> +1
[18:04] <smoser> blackboxsw, i dont really have a strong feeling on what to do
[18:05] <smoser> i just knew that larsks had committed some things with a reference, and wanted to keep that going in case he was using anything to read such messages.
[18:06] <powersj> smoser: "default to /srv/citest/kvm-nocloud or something." what about "/srv/images"?
[18:08] <powersj> because that's what I already updated it to ;)
[18:11] <smoser> powersj, well, then what happens when you run on the same system as curtin ?
[18:12] <powersj> well I am pushing to /srv/images/server
[18:13] <blackboxsw> ok so powersj smoser the 'make deb' 'packages/bddeb'  error I ran into with resizefs was because I was running as root.
[18:14] <powersj> that used to work though
[18:14] <blackboxsw> so our os.access(<path>, W_OK) always returns True regardless of file perms
[18:14] <powersj> ah
[18:14] <blackboxsw> I'll tweak the test to mock os.access return val
[18:14] <blackboxsw> so we don't care if root runs it versus unpriv user
[18:15]  * blackboxsw will also fix the OMNIBUS/chef bug if I can  in this branch
[18:15] <powersj> smoser: ok so I'll go with mirror_dir: '/srv/citest/kvm-nocloud'
[18:15] <blackboxsw> test bug
[18:17] <powersj> smoser: as far as renaming the class you suggested class KVMNoCloudInstance right? 1) do this to the images, platform, and snapshot files as well? 2) do I need to rename the files as well?
[18:17] <smoser> blackboxsw, so why did it used to work ?
[18:18] <smoser> powersj, i think for now rename them all.
[18:19] <smoser> one could argue that the images that you have could be generic for all kvm
[18:19] <smoser> but then one could argue that they were generic for even more than kvm.
[18:19] <powersj> right
[18:19] <powersj> file name stay kvm.py?
[18:19] <smoser> i say we rename to KVMNoCloud<Stuff> and then at a later date we can genericize them to just kvm.
[18:20] <smoser> what do you think about the file ?
[18:20] <smoser> i think at some point we're going to say "this should get renamed"
[18:21] <powersj> part of me says be verbose now, and make filename as well kvmnocloud
[18:24] <smoser> i think that makes sense.
[18:24] <powersj> ok I'll update the image dir, filenames and class names and resubmit
[18:25] <powersj> smoser: one more, on the CLI when we invoke which backend, should it also be kvmnocloud or remain kvm?
[18:27] <smoser> bikesheds are fun
[18:27] <smoser> so, now i'm thinking... sorry.
[18:27] <powersj> lol
[18:27] <smoser> NoCloudKVM
[18:27] <smoser> as i think it makes sense to have the platform first
[18:27] <smoser> as the backend is somewhat arbitrary
[18:27]  * powersj will do pretty much whatever smoser says at this point 
[18:28] <powersj> do whatever*
[18:28] <smoser> ie, we could potentially have an AWSInstanceStore
[18:28] <smoser> and AWSEbs
[18:29] <powersj> smoser: ok and the platform name for the cli? nocloudkvm as well?
[18:30] <powersj> or just kvm
[18:30] <powersj> I'm thinking nocloudkvm
[18:30] <smoser> i think 'nocloud' there
[18:30] <smoser> yeah
[18:30] <smoser> or nocloud-kvm
[18:30] <powersj> nocloud-kvm it is (I like the best)
[18:41] <powersj> smoser: creating a dir in /srv requires root?
[18:48] <powersj> I guess my past runs have worked using /srv/images since it was already there and setup.
[18:48] <powersj> but if someone runs this the first time it will fail
[18:54] <smoser> powersj yeah.
[18:54] <smoser> we can make it ~/citest
[18:54] <smoser> default to that.
[18:54] <smoser> curtin defaulted to the /srv path
[18:54] <nacc> yeah i did somethingn similar on my home machinne
[18:55] <smoser> as we'd like them generally shared
[18:55] <nacc> for the purpose of testing as my user
[18:55] <smoser> but make it configurable
[18:55] <smoser> (environment is "configurable" enough for me)
[18:55] <smoser> that is what curtin uses
[18:55] <powersj> smoser: it is configurable in the yaml file, do you want more than that?
[18:55] <smoser> ah. thats fine enough.
[18:55] <powersj> ok
[18:55] <smoser> but /tmp not ok.
[18:55] <powersj> pushed renames and that fix to https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/327646
[18:56] <smoser> powersj, ok. i'm reviwing 'collect-logs' from blackboxsw now. then to you
[18:56] <powersj> ok thanks
[18:57] <blackboxsw> powersj: man, I'm on an artful container ./packages/bddeb still succeeds for me
[18:58] <blackboxsw> no chef unit test failures
[18:58] <blackboxsw> I had run 'make ci-ubuntu-deps' first though
[18:59] <blackboxsw> https://www.irccloud.com/pastebin/1tJ4ifK7/
[19:02] <powersj> blackboxsw: trying a artufl container now
[19:06] <powersj> blackboxsw: agreed, so something on my local system is causing it
[19:06] <smoser> how's it fail on your system ?
[19:06] <blackboxsw> https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330774 here's the fix for leaking os.access as root user
[19:07] <powersj> smoser: https://paste.ubuntu.com/25535666/
[19:08] <powersj> Using python3-httpretty 0.8.14-1
[19:08] <blackboxsw> hrm
[19:11] <smoser> blackboxsw, question at https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330774
[19:12] <blackboxsw> eww. right-o
[19:12] <blackboxsw> smoser: I think you are right. we'd always see it as writeable in cloud-init
[19:14] <blackboxsw> so, I need to fix that logic
[19:19] <blackboxsw> fix or drop... hmm /me attempts to specify resize user-data to lxc to watch it break
[19:20] <smoser> blackboxsw, well, it will say it can't tehre
[19:20] <smoser> in a container
[19:21] <smoser> hm..
[19:21] <smoser> well, it doesnt seem to do that well (looking at /var/log/cloud-init from a container)
[19:21] <blackboxsw> smoser: I'm not seeing the check that would trigger that rejection
[19:21] <smoser> ?
[19:21] <blackboxsw>     if not os.access(devpath, os.W_OK):   # will never be true right?
[19:22] <blackboxsw> because we are root
[19:23] <blackboxsw> Device '/dev/mapper/ubuntu--vg-root' did not exist in container. cannot resize: dev=/dev/mapper/ubuntu--vg-root mnt_point=/ path=/
[19:23] <blackboxsw> ahh
[19:23] <smoser> http://paste.ubuntu.com/25535747/
[19:23] <smoser> yeah. ends up getting there somewhere.
[19:23] <blackboxsw> so we don't get down that far
[19:24] <blackboxsw> we bail in the ENOENT handling
[19:24] <blackboxsw> ok so that said, container logic below line 197 is probably unused paths right?
[19:25] <blackboxsw> only thing non-container  read-only devpath, which likely can't be triggered as root user
[19:25] <smoser> well, it'd mostly be bogus path.
[19:39] <smoser> blackboxsw, in an iscsi root
[19:39] <smoser> with overlayroot
[19:39] <smoser> (like maas)
[19:39] <smoser> we see
[19:39] <smoser> 2017-09-14 19:33:42,855 - cc_resizefs.py[WARNING]: Device 'overlayroot' did not exist. cannot resize: dev=overlayroot mnt_point=/ path=/
[19:39] <smoser> http://paste.ubuntu.com/25535835/
[19:40] <smoser> oh. hmm.. but that is 'overlayroot' that it finds as its dev.
[19:41] <smoser> oh. hmm.. but that is 'overlayroot' that it finds as its dev.
[19:46] <blackboxsw> strange, I'm not quite sure what we should do here https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330774/comments/866258
[19:53] <smoser> blackboxsw, ok. i verified that the check for writable is pointless
[19:53] <smoser> as root at least
[19:53] <smoser> in an overlay root on read-only iscsi
[19:54] <smoser> i just hacked and changed devpath to be /dev/disk/by-label/cloudimg-rootfs
[19:55] <blackboxsw> in that case what was the dev path perm?
[19:56] <blackboxsw> 400?
[19:56] <blackboxsw> s/perm/mode
[19:57] <smoser> http://paste.ubuntu.com/25535925/
[19:57] <smoser> so yeah, that check is pointless
[19:57] <smoser> in the overlayroot path, it norlaly gets 'overlayroot' as its root device
[19:57] <smoser> from /proc/1/mounts
[19:58] <smoser> and that doesnt exist
[19:58] <smoser> so it warns
[19:58] <smoser> hack that so it used /dev/sda1 (the actual device on the filesystem)
[19:58] <smoser> and it sees its a overlayroot filesystem type
[19:58] <smoser> which it doesnt know how to resize
[19:58] <smoser> hack *that* so it gets ext4
[19:58] <smoser> and in this particular case, resize2fs realizes there wasnt a resize to be done :)
[19:59] <smoser> so it just exited
[19:59] <blackboxsw> gotcha ok, so we can drop the os.access check then right?
[20:00] <blackboxsw> that whole conditional
[20:05] <blackboxsw> pushed the update dropping that check
[20:06] <blackboxsw> --force pushed
[20:09] <blackboxsw> smoser: is that description valid? https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330774
[21:24] <smoser> blackboxsw, i have to run.
[21:24] <smoser> but i had a ahah
[21:24] <blackboxsw> see ya smoser have a good one
[21:24] <smoser> i remembered one other time i tried to test if something was writable
[21:24] <smoser> and the filesystem actually was lying to me
[21:25] <smoser> if you want to test if something is writable , you can say:
[21:25] <smoser>  hey filesystem can i write to that file?
[21:25] <smoser> (thats what we're doing now)
[21:25] <smoser> or you can open it for writing
[21:25] <smoser> and not write anything
[21:25] <smoser> the second path gives you more reliable data :)
[21:26] <smoser> i think probably safe if you do that notrunc and then dont write anything.
[21:36] <blackboxsw> sorry 1:1 just ended
[21:38] <blackboxsw> "i think probably safe if you do that notrunc and then dont write anything."  I parsed that safe, if you do 'that' (perform the os.access call?) on trunk and don't write anything