[00:03] <openstackgerrit> Joshua Harlow proposed stackforge/cloud-init: Bring over the 'templater' from bzr  https://review.openstack.org/170257
[00:03] <harlowja> smoser ok updated with fixtures usage
[00:11] <openstackgerrit> Joshua Harlow proposed stackforge/cloud-init: Expose api response properties and cache buffer decoding  https://review.openstack.org/195800
[14:29] <erhudy> good morning, i'm having a bit of an issue with some openstack vendordata that's clobbering /etc/cloud/cloud.cfg and breaking things when launching instances from ubuntu cloud images
[15:30] <smoser> erhudy, example ?
[15:30] <smoser> i'd consider it a bad idea for vendordata to clobber /etc/cloud/cloud.cfg
[15:52] <erhudy> i've been digging into it more and it seems like something is happening with the vendordata that's causing freakouts in user_data.py
[15:52] <erhudy> it writes out the raw data to /var/lib/cloud/instance/vendor-data.txt
[15:52] <erhudy> and then when it goes to write it out in mime multipart format to vendor-data.txt.i
[15:53] <erhudy> something along the call stack silently malfunctions and it never writes that file out and other things seems to short-circuit
[15:53] <erhudy> it's really weird
[15:56] <erhudy> not familiar with cloud-init as a whole but i'm guessing the whole thing is wrapped in some sort of exception handler that could be silently consuming some exception that's getting thrown
[15:58] <erhudy> specifically in user_data.py, in convert_string() where it tries to evaluate data[0:4096].lower(), at that point it seems like execution just stops and it unwinds the entire call stack all the way back to the top
[16:01] <erhudy> i pickled out data and then loaded it in python and it looks like TypeError: unhashable type is getting thrown and silently eaten
[16:01] <erhudy> that was ambiguous, i pickled out the data variable and loaded it in the REPL
[16:03] <erhudy> basically it looks like util.decomp_gzip() is returning a dict and things below it expect it to be returning a string
[16:14] <erhudy> actually, raw_data is already a dict, so util.decomp_gzip() doesn't do anything with it
[16:22] <erhudy> this is with 0.7.5 which is what canonical is still including, it looks like the code i'm looking at has changed slightly in 0.7.7
[17:07] <smoser> erhudy, please open a bug.
[17:07] <smoser> if it is non-sensitive, if you can give the content of vendor_data that'd be wonderful.
[17:15] <erhudy> i think it was fixed in 0.7.7
[17:37] <smoser> erhudy, open a bug and we can get it fixed in 14.04 images.
[17:37] <smoser> via SRU to 14.04.
[17:48] <smoser> harlowja, so with safe_yaml just 'yaml.load'... any reason to not just abandon that at the moment ?
[17:49] <harlowja> as long as people use 'yaml.safe_load' and not yaml,load
[17:50] <harlowja> let me tweak it a little
[17:51] <smoser> here. let me push first
[17:51] <openstackgerrit> Scott Moser proposed stackforge/cloud-init: Bring over the 'safeyaml' from bzr  https://review.openstack.org/170252
[17:52] <smoser> there. thats what i did. basically just used safe_load and assert that loading python should fail
[17:53] <smoser> things i need to fix:
[17:54] <harlowja> oh man, u to fast
[17:54] <harlowja> haha
[17:57] <smoser> so i want to test fixtures with the fixtures on 14.04
[17:57] <smoser> and basically want to put in some way to test "will this work on 14.04"
[17:58] <smoser> even build/test
[18:05] <erhudy> smoser: will do
[18:06] <erhudy> smoser: i actually opened a canonical case already asking for cloud-init to get updated to 0.7.7 in UCI, but i'll file a separate bug on cloud-init
[18:07] <smoser> erhudy, ok. thank you.
[18:07] <smoser> we wont update 14.04 -> 0.7.7, but we can/will SRU the fixes.
[18:08] <openstackgerrit> Joshua Harlow proposed stackforge/cloud-init: Bring over the 'safeyaml' from bzr  https://review.openstack.org/170252
[18:08] <harlowja> smoser ok did some other additions
[18:09] <harlowja> sucked over the logging module from taskflow, which sets up a new less than DEBUG level, and a nullhandler and all that crap
[18:10] <smoser> thank you for the YAMLError
[18:10] <harlowja> :-P
[18:10] <smoser> what is shell ?
[18:11] <smoser> i thikn yoru load_file is 'util', no ?
[18:11] <harlowja> idk, i guess common shell stuff
[18:11] <harlowja> unsure, ha, i'll put it either place :-P
[18:11] <harlowja> shell-like stuff in shell?
[18:11] <harlowja> or util, or both, idk
[18:12] <smoser> k. but shell, i tihnk i copied the meaning from the python-<openstacktool>
[18:12] <smoser> the intent was that would be where the main was
[18:12] <harlowja> ah
[18:12] <harlowja> ok
[18:12] <harlowja> gotcha
[18:12]  * harlowja moving
[18:13] <smoser> moving as in houses ?
[18:13] <smoser> as in you're moving to michigan now?
[18:13] <smoser> leaving that west coast non-sense.
[18:15] <harlowja> lol
[18:15]  * harlowja is coming to stay with u man
[18:16] <harlowja> in your basement
[18:16] <harlowja> *we can share the basement*
[18:16] <smoser> sweet. party on wayne.
[18:16] <openstackgerrit> Joshua Harlow proposed stackforge/cloud-init: Bring over the 'safeyaml' from bzr  https://review.openstack.org/170252
[18:17] <harlowja> party on gerrit
[18:17] <harlowja> lol
[18:25] <erhudy> filed https://bugs.launchpad.net/cloud-init/+bug/1469260 for you smoser, let me know if you need/care about tickets in salesforce
[18:26] <smoser> erhudy, i'll mostly let them take care of it. they may or may not bother me.
[18:26] <smoser> but thank you.
[18:27] <erhudy> okay, i referenced the LP ticket in the other one
[18:27] <erhudy> thanks for your time
[18:27] <harlowja> tickets in salesforce?
[18:28] <smoser> canonical support
[18:28] <smoser> if you ahve a support contract it goes up that way
[18:32] <smoser> harlowja, i think we want to ditch requests.
[18:32] <harlowja> oh man
[18:32] <harlowja> ha
[18:32] <smoser> https://bugs.launchpad.net/simplestreams/+bug/1461181
[18:33] <smoser> quite possibly my code that was leaking.
[18:33] <harlowja> u got alot of openfiles
[18:33] <smoser> (not explicitly requests, but we couldn't reproduce it)
[18:33] <harlowja> open less files?
[18:33] <harlowja> 1 file per day, keeps the doctor away
[18:33] <smoser> it was leaking file descriptors.
[18:33] <smoser> many more than 1 per day
[18:33] <smoser> :)
[18:33] <harlowja> 2 files per day keeps the doctor away
[18:34] <harlowja> lots of peopel use requests (all of openstack) so it makes me wonder...
[18:34] <smoser> hm.. maybe it is just mine then.
[18:34] <smoser> but for cloud-init id ont really think we gained a lot from it.
[18:34] <harlowja> agreed, we probably don't
[18:35] <harlowja> except for ssl support that works
[18:35] <smoser> urllib3 has that.
[18:35] <harlowja> ya, pretty sure requests uses urllib3333
[18:35] <smoser> right.
[18:35] <harlowja> and vendorizes it :-/
[18:35] <harlowja> https://github.com/kennethreitz/requests/tree/master/requests/packages/urllib3/
[18:36] <harlowja> which afaik most packagers un-vendorize, lol
[18:41] <smoser> oh nice.
[18:41] <smoser> i didntk now that.
[18:41] <smoser> wow
[18:43] <harlowja> :-P
[20:01] <larsks> smoser: what decides between DataSourceConfigDrive vs DataSourceConfigDriveNet?
[20:02] <larsks> I want cloud-init (0.7.x where x < 6) to consider injected network information on openstack, but it checks for 'self.dsmode = "local"', and with DataSourceConfigDriveNet self.dsmode is always 'net' (and this check ignores any user-set dsmode).
[20:03] <larsks> Or harlowja ? ^^^^
[20:04] <smoser> when it runs.
[20:05] <smoser> larsks, the idea is that config drive net would run in 'cloud-init init --local'
[20:05] <smoser> and *should* have the opportunity there to apply networking config
[20:05] <larsks> smoser: but it won't, if self.dsmode == 'net'.
[20:06] <larsks> ...which appears to always be the case when it checks before trying to run on_first_boot(), which is what seems to handle network config...
[20:06] <larsks> I tried forcing datasource_list: [ ConfigDrive ], but I still end up with dsmode == 'net'..
[20:08] <larsks> (I am looking in particular at http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/sources/DataSourceConfigDrive.py#L119)
[20:13] <larsks> smoser: Oh wait, I get it; the call with --local changes self.dsmode. Okay.
[20:13] <larsks> So, let's see what happens in that case...