[00:11] <ndonegan_> harlowja: Is Heat making use of cloud-init? Haven't looked into Heat all that much yet
[00:11] <harlowja> ndonegan_ seems like a very heavy user
[00:16] <harlowja> https://github.com/openstack/heat-templates/blob/master/hot/software-config/example-templates/example-cloud-init.yaml
[00:16] <harlowja> and others...
[00:16] <harlowja> most of that template is just formed into cloudinit blobs, lol
[00:16] <harlowja> even the crappy merging format
[00:16] <harlowja> https://github.com/openstack/heat-templates/blob/master/hot/software-config/example-templates/example-cloud-init.yaml#L40
[01:15] <smoser> harlowja, oh. wow. someone using the crappy merging format.
[01:15] <smoser> thats neat i ithink :)
[01:15] <harlowja> smoser its something, hahah
[01:15] <harlowja> *scary*
[01:15] <harlowja> lol
[01:16] <smoser> well, to be fair, anyone using anything that i wrote is scary
[01:16] <harlowja> ;)
[01:16] <harlowja> +3
[01:16] <harlowja> haha
[01:16] <smoser> and then, when you throw in that josh harlow contributed
[01:16] <harlowja> +10
[01:16] <harlowja> lol
[01:16] <smoser> it just gets down right horrific
[01:16] <smoser> :)
[01:16] <harlowja> def
[01:17] <smoser> have a nice night man. and thanks for all your help always.
[01:18] <harlowja> np
[07:16] <harmw> so, people are actually *using* cloud-init! 
[07:16] <harmw> woohoo :p
[14:24] <harmw> smoser: thanks!
[14:24] <harmw> reading the cirros announcement
[14:25] <smoser> since i suck in being responsive and in so many other ways, the least i can do is publicly thank you
[14:25] <harmw> lol
[14:26] <harmw> well, thanks :)
[14:26] <harmw> looks nice :p
[14:27] <harmw> btw, when is a new version of c-i due?
[15:36] <smoser> harmw, a few things hav eto be fixed.
[15:36] <smoser> openstack things
[15:36] <smoser> vendor-data
[15:36] <smoser> and logg woarn. that is currently there.
[15:36] <smoser> going to look more of that today
[15:58] <JayF> smoser: Is there any vendor_data handling in any of the current data sources?
[16:01] <JayF> smoser: hmm... I think I may have thought what I had to do for the first step was harder than I thought
[16:16] <hoban> Hello everyone
[16:18] <smoser> JayF, i'm gonna look at that now.
[16:19] <JayF> I'm working on it as we speak
[16:19] <hoban> I'm working towards migrating from CentOS 6.x to 7.0. Previously, we've done post-installation scripting via /etc/rc.d/init.d/firstboot, which no longer exists. I presume I could use /etc/rc.local. but I only want this code to run on the first time the system boots. Would cloud-init be a good fit for this? I also see a fairly immediate need to use something similar for OpenStack instances we'll be exploring soon. Any reason cloud-init couldn't handle my nee
[16:19] <hoban> Mainly, I'm just manually applying some puppet manifests & running some scripts.
[16:21] <JayF> smoser: what keeps things from blwoing up if vendor_data.json isn't actually json?
[16:21] <JayF> smoser: that's what I've been trying to figure out w/r/t https://etherpad.openstack.org/p/cloud-init-vendor-data
[16:21] <JayF> it looks like the current parsing would raise an exception if anyone put anything but JSON in that file anyway
[16:22] <smoser> "vendor_data.json must be valid json, or cloud-init will warn and ignore"
[16:22] <JayF> OH
[16:22] <JayF> so that behavior is based on what's under the 'cloud-init' key
[16:22] <JayF> so if I have DataSourceConfigDrive loading in vd similarly to how the other sources do it, that should be one chunk of work that can merge separately
[16:23] <smoser> ok.
[16:23] <smoser> so that pad is what i thikn it should work like
[16:23] <smoser> not what it does (we're both aware its buggy)
[16:23] <JayF> Yeah, I'm trying to break work up into steps
[16:23] <smoser> but basically, if 'json.loads(open("vendor-data.json"))' returns a 
[16:24] <smoser>  string: then cloud_init_vendor_data = vendor_data
[16:24] <JayF> https://code.launchpad.net/~jason-oldos/cloud-init/upgrade-configdrive/+merge/232312 that should get us to the point of ConfigDrive feeding in vd the same as the OpenStack data source (md service) does
[16:24] <smoser>    (ie, this is  all for me)
[16:24] <JayF> gotcha
[16:24] <smoser> you said that ruby would have a hard time getting a string, but python-json does allow that
[16:24] <smoser> but it'd be odd
[16:24] <JayF> If I were to do it in python, I'd almost certainly just do a try: except:
[16:24] <smoser> then if its a list... we assume its clond-config-archive, and inspect each of the elements of the list.
[16:25] <JayF> bceause afiact json.loads() in python throws a valueerror if it's not valid json
[16:25] <smoser> right
[16:25] <smoser> it will.
[16:25] <smoser> i showed you this before
[16:25] <smoser> here... just aminute
[16:25] <JayF> yeah I remember :)
[16:25] <smoser> $ python -c 'import json; print(json.dumps(""))'
[16:25] <smoser> ""
[16:25] <smoser> er.. maybe better.
[16:26] <smoser> $ python -c 'import json; print(json.dumps("this is just a string"))'
[16:26] <smoser> "this is just a string"
[16:26] <JayF> What in the blue hell did I do earlier to make it throw an exception?!
[16:26] <smoser> $ python -c 'import json; print(json.loads(json.dumps("this is just a string")))'
[16:26] <smoser> this is just a string
[16:27] <smoser> $ python -c 'import json; json.loads("this is just a string")'
[16:27] <JayF> https://gist.github.com/jayofdoom/efd85d68df5bc9c74217
[16:27] <smoser> ^^ that will show value error
[16:27] <JayF> smoser: ^
[16:27] <smoser> but
[16:27] <JayF> OH
[16:27] <JayF> dumps vs loads
[16:27] <JayF> that's the piece I was missing
[16:27] <smoser> $ python -c 'import json; json.loads("\"this is just a string\"")'
[16:27] <smoser> a quoted string is valid json
[16:28] <smoser> a non-quoted string is not.
[16:28] <JayF> aha
[16:28] <JayF> That makes a lot of sense, thank you very much for walking me through it
[16:28] <smoser> :)
[16:29] <JayF> OK
[16:29] <JayF> so assuming my existing patch is fine
[16:29] <JayF> What's the next step for getting the network configuration support in
[16:29] <JayF> am I dependant on the overall vd fixes?
[16:34] <smoser> ok. give me a bit . and i'll look at this more. have to finish something up.
[16:35] <JayF> OK, that works. I need to commute into the office anyway. When you look at it comments on the merge req would be awesome too :)
[16:35] <JayF> ty
[18:28] <smoser> harmw, 
[18:28] <smoser> cloudinit/distros/freebsd.py:267: undefined name 'searchservers'
[18:29] <harmw> eerrrrr
[18:29] <harmw> how come 
[18:31] <smoser> i'm virtually certain it should be searchdomains
[18:31] <harmw> sounds familiar, since it should probably should be searchdomains
[18:31] <harmw> yep
[18:31] <smoser> as searchdomains is assigned to []
[18:31] <harmw> indeed
[18:31] <smoser> and never set otherwise
[18:31] <smoser> k.
[18:31] <smoser> i'll make that change
[18:32] <harmw> this does ring a bell though, I'm sure i've fixed that once already
[18:32] <harmw> perhaps never got merge, or I'm just way off :p
[18:32] <harmw> thanks though smoser 
[18:33] <smoser> well, trunk is up to date wrt to lp:~harmw/cloud-init/freebsd-fixes
[18:34] <smoser> but who cares. fixed.
[18:35] <harmw> I haven't seen it recently, so probably never got merged somewhere in the past few months - if I'm even correct
[18:36] <harmw> which ubuntu version should I use to build cirros btw?
[18:36] <harmw> 12.04 or somthing?
[18:45] <harmw> ndonegan_: jenkins and jclouds plugin, is that fun?
[18:46] <harmw> from the looks of it that'll spawn a new vm on any supported cloud to just complete the buildtask
[18:50] <smoser> harmw, 0.3 will only build on 12.04
[18:50] <smoser> there is a buildroot bug that causes that.
[18:50] <harmw> great thanks
[18:52] <smoser> i will plan to build 0.4 on trusty though
[18:52] <smoser> so as soon as we get a up to date buildroot we can move there.
[18:52] <harmw> cool
[18:55] <smoser> JayF, where is your work for vendor data stuff ?
[18:55] <JayF> smoser: um I linked it earlier
[18:55] <JayF> https://code.launchpad.net/~jason-oldos/cloud-init/upgrade-configdrive/+merge/232312
[18:56] <JayF> like I said it's not a lot, because I don't think a lot was needed to get to the point where DataSourceConfigDrive does the same stuff for vd that DataSourceOpenStack does
[18:57] <smoser> ok. will look.
[19:07] <smoser> JayF, the one issue there 
[19:07] <smoser> hm.. maybe not.
[19:07] <smoser> never mind. 
[19:10] <smoser> JayF, i like how you did the search.
[19:10] <smoser> we should go backwards on OS_VERSIONS until we find on.
[19:12] <smoser> i think "chronological order by time" is redundant :)
[19:26] <JayF> smoser: I write comments in JayF -vvvvvv
[19:26] <JayF> heh
[19:26] <JayF> smoser: you want me to reword that or any other comments?
[19:27] <smoser> no. i 'm llooking 
[19:27] <smoser> and i think i can simplify it a bit
[19:33] <smoser> JayF, http://paste.ubuntu.com/8312141/
[19:33] <smoser> that is diff versus trunk at 1009
[19:34] <JayF> When you said you could simplify, I suspected that'd be what you'd do
[19:34] <JayF> it lgtm
[19:34] <smoser> it makes the BaseReader just select the latest supported version
[19:34] <smoser> ok. can you give it a quick test ? i will too.
[19:34] <smoser> the other test i have to do is test ec2 right nwo
[19:35] <smoser> and make sure we didn't break it or make annoying warnings if openstack looks like ec2 and that ds claims it
[19:35] <JayF> I don't have a great test environment for this right now
[19:35] <JayF> do you have a suggestion?
[19:36] <JayF> We tested our local changes by using cloud-init to install a newer cloud-init rpm/deb via user_data then rebooting to see if it worked right, heh
[19:36] <JayF> but I obviously can't do that until my changes get far enough that trunk will work on my servers
[19:37] <smoser> ah. ok. i was just going to build from trunk
[19:38] <smoser> and boot an instanance
[19:38] <smoser> then install that
[19:38] <smoser> and run cloud-init --local an dcloud-init
[19:38] <smoser> and see that it seemed to have made sense.
[19:39] <JayF> smoser: boot and instance where? for ec2 obviously aws, but what cloud uses ConfigDrive all the way around except OnMetal?
[19:40] <smoser> updated http://paste.ubuntu.com/8312203/
[19:41] <smoser> JayF, well, if you boot with --configdrive=1
[19:41] <smoser> then you will get a config drive
[19:41] <smoser> you'll still have the metadata service, but cloud-init should choose the configdrive
[19:42] <JayF> smoser: I think harlowja and I both wanted to get the version= kwarg out of the function signature for read_config_drive() since it's not used
[19:42] <JayF> smoser: you didn't appear to keep that change in your diff?
[19:43] <smoser> well, i kept it.
[19:43] <smoser> so you could explicitly state "i want this one"
[19:43] <JayF> nowhere in existing code is that used at all though
[19:43] <smoser> right.
[19:43] <smoser> ok. lets drop that then
[19:45] <JayF> cool
[19:45] <smoser> i do think theres good reason to have it
[19:45] <JayF> I can absolutely see situations where it /could/ be valuable
[19:45] <smoser> i'd like to be able to configure it in the dsconfig
[19:46] <smoser> yeah.
[19:46] <JayF> but I was always told to not include args for stuff like that until it's used
[19:46] <smoser> :)
[19:46] <JayF> but I don't know or have strong opinions in this case
[19:46] <JayF> although being able to tell cloud-init which configdrive version to read via config could be interesting
[19:47] <smoser> biggest use case would be able to reproduce a bug on OS_HAVANA cloud
[19:47] <smoser> when all i had access to was an OS_JUNO
[19:47] <smoser> or something
[19:47] <JayF> I don't /think/ ConfigDrive is version bumping in Juno?
[19:47] <JayF> is it?
[19:53] <smoser> just a bad example
[19:55] <JayF> I mean, it's OK if it's a bad example, just wanted to make sure I was up on what was actually happening :)
[19:55] <JayF> although there will certainly be one for KILO
[20:14] <smoser> JayF, so it looks good to me so far.
[20:15] <JayF> Is there any reason not to land the changes with your modifications?
[20:15] <smoser> the thing annoying right now is that 
[20:15] <smoser> on amazon:
[20:15] <JayF> Land those then we can talk about next steps?
[20:15] <smoser> 2014-09-10 20:10:30,588 - DataSourceOpenStack.py[DEBUG]: Giving up on OpenStack md from ['http://169.254.169.254/openstack/2012-08-10/meta_data.json', 'http://169.254.169.254/openstack/2013-04-04/meta_data.json', 'http://169.254.169.254/openstack/2013-10-17/meta_data.json', 'http://169.254.169.254/openstack/latest/meta_data.json'] after 0 seconds
[20:15] <smoser> i just landed. 
[20:15] <JayF> Woo
[20:15] <JayF> I don't think it's that bad
[20:15] <JayF> that's one log line, in debug
[20:15] <JayF> most normal people won't run in debug ... 
[20:15] <smoser> well, it annoying that it does 4 GETs though.
[20:15] <JayF> ... we do
[20:15] <JayF> Ah, I see that
[20:15] <JayF> also ssomething kinda strange
[20:16] <smoser> we could have it just first poke '/latest/meta_data.json' or even just '/latest/'
[20:16] <JayF> is you're either logging in reverse
[20:16] <smoser> and assume that would always b there.
[20:16] <JayF> or going through them in reverse
[20:16] <smoser> that was just from the initial search
[20:16] <smoser> after it finds one
[20:16] <smoser> then it goes in reversed order like you did
[20:16] <smoser> its just looking for presense of a MD server at all
[20:16] <JayF> Hmm
[20:16] <JayF> smoser: if a cloud is running md service
[20:17] <JayF> smoser: does a GET on /openstack/ succeed?
[20:17] <JayF> If so that's a real quick short circuit for that data source
[20:17] <smoser> right
[20:19] <smoser> i'm gonna change that now to just check for /opentsack/
[20:23] <JayF> Sweet
[20:47] <smoser> we've made some good progress today.
[22:54] <JayF> smoser: I'd still like to chat at some point about what the best next step is for me to get the bigger pieces I want to merge in
[22:55] <JayF> smoser: also, RFE (and something I might try and implement if it's easy), an option to point cloud-init at a config drive dir
[22:55] <JayF> smoser: for running it against "fake" config drives in a chroot
[22:56] <JayF> smoser: CoreOS's cloud-init thinger does this, and it's been very useful in troubleshooting
[23:41] <smoser> JayF, you can do that.
[23:41] <smoser> almost certain it takes a 'seed' dir.
[23:42] <JayF> ah, nice. I'll look at that, if so that'll help testing
[23:43] <smoser> yeah, it works.
[23:43] <smoser> or should
[23:43] <smoser> see DataSourceConfigDrive
[23:43] <smoser>         if os.path.isdir(self.seed_dir):