[00:11] harlowja: Is Heat making use of cloud-init? Haven't looked into Heat all that much yet [00:11] ndonegan_ seems like a very heavy user [00:16] https://github.com/openstack/heat-templates/blob/master/hot/software-config/example-templates/example-cloud-init.yaml [00:16] and others... [00:16] most of that template is just formed into cloudinit blobs, lol [00:16] even the crappy merging format [00:16] https://github.com/openstack/heat-templates/blob/master/hot/software-config/example-templates/example-cloud-init.yaml#L40 [01:15] harlowja, oh. wow. someone using the crappy merging format. [01:15] thats neat i ithink :) [01:15] smoser its something, hahah [01:15] *scary* [01:15] lol [01:16] well, to be fair, anyone using anything that i wrote is scary [01:16] ;) [01:16] +3 [01:16] haha [01:16] and then, when you throw in that josh harlow contributed [01:16] +10 [01:16] lol [01:16] it just gets down right horrific [01:16] :) [01:16] def [01:17] have a nice night man. and thanks for all your help always. [01:18] np === harlowja is now known as harlowja_away === praneshp_ is now known as praneshp [07:16] so, people are actually *using* cloud-init! [07:16] woohoo :p [14:24] smoser: thanks! [14:24] reading the cirros announcement [14:25] since i suck in being responsive and in so many other ways, the least i can do is publicly thank you [14:25] lol [14:26] well, thanks :) [14:26] looks nice :p [14:27] btw, when is a new version of c-i due? === zz_gondoi is now known as gondoi [15:36] harmw, a few things hav eto be fixed. [15:36] openstack things [15:36] vendor-data [15:36] and logg woarn. that is currently there. [15:36] going to look more of that today [15:58] smoser: Is there any vendor_data handling in any of the current data sources? [16:01] smoser: hmm... I think I may have thought what I had to do for the first step was harder than I thought [16:16] Hello everyone [16:18] JayF, i'm gonna look at that now. [16:19] I'm working on it as we speak [16:19] I'm working towards migrating from CentOS 6.x to 7.0. Previously, we've done post-installation scripting via /etc/rc.d/init.d/firstboot, which no longer exists. I presume I could use /etc/rc.local. but I only want this code to run on the first time the system boots. Would cloud-init be a good fit for this? I also see a fairly immediate need to use something similar for OpenStack instances we'll be exploring soon. Any reason cloud-init couldn't handle my nee [16:19] Mainly, I'm just manually applying some puppet manifests & running some scripts. [16:21] smoser: what keeps things from blwoing up if vendor_data.json isn't actually json? [16:21] smoser: that's what I've been trying to figure out w/r/t https://etherpad.openstack.org/p/cloud-init-vendor-data [16:21] it looks like the current parsing would raise an exception if anyone put anything but JSON in that file anyway [16:22] "vendor_data.json must be valid json, or cloud-init will warn and ignore" [16:22] OH [16:22] so that behavior is based on what's under the 'cloud-init' key [16:22] so if I have DataSourceConfigDrive loading in vd similarly to how the other sources do it, that should be one chunk of work that can merge separately [16:23] ok. [16:23] so that pad is what i thikn it should work like [16:23] not what it does (we're both aware its buggy) [16:23] Yeah, I'm trying to break work up into steps [16:23] but basically, if 'json.loads(open("vendor-data.json"))' returns a [16:24] string: then cloud_init_vendor_data = vendor_data [16:24] https://code.launchpad.net/~jason-oldos/cloud-init/upgrade-configdrive/+merge/232312 that should get us to the point of ConfigDrive feeding in vd the same as the OpenStack data source (md service) does [16:24] (ie, this is all for me) [16:24] gotcha [16:24] you said that ruby would have a hard time getting a string, but python-json does allow that [16:24] but it'd be odd [16:24] If I were to do it in python, I'd almost certainly just do a try: except: [16:24] then if its a list... we assume its clond-config-archive, and inspect each of the elements of the list. [16:25] bceause afiact json.loads() in python throws a valueerror if it's not valid json [16:25] right [16:25] it will. [16:25] i showed you this before [16:25] here... just aminute [16:25] yeah I remember :) [16:25] $ python -c 'import json; print(json.dumps(""))' [16:25] "" [16:25] er.. maybe better. [16:26] $ python -c 'import json; print(json.dumps("this is just a string"))' [16:26] "this is just a string" [16:26] What in the blue hell did I do earlier to make it throw an exception?! [16:26] $ python -c 'import json; print(json.loads(json.dumps("this is just a string")))' [16:26] this is just a string [16:27] $ python -c 'import json; json.loads("this is just a string")' [16:27] https://gist.github.com/jayofdoom/efd85d68df5bc9c74217 [16:27] ^^ that will show value error [16:27] smoser: ^ [16:27] but [16:27] OH [16:27] dumps vs loads [16:27] that's the piece I was missing [16:27] $ python -c 'import json; json.loads("\"this is just a string\"")' [16:27] a quoted string is valid json [16:28] a non-quoted string is not. [16:28] aha [16:28] That makes a lot of sense, thank you very much for walking me through it [16:28] :) [16:29] OK [16:29] so assuming my existing patch is fine [16:29] What's the next step for getting the network configuration support in [16:29] am I dependant on the overall vd fixes? [16:34] ok. give me a bit . and i'll look at this more. have to finish something up. [16:35] OK, that works. I need to commute into the office anyway. When you look at it comments on the merge req would be awesome too :) [16:35] ty === harlowja_away is now known as harlowja [18:28] harmw, [18:28] cloudinit/distros/freebsd.py:267: undefined name 'searchservers' [18:29] eerrrrr [18:29] how come [18:31] i'm virtually certain it should be searchdomains [18:31] sounds familiar, since it should probably should be searchdomains [18:31] yep [18:31] as searchdomains is assigned to [] [18:31] indeed [18:31] and never set otherwise [18:31] k. [18:31] i'll make that change [18:32] this does ring a bell though, I'm sure i've fixed that once already [18:32] perhaps never got merge, or I'm just way off :p [18:32] thanks though smoser [18:33] well, trunk is up to date wrt to lp:~harmw/cloud-init/freebsd-fixes [18:34] but who cares. fixed. [18:35] I haven't seen it recently, so probably never got merged somewhere in the past few months - if I'm even correct [18:36] which ubuntu version should I use to build cirros btw? [18:36] 12.04 or somthing? [18:45] ndonegan_: jenkins and jclouds plugin, is that fun? [18:46] from the looks of it that'll spawn a new vm on any supported cloud to just complete the buildtask [18:50] harmw, 0.3 will only build on 12.04 [18:50] there is a buildroot bug that causes that. [18:50] great thanks [18:52] i will plan to build 0.4 on trusty though [18:52] so as soon as we get a up to date buildroot we can move there. [18:52] cool [18:55] JayF, where is your work for vendor data stuff ? [18:55] smoser: um I linked it earlier [18:55] https://code.launchpad.net/~jason-oldos/cloud-init/upgrade-configdrive/+merge/232312 [18:56] like I said it's not a lot, because I don't think a lot was needed to get to the point where DataSourceConfigDrive does the same stuff for vd that DataSourceOpenStack does [18:57] ok. will look. [19:07] JayF, the one issue there [19:07] hm.. maybe not. [19:07] never mind. [19:10] JayF, i like how you did the search. [19:10] we should go backwards on OS_VERSIONS until we find on. [19:12] i think "chronological order by time" is redundant :) === mgagne is now known as Guest75250 [19:26] smoser: I write comments in JayF -vvvvvv [19:26] heh [19:26] smoser: you want me to reword that or any other comments? [19:27] no. i 'm llooking [19:27] and i think i can simplify it a bit [19:33] JayF, http://paste.ubuntu.com/8312141/ [19:33] that is diff versus trunk at 1009 [19:34] When you said you could simplify, I suspected that'd be what you'd do [19:34] it lgtm [19:34] it makes the BaseReader just select the latest supported version [19:34] ok. can you give it a quick test ? i will too. [19:34] the other test i have to do is test ec2 right nwo [19:35] and make sure we didn't break it or make annoying warnings if openstack looks like ec2 and that ds claims it [19:35] I don't have a great test environment for this right now [19:35] do you have a suggestion? [19:36] We tested our local changes by using cloud-init to install a newer cloud-init rpm/deb via user_data then rebooting to see if it worked right, heh [19:36] but I obviously can't do that until my changes get far enough that trunk will work on my servers [19:37] ah. ok. i was just going to build from trunk [19:38] and boot an instanance [19:38] then install that [19:38] and run cloud-init --local an dcloud-init [19:38] and see that it seemed to have made sense. [19:39] smoser: boot and instance where? for ec2 obviously aws, but what cloud uses ConfigDrive all the way around except OnMetal? [19:40] updated http://paste.ubuntu.com/8312203/ [19:41] JayF, well, if you boot with --configdrive=1 [19:41] then you will get a config drive [19:41] you'll still have the metadata service, but cloud-init should choose the configdrive [19:42] smoser: I think harlowja and I both wanted to get the version= kwarg out of the function signature for read_config_drive() since it's not used [19:42] smoser: you didn't appear to keep that change in your diff? [19:43] well, i kept it. [19:43] so you could explicitly state "i want this one" [19:43] nowhere in existing code is that used at all though [19:43] right. [19:43] ok. lets drop that then [19:45] cool [19:45] i do think theres good reason to have it [19:45] I can absolutely see situations where it /could/ be valuable [19:45] i'd like to be able to configure it in the dsconfig [19:46] yeah. [19:46] but I was always told to not include args for stuff like that until it's used [19:46] :) [19:46] but I don't know or have strong opinions in this case [19:46] although being able to tell cloud-init which configdrive version to read via config could be interesting [19:47] biggest use case would be able to reproduce a bug on OS_HAVANA cloud [19:47] when all i had access to was an OS_JUNO [19:47] or something [19:47] I don't /think/ ConfigDrive is version bumping in Juno? [19:47] is it? [19:53] just a bad example [19:55] I mean, it's OK if it's a bad example, just wanted to make sure I was up on what was actually happening :) [19:55] although there will certainly be one for KILO [20:14] JayF, so it looks good to me so far. [20:15] Is there any reason not to land the changes with your modifications? [20:15] the thing annoying right now is that [20:15] on amazon: [20:15] Land those then we can talk about next steps? [20:15] 2014-09-10 20:10:30,588 - DataSourceOpenStack.py[DEBUG]: Giving up on OpenStack md from ['http://169.254.169.254/openstack/2012-08-10/meta_data.json', 'http://169.254.169.254/openstack/2013-04-04/meta_data.json', 'http://169.254.169.254/openstack/2013-10-17/meta_data.json', 'http://169.254.169.254/openstack/latest/meta_data.json'] after 0 seconds [20:15] i just landed. [20:15] Woo [20:15] I don't think it's that bad [20:15] that's one log line, in debug [20:15] most normal people won't run in debug ... [20:15] well, it annoying that it does 4 GETs though. [20:15] ... we do [20:15] Ah, I see that [20:15] also ssomething kinda strange [20:16] we could have it just first poke '/latest/meta_data.json' or even just '/latest/' [20:16] is you're either logging in reverse [20:16] and assume that would always b there. [20:16] or going through them in reverse [20:16] that was just from the initial search [20:16] after it finds one [20:16] then it goes in reversed order like you did [20:16] its just looking for presense of a MD server at all [20:16] Hmm [20:16] smoser: if a cloud is running md service [20:17] smoser: does a GET on /openstack/ succeed? [20:17] If so that's a real quick short circuit for that data source [20:17] right [20:19] i'm gonna change that now to just check for /opentsack/ [20:23] Sweet [20:47] we've made some good progress today. === jfontan_ is now known as jfontan === gondoi is now known as zz_gondoi [22:54] smoser: I'd still like to chat at some point about what the best next step is for me to get the bigger pieces I want to merge in [22:55] smoser: also, RFE (and something I might try and implement if it's easy), an option to point cloud-init at a config drive dir [22:55] smoser: for running it against "fake" config drives in a chroot [22:56] smoser: CoreOS's cloud-init thinger does this, and it's been very useful in troubleshooting [23:41] JayF, you can do that. [23:41] almost certain it takes a 'seed' dir. [23:42] ah, nice. I'll look at that, if so that'll help testing [23:43] yeah, it works. [23:43] or should [23:43] see DataSourceConfigDrive [23:43] if os.path.isdir(self.seed_dir):