[09:55] <yann2> Hello! I am trying to use cloud-init with FOG and rackspace, but am failing miserably. It seems that my  mime encoded cloudinit script is properly copied to /var/lib/cloud/seed/nocloud-net/user-data  , but it doesnt appear to be run... not sure where I should look to see whats wrong... any idea? the rackspace examples I found on the web didnt work
[14:23] <smoser> SpamapS, yeah, i have creds to that private key.
[14:23] <smoser> but it is separate by design from my private key.
[14:23] <smoser> i'll sign it and upload
[14:26] <jclift> yann2: Does the script work when it's not mime encoded?
[14:27] <jclift> yann2: Also, there's a log file in /var/log/cloud-init.log
[14:27] <yann2> jclift, yes, actually I got it to run using nova client 
[14:28] <jclift> k, so it sounds like the mime encoding is busting it somehow
[14:28] <jclift> yann2: I've not tried using mime encoded version
[14:28] <jclift> yann2: At a guess, you'll probably need to look through /var/log/cloud-init.log to see if there's anything there
[14:29] <yann2> trying this now http://pastealacon.com/34262  - I think the problem is that rackspace only tries to get cloud init files from config disks by default and not from /var
[14:30] <yann2> jclift, the only error seem to be unrelated and linked to a "ubuntu" user that is not found.. the rest seems to be rackspace stuf
[14:31] <yann2> 2014-04-07 14:27:13,937 - __init__.py[WARNING]: Unhandled non-multipart userdata '   < maybe that
[14:31] <jclift> yann2: Hmmm, I don't use Ubuntu so no idea there
[14:31] <jclift> Yeah, that warning could be significant
[14:32] <jclift> yann2: Are you able to put your proper #cloud-config file on a webserver somewhere, and then point to it using an #include?
[14:32] <jclift> yann2: This is what I use as a #cloud-config file: https://forge.gluster.org/glusterfs-rackspace-regression-tester/glusterfs-rackspace-regression-tester/blobs/raw/master/remote_centos6.cfg
[14:32] <yann2> There is also this file in /etc/cloud/cloud.cfg.d/90_dpkg.cfg that contains this datasource_list: [ ConfigDrive ]
[14:33] <yann2> so I assume my only way to include additional cloud init config would be via the configdrive...
[14:33] <jclift> yann2: I pass that remote_centos6.cfg in to the local VM via config drive, and then cloud-init goes and retrieves the external URLs itself
[14:34] <jclift> yann2: Is the mime-encoded script showing up in /var/lib/cloud/instance/user_data ?
[14:34] <jclift> Or /var/lib/cloud/instance/user_data.i ?
[14:35] <yann2> jclift, cf my last paste, I am now trying to see if I can pass the cloud init file directly to the config file, a bit like the nova client does here  http://developer.rackspace.com/blog/using-cloud-init-with-rackspace-cloud.html
[14:37] <jclift> yann2: I'm using cloud-init with Pyrax in rackspace
[14:37] <yann2> but otherwise, yeah, when i used to copy files using personality, the file showed up mime encoded at the right place ('/var/lib/cloud/seed/nocloud-net/user-data' ) but I think cloud init is not configured to read it
[14:38] <yann2> jclift, and where do you put the cloud init configuration?
[14:38] <jclift> Unfortunately, it's not documented (so I should write up some docs for it to help the next guy)
[14:39] <jclift> yann2: Are you ok with Python?
[14:39] <yann2> I can read, yes, but I need ruby
[14:40] <jclift> k.  This might be helpful for reading then.  It's kind of spagetti code atm though... I need to clean it up.  But it works. https://forge.gluster.org/glusterfs-rackspace-regression-tester/glusterfs-rackspace-regression-tester/blobs/master/create_servers.py
[14:40] <jclift> yann2: Line 179 is where the new server instance is created
[14:41] <jclift> yann2: One of the parameters to the call which creates the new server is "userdata".
[14:41] <yann2> thanks a lot for your help, I ll read this :)
[14:41] <jclift> yann2: I provide the #cloud-config file to that, which passes it automatically to cloud-init using config drive
[14:41] <jclift> (note that I have config_drive=True as another parameter)
[14:42] <jclift> yann2: The #cloud-config file I use, is that "remote-centos6.cfg" thing
[14:42] <jclift> So, when cloud-init starts up, it gets given that remote-centos6.cfg file.  It reads that, noticing it starts with #include, then goes and gets the URLs on the next two lines.
[14:43] <jclift> Those URL contain what actually needs to be done
[14:43] <jclift> The first is a proper #cloud-config thing
[14:43] <jclift> The seconds is a script to execute after the #cloud-config thing has fully run
[14:43] <jclift> s/seconds/second/
[14:44] <yann2> so you use config_drive as well
[14:44] <yann2> and userdata = ci_config... should be very similar to what I m doig...
[14:45] <jclift> yann2: As a thought, when I was getting my head around cloud-init for the first time last week, I spun up the various VM's remotely (like you), then logged into them to try and figure out wtf was happening
[14:45] <jclift> yann2: Yeah, it sounds similar
[14:45] <yann2> so ci_config is not mime encoded, nor base64 encoded
[14:46] <jclift> Correct. Have you logged into a VM, and mounted /dev/xvde to a temp spot to see what's being passed via config_drive?
[14:46] <jclift> yann2: The userdata= argument here can actually be just a newly opened file handle.  You don't even need to read in the contents.
[14:46] <yann2> nope, good idea
[14:47] <jclift> yann2: Yeah, mount /dev/xvde to a temp spot, and take a look through it's structure
[14:47] <jclift> yann2: Helps to get a better conceptual understanding of wtf is going on :)
[14:47] <yann2> indeed.. its xvdd here, where would my userdata be?
[14:48] <jclift> Ahh yeah. xvdd is right
[14:48] <jclift> xvde is the data drive
[14:48] <jclift> Sorry
[14:48] <jclift> Been using xvde a lot over the weekend :)
[14:49] <yann2> mhhh
[14:49] <yann2> so I have a xvdd with bunch of jsons, and no xvde
[14:49] <jclift> That just means you're using a VM type without an additional data disk
[14:49] <jclift> eg 512 Standard or 1GB Performance
[14:50] <jclift> Don't get hung up on xvde thing.  That was my mistage
[14:50] <jclift> mistake
[14:50] <yann2> ok
[14:50] <jclift> My typing sucketh today ;)
[14:50] <yann2> cant see my config copied anywhere on the config disk though
[14:50] <jclift> yann2: Is this similar to what you're seeing? http://fpaste.org/92259/96882152/
[14:51] <yann2> yep
[14:52] <yann2> is my  user_data supposed to be copied somewhere there?
[14:52] <jclift> yeah
[14:52] <jclift> New paste: http://fpaste.org/92262/13968823/
[14:53] <jclift> That shows the same mount, but also shows the user_data file that was pulled in by the userdata= parameter in my Pyrax call
[14:54] <jclift> That file gets placed into /var/lib/cloud/instance/user_data
[14:54] <jclift> After the remote urls are grabbed, the result is /var/lib/cloud/instance/user_data.i, which is then run/processed/whatever
[14:54] <yann2> yeah your ci_config variable actually contains the content of the file
[14:54] <jclift> Yeah
[14:54] <yann2> so this file is a cloud init config file
[14:55] <jclift> Yeah
[14:55] <yann2> and if i read it right you do not base64encode it nor is it mime encoded
[14:55] <jclift> Correct
[14:55] <yann2> I ll show you my script give me a sec
[14:55] <jclift> I don't base64 or otherwise touch it
[14:55] <jclift> k
[14:57] <yann2> http://pastealacon.com/34263
[14:58] <jclift> Does it work like that?
[14:58] <yann2> I dont have a user_data in openstack/latest though
[14:58] <yann2> nope
[14:59] <yann2> I mean it runs and creates the vm, but no userdata in the config disk
[14:59] <yann2> maybe its a question for fog...
[14:59] <jclift> What happens if you try "userdata" instead of "user_data" ?
[14:59]  * jclift noticed some inconsistency in naming around user_data and userdata
[15:00] <jclift> Some parts of Nova require "user_data"
[15:00] <jclift> But Pyrax definitely needs it as userdata
[15:00] <yann2> https://github.com/fog/fog/blob/f531f0bdd715e6f0e3008b35535834ad8e854ec1/lib/fog/cloudstack/models/compute/server.rb 
[15:00] <yann2> seems to be an alias, lets try it..
[15:10] <yann2> still no success :'(
[15:10] <jclift> Damn
[15:10] <jclift> Hmmm, maybe time to ask the Fog guys then
[15:10] <yann2> yep..
[15:10] <jclift> Do they have an example of working Fog code with userdata you could copy to experiment with?
[15:12] <yann2> dont know, I just asked on #ruby-fog
[15:15] <yann2> found this, but its old http://pastebin.com/n6fRuF0C  :D 
[15:18] <jclift> yann2: Heh, well "Good Luck" I guess.  I'm not into Ruby... hopefully the Fog guys can help. :)
[15:19] <yann2> yeah thanks, I think it's supported by the openstack compute but not the rackspace one...
[15:20] <yann2> I ll try my luck with a bug report
[15:25] <yann2> opened a bug there https://github.com/fog/fog/issues/2824  let's wait and see
[15:53] <yann2> jclift, trying my luck patching fog, wish me luck ;)
[15:58] <jclift> :)
[15:58] <jclift> Definitely good luck!
[16:00] <yann2> yeah man, epic skills here :) I ll do a pull request this evening
[16:11] <jclift> :)
[16:35] <sauce> i am using cloud-init for the first time. using it with EC2. i created a very simple user-data #cloud-config file (see here http://pastebin.com/gE8C0M1d).  i am using centos6.4 AMI's from cloudmarket.com that say "with cloud-init installed".  when I do "ec2-run-instances -f myuserdata.yaml", it boots up fine, i can see cloud-init is installed, but my user-data file was not taken into consideration. anyone know why?
[16:55] <smoser> sauce, what is the version of cloud-init ?
[16:57] <yann2> https://github.com/yannh/fog/commit/e335084b200e88263f739a867366afd97fc6b0cb  jclift , for what its worth
[16:57] <yann2> testing a bit more before sending my pull request :)
[16:57] <harlowja> smoser so when we switching cloud-init to git? ;)
[16:59] <smoser> harlowja, i try to avoid flamewars.
[16:59] <harlowja> lol
[17:04] <SpamapS> smoser: cool. I just fetched the key from keyserver.ubuntu.com and it does not have your signature on it.
[17:04] <SpamapS> really any signatures on it
[17:05] <yann2> https://github.com/fog/fog/pull/2826/files now just need a nice soul to merge it and I should be good
[17:06] <smoser> SpamapS, i can sign it. thats fine. the thing i didn't know is what i should sign.
[17:06] <smoser> its a subkey
[17:08] <yann2> alright, off for today, thanks a lot for all your help jclift that was immensely useful
[17:08] <yann2> have a nice day/evening
[17:20] <smoser> SpamapS, i signed now. and put onto keyserver.ubuntu.com . 
[17:29] <SpamapS> smoser: sign the main key. The point is to allow downloaders to verify the key without prayer. ;)
[17:30] <smoser> SpamapS, yeah. i did, right?
[17:38] <SpamapS> smoser: indeed, well done :)
[17:45] <sauce> smoser cloud-init-0.5.15-68.el6_bashton1.noarch
[17:47] <sauce> smoser nevermind, cloud-init did run. i tried it with some low level stuff like touch /tmp/file1 and it did.  now i gotta figure out why the puppet module didnt work
[17:47] <sauce> while i am here, whats the best way to add a host to /etc/hosts with cloud-init?
[17:50] <smoser> sauce, that is old
[17:51] <smoser> i really dont know what support there would be in something called that version.
[17:51] <smoser> without looking. so that i'd be my first guess.
[17:51] <smoser> wrt /etc/hosts, you can modify the apprpriate template file with a boothook.
[17:52] <SpamapS> smoser: note that it would be quite useful now if you started either signing the images, or producing SHA256SUMS
[17:52] <SpamapS> smoser: MD5 is broken
[17:53] <smoser> i am signing the images.
[17:53] <smoser> http://cloud-images.ubuntu.com/releases/streams/v1/com.ubuntu.cloud:released:download.sjson
[17:53] <smoser> thats the preferred way.
[17:53] <SpamapS> preferred by who?
[17:54] <harlowja> 0.5.15, woah
[17:54] <smoser> err.. wait. bad link.
[17:54] <SpamapS> I can use jq to grab those sha256sums tho .. so that works
[17:54] <harlowja> sauce u are on rhel6?
[17:54] <smoser> http://download.cirros-cloud.net/streams/v1/net.cirros-cloud:devel:download.sjson
[17:54] <sauce> 6.4, about 1 year old
[17:54] <harlowja> http://repos.fedorapeople.org/repos/openstack/cloud-init/ should be newer :)
[17:55] <harlowja> although i thought u could install a newer one
[17:55] <harlowja> 0.7.2 is much better ;)
[17:55] <SpamapS> smoser: color me old fashioned.. but a .asc file for each image would be um.. a bazillion times preferrable to that for scripting. :-P
[17:55] <SpamapS> smoser: but I can work with it. :-P
[17:56] <smoser> i agree that having FILE.asc is simpler for scripting.
[17:57] <SpamapS> jq has at least made json available to bash
[17:59] <SpamapS> smoser: would be quite helpful if this format actually had a _list_ for the products. :-P
[18:00] <smoser> list ?
[18:00] <smoser> its a dict
[18:00] <smoser> because the products is a key
[18:01] <SpamapS> smoser: so to find latest version, I should still use /version/released ?
[18:01] <SpamapS> smoser: Yeah, but I have to sort the keys to find the latest. :-P
[18:01]  * SpamapS will call the whambulance
[18:01] <SpamapS> smoser: just whining. :)
[18:02] <jclift> sauce: There's modern cloud-init in EPEL isn't there?
[18:03] <sauce> jclift im just being lazy and using someone elses AMI
[18:03] <sauce> rather than create my own
[18:03] <smoser> SpamapS, http://paste.ubuntu.com/7218123/
[18:03] <jclift> Heh.  Doesn't seem to be working for you in this instance. :(
[18:03] <smoser> that is secure and allows you to easily point it at a mirror if you want.
[18:03] <smoser> and '--max=1' does the sorting for you.
[18:04] <harlowja> sauce i'd recommend at least upgrading the AMI cloud-init version :)
[18:04] <SpamapS> $ sstream-query
[18:04] <SpamapS> sstream-query: command not found
[18:04] <SpamapS> smoser: no package suggestion. :(
[18:05] <SpamapS> smoser: also I have a really giant set of Fedora users so I have to be mindful of hard it will be for them to use.
[18:05] <smoser> $ dpkg -S `which sstream-query`
[18:05] <smoser> simplestreams: /usr/bin/sstream-query
[18:05] <smoser> SpamapS, that giant set of users installs random garbage from pip all the time.
[18:06] <smoser> so... not being packaged isn't a *huge* thing.
[18:06] <harlowja> bad users
[18:06] <SpamapS> smoser: oh we're quite happy to abuse pip :)
[18:06] <smoser> so i should get a simpelstreams into pip.
[18:07] <sauce> ubuntu's official EC2 ami's use: 0.6.3-0ubuntu1.10
[18:08] <SpamapS> smoser: yes. but for now, jq will suffice
[18:08] <smoser> jq ?
[18:10] <harlowja> http://stedolan.github.io/jq/
[18:10] <harlowja> those bash people
[18:10] <harlowja> lol
[18:11] <SpamapS> --[18:11] <harlowja> lol
[18:11] <smoser> its C ?
[18:11] <SpamapS> smoser: also, you have files that are '.gpg' but they have .asc content in them
[18:11] <SpamapS> smoser: yeah tiny little utility, we use it everywhere to process json
[18:12] <smoser> thats nice.
[18:14] <smoser> SpamapS, i dont think i started that.
[18:14] <smoser> the little i know came from modelling ubuntu
[18:14] <smoser> http://cdimage.ubuntu.com/ubuntu-server/daily/current/MD5SUMS.gpg
[18:14] <SpamapS> sure
[18:14] <SpamapS> they did it wrong too :)
[18:15] <SpamapS> it confuses gpg, you have to tell it that it is ascii armored detached
[18:16] <harlowja> i left my ascii armor at home :-/
[18:17] <smoser> SpamapS, well that sucks.
[18:17] <smoser> my damned ignorance keeps getting in the way.
[18:17] <smoser> the sstreams code only really handles the inline signed anyway.
[18:28] <SpamapS> smoser: Yeah, and the inlined signed is kind of hard to use w/ jq.. as I have to strip off the signature.. :-P
[18:29] <smoser> right. so we need to improve jq for inline signed json
[18:31] <SpamapS> smoser: jq example:
[18:31] <SpamapS> $ jq '.["products"]["net.cirros-cloud:standard:0.3:i386"]["versions"] | keys | sort | max' net.cirros-cloud\:released\:download.json
[18:31] <SpamapS> "20140317"
[18:32] <SpamapS> see how hard that is. :-P
[18:33] <SpamapS> $ jq '.["products"]["net.cirros-cloud:standard:0.3:i386"]["versions"]["20140317"]["items"]["disk.img"]["sha256"]' net.cirros-cloud\:released\:download.json
[18:33] <SpamapS> "f0803c2d179c8a02d029239d35fc3e752cc81ad3436ea52b757e11685ca7c074"
[18:34] <smoser> see how much easier it was with the tool i gave you ?
[18:35] <SpamapS> smoser: is it on pypi yet?
[18:35] <smoser> its not. i can put it there, although i've never done that before.
[18:36] <smoser> so again with the damned ignorance getting in my way.
[18:39] <SpamapS> smoser: I've got a jq solution. It's not horrible. But if/when you do get it on pypi, please let me know so I can switch.
[18:39] <smoser> SpamapS, are you looking to put that into devstack ?
[18:39] <smoser> or elsewhere
[18:42] <SpamapS> devstack is dead to me :)
[18:42] <SpamapS> but it should go there too yes
[18:42] <SpamapS> smoser: TripleO devtest yes
[18:42] <SpamapS> smoser: https://review.openstack.org/#/c/83347/
[18:44] <smoser> well, its interesting actually. both of them are.
[18:44] <smoser> from the "continuous integration" perspective.
[18:45] <smoser> in that I can randomly change / break things. 
[19:00] <sauce> where can i get full cloud-init docs other than http://cloudinit.readthedocs.org/en/latest/?  the Modules section is empty, and i feel that the examples don't list everything.
[19:01] <sauce> i.e. it's not clear how to add a 3rd party apt repo
[19:02] <sauce> i would even settle for code to look through.  where are the modules stored?
[19:08] <smoser> sauce, you get docs for the version you'r using in the package you're using.
[19:09] <smoser> the config modules are stored in cloudinit/modules/cc_*
[19:09] <smoser> (except they might be different in 0.5.x)
[19:09] <smoser> to add a 3rd party apt repo is trivial 
[19:09] <smoser> and is shown at http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config.txt#L21
[19:10] <harlowja> SpamapS u say devstack is dead??
[19:10] <harlowja> intersting
[19:16] <harlowja> SpamapS u know u want to use anvil, lol
[19:23] <SpamapS> harlowja: I say that devstack is dead _to me_
[19:24] <SpamapS> harlowja: we use devtest.. because multi-node matters. :)
[19:24] <harlowja> lol
[19:24] <harlowja> ya
[19:34] <sauce> thanks smoser !
[19:34] <sauce> smoser you say the docs come with the package, i have no man cloud-init though
[19:36] <smoser> sauce, in ubuntu they're in /usr/share
[19:37] <smoser> in centos or whatever, id' rpm -qa | grep -i doc
[19:42] <sauce> thanks sir
[19:43] <sauce> i switched to ubuntu for now. i want to learn cloud-init in ubuntu first, then i'll go to centos
[19:48] <smoser> or just stay on a sane OS :)
[19:49] <smoser> although, to be fair, 2.6.18 was a good kernel.
[20:07] <smoser> sauce, i was making fun of centos.
[20:07] <sauce> i know i know
[20:07] <smoser> :)
[20:07] <sauce> we are working on building a new app at work, and i have the opportunity to choose the linux distro
[20:08] <sauce> i want to use ubuntu, but that's just the kid in me talking
[20:25] <sauce> hey smoser, i'm gonna paste somethin in chan, forgive me
[20:25] <sauce> apt_sources: - source: deb http://apt.puppetlabs.com $RELEASE main dependencies
[20:25] <sauce>    keyid: 4BD6EC30    # GPG key ID published on a key server
[20:25] <sauce>    filename: puppetlabs.list
[20:26] <sauce> i can't get that to work. i troubleshooted down to the "keyid" line. if i comment it out, cloud-init works
[20:32] <smoser> sauce, indentation is important.
[20:32] <smoser> youc an see that what you're going to get is sane by just doing somethin glike:
[20:33] <smoser> python -c 'import yaml, pprint, sys; pprint.pprint(yaml.load(open(sys.argv[0])))' my.file
[20:33] <sauce> also maybe yamllint.com ?
[20:34] <smoser> http://paste.ubuntu.com/7218786/
[20:34] <smoser> sure. 
[20:36] <sauce> yamllint.com says the yaml is OK but outputs it vastly different than it appears in examples
[20:36] <sauce> lemme try python
[20:39] <sauce> i don't think indentation is the problem here
[20:44] <smoser> sauce, its not that its valid or not.
[20:44] <smoser> its that the result is int he right format
[20:44] <sauce> i hear ya
[20:44] <smoser> apt_sources is a list of dictionaries
[20:44] <sauce> i don't see how i can be messing up the indentation
[20:44] <sauce> its just a few spaces, everything is lined up
[20:44] <sauce> i deleted the line and rewrote it char by char
[20:45] <sauce> "filename" works with the same indentation
[20:45] <sauce> keyid doesn't
[20:45] <smoser> hm.
[20:45] <sauce> i know right
[20:45] <smoser> pastebin /var/log/cloud-init.log ?
[20:46] <sauce> absolutely hang on
[20:47] <sauce> welp there ya go: Apr  7 20:42:26 ip-10-30-4-54 [CLOUDINIT] cc_apt_update_upgrade.py[WARNING]: Source Error: deb http://apt.puppetlabs.com precise main dependencies:failed to get key from keyserver.ubuntu.com
[20:49] <smoser> sauce, its actuall preferable from both security and reliability perspective to insert the full key
[20:49] <sauce> running this again, lemme check
[20:50] <sauce> smoser ok i will try that
[20:50] <smoser> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config.txt#L120
[20:50] <smoser> hte example there.
[20:50] <smoser> its more reliable because you dont depend on the keyserver
[20:51] <smoser> and int his case all you're really using it for is a object store to map <short_key_id> to <public key>
[20:56] <sauce> i gotcha.  i pasted the key block and it validated as yaml. running it now.
[20:57] <sauce> many ec2 instances have died at my hand stoday
[20:57] <sauce> hands today
[20:58] <smoser> sauce, for playing, you can use lxc or kvm 
[20:58] <smoser> lxc you can use quite performantly inside a ec2 instance
[20:58] <smoser> see example here: http://ubuntu-smoser.blogspot.com/2013/08/lxc-with-fast-cloning-via-overlayfs-and.html
[20:58] <sauce> while this is running i have another question: can i specify the order that things are run? i.e. i want to install puppetlabs repo, then use the puppet module
[20:59] <sauce> nice the key worked!!  
[20:59] <sauce> and it installed in the right order, that was a 50/50 chance probably right?
[20:59] <smoser> well, no. :)
[20:59] <smoser> puppet module runs after apt_config
[20:59] <smoser> and that is by design
[20:59] <smoser> you can specify the order of things
[21:00] <smoser> just define the list in your user-data
[21:00] <smoser> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/config/cloud.cfg
[21:00] <smoser> the easiest thing to do is just re-define the whole list
[21:00] <smoser> cloud_init_modules runs first, then cloud_config_modules, then cloud_final_modules
[21:01] <smoser> nd the listst there in order.
[21:01] <smoser> you really should take a look at that blog post, it makes testing this stuff really easy and much faste rthan waiting on instances in amazon 
[21:02] <sauce> awesome smoser 
[21:03] <sauce> just awesome
[21:06] <smoser> thanks.
[21:22] <sauce> why do i even need puppet, if i have cloud-init?