[07:59] <ashishjain> hello
[07:59] <ashishjain>  a newbie here .. trying to use cloud-init with libvirt
[07:59] <ashishjain> however data injection is not happening :(
[08:01] <ashishjain> cloud-init.log gives some errors like "Failed at attempted import of DataSourceNone"
[08:01] <ashishjain> No local datasource found etc
[08:02] <ashishjain> I am using cloud-init 0.7.5
[08:02] <ashishjain> Kindly help.
[10:48] <ashishjain> hello...
[10:49] <ashishjain> any samaritan here  who can help me
[11:55] <arkin> ashishjain: what with?
[11:55] <arkin> Ah no, thats outside of my expertise
[11:57] <ashishjain> arkin: cloud-init with libvirt
[12:00] <ashishjain> cloud-init with kvm
[12:57] <smoser> ashishjain, what is it you're wanting ?
[12:57] <smoser> explain how yu're try;ing to do it.
[12:57] <ashishjain> @smoser: Data injection does not work for me
[12:57] <smoser> and look in cloud-init.log for WARN
[12:57] <ashishjain> sure
[12:57] <ashishjain> Here is what I do
[12:58] <ashishjain> I have got a base ubuntu image say base.img and a iso say init.iso.
[12:58] <ashishjain> this init.iso has been created using genisoimage using user-data and meta-data files
[12:59] <ashishjain> Now I have tried using virt-manager as well as virt-install to create a vm, while my VM is created fine but it does not pick any data from init.iso
[12:59] <ashishjain> I am not sure if thei is some field which is not being correctly passed
[13:00] <ashishjain> I will paste the command which I use
[13:02] <ashishjain> sudo virt-install -r 1024   -n test   --vcpus=1  --import --disk vol=test-pool/coe-lab-002-Backup.img,format=vmdk,bus=virtio --boot hd --disk vol=test-pool/init.iso,bus=virtio
[13:03] <ashishjain> here test-pool is the custom pool created by me
[13:04] <ashishjain> .img file is of type vmdk
[13:32] <smoser> ashishjain, its probably the type of data you're putting on it.
[13:32] <smoser> i suggest you use cloud-localds
[13:32] <smoser> (or look at what it does)
[13:33] <smoser> the type of the disk should not matter, but the label on the volume must be 'cidata'
[13:34] <smoser> http://bazaar.launchpad.net/~cloud-utils-dev/cloud-utils/trunk/view/head:/bin/cloud-localds
[13:42] <ashishjain> @smoser: This is the command I use to generate the iso which has got cidata
[13:42] <ashishjain> genisoimage -output init.iso -volid cidata -joliet -rock user-data meta-data
[13:44] <ashishjain> @smoser: How to use cloud-localds?
[13:44] <ashishjain> @smoser: Is their a binary which can be invked
[13:45] <smoser> apt-get install cloud-image-utils
[13:45] <smoser> cloud-localds (see the help there)
[13:45]  * smoser has to reboto
[13:45] <ashishjain> ${0##*/} my-seed.img my-user-data my-meta-data
[13:45] <ashishjain> ya I could see here
[13:45] <ashishjain> what is "${0##*/}"?
[13:59] <smoser> ashishjain, basename $0
[13:59] <smoser> if you have ubuntu, just apt-get install it and use it.
[14:00] <ashishjain> @smoser: Yes I got ubunutu I will  try it out and let you know.
[14:31] <ashishjain> @smoser: It does not help I still have the same issue
[14:31] <ashishjain> kvm -net nic -net user,hostfwd=tcp::2222-:22 -drive file=coe-lab-002-Backup.img,if=virtio -drive file=init.iso,if=virtio
[14:32] <ashishjain> @smoser: I used the following command
[14:33] <ashishjain> cloud-localds init.iso user-data meta-data
[14:34] <smoser> hm.
[14:34] <smoser> and you're booting a ubuntu image ?
[14:38] <ashishjain> yes ubuntu trusty
[14:38] <ashishjain> can it be a cloud-init bug
[14:40] <ashishjain> http://paste.openstack.org/show/472810/
[14:40] <ashishjain> this is my user-data and meta-data
[14:43] <ashishjain> some messages in my cloud-init.log are as follows:
[14:45] <ashishjain> Failed at attempted import of "DataSourceNone" due to: No module named DataSourceNone
[14:45] <ashishjain> No local datasource found
[14:49] <ashishjain> one bug which I found just now https://bugs.launchpad.net/cloud-init/+bug/1356855
[15:00] <smoser> ashishjain, can you log in and get /var/log/cloud-init.log ?
[15:00] <smoser> you can "backdoor" the image with 'backdoor-iamge' easily so you'll be able to log in
[15:02] <smoser> https://code.launchpad.net/~smoser/+junk/backdoor-image
[15:02] <ashishjain> https://code.launchpad.net/~smoser/+junk/backdoor-image
[15:03] <ashishjain> aah google search helps a lot
[15:03] <smoser> right. basically it just ads a user and a password , makeing sure you can log in
[15:03] <ashishjain> I am able to log-in
[15:04] <ashishjain> because the base image has got a username and password
[15:28] <ashishjain> @smoser: I am able to login but no idea how can copy the log file :(
[15:28] <ashishjain> the login is possible only through virt-manager console or kvm UI
[15:28] <ashishjain> but none of these interfaces are allowing me to copy the log file
[15:29] <smoser> sudo apt-get install pastebinit
[15:30] <smoser> pastebinit /var/log/cloud-init.log
[15:30] <smoser> if you have networking then you can scp it out too.
[15:30] <smoser> or, you can use 'mount-image-callback'' and mount the image and get it that way.
[15:31] <smoser> sudo mount-image-callback /path/to/that/image.img -- sh -c 'cat $MOUNTPOINT/var/log/cloud-init.log'
[15:40] <ashishjain> @smoser: here is the complete log http://paste.openstack.org/show/472903/
[15:44] <smoser> ashishjain, i think you have some config that tells it only to look for datasource none
[15:45] <ashishjain> @smoser, here is one  bug https://bugs.launchpad.net/cloud-init/+bug/1356855
[15:45] <ashishjain> which says cloud-init always looks for some particular datasource
[15:46] <ashishjain> @smoser: Any ideas where  this configuration can be?
[15:47] <smoser> $ cat /etc/cloud/cloud.cfg.d/90_dpkg.cfg
[15:47] <smoser> # to update this file, run dpkg-reconfigure cloud-init
[15:47] <smoser> datasource_list: [ NoCloud, ConfigDrive, OpenNebula, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Ec2, CloudStack, None ]
[15:47] <smoser> that is what you'd normally see, and you can configure it by (as it says) running dpkg-reconfigure
[15:48] <smoser> i suspect somewhere in /etc/cloud/cloud.cfg.d or /etc/cloud/cloud.cfg you have 'datasource_list' defined
[15:51] <ashishjain> In 90_dpkg.cfg I have only one entry " datasource_list: [ None ]"
[15:53] <ashishjain> @smoser: I configure it to have all the entries I will try it now
[15:53] <ashishjain> thanks for this pointer
[15:53] <smoser> for this purpose you only need NoCloud . fwiw. and you shoudl always enable 'None' as fallback.
[15:59] <ashishjain> @smoser: voila !!
[15:59] <ashishjain> @smoser: It worked.
[16:00] <ashishjain> @smoser: thanks a lot for your time and help
[16:08] <smoser> no problem.
[16:09] <ashishjain> @smoser: One more question is their a python api to automate the creation of disk for cloud-init...same as what is done by genisoimage or cloud-localds
[16:12] <smoser> there is not. i'd just use cloud-localds and subprocess.
[16:19] <arkin> Hi all, I've just started getting this error on my provider— __init__.py[WARNING]: Unhandled non-multipart (text/x-not-multipart) userdata: 'hostname: skasu2\nfqdn: s...'
[16:20] <arkin> its preventing the server from being setup, can someone help me get past it ??
[16:21] <ashishjain> @smoser: Thanks, are their any other alternatives to inject data into a VM..something whcih is done by openstack by hosting on a 169.254.169.254
[16:22] <smoser> arkin, '#cloud-config' at top.
[16:22] <smoser> its ignoring your data by design
[16:22] <smoser> you need to tell it it is for cloud-init.
[16:23] <smoser> ashishjain, https://gist.github.com/smoser/1278651/
[16:23] <arkin> smoser: I have that— https://www.dropbox.com/s/08nduxrxsh0dwia/Screenshot%202015-09-21%2017.23.22.png?dl=0
[16:24] <smoser> arkin, that would seem like it shoudl be ok
[16:24] <arkin> It was working, I thought it was something about my host updating cloud-init :/
[16:24] <arkin> maybe it had to be changed, I guess not
[16:24] <ashishjain> @smoser: Thanks for that link. I will look into it.
[16:25] <arkin> smoser: I'll debug whether #cloud-config gets passed as I do replacements on the {{ alias }} etc
[16:26] <smoser> arkin, yeah, i wonder if your templter is stripping that as a comment
[16:26] <arkin> smoser: You are spot on actually, I recently added that
[16:26] <arkin> doh
[16:26] <arkin> smoser: Trimming the config files before uploading them ;)
[16:27] <smoser> arkin, you can view what cloud-init saw inside by:
[16:27] <smoser>  at /var/lib/cloud/instance/user-data.txt
[16:28] <arkin> smoser: perfect, thanks
[16:28] <arkin> smoser: don't suppose you are any good at regex ? :D
[16:28] <smoser> i can try quickly
[16:28] <smoser> but only quickly :)
[16:28] <arkin> smoser: I have no issues coming up with ^\s*#.+$ to strip comments
[16:29] <arkin> I want to add an exception for cloud-config using a negative lookahead
[16:29] <smoser> do you have a build-your-own template engine ?
[16:30] <smoser> rather than cheetah or jina or something
[16:30] <smoser> jinja
[16:31] <arkin> smoser: Yeah only needs to be super simple
[16:31] <smoser> i dont know.
[16:31] <smoser> if you know that you're always producing cloud-config,
[16:32] <smoser> one thing that you coudl do is just let the renderer render
[16:32] <smoser> and then add the cluod-config
[16:32] <smoser> and then you could also verify that the renderer produced valid yaml (which is oftne non-obvious0
[16:32] <smoser> by just verifying: yaml.load(that_rendered_text)
[16:32] <arkin> smoser: good suggestion actually, thanks. I'm new to yaml
[16:34] <arkin> smoser: and thanks for your help/advice
[17:56] <sputnik13> hah, sweet
[17:56] <sputnik13> just thought I'd try and see if cloud-init had a channel, of course it does :-D
[17:59] <harlowja> sputnik13 whats uppp
[17:59] <harlowja> lol
[18:00] <harlowja> i know u!
[18:00] <harlowja> ha
[18:00] <harlowja> harlowja is everywhere!
[18:00] <sputnik13> yes, yes you are
[18:00] <sputnik13> when do you start the harlowja for president of the universe campaign
[18:00] <sputnik13> :)
[18:01] <harlowja> oh, i already won that
[18:01] <sputnik13> cloud-init doesn't support lvm resizing right?
[18:01] <harlowja> i don't think so,
[18:02] <sputnik13> disk-image-builder has a -l option to set up the root volumes on lvm
[18:02] <harlowja> https://github.com/stackforge/cloud-init/blob/0.7.x/cloudinit/config/cc_resizefs.py#L78 resize stuffs/module
[18:02] <harlowja> i may or may not have touched that code at some point
[18:02] <sputnik13> but such images fail to resize at boot for the obvious reason that cloud-init don't handle it :)
[18:03] <harlowja> damn cloud-init
[18:03] <harlowja> lol
[18:03] <harlowja> smoser might know more about all this though, if he's around
[18:08] <smoser> i know all about all
[18:09] <smoser> cloud-init does not due lvm resizing
[18:09] <smoser> why do you build images with lvm ?
[18:09] <harlowja> sputnik13 why did u do such a thing????!!!?
[18:09] <smoser> (im not opposed to supporting that)
[18:09] <harlowja> Odd_Bloke smoser sputnik13 is a taskflow-core, so can also be bugged about taskflow
[18:09] <harlowja> :-P
[18:10] <harlowja> we are all about cross-pollination, ha
[18:30] <sputnik13> uh oh
[18:30]  * sputnik13 goes in to hiding
[18:31] <sputnik13> smoser: mine is not to ask why, mine is to do or die...  in other words I was told make lvm work :)
[18:32] <sputnik13> for the time being we're probably going to use some custom scripts or whatever at boot time via userdata, but I was holding out hope that there's some hidden feature in cloud-init that solves all my problems
[18:32] <sputnik13> and makes me coffee and a corned beef hash with poached eggs for breakfast
[18:32] <sputnik13> looks like I'm not getting any of those :(
[18:33] <smoser> sputnik13, i think have to add that support to growpart
[18:34] <smoser> cloud-initm ight be trying to run resize2fs correctly
[18:34] <smoser> but if nothing made the root volume any larger, then that doesn't do anything.
[18:34] <smoser> and the thing that does that on non-lvm is growpart
[18:39] <sputnik13> ic
[18:39] <harlowja> corned beef module isn't merged yet, lol
[18:39] <sputnik13> can it make me corned beef hash too?
[18:39] <harlowja> sure
[18:39] <sputnik13> doh, beat me to the punch :)
[18:40] <sputnik13> so growpart normally does parted or something similar yes?
[18:40] <sputnik13> for lvm support, the partition probably needs to be resized, then pvresize, vgresize, and lvresize
[18:40] <sputnik13> or some combination thereof
[18:41] <smoser> growpart uses sfdisk
[18:41] <smoser> so yeah, growpart would have to recognize that this was an lvm volume and use those tools
[22:01] <sputnik13> hmm, so I would expect that at least the partition resize to still go OK, but growpart fails with the following... http://paste.openstack.org/show/473343/
[22:18] <sputnik13> ahhh, need to run partprobe to re-read partition table but growpart is failing and reverting because sfdisk is returning an error
[23:17] <sputnik13> orrrr, not
[23:17]  * sputnik13 is confused
[23:20] <sputnik13> errr orrr yes...  yes, it is failing and reverting
[23:24] <harlowja> hmmm, that seeems bad
[23:24] <harlowja> lol
[23:24] <harlowja> don't do that
[23:26] <sputnik13> I think what I want is for it to do a partprobe, re-read the part table, and if it's not what it should be then attempt a revert
[23:26] <sputnik13> maybe?
[23:26] <sputnik13> :)
[23:26] <sputnik13> it's not that the update actually failed, sfdisk isn't doing a partprobe for you, so it returns an error saying you need to do it
[23:26] <sputnik13> I think
[23:26] <sputnik13> too many "i think"s here for my comfort
[23:30] <harlowja> ya
[23:30] <harlowja> exactly
[23:30] <harlowja> lol
[23:30] <harlowja> :-/