#cloud-init 2013-11-18
<nate`> anyone around?
<kwadronaut> nate`: best to phrase your question, and not just ask for asking.
<nate`> what's wrong with this? http://pastebin.com/PQrZp8R9
<nate`> RHEL uses 0.7.1 for it's cloud init on the RHEL provided AMIs.  When using chef: portions from the examples the log dumps out that it can't figure out any cheffyness or puppeteering.
<nate`> well other than the -0 that was a typo that I fixed and ran another instance.
<nate`> http://pastebin.com/XCz1ndSw is the one without the typos.
<nate`> http://pastebin.com/tnBecZk8 is the output of /latest/user-data
<nate`> also here's the error when using the chef: directives: http://pastebin.com/gtM22x2R
<nate`> Final question someone might answer for me.  Is there any decent way to test cloud init scripts without having to fire up a new ami every time?
<harmw> cloud-init --bla from inside the instance?
<harmw> and rm -rf /var/lib/cloud
<nate`> screw it on the using the yaml stuff.  shell scripts it is.
<harlowja> woot, smoser other branch not needed, tim got approved by legal
<swaT30> hey guys, any particular reason that CentOS hosts would be getting a 169.254.0.0 route to eth0, while Ubuntu hosts are not? Seems the route breaks metadata on neutron/quantum
#cloud-init 2013-11-19
<harlowja> hmmm, i don't know of any reason that would happen swaT30 
<harlowja> depends on whats in /etc/sysconfig/network-scripts/ifcfg-eth0
<harlowja> and also what came in on the ec2 metdata/configdrive
<swaT30> harlowja: let me take a look at the scripts
<swaT30> harlowja: eth0 config script is pretty standard dhcp.. and both instances are getting the same metadata/configdrive info
<harlowja> hmmm, not sure then :-/
<harmw> swaT30: an unrelated question, but do your centos instances get configured with static networking?
<smoser> nate`, lxc
<smoser> http://ubuntu-smoser.blogspot.com/2013/08/lxc-with-fast-cloning-via-overlayfs-and.html
<smoser> or kvm
<smoser> http://ubuntu-smoser.blogspot.com/2013/02/using-ubuntu-cloud-images-without-cloud.html
<nate`> I'm forced into using RHEL because of contractual obligations.
<swaT30> harmw: nope, all DHCP
<harmw> oh ok
<smoser> hey...
<smoser> harmw, ping
<smoser> harlowja, 
<smoser> https://bugs.launchpad.net/cloud-init/+bug/1241834
<smoser> can you verify that bug  and fix ?
<smoser> i'll yank that.
<smoser> wanting to do 0.7.4
<smoser> and that looked reasonable if it "works and fixes" for you.
<harmw> sup smoser 
<harmw> don't tell me you tabcompleted the wrong person :p
<smoser> i did.
<smoser> yeah, sorry.
<harmw> np :)
<harlowja> hey smoser , just got in, will verify that
<smoser> i think the issue is just that 0.6.4 is listed there.
<smoser> and there is no tag
<smoser> ie, i think this: http://paste.ubuntu.com/6444094/
<smoser> i can see the failure without it
<harlowja> ah
<harlowja> hmmm
<harlowja> let me try that out
<smoser> harlowja, i just pushed up a fix. just joined 0.6.4 into 0.7.0 (which is actually "correct")
<harlowja> kkk
<harlowja> was just about to try it out :)
<smoser> and now 'make rpm' fails for me with command not found on 'rpmbuild'
<smoser> (which is compltely reasonable)
<harlowja> :)
<harlowja> i'll double check, getting bugged to review some stuff
<harlowja> i've been neglecting some internal reviews, haha
<smoser> harlowja, https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1250398
<smoser> very tempted to drop boto requirement
<harlowja> botooo
<harlowja> oooo
<harlowja> that code still exists somewhere i think
<utlemming> smoser: also, boto is now maintained by AWS
<harlowja> intersting it is utlemming 
<harlowja> was there a public announcement about that?
<utlemming> harlowja: I don't think so. Mitch went to work for AWS a while back. The new EC2 CLI python tool is based on Boto.
<harlowja> ah
<utlemming> https://aws.amazon.com/sdkforpython/
<harlowja> so by inference its supported by amazon then :)
<utlemming> well, that last link is about as official as AWS can get on that
<harlowja> but that seems like they are replacing boto -> boto3?
<harlowja> https://github.com/boto/boto3 
<utlemming> that's news to me....
<harlowja> "This is not a port of boto, but a ground-up rewrite." 
<utlemming> (admittately I don't follow boto closely)
<harlowja> *from readme.rst there
<harlowja> np
<harlowja> good to know anyway (i didn't know it either)
<utlemming> at least it is a MIT license....with AWS you never know what you'll get
<harlowja> smoser on a RHEL 6.4 box, http://paste.ubuntu.com/6445336/, all seems ok
<harlowja> *on a fresh VM/box
<harlowja> seems ok to me
#cloud-init 2013-11-20
<smoser> thanks harlowja .
<harlowja> np
<smoser> did you see my release of 0.7.4 ?
<harlowja> ya
<harlowja> thx! hopefully internally people can pick that up and bundle it in
<smoser> yeah. and hopefully RH folk could grab it.
<harlowja> def
<smoser> https://bugs.launchpad.net/cloud-init/+bug/1220461 
<smoser> ?
<harlowja> whats that about, ha
<harlowja> so little context, ha
<harlowja> :-/
<harlowja> that almost seems like a savannah bug?
<harlowja> but i'm not sure
<harlowja> *openstack savanna (which the bug poster is part of)
<harlowja> maybe it got assigned wrong or something
#cloud-init 2013-11-21
<eric0> hi all
<eric0> is there a way to use ec2 metadata in cloud-config actions? i read the docs on datasources but don't understand whether it automatically pulls down all the stuff available in /latest for use somehow, or whether i need to shell out to ec2-metadata or curl to make use of it
<smoser> no. its not readily available, eric0
<eric0> ok i'll just use a little shell snippet, thx. poking at some additional puppet integrations with it and puppet's trained me too well to treat shell as the last resort :)
<abk> ping... can anyone here answer what all distros compatibility does cloud-init already has... or point to a resource which does
<kwadronaut> http://cloudinit.readthedocs.org/en/latest/topics/availability.html?highlight=centos
<kwadronaut> ignore the highlight.
<abk> thanks 
<abk> I'm trying to get clear picture around cloud-init... using it with lxc or virtualbox containers shall work as with any advance hypervisor-based containers right
<smoser> you needto give it a datasource.
<smoser> but yes.
<abk> ok, and the link says "and more..."; I'll specify the other distros than listed which are of my concers... please let me know if any of those are not supported... Gentoo, Arch, OpenSuse, FreeBSD
<abk> and also is there any tutorial (getting started like) to tryout these on lxc or virtualbox
<kwadronaut> hmmm if there aren't any hello world examples, looking at the cloud config examples could get you started.
<kwadronaut> i remember that around may someone said he wanted to start hacking on fbsd support, don't know the current status.
<abk> thanks, any ack on Gentoo/Arch/Suse
<kwadronaut> i don't personally use any of those. you might want to try the mailinglist or something like that. or stick around for a longer time until someone else pops by who happens to know answers.
<abk> kwadronaut, sure... thanks... just one more Q. do you have any reference links for creating a cloud-init enabled (cloud) image... or to re-phrase it how cloud-images fetch for user data
<Chocobo> Does the "user: <USERNAME>" configuration option create <USERNAME> or does it already have to exist?
<Chocobo> Nevermind... I found it in the examples
<Chocobo> errr.. I found "users:" not "user:"  
<abk> ny reference links for creating a cloud-init enabled (cloud) image... or to re-phrase it how cloud-images fetch for user data?
<Chocobo> is it lock_passwd or lock-passwd?  The example has it both ways.
<harlowja> smoser woah, just noticed that http://cloudinit.readthedocs.org/ format changed
<harlowja> did u do that, or was that the readthedocs.org people
<smoser> i think that was rtd
<smoser> i didn't do it
<smoser> hm..
<smoser> look at the bottom 'latest' (for versions)
<smoser> i wonder if there is more on that.
<smoser> as that is something people have asked for.
<smoser> (of cloud-init, version specific doc)
<harlowja> ya, remember how i think i found a link that version support for bzr not so good :-/
<harlowja> http://read-the-docs.readthedocs.org/en/latest/features.html#versions
<harlowja> i think cloud-init uses tags, not branches
<harlowja> which is under [No
<harlowja> if u created fake-branch that might make a version appear (not sure really)
<smoser> hm.. yeah.
<harmw> looking fine, that new page
<harlowja> ya, the search is neat
<harlowja> versions would be nice to (but see above)
<harmw> yea, im kinda missing a changelog
<harlowja> agreed
<harlowja> we can probably plug that in pretty easily
<harlowja> weird smoser , http://anvil.readthedocs.org/en/latest/ didn't get updated
<harmw> plus, is it just the look&feel thats changed or did the docs themselves got refreshed aswell?
<harlowja> look & feel afaik
<harmw> k
<harlowja> although its odd that my other rtd site didn't change
<harmw> job wel done i'd say :)
<harlowja> maybe they are selectively rolling out that upgrade
<abk> if I need to make a particular distro cloud-init ready... what are the steps... are Gentoo/FreeBSD/Suse supported
<harlowja> freebsd is the odd one there :-P
<harlowja> 1. make sure python exists in distro
<harlowja> 2. make sure basic cloud-init works there (likely remove most of the modules that run)
<harlowja> 3. figure out how cloud-init will get started on boot
<harlowja> 4. adjust any code in cloud-init (especially for freebsd)
<harlowja>   - repeat 2 -> 4
<harlowja> 5. package cloud-init for distro
<harlowja> 6. profit
<smoser> 7. share profit with poor smoser
<abk> @harlowja, @smoser thanks :) ...my main confusion is how system knows to run cloud-init as prior boot task before any, is there any article/wiki from Ubuntu Cloud Images or others that I can refer
<harlowja> hmmm
<harlowja> its very system specific unfortunately abk 
<harlowja> each system uses somethign different
<harlowja> systemd upstart sysvinit
<harlowja> ...
<harlowja> upstart -> ubuntu
<harlowja> systemd -> fedora
<harlowja> sysvinit -> most everything else (old school)
<harlowja> freebsd -> (??)
<smoser> abk, the big thing is that cloud-init local runs first, then cloud-init network (and that should run only after all networking has come up)
<abk> openrc -> freebsd
<smoser> and then the same order for the other upstart/ jobs.
<harlowja> ya, what smoser said is the big ordering
<abk> if I can clealry understand application of it in terms of any sysVinit/systemD/upstart/openrc; I'll decipher it for others :(
<harlowja> abk u can look at the upstart/sysvinit and such in http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/sysvinit/redhat/cloud-init
<harlowja> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/files/head:/upstart/
<abk> ohk, thanks
<harlowja> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/files/head:/systemd/ :)
<harlowja> bb
<abk> hopefully if I get it going... I'll publish article so I don't forget and come asking and others can refer :D thanks folks
<smoser> abk, that'd be great.
<smoser> there are certainly some linux-isms there.
<smoser> but we're open to finding more cross platform solutions.
#cloud-init 2013-11-24
<harmw> are there plans to have cloud-init run on freebsd?
<harmw> hm, I'm guessing a freebsd.py in cloudinit/distros could suffice...
#cloud-init 2014-11-18
<harlowja> smoser when u get some time, can u lookover the following 2 
<harlowja> https://code.launchpad.net/~harlowja/cloud-init/setup-virtualenv-installable
<harlowja> sorry, 3 , ha
<harlowja> https://code.launchpad.net/~harlowja/cloud-init/fix-digital-ocean-test-py26
<harlowja> https://code.launchpad.net/~daniel-thewatkins/cloud-init/ssh-equals
<broskos> Hello - anybody on with a moment to help me troubleshoot?
<broskos> I am trying to pass a chef config via os::heat::cloudconfig.  After boot, my instance logs a cloud init error :
<broskos> 2014-11-18 12:05:15,134 - util.py[WARNING]: Running chef (<module 'cloudinit.config.cc_chef' from '/usr/lib/python2.7/site-packages/cloudinit/config/cc_chef.pyc'>) failed
<broskos> I get what seems like a properly formatted cloud config file
<broskos> cat /var/lib/cloud/instance/cloud-config.txt  #cloud-config  # from 1 files # part-001  --- chef:     environment: LOCAL     run_list:     - recipe[rms-app-go_server]     server_url: https://dpyc111f98ef.lab.rmsonecloud.net:4040     validation_key: '-----BEGIN RSA PRIVATE KEY-----
<broskos> In fact, If I copy the 'chef' block out of cloud-config.txt and paste it into /etc/cloud/cloud.cfg and remove the instances cache
<broskos> I can run cloud-init -d modules and my chef config builds fine in /etc/chef
<broskos> I can't figure out how to troubleshoot further
<broskos> disregard - whatever the issue, it seems resolved in 0.7.7
<harmw> god.. does greendns.py realy -not- support ipv6?
<harmw> ERROR oslo.messaging._drivers.impl_rabbit [-] AMQP server on bla:5672 is unreachable: [Errno -9] Address family for hostname not supported. Trying again in 13 seconds.
<harmw> seriously, Rabbit..
#cloud-init 2014-11-19
<anotheral> question: what do I need to do to get cloud-init to reload the ubuntu user's authorized keys from AWS metadata, when i'm creating a new AMI?
<harlowja> anotheral typically clearing out the cloud-init data folder should do it
<harlowja> http://cloudinit.readthedocs.org/en/latest/topics/dir_layout.html 
<harlowja> if u are making a new AMI i would also do other cleanup to
<harlowja> cleaning out any other settings (your own ssh keys that were installed...)
<anotheral> yeah it's very weird - i'm starting with an official ubuntu 12.04 daily AMI
<anotheral> instantiate it, clear out /var/lib/cloud and ~ubuntu/.ssh/authorized_keys
<anotheral> and make a new AMI.  the authorized_keys file remains empty when I instantiate the new AMI
<harlowja> hmmm
<harlowja> seems odd, that shouldn't happen
<harlowja> do u have the cloud-init log files from when u make a new instance from your new AMI?
<anotheral> lemme check those
<broskos> I'm using cloud-init with a chef block to install and run chef. How can I notify cfn or heat when the chef run has completed?  Is there a way to do this with cloud-init?
<harlowja> broskos i thought the heat folks had some callback mechanism? perhaps asking in #heat channel ?
<broskos> harlowja: thanks, I'll try.  
<broskos> I see their waithandles and such, and cfn-signal which is pretty straight forward.  Just working on sorting out how to tell when the chef run is finished at the moment
<harlowja> ya, i know one way cloud-init can do it, but i thought the heat folks used a different strategy
<harlowja> aka, http://cloudinit.readthedocs.org/en/latest/topics/examples.html#call-a-url-when-finished is one way, but i didn't think this was the way
<harlowja> that more of signals when cloud-init is done, which could be used to also say 'chef' is done
<broskos> saw that thanks.
<broskos> can't seem to get it to send the right bits back to heat
<harlowja> let me know if u find out, i've always wondered how to and never had enough time to investigate
<broskos> still poking - I'll come up with somehting... I just mostly wanted to make sure I wasn't overlooking something more obvious
<harlowja> broskos u might be interested in https://code.launchpad.net/~harlowja/cloud-init/better-chef-module/+merge/238040 which yahoo is using (until that merges)
<broskos> k
<harlowja> adds more configurability...
<broskos> heh - had to hack in a small portion of that myself.
<broskos> current version doesn't run chef-client except on a gem install
<broskos> I'll keep an eye on that branch, it's much better than what Ive got going
<harlowja> :)
<broskos> harlowja: ok - from within heat you create your wait_condition and your wait_handle
<broskos> like this guy does here:
<broskos> https://developer.rackspace.com/blog/openstack-orchestration-in-depth-part-2-single-instance-deployments/
<broskos> very basic, name them what you will
<harlowja> k, then heat just polls until that stuff is done?
<broskos> the wait_handle when you reference it will return a url that you can post to
<broskos> so now you can use phone_home
<harlowja> ah
<broskos>           phone_home:             url:               Ref:  "GoServerWaitHandle"             post: [ instance_id ]
<broskos> you can also daisy chain config blocks and user data together via mulipart mime
<broskos> and they will process sequentially, 
<broskos> so chef config first, then one that does a notfiy back using heat stuff...
<harlowja> right
<broskos> but that is more complicated than what I needed, since I only wanted the chef config to notify
<harlowja> not many people know about that mime multipart stuff, the heat folks do, ha
<broskos> I found it confusing, mostly because there are 3 different template formats and various posts will have used a format slightly different that what you are using causing a lot of time interpreting and converting
<harlowja> ya :-/
<broskos> in the end I got the multipart mime to work, but then backed it out and when back to phone_home
#cloud-init 2014-11-20
<lindenle> I/j #packer
<anotheral> i'd asked on tuesday about an AMI that won't populate ssh keys
<anotheral> ~ubuntu has an empty authorized_keys file in the AMI, and it's not getting filled out when I launch an instance
<anotheral> here's the log entry:
<anotheral> Nov 20 21:15:41 ip-10-10-4-181 [CLOUDINIT] __init__.py[DEBUG]: handling ssh-import-id with freq=None and args=[]
<anotheral> nothing after it
<anotheral> full log is here: http://paste.ubuntu.com/9132258/
<anotheral> and ec2metadata has the correct keys
<sauce> which AMI? 12.04?
<sauce> the cloud-init on the 12.04 is so old, mostly nothing works, so i basically use runcmd echo a ton.  "echo mydata >> /home/ubuntu/.ssh/authorized_keys" for example
<anotheral> sauce: yeah, 12.04
<anotheral> ok, that helps a bunch
<sauce> thats your problem
<anotheral> at least i'm not crazy
<sauce> whatever ssh module you're using won't work
<anotheral> guess i get to tell my users we're upgrading :-D
<sauce> well before you do that, check out how 14.04 AMIs behave. i have no experience with 14.04 yet
<anotheral> what I'm trying to do is just extend the base AMI with some company-specific stuff
<sauce> oh. so take 12.04 and upgrade the cloud-init
<sauce> then do your stuff
<sauce> then AMI it
#cloud-init 2014-11-21
 * mhayden tips his hat to JayF
 * gholms waves at mhayden
<JayF> Hey smoser, mhayden is another long-time racker
<harlowja> mhayden hey, for https://code.launchpad.net/~rackerhacker/cloud-init/add-ipv6-static-address-support a coworker at y! is also working on this, https://bugs.launchpad.net/cloud-init/+bug/1391695 ; she should have that up for review today, hopefully we can look at both and merge them or something
<JayF> he's going to be tossing a patch your way for IPv6 static support
<harlowja> JayF i saw that :-P
<mhayden> harlowja: good timing ;)
<harlowja> ;)
<mhayden> harlowja: should i add a mention of my branch in that ticket?
<harlowja> sure, although both of u guys i think started a solution at the same time, ha
<harlowja> concidence :-P
<mhayden> i'm curious to see the other solution as this is my first time working with cloud-init ;)
<mhayden> well, working with the code itself i mean
<harlowja> agreed
<harlowja> she'll be in shortly i think
<harlowja> mhayden another intersting question is whether ipv6 works for the other distros also
<harlowja> that network conversion (from ubuntu -> internal format) is also used in freebsd for example
<mhayden> my changes only add in some new dict keys
<mhayden> so it shouldn't affect them unless they blindly write whatever is in the dict
<mhayden> i didn't check around for that
<harlowja> ya i don't think they do, but my guess is they are also not ipv6 compat
<harlowja> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/distros/sles.py#L56 
<harlowja> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/distros/arch.py#L63
<harlowja> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/distros/freebsd.py#L273
<mhayden> ah so yeah, the sles one would just write IPv4 config only
<harlowja> ya
<mhayden> would it make sense in the future to put all the network config into json and avoid parsing the debian-style network config?
<JayF> mhayden: That's what I'm doing :)
<harlowja> +1
<harlowja> ;)
<mhayden> then i owe you tacos, JayF 
<JayF> mhayden: https://github.com/racker/cloud-init/pull/1 is the downstream patch we use for OnMetal now (I think it's enabled on the whole cloud at this point, perhaps?)
<JayF> mhayden: we put some info in vendor_data that's based on a spec upstream for openstack to put in json network data, and read that today
<harlowja> https://review.openstack.org/#/c/85673/16/specs/kilo/approved/metadata-service-network-info.rst right?
<JayF> mhayden: when openstack's upstream one comes, we'll migrate off our vendored one and go to the upstream one
<JayF> aye, capn
<JayF> although tbh I'm not working on it actively right now
<harlowja> whose 'Claxton Correya' i don't think thats an alias for u
<harlowja> lol
<harlowja> maybe it is though
<mhayden> it's his pen name
<harlowja> cools
<mhayden> i'm kidding ;)
<harlowja> me too
<harlowja> :)
<smoser> hey all. i'm sorry for not being present.
<smoser> been traveling / working last three weeks and not much else.
<harlowja> spandhe mhayden welcome, both fixing ipv6
<harlowja> :-P
<mhayden> yay
<spandhe> haha
<harlowja> mhayden spandhe  i think uploaded https://code.launchpad.net/~shraddha-pandhe/cloud-init/cloud-init-ipv6-support if u want to check thato ut
<mhayden> taking a look
<mhayden> looks reasonable except i might toss in a "IPV6_AUTOCONF=no" into cloudinit/distros/rhel.py
<mhayden> just for completeness
<mhayden> since there's going to be a static assignment
<mhayden> but i like that spandhe remembered NETWORKING_IPV6 in /etc/sysconfig/network -- i forgot
<spandhe> mhayden: I thought BOOTPROTO=static will take care of it.. 
<spandhe> Ill look up more about autoconf
<JayF> spandhe: not always ime
<JayF> spandhe: IPV6_AUTOCONF=no basically maps directly to the sysctl iirc
<spandhe> mhayden: I also put the routing info and ifconfig support.. 
<mhayden> good stuff, spandhe 
<mhayden> i'm not sure about the issues caused by leaving out IPV6_AUTOCONF... but i haven't seen any providers that due IPv6 via SLAAC yet
<mhayden> it would hopefully protect the VM against router advertisement attacks though
<spandhe> thanks mhayden ! will add unittests soon too.. also, for now, makes sense to keep autoconf = no.. nova currently doesnt provide any information about it.. so safe it keep it no.. 
<mhayden> cool
<mhayden> i still haven't wrapped my head around writing tests in python :)
<spandhe> mhayden: cool :) need to addcsupport for IPV6ADDR_SECONDARIES later.. 
<mhayden> that will be fun
<mhayden> do you have a use case for it? i don't
<mhayden> we assign a single IPv6 address per instance for now
<mhayden> anything other than that would be a floating/shared IP
<spandhe> right.. not for now.. but when we have multiple later with global scope, etc, we can add it.. 
<mhayden> makes sense
<mhayden> harlowja: i'm a launchpad dunce -- how do i kick out my branch proposal?
<harlowja> i believe there is a delete branch that i think u can access on https://code.launchpad.net/~rackerhacker/cloud-init/add-ipv6-static-address-support if thats what u want to do
<JayF> mhayden: ++ absolutely hardening against RA attacks is perfect
<mhayden> harlowja: yeah, i'll yield to spandhe's work there
<mhayden> done
<mhayden> JayF: yay
<harlowja> kk
<harlowja> smoser alright got http://cloudinit.readthedocs.org/en/latest/topics/modules.html to generate more goodness
<harlowja> example @ http://cloudinit.readthedocs.org/en/latest/topics/modules.html#debug
<harlowja> woot
<harlowja> looks good this time
<harlowja> http://cloudinit.readthedocs.org/en/latest/topics/modules.html#ubuntu-init-switch also
#cloud-init 2015-11-16
<smoser> Odd_Bloke, you around ?
<smoser> hm.. general python / nose question.
<smoser> i'm usinging setUpClass and tearDownClass. and running tests with nose.
<smoser> i'd like to have something that runs at the end of all the 'nosetests' run.
<smoser> and woudl have access to see what the pass/fail were and such.
<Odd_Bloke> smoser: Hmm, not sure how to do that.
<Odd_Bloke> smoser: What are you trying to do?
<smoser> curtin's 'vmtest' runs a bunch of vms, and does install and then reboot and collect output of each install.
<smoser> for better or worse we use setupClass to do the install and collect of output.
<Odd_Bloke> And you want to move away from that?
<Odd_Bloke> Or there's something else that you want to do?
<smoser> the thing i want to do though is basically keep arond the disks and console logs and such of each failure
<smoser> but clean all that up for successful runs
<Odd_Bloke> smoser: Maybe a plugin: http://nose.readthedocs.org/en/latest/plugins/writing.html ?
<smoser> yeah, probably.
<Odd_Bloke> smoser: Or maybe just a wrapper script?
<smoser> yeah. that too might be even better. as then i'm not dependent on nose specifically.
#cloud-init 2015-11-17
<jdcaddie> Is there any documentation on how to configure a datasource for openstack
<jdcaddie> newb here
<jdcaddie> trying to figure out where to exactly configure my cloud-config files
<smoser> jdcaddie, you should not erally have to configure anything
<smoser> the built in list will include openstack search
<smoser> you can optionally explicitly set it in a file in /etc/cloud/cloud.cfg.d/your.cfg
<smoser> datasource_list: [ MAAS ]
<jdcaddie> smoser, I guess im just trying to really figure out how to set custom configurations using cloud-init... such as mount points, running scripts, etc
<smoser> ah.
<jdcaddie> smoser, do I just add them to the cloud.cfg or should I create cloud-config metadata scripts
<smoser> if you want to config things and make a "golden image" with those built in
<smoser> then put whatever you'd like in /etc/cloud/cloud.cfg.d/your.cfg
<smoser> but generally, you dont have to do that and you should just pass whatever you want to happen through user-data
<smoser> https://help.ubuntu.com/community/CloudInit
<smoser> and then http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config.txt
<smoser> and other files in that directory are really best documentation
<smoser> and i admittedly use "best" loosely
<jdcaddie> great, I will do some reading! thanks for the insite
<jdcaddie> in openstack user-data are scripts that get passed when building a instance correct? of can there be centalized set of user-data scripts?
<smoser> when you create an instance
<smoser> (as opposed to build an image)
<smoser> user-data is per-instance and provied on 'nova boot'
<jdcaddie> thanks, thats exacty what i needed to know
<agy> if i want to create an additional DataSource for cloud-init, can i provide cloud-init an additional path to load this new DS?
<agy> i'm not overly keen to dump the new DS in the dist-packages directory if i can help it
<agy> or at least, the dist-packages/cloud-init directory
#cloud-init 2015-11-18
<smoser> agy, no, you cant. :-(
<smoser> agy, but patches are definitely welcome for that.
<BarnacleBob> hi there.  i'm trying to use cloud-init to partition and mount a disk using the latest ubuntu 14.04 official images on aws and i'm getting a stack trace.  https://gist.github.com/BarnacleBob/ef00cbed7d49bd97a058
<BarnacleBob> anyone see what i might be doing wrong
<Odd_Bloke> BarnacleBob: I think you're hitting https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1311463
<BarnacleBob> ah excelent
<BarnacleBob> does cloud-init rerun everytime the machine boots up or is there some protection against that?
<Odd_Bloke> BarnacleBob: So it does run, but modules specify if they are per-instance or per-boot.
<Odd_Bloke> BarnacleBob: So generally it comes up and finds there isn't anything to do.
<BarnacleBob> ok great.  i'm guessing disk partitioning and filesystem are once per instance but i'll test it
<Odd_Bloke> I believe they are.
<Diplomat> hey guys, I have a little issue.. my code fails to create users and set passwords: http://paste.ubuntu.com/13333626/
<Diplomat> and here's my script: http://paste.ubuntu.com/13333632/
<Odd_Bloke> Diplomat: Is that the output in /var/log/cloud-init.log, or is there more info there?
<Diplomat> well, I can't access my vm unfortunately
<Diplomat> oh
<Diplomat> nvm
<Diplomat> no it's from the console
<Diplomat> console log from openstack
<Odd_Bloke> OK, getting cloud-init.log would probably make it much easier to debug.
<Odd_Bloke> Diplomat: Can you mount that disk on another VM and access it that way?
<Diplomat> Not really
<Diplomat> I'm seeing this too
<Diplomat> [WARNING]: Running ssh-authkey-fingerprints (<module 'cloudinit.config.cc_ssh_authkey_fingerprints' from '/usr/lib/python2.6/site-packages/cloudinit/config/cc_ssh_authkey_fingerprints.pyc'>) failed
<Diplomat> I'm using CentOS 6.7 cloud image
<Diplomat> from their website
<Odd_Bloke> Diplomat: OK, if you aren't going to get cloud-init.log, then your best bet is probably to strip down to an empty configuration and add a line at a time.
<Odd_Bloke> Diplomat: I'm curious to know why you can't mount the disk though. :)
<Diplomat> Because I'm not using a volume
<Diplomat> If i'd boot using a volume then i could do that
<Odd_Bloke> Diplomat: What are you booting from?
<Diplomat> From that disc it creates into a compute node
<Odd_Bloke> Diplomat: "that disc it creates" <-- what is creating the disk?
<Diplomat> Umm.. nova?
<Diplomat> I'm not sure what you mean
<Odd_Bloke> So my understanding is that nova creates a volume from the image; you should be able to mount that volume on another instance, no?
<Odd_Bloke> (I'm not super au-fait with OpenStack though, so I could be completely wrong :p)
<Diplomat> Well, you can't do that.. if you would use a volume (Cinder) then you can remove it and add to another VM, mount it and then access those files
<Diplomat> Also.. I just commented out "customer" account creation and used a key pair and now it works
<Diplomat> Changing password for user root.
<Diplomat> passwd: all authentication tokens updated successfully.
<Diplomat> Locking password for user root.
<Diplomat> passwd: Success
<Odd_Bloke> Cool. :)
<Diplomat> Yes, but now I have to figure out why it doesn't want to create an account
<Odd_Bloke> Diplomat: Just add in each line of config there one at a time, and you'll discover which one is the problem.
<Diplomat> I think I already found it.. its that account creation thingy
<smatzek> so you want to log in to an account that is non-root using password after creation?
<Odd_Bloke> Diplomat: Right, I'm saying see if adding "- name: customer" works and iterate just through that section.
<Diplomat> oh
<Odd_Bloke> Diplomat: Having said that, I suspect lock-passwd should be lock_passwd.
<Diplomat> smatzek: yes
<Odd_Bloke> (Not 100% sure)
<smatzek> I was going exactly where Odd_Bloke went.  lock_passwd is wrong in the public cloud-init docs.
<smatzek> that's burned hours for me in the past.
<Odd_Bloke> Oh, really?
<Odd_Bloke> </3
<Diplomat> I removed it.. but it still failed :/
<smatzek> http://cloudinit.readthedocs.org/en/latest/topics/examples.html#yaml-examples  has lock-passwd which is wrong.
<Diplomat> yea i took it from there lol
<smatzek> if you want to be able to log in with password it must be lock_passwd: False
<smatzek> lock-passwd: False doesn't get picked up because of the - _ difference and the default for lock_passwd is True.
<Diplomat> groups: sudo
<Diplomat> maybe that's a problem too ?
<Odd_Bloke> Diplomat: Try a line at a time. :)
<smatzek> also, it's not covered much in the docs but you could try plain_text_passwd: passw0rd  instead of the hashed one to see if its a problem with a SHA / salt difference.
<Diplomat> Good idea
<Diplomat> yea
<Diplomat> groups: sudo was wrong
<Diplomat> it failed then
<Diplomat> but I still can't login with customer:password
<Diplomat> there are no error messages :/
<Diplomat> also it only says Changing password for user root.
<Diplomat> nothing about customer..
<Diplomat> oh, because i forgot to add that line :>
<Diplomat> wooo
<Diplomat> works :D
<Diplomat> Now I have to figure out why it doesn't resize the disc
<Diplomat> growpart:
<Diplomat>  mode: auto
<Diplomat>  devices: ["/"]
<Diplomat>  resize_rootfs: True
<Diplomat>  resize_rootfs_tmp: /dev
<Diplomat> sorry for spam
<Diplomat> but shouldn't this part do that?
<Diplomat> or i have to do under runcmd growpart /dev/vda 1
<Diplomat> or whatever it was
<Diplomat> Actually.. I can see that something is running this: fsck.ext4 -a /dev/vda1
#cloud-init 2015-11-19
<ezeh0001_> Trying to change openstack single disk partition mounted to / with cloud-int to repartition to have multiple mountpoints (/var) at least
<ezeh0001_> Tried using disk_setup and fs_setup in user-data but that is not working
<ezeh0001_> Is it even possible?
<ezeh0001_> Have question about partitioning disk with cloud-init
<Odd_Bloke> ezeh0001_: That should be possible; what do you mean by "not working"?
<Odd_Bloke> (And what distro/version are you using?)
<ezeh0001_> I am running on RHEL 7
<ezeh0001_> I used the disk_setup inside user-data, but when it runs the partition is not yet there
<Odd_Bloke> ezeh0001_: Can you paste your config somewhere?
<ezeh0001_> Odd_Bloke: Config http://pastebin.com/yv7MKB7H
<Odd_Bloke> ezeh0001_: You have "ephmeral" not "ephemeral" under disk_setup; I don't know if that's causing the problem.
<Odd_Bloke> ezeh0001_: If that doesn't fix it, /var/log/cloud-init.log is also extremely useful for debugging problems.
<Odd_Bloke> ezeh0001_: So either have a look at that yourself, or pastebin it and I can have a look.
<Odd_Bloke> (Though I'm about to go to lunch, so I won't be able to help for 90-120 minutes)
<harlowja_at_home> 120 minute lunch, questionable
<harlowja_at_home> lol
<ezeh0001_> Obb_Bloke: Thanks.  Will try that.
<ezeh0001_> The error I get for disk_setup can be seen here: http://pastebin.com/pWYJ7fbJ
<manuq> hi! how can I enable debug logging to cloud-init ? version is 0.7.5
<ezeh0001_> Odd_Bloke: Looking at the error, the partition table is not changed before cloud-init gets to it.  I am trying to repartition the boot disk.  Is that possible with cloud-init
<Odd_Bloke> harlowja: I had a call as well!
<Odd_Bloke> harlowja: NO YOU SHUT UP
<Odd_Bloke> ezeh0001_: Cool, having a quick look now.
<Odd_Bloke> ezeh0001_: Can you paste the full log? (Having context can sometimes make all the difference when debugging this sort of thing)
<ezeh0001_> Odd_Bloke: whole log is at http://pastebin.com/ZsyNwmYb
<Odd_Bloke> ezeh0001_: Ohhhhh, I only just worked out you're trying to partition /.
<Odd_Bloke> ezeh0001_: I'm not sure we support that.
<Odd_Bloke> smoser: ^ ?
<smoser> whats going on ?
<Odd_Bloke> smoser: Is there anyway to partition the disk that is mounted on /?
<smoser> i dont think linux supports it really.
<smoser> hm.. i guess maybe if / was an lvm
<smoser> but if its a partitioned disk, then you can tell cloud-init not to resize and take the rest, but i dont know that you can add partitions wihtout unmounting
<smoser> i'd have to think more though
<ezeh0001_> @smoser and Odd_Bloke: Thanks for your assistance
<corb00> Hi is there any way to specify uid/gid for a user created in the cloud-init users: section?
#cloud-init 2015-11-20
<leeuwenrjj> Hi, I have a simple question but I cannot find it anywhere in the docs: How can I set the security repositories through cloud-init? apt_mirror for the mirror works fine but I cannot find the setting for the security repo's (ubuntu)
<zero_shane> hi all - I have a question on cloud-init, used in conjunction with the Ubuntu "vmbuilder" tool - everything is working perfectly for me - **except** the default config for /etc/cloud/cloud.cfg.d/90_dpkg.cfg (datasource_list) wires in a bunch of datasources that causes a LONG delay on initial boot of my VMs
<zero_shane> I change it afterwards - to remove all datasources I don't need - but it takes a long time before my initial VM comes up and is configured
<zero_shane> are there any pointers on how to force cloud-init configuration on *initial* package install to wire the datasoures - in an automated way - since this is not a "human involved" install process (eg the configure step in the packaging)
<smoser> zero_shane, you can preseed that.
<smoser> the key is cloud-init/datasources
<smoser> its preseed like any other preseed is done.
<zero_shane> smoser: thx for reply - in conf. call right now - will look at that in a short bit
<zero_shane> smoser - I'm not seeing how I'd pass in a preseed to the vmbuilder build process?  I see the default /etc/vmbuilder/*/*.tmpl template files ....
<zero_shane> and can make changes there ... also (re)read through the docu I can find (https://help.ubuntu.com/12.04/serverguide/jeos-and-vmbuilder.html) but haven't found specifics on overrideing anything outside of those existing template files
<zero_shane> I suppose I could *NOT* define cloud-init as a build-time package insall, and do that as a post-build task, then override the config, thus avoiding the 15 minute delay waiting on initial boot
<zero_shane> but it'd be nice to figure out how to override this via vmbuilder directly
#cloud-init 2016-11-21
<johnny_bravo> hey guys, how can i set the hostname with cloud-init on ubuntu 16.04 so that it sends itâs new name to the dhcp server
<johnny_bravo> this âsend host-name = gethostname();â in dhclient.conf gets the old name
<johnny_bravo> once i reboot the VM the hostname is sent correctly
<johnny_bravo> is there a way to do this?
<johnny_bravo> im using qemu-kvm and the user-data is on a cdrom iso image
<rharper> powersj: you poked on cloud-init snappy and package install, I noticed that if you ssh in , the config-final modules aren't done yet; so initially snap list didn't show those py-numpy and such, but after the module completes they are present
<powersj> rharper: interesting I'll give it another round this afternoon
<powersj> maybe I didn't wait long enough :(
<powersj> I usually wait for /var/lib/cloud/instance/boot-finished to show up
<rharper> powersj: ok
<rharper> powersj: http://paste.ubuntu.com/23512996/  ; the 'ready' state of the last snap installed corresponds pretty much to the timestamp on boot-finished ;  snap install doesn't appear to be async; but I suppose it's possible that a snap install has completed, but it doesn't appear in snap list until it's ready and that may be async from cloud-init finish
<powersj> rharper: I'm starting to wonder if all my issues are with the fact that I'm using LXD as a backend
<powersj> is that what you are using? or VMs?
<rharper> powersj: vms
<rharper> it's possible there's an issue with snappy under lxd, you may need to update your host (lxd updates)
<rharper> and/or ensure we launch with a profile that supports snappy under lxd (nested security support is needed)
<rharper> that may be kernel (4.8 or updates to 4.4) etc
<rharper> I suggest asking in #lxd for the right levels/config options needed
#cloud-init 2016-11-22
<smoser> Odd_Bloke, fyi, i was looking at https://code.launchpad.net/~daniel-thewatkins/cloud-init/lp1460715
<smoser> i think  you created a git repository there. which is kind of bzr-like behavior
<smoser> (versus just pushing a repo with multiple branches to
<smoser>  git.launchpad.net/~daniel-thewatkins/cloud-init
<smoser> oh. funny.
<smoser> it *is* a bzr branch.
<rharper> it was pushed before cloud-init switched to git IIUC
<smoser> Odd_Bloke, are you around ?
<smoser> your branch, the tests you added i dont think correctly unpatch subp
<smoser> powersj, around ?
<smoser> you interested in reaching out to tore on https://code.launchpad.net/~tore.lonoy/cloud-init/+git/cloud-init/+merge/310051 and helping with unit tests ?
<powersj> smoser: sure
<powersj> I'll take a look
<powersj> smoser: you gone yet?
<powersj> smoser: filed LP: #1644043 to fix master's unit tests
<powersj> has a patch :)
#cloud-init 2016-11-23
<smoser> powersj, :-(. odd that i dont see that. i think because you have (or build systemk has) a eth0
<smoser> and i do not.
<smoser> powersj, fixing and pushing and walking away. thank youo.
<powersj> smoser: thank you
<Dominique> Hi, is there a 0.7.8 cloud-init for EL7? The one included with EL7 seems a bit old and has some issues.
<Odd_Bloke> smoser: Which tests?
<harlowja> Dominique i don't think so, but u can just build your own
<harlowja> RH i guess will someday get around to it :-P
<harlowja> someone from this community perhaps should take over building of that package for them
<harlowja> so that its not always behind
<harlowja> 0.7.5 seems to be the latest @ https://apps.fedoraproject.org/packages/cloud-init
<kalx2> Hi all. Is there a way to disable ssh key injection in cloud-init? This happens automatically on the Ubuntu 14.04 AMI in AWS. Would like to disable it without disabling SSH entirely
#cloud-init 2016-11-24
<powersj> kalx2: are you trying to prevent the image from generating SSH keys automatically?
<kalx2> powersj: I still want it to generate the local RSA/DSA/ECDSA/etc keys. I don't want it to take the key provided by EC2 Metdata and add it to the authorized_keys file
<kalx2> powersj: Basically, looking to have a hardcoded key in authorized_keys, and just have cloud-init do the RSA/DSA/ECDSA/etc key generation and not touch the authorized_keys file.
<kalx2> I assume the SSH module is what's grabbing the key from EC2 metadata and adding it to authorized_keys. However, I can't find any config related to this or to disable it
<powersj> there is a disable_ec2_metadata option, but I'm not sure what other impacts that would have.
<kalx2> powersj: from the logs, it looks like it grabs a lot of other info from the ec2 datasource like details on storage devices to auto-mount, etc.
<powersj> ah that does make sense, probably not something you want to wholesale blow away :)
<kalx2> powersj: so not so desirable here., yes : ) This disable SSH key seems like it would be a somewhat common usecase, will try to search more
<kalx2> thx for your help though
<kalx2> very odd. I don't find anything and the SSH module doc doesn't mention anything about performing this function. Best choice seems to disable the ssh module and take care of the host key generation manually via script
#cloud-init 2016-11-25
<vans163> Anyone know the correct way to use cloud-init with qemu?
<vans163> If I pass the .iso I created to qemu as -drive then remove it.  I fail to boot up the second time with errors like this  blk_update_request: I/O error, dev fd0, sector 0
<vans163> reading google seems to indicate maybe the cloud image expects that drive to be there
<powersj> vans163: I have been using something similar to: https://ubuntu-smoser.blogspot.cl/2013/02/using-ubuntu-cloud-images-without-cloud.html
<vans163> powersj: doing -hda/hdb like that does not allow to pass virtio
<vans163> im gonna try using -cdrom i guess
<vans163> or i guess will have to keep the cloud-init.iso always mounted/attached
<vans163> everytime that vm boots
<powersj> Well so here is what I end up using, which you can see the virtio cmd: https://gist.github.com/powersj/987069c407c5115acfd72a09ce1b9be9
<vans163> powersj: wil investigate that fully thanks, but that last command to run qemu looks identical to mine. My trouble starts is on the second boot I leave out -drive file=$SEED_FILE,format=raw,if=virtio
<powersj> very specific for my use-case of going through cloud configs quickly, but you can play with it
<powersj> ah
<vans163> do you leave out this line "-drive file=$SEED_FILE,format=raw,if=virtio" on second boot and further?
<powersj> ah second boot? let me try
<vans163> yea like make the seed.iso, attach it, boot, works. Shutdown QEMu fully, start qemu again the same main disk, BUT leave out "-drive file=$SEED_FILE,format=raw,if=virtio". It does not boot, gives those errors "blk_update_request: I/O error, dev fd0, sector 0"   Do you see these at all?
<vans163> By default I get 3 of those using the ubuntu xenial 16.04 cloud image
<vans163> but it does not affect anything
<vans163> without the $SEED_FILE theres like 10-12 of them, and Ubuntu does not boot
<powersj> vans163: agreed, seeing same thing
<vans163> powersj: fails to boot fully too? Gets stuck after those blk_update_request errors?
<powersj> I wonder if it is stuck waiting for userdata or something like that
<vans163> ah
<powersj> vans163: yes just sits there
<powersj> when I added the seed back in it boots as expected like you mentioned
<vans163> i was going crazy at first, i thought my filesystem was corrupting the image and im using ZFS x.x
<powersj> ha! glad I could at least confirm it for you
<vans163> yea thanks. so would this be a bug?
<vans163> im not sure if its local to xenial ubuntu or if itl happen on other images
<powersj> I'm not sure, I don't know enough if there is some setting you would need to disable or mark on the cloud-img I'm using to allow 2nd boot
<vans163> i see. I think il just run with always having the seed attached for now
#cloud-init 2017-11-20
<GreatSnoopy> hello. Can anyone help me debug a cloud init issue ? I am trying to get cloud-init to work with centos on Azure. The official cloudinit in centos (0.7.9) hangs for a while and does nothing, eventually. So i installed the very last and greatest cloudinit from sources, however now it gives me "util.py[WARNING]: No instance datasource found! Likely bad things to come!"
<GreatSnoopy> although i have a datasource_list: [Azure] in a config file in conf.d
<GreatSnoopy> what changed in the newer releases of cloudinit ?
<GreatSnoopy> it seems I do have a /usr/lib/python2.7/site-packages/cloudinit/sources containing a DataSourceAzure.py
<morgana2313> Hello. Is there an easy way to use macro's/variables that contain the cloud-instance ip-adress and hostname in a write_files module?
<blackboxsw> mornin folks.  GreatSnoopy have you tried our daily copr builds? https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/
<GreatSnoopy> actually yes
<GreatSnoopy> that was before trying the source
<GreatSnoopy> the same behavior applies with those builds
<GreatSnoopy> I tried the source version as a last resort
<blackboxsw> that is version 17.1.x off of our master.  The no datasource found is intriguing. I'll look over our Azure changes to see if there's something indicative of a prob there.  GreatSnoopy, so I'd be curious to see a paste of your /var/log/cloud-init.log. It should report that it attempts to run Azure datasource and complains if something is amiss
<GreatSnoopy> just a moment
<GreatSnoopy> http://pastebin.centos.org/435761/15111921/
<GreatSnoopy> for the record, the above is produced by running cloud-init --debug init
<blackboxsw> GreatSnoopy: +1 on that. ok so you upgraded from 0.7.5 -> 0.7.9 originally and saw this problem too?
<GreatSnoopy> blackboxsw: so the succession of events is the following:
<GreatSnoopy> first, used 0.7.9 that is availabpe in epel. That hangs for like 2 minutes, does nothing in the end when ran manually. And if the machine is rebooted, it just hangs there indefinitely and with ssh stopped is basically a bricked VM
<GreatSnoopy> so i went trying newer cloud-init
<GreatSnoopy> first used that copr repo
<GreatSnoopy> which gave me the behavior of finishing fast but also doing nothing and complaining about unavailable data sources
<GreatSnoopy> as in the log
<GreatSnoopy> then i tried the actual sources
<GreatSnoopy> installed dependencies and the cloud-init distribution with pip install .
<GreatSnoopy> (and -r requirements.txt)
<GreatSnoopy> but this solved nothing
<GreatSnoopy> so basically, the copr build and the raw, unpackaged build installed by hand (mis)behave in the same way
<GreatSnoopy> but differently than 0.7.9 :)
<GreatSnoopy> which is the current epel version available
<blackboxsw> yeah the copr report cloud-init-dev is actually only 4 hours old. Our CI builds after every commit I think.    sorry for the thrashing you've experienced here. Would you be able to "cloud-init collect-logs"  on the commandline and attach it to a new cloud-init bug @ https://bugs.launchpad.net/cloud-init/+filebug
<blackboxsw> I find this peculiar in your logs https://bugs.launchpad.net/cloud-init/+filebug
<blackboxsw> oops I mean this: 2017-11-20 15:34:09,308 - __init__.py[DEBUG]: Searching for network data source in: []
<blackboxsw> I'd have expected to see Azure in that empty list
<GreatSnoopy> the config I added is the following, maybe I am using it wrong:
<blackboxsw> azure datasource though in cloud-init version 17.1 looks to be only run in init-local timeframe
<GreatSnoopy> http://pastebin.centos.org/435771/15111932/
<blackboxsw> and in your logs I only see the stage labelled 'init' which is actually cloud-init's 'init-network' stage
<blackboxsw> which means Azure (as init-local stage Datasource) doesn't match as a network data source.
<blackboxsw> I *think8
<blackboxsw> I *think*... though haven't had my coffee yet to confirm
<GreatSnoopy> because i ran it manually
<GreatSnoopy> maybe ?
<GreatSnoopy> if i let the machine boot, it will hang, actually
<GreatSnoopy> s/boot/reboot
<GreatSnoopy> cloud-init collect-logs seems to do nothing, /var/log/cloud-init* remain empty
<blackboxsw> GreatSnoopy,  there are semaphore files blocking cloud-init fresh re-runs. If you are willing to let cloud-init re-run on your system in entirety, you could 'sudo rm -rf /var/log/cloud-init* /var/lib/cloud; sudo reboot'
<blackboxsw> that gives cloud-init the perception that it has never run before
<GreatSnoopy> i can do that, but that would brick the machine
<GreatSnoopy> var/lib/cloud/instances/ is actually empty
<GreatSnoopy> i can ofc delete every trace of them, but shouldn't it be able to run in the cli otherwise ?
<GreatSnoopy> i mean so that i do not need to reboot
<GreatSnoopy> and lose control of the machine ?
<GreatSnoopy> [root@democentfihn ~]# rm -Rf /var/log/cloud-init* /var/lib/cloud [root@democentfihn ~]# reboot
<blackboxsw> GreatSnoopy: so Azure datasource is init-local only, so running on the commandline you'd need to try "cloud-init init --local" since Azure is only run during init-local.  I'm thinking though this might be that hang you were talking about though
<blackboxsw> ok, I'll try firing up a Centos image on Azure today to see if I can reproduce the prob
<GreatSnoopy> well, i rebooted the machine, and quite as expected ssh now is stopped so i cannot log any more
<GreatSnoopy> dunno if this is due to cloudinit or waagent
<smoser> for s in cloud-init-local cloud-init cloud-config cloud-final; do echo == $s.service ==; systemctl restart $s.service || break; done
<smoser> that would run everything in order that they would run.
<GreatSnoopy> i will pick the next VM and try again, but basically I am back to square one when using 0.7.9: basically i boot the machine but although in the boot diagnostic i can see the familiar cloudinit table with network configuration, the rest of the config does not run and ssh does not get to be started - basically i cannot log into the machine
<blackboxsw> GreatSnoopy: check this out. https://bugs.launchpad.net/cloud-init/+bug/1717611 this change did land in azure which might be affecting you
<ubot5> Launchpad bug 1717611 in cloud-init "Azure: Azure datasource needs to wait longer for SSH pubkey to be dropped by waagent" [Medium,Fix released]
<blackboxsw> GreatSnoopy: do you have a pointer to a public centos image we can use on Azure?
<blackboxsw> or is this custom
<GreatSnoopy> I am not sure I understand what you are asking from me :) It is supposed to be a custom image that we are building to have cloudinit pre-enabled so that we can then provision other machines
<GreatSnoopy> but we are not even there yet
<GreatSnoopy> now we have a vm created from a regular Azure Centos baze
<GreatSnoopy> which I think is provided by OpenLogic
<blackboxsw> gotcha, I was just wondering if you were using stock CentOS in azure or a custom image. I'll spin up an instance on azure to checkout
<GreatSnoopy> for now I'm just trying to get to nonstandard but i cannot pass the standard phase :))
<GreatSnoopy> ideally this should work with 0.7.9 from epel, but if needed I can make my images with a newer cloudinit as long as it works
<blackboxsw> I'm with you, will ping you when I have progress on this. if you could file a bug in launchpad that'd help us reference progress on this.
<blackboxsw> https://bugs.launchpad.net/cloud-init/+filebug   your simple paste of cloud.cfg.d & cloud-init.log with the steps to reproduce the hang would be sufficient
<GreatSnoopy> Just to simplify things, can we start by investigating the 0.7.9 issue ? Point being made is that ideally i should get it working with what is provided in the more mainstream OS repos
<GreatSnoopy> also, can you validate the soundness of the config I gave to cloudinit ?
<GreatSnoopy> i mean this http://pastebin.centos.org/435771/15111932/
<GreatSnoopy> blackboxsw: or let's ask the other way around: what would be the preferred/recommended way to install cloud-init on centos in azure?
<blackboxsw> GreatSnoopy: I know cloud-init in certain clouds is already baked into centos images. Trying to confirm on azure now.
<GreatSnoopy> unfortunately, not the case in azure - at least up to centos 7.3
<GreatSnoopy> they only have cloudinit for ubuntu and coreos
<blackboxsw> if a given cloud doesn't have an image that contains cloud-init. I'd install it then shut it down and take a snapshot or make an image of it in that cloud so I could reference that un subsequent VM creations
<GreatSnoopy> that is exactly what I am trying to do :)
<GreatSnoopy> hence the question, which is the recommended way to get cloudinit on that machine : the package in epel, slightly older - 0.7.9 or should i go and install the latest from source ?
<blackboxsw> and I wouldn't want to run cloud-init on that image before I snapshotted it (or I'd remove /var/log/cloud-init* /var/lib/cloud before snapshotting)(
<GreatSnoopy> that's understood, i always delete those items
<blackboxsw> GreatSnoopy: probably easiest for your to try to use 0.7.9 as it's in epel. But, if there are bugs with 0.7.9 the only fix we'd propse would land in upstream 17.X
<blackboxsw> we don't backport fixes to centos epel (and it's up to centos when they want to pull in latest cloud-init)
<blackboxsw> and it sounds like 0.7.9 and 17.1 are both causing probs for you. let's see what's up with that. I do think you might be hitting that infinite wait on ssh keys though  on 0.7.9
<blackboxsw> on systems like that, I'd expect you'd see"waiting for SSH public key files" in the logs. if you ever got there.
<blackboxsw> GreatSnoopy: looks like that ssh key times out at 900 seconds
<blackboxsw> so that's a 15 minute wait
 * blackboxsw had to hit the calculator
<GreatSnoopy> what ssh keys does it expect and who is supposed to create those ? because although the source is a standard source image, i spin the instances via terraform
<GreatSnoopy> and the only thing I pass to the instance is custom data with the cloudinit yaml
<blackboxsw> GreatSnoopy: it looks like DatasourceAzure.py is waiting for ssh key from azure fabric to configure the instance (as the UI/api provides an ssh key that is used to contact the instance)
<GreatSnoopy> that should not be normal behavior :
<GreatSnoopy> because even in the GUI i can create a vm that has only password - no key
<GreatSnoopy> or is it a different one ? system only ?
<GreatSnoopy> that is provided no matter what the user actually provides ?
<smoser> GreatSnoopy: it only waits for files to appear which are listed on the cdrom in the metadata there.
<smoser> so... password only wont have any ssh keys listed in the metadata so it wont wait for anything
<GreatSnoopy> can i manually retrieve the file so that i can check the data received before i reboot the machine ?
<GreatSnoopy> where does that file "land" initially ?
<smoser> walinux-agent would put it into /var/lib/waagent
<smoser> for *.crt files in that directory
<GreatSnoopy> one more question : waagent should be disabled so that its cloud-init the one that starts it, or should be left enabled ?
<GreatSnoopy> cloudinit's relationship with waagent seems a little bit of chicken and egg dillema
<smoser> GreatSnoopy: cloud-init no longer needs walinux-agent.
<smoser> and so its default behavior is suggested.
<smoser> which is 'agent_command' of '__builtin__'
<GreatSnoopy> interesting, but won't azure "see" the instance as failed if the cloud fabric cannot communicate with the agent ? or does cloudinit also create a replacement for that ?
<GreatSnoopy> in my previous experience, not having waagent running results in the instance being marked as failed after reboot
<GreatSnoopy> because it cannot communicate with the agent
<smoser> GreatSnoopy: i' not sure when it went in, but yeah, you dont need walinux-agent anymore.
<smoser> yeah. and newer ubuntu instances do not use it... let me check fof sure
<GreatSnoopy> filed this https://bugs.launchpad.net/cloud-init/+bug/1733403
<ubot5> Launchpad bug 1733403 in cloud-init "cloud-init does not work reliably in Azure with Centos" [Undecided,New]
<blackboxsw> thanks for this bug GreatSnoopy and the good context.
<GreatSnoopy> I will come back tomorrow...for now I am out of VM's to brick :D
<GreatSnoopy> you know the most stupid part.... we managed to get this step BEFORE for both centos7 and debian
<GreatSnoopy> why this is not working any more I don't know
<blackboxsw> GreatSnoopy: thx again, one thing I wonder is your datasource config represents agent_command :['systemctl', 'start', 'waagent' ]   ...   I wonder if it'd work with   ['service', 'walinuxagent', 'start'] instead
<blackboxsw> the datasource itself checks to see if agent_command  == ['service', 'walinuxagent', 'start'] and grabs content from metadata in that case.
<GreatSnoopy> lets see, although that would be, well... ugly :)
<blackboxsw> yeah, think I misread the code. I think it checks to see if agent_command == '__builtin__' and then tries to get to metadata to pull in any ssh keys etc.
<blackboxsw> I'm referencing docs at https://cloudinit.readthedocs.io/en/latest/topics/datasources/azure.html as well
<blackboxsw> ... as I don't use azure too often :/
<GreatSnoopy> for s in cloud-init-local cloud-init cloud-config cloud-final; do echo == $s.service ==; systemctl restart $s.service || break; done == cloud-init-local.service == == cloud-init.service == Job for cloud-init.service failed because the control process exited with error code. See "systemctl status cloud-init.service" and "journalctl -xe" for details.
<GreatSnoopy> 2017-11-20 19:40:48,245 - util.py[DEBUG]: Running command ['blkid', '-tTYPE=udf', '-odevice'] with allowed return codes [0, 2] (shell=False, capture=True) 2017-11-20 19:40:48,378 - handlers.py[DEBUG]: finish: init-network/search-AzureNet: SUCCESS: no network data found from DataSourceAzureNet 2017-11-20 19:40:48,378 - util.py[WARNING]: No instance datasource found! Likely bad things to come! 2017-11-20 19:40:48,378 - util.p
<GreatSnoopy> i mean http://pastebin.centos.org/435866/
<GreatSnoopy> in any case, i will be back tomorrow, and this time I will also rerun the whole process again (including the vm creation)
<blackboxsw> good deal thx GreatSnoopy
<GreatSnoopy> currently that was not made by me, and i will have to check if something got left out, just to be sure
<GreatSnoopy> thanks a bunch, guys. See you tomorrow, have a nice day !
<blackboxsw> you too
<smoser> blackboxsw: i'm grabbing merge of fix-ec2-fallback-nic now
<blackboxsw> sweet, I'm on the #jinja2 stuff. no other changes needed
<blackboxsw> ?
<blackboxsw> smoser: with that fix-ec2-fallback-nic branch landed, shall we do a minor SRU?
<smoser> we could.
<blackboxsw> we have 2 fixes for ec2 that'd be helpful.
<blackboxsw> and it'd make the SRU simple
<smoser> are you thinking cherry-pick ?
<smoser> blackboxsw: ?
<smoser> https://hastebin.com/ugoyozasuz
<smoser> that is trunk -> bionic riht now
<smoser> which we absolutely should do
<blackboxsw> smoser: I was thinking master !cherry-pick
<blackboxsw> forgot about the others
<blackboxsw> but the thing about doing something painful, is to repeat the process often :)
<blackboxsw> it can only get better with practice.
<smoser> blackboxsw: i'm fine with SRU
<smoser> but we need to do bionic first
<smoser> and that can happen "right now" if you want to propose, i'll upload
<blackboxsw> ok will do smoser
<smoser> blackboxsw: i'm pusing the integration test one
<smoser> so wait on that ?
<smoser> i'm tox && git push on it
<smoser> pushed
<smoser> 7624348712b4502f0085d30c05b34dce3f2ceeae
<blackboxsw> fire awaay
<smoser> thats in now
<blackboxsw> ok grabbing
<blackboxsw> smoser: this is what I see http://pastebin.ubuntu.com/
<blackboxsw> should the AliYun have been "bionic"
<blackboxsw> ?
<blackboxsw> instead of UNRELEASED?
<smoser> blackboxsw: ah.
<smoser> just squahs it into your new commit
<smoser> it was committed as UNRELEASED as it was not released.
<smoser> adn then the next release would just pick it up
<smoser> kind of queueing things
<smoser> so you  just drop that old changelog entry and pull the AliYum comment up
<smoser> make sense ?
<blackboxsw> yeah squash, gotcha
<smoser> put the debian/ ones at the top.
<smoser> no real reason
<smoser> just how i've done it before
<smoser> https://hastebin.com/sezipunibi
<smoser> thos will be your top two entries
<smoser> with your name instead of mine
<smoser> i'll get that in and uploaded later tonight if you MP it
<smoser> but have to run for now.
<blackboxsw> smoser: https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/333998
<blackboxsw> moving on to artful,zesty,xenial
<blackboxsw> smoser: forgot with bionic(devel) should I remove bug #'s which don't affect ubuntu? or just on SRU series (artful, zesty, xenial)
<blackboxsw> repushed with my name removed from changelog
<blackboxsw> https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/333999
<smoser> blackboxsw: ah. i just leave them in for ubuntu-devel
<blackboxsw> ok thx. seeing merge conflict on artful for some reason
<smoser> so re-push with the bug numbers on devel
<blackboxsw> smoser: re-push contains all bug #'s only removed my name in brackets
<smoser> k
<smoser> blackboxsw: when you do artful, zesty, xenial
<smoser> cherry-pick the templates fix
<smoser> and i'mo going to move your debian/cloud-int.templates to before the upstrema snapshot comment
<blackboxsw> +1
<blackboxsw> I'm in hangout for a quick resolution
<blackboxsw> cherry pick is good. just not sure about why I'm seeing merge conflic
<blackboxsw> cherry pick is good. just not sure about why I'm seeing merge conflict
<smoser> hm..
<smoser> ckonstanski
<smoser> i'll fix that
<smoser> he has username in changelog
<smoser> or, just leave it
<smoser> lets just levae it
<smoser> but we should lint those sorts of things on merge proposal
<smoser> blackboxsw: i'm not in a hurry to do the others tonight.
<smoser> just uploaded bionic
<blackboxsw> kthx. sounds good
<blackboxsw> have a good one
#cloud-init 2017-11-21
<smoser> blackboxsw: i found the artful issue.
<smoser> smoser error
<smoser> i had made one fix to yours, and i think i must have accidently/ignorantly un-done the merge commit
<blackboxsw> smoser: https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/334051
<smoser> when i added 1a252d46cc6739bc501fc06bde0b5e25e803266b
<blackboxsw> ahh oops
<blackboxsw> looking
<blackboxsw> first rule of cloud-init-club is: "smoser doesn't make errors"
<blackboxsw> wait that's 2nd rule. First rule is "there is no cloud-init-club"
<blackboxsw> smoser: by "undone the merge commit" you mean like a rebase -i HEAD~2   and a 'fixup' which squashed the merge-commit or something?
<smoser> eithe rthat ir o thik i might have just reset --hard HEA^
<smoser> i'im not sure.
<smoser> upstream now has ubuntu/17.1-27-geb292c18-0ubuntu1_17.10.1 correctly
<smoser> and merge-base shows the same for artful and xenial and zesty
<smoser> blackboxsw: you did not strip the bug numbers from the changelog
<smoser> i guess we have 2 options.
<smoser> a.) megabug route
<smoser> b.) individual bugs
<blackboxsw> oops quick touch one min
<blackboxsw> hrm
<blackboxsw> let's do megabug, I like one bug to point people at
<smoser> either way, we were dropping the non-ubuntu bugs (sles ones are in there)
<blackboxsw> I'll create a bug
<smoser> and gentoo
<blackboxsw> +12
<blackboxsw> +2
<yeazelm> I need a little help with the CI for my PR
<yeazelm> https://code.launchpad.net/~yeazelm/cloud-init/+git/cloud-init/+merge/331897 - I'm not sure what to do about  FAILED: Ubuntu LTS: Integration
<blackboxsw> https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1733653 filed
<ubot5> Launchpad bug 1733653 in cloud-init (Ubuntu) "sru cloud-init (17.1-27-geb292c18) update to (17.1-42-gbbe91cdc)" [Undecided,New]
<blackboxsw> yeazelm: looking
<blackboxsw> looks like may have had a slave failure installing the testable deb http://pastebin.ubuntu.com/26013415/
<blackboxsw> I'll re-run your tests yeazelm
<blackboxsw> if that's not it I'll look locally at the prob
<yeazelm> awesome thanks!
<blackboxsw> np yeazelm thank you
<blackboxsw> smoser: https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/334063 for you
<blackboxsw> xenial with just the SRU bug and debian packaging bug
<blackboxsw> smoser, and I pulled in mccabe's  should I leave that out
<blackboxsw> ?
<smoser> blackboxsw:  you can just leave it out
<blackboxsw> smoser: will leave it out to match bionic
<smoser> new-upstream-snapshot HEAD^
<smoser> er...
<smoser> you get the point
<smoser> youcan give it anything just give it the commit that went into bionic. then easier.
<blackboxsw> smoser: good deal. repushed zesty https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/334065
<blackboxsw> using SRU megabug
<blackboxsw> and fixing names, dropping bugs
<blackboxsw> xenial up https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/334066
<blackboxsw> I thought I saw a CI failure fly by on zesty.
<blackboxsw> checking
<blackboxsw> tox locally wfm
<smoser> blackboxsw: i was wrong about not collecting consoel logs
<smoser> we do get them
<smoser> https://jenkins.ubuntu.com/server/view/cloud-init/job/cloud-init-integration-x/377/artifact/cloud-init/results/lxd/xenial/modules/apt_configure_conf/
<smoser> i think that i saw them fail in -proposed, which is missing my cleanup commit that added those.
<smoser> blackboxsw: https://jenkins.ubuntu.com/server/job/cloud-init-ci/530/console
<smoser> oh shoot.
<blackboxsw> smoser: so cherry-pick needed me to touch it?
<blackboxsw> to avoid patch fuzz?
<blackboxsw>         Reversed (or previously applied) patch detected!  Skipping patch.
<smoser> yeah
<smoser> hm..
<smoser> so ... what happened here.
<blackboxsw> ahh oops
<smoser> hm..
<smoser> you used 'git cherry-pick 1110f30ee9b4703a3c66fc678c0df3f0862cc7eb' ?
<blackboxsw> ./debian/cherry-pick 1110f30ee9b4703a3c66fc678c0df3f0862cc7eb
<smoser> or './debian/cherry-pick'
<smoser> yeah, you want he former
<smoser> it is confusing. sorry.
<blackboxsw> bummer.
<blackboxsw> ok recreating
<smoser> ./debian/cherry-pick takes an upstream commit and applies it via patches/debian/cpick-XXX-XXX to an ubuntu branch
<blackboxsw> my bad
<smoser> but we manage the ubuntu branches directly with git
<smoser> that difference make sense ?
<smoser> and when you do 'cherry-pick 1110f30ee9b4' use '-x'
<smoser> (which i just nwo learned about). so we have a record in the commit, even though its probably obvious
<blackboxsw> smoser: so why do we want cpick files then?
<blackboxsw> only when we don't manage the branches directly?
<blackboxsw> smoser: also when I git cherry-pick, it doesn't add anything to the changelog.
<blackboxsw> so I should add that back in right?
<smoser> blackboxsw: ...writing something, and then i'll update https://gist.github.com/smoser/6391b854e6a80475aac473bba4ef0310#file-ubuntu-release-process-md
<smoser> https://hackmd.io/IwYwRgzALAZjDsBaEwCsBORUAMA2AHIvsPIWOsNgCb4Cmp+E+QA=
<blackboxsw> smoser I'm working on adding test/skip references for trello for this SRU
<blackboxsw> https://hackmd.io/BwUwrAhgRg7ADFAtMCBOJAWCDHQGwyIBmcAJqiBnhiAExRRA?both
 * smoser feels like he should hit the 'buy us coffee' button for  hackio.md
<blackboxsw> yeah I was thinking the same thing. $5 latte here you come (Thanksgiving and all)
<blackboxsw> âthanks hackmd
<smoser> blackboxsw: ^ is updated now. should hopefully make some sense.
<smoser> ugh. going to figure out why https://jenkins.ubuntu.com/server/job/cloud-init-ci/531/console
<blackboxsw> thx smoser
<powersj> I have a hunch it is because the version in the container is newer than the package you are trying to install
<blackboxsw> thanks for the gist update smoser
<shewless> hello there. does anyone know if I can use the phone_home module to call multiple urls?
<blackboxsw> xenial, zesty and artful sru review pushed https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/334072, https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/334071 and https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/334070
<smoser> blackboxsw: grabbing
<blackboxsw> shewless: doesn't look like the module supports it.  I wonder if you could either provide multiple mime type /cloud-config text/cloud-config parts. per http://cloudinit.readthedocs.io/en/latest/topics/format.html?highlight=include
<blackboxsw> but I'm thinking the merge of all those cloud-init parts would actually just give you whatever cc_phone_home section was defined last..
<shewless> blackboxsw: thanks for confirming
<shewless> maybe you can help a different way
<blackboxsw> hmm shewless shortly we'll have the ability to run "cloud-init status --wait" in scripts and your script your provide could do whatever you want
<shewless> I want to run a command, like in "runcmd" but I want to do it in the cloud_final_modules stage.. any hints?
<blackboxsw> shewless: per https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/333513 (the status subcommand I referenced)
<shewless> blackboxsw: looks cool.. I think I was trying to do too much in the phone_home module perhaps
<blackboxsw> shewless: http://cloudinit.readthedocs.io/en/latest/topics/boot.html?highlight=stages
<blackboxsw> runcmd is run during final cloud-init stage
<blackboxsw> if that helps. it's when every other module has run
<blackboxsw> *has finished running  rather
<blackboxsw> same with any provided user-scripts you provide
<blackboxsw> it's run during final cloud-init stage
<shewless> blackboxsw: thanks!
<blackboxsw> np (and another link to continue spamming you :) ) http://cloudinit.readthedocs.io/en/latest/topics/format.html#user-data-script
<shewless> blackboxsw: thank link confused me. The user-data can be code?
<blackboxsw> shewless: yeah you can actually provide any script to cloud init that starts with #!
<blackboxsw> and cloud-init writes that script to a file that is run in final stages
<blackboxsw> write_files: is another alternative that lets you be more explicit about where you want your script written (and how you want to call it using runcmd:)
<blackboxsw> I'm probably providing too many alternatives. sorry to confuse
<shewless> blackboxsw: mind explodes.. but thanks I'm good for now
<blackboxsw> hehe, turkey will help if you are US based
<shewless> nope.. had my thanksgiving in Oct
<shewless> maybe a good black friday deal will help :P
<smoser> shewless: feel free to buy me things too. i like stuff.
<shewless> don't we all
<smoser> blackboxsw or powersj https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/334074
<powersj> smoser: thx +1'ed
<smoser> ok. i'll wait on ci to happy on nit and then pull
<smoser> blackboxsw: https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/334065 . in your changelog, i think i'd drop the 'cherry-pick 1110f30:'
<smoser> since its not an upstream change that we're cherry-picking.
<smoser> i can fix that here.
<blackboxsw> smoser: ahh ok gotcha smoser
<blackboxsw> smoser: tests/cloud_tests/setup_image.py:63:9: E123 closing bracket does not match indentation of opening bracket's line
<blackboxsw> on your test branch
<smoser> ah. thanks.
<smoser> blackboxsw: uploaded
<smoser> we  need a sru template for https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1728186
<ubot5> Launchpad bug 1728186 in cloud-init (Ubuntu Artful) "AliYun datasource has wrong case in config" [Medium,In progress]
<smoser> and update to the megabug bug 1733653
<ubot5> bug 1733653 in cloud-init (Ubuntu Artful) "sru cloud-init (17.1-27-geb292c18) update to (17.1-42-gbbe91cdc)" [Medium,In progress] https://launchpad.net/bugs/1733653
<smoser> bugger.
<smoser> that really sucks
<smoser> so the hold and apt-get install works if the package is older than the one available
<smoser> but if it is newer, then it fails like in https://jenkins.ubuntu.com/server/job/cloud-init-ci/536/console
<blackboxsw> Smoser will do
<blackboxsw> Also I'm thinking we need to automatically add nominations for the Ubuntu related bugs for the xenial,zesty and artful
<smoser> blackboxsw: well, we can do that, yes. that ws the extra tracking that we've talked aobut.
<blackboxsw> yeah I'll just take care of it later (when the SRU lands)
<blackboxsw> as an accounting step
<smoser> blackboxsw: unless you object shortly...
<smoser>  https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/334074
<smoser> gonna pull that.
<blackboxsw> nice smoser was that a recommendation from ubuntu-devel?
<blackboxsw> testing the cmdline now
<blackboxsw> though already merged
<blackboxsw> smoser: sru template written https://bugs.launchpad.net/ubuntu/artful/+source/cloud-init/+bug/1733653
<ubot5> Launchpad bug 1733653 in cloud-init (Ubuntu Artful) "sru cloud-init (17.1-27-geb292c18) update to (17.1-42-gbbe91cdc)" [Medium,In progress]
#cloud-init 2017-11-22
<powersj> smoser: https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/334141
<powersj> enables bionic for integration tests
<smoser> powersj: gracias.
<powersj> looking at the image modification issue you brought up now
<powersj> We use qemu-image create with the original qcow as a backing file
<powersj> heh it does indeed look like we setup the image (i.e. install new deb) before creating the copy doh!
<smoser> powersj: so i had thought someon this.
<powersj> all ears :)
<smoser> when you create an image it should create a qcow2 backed by the /srv/ file
<smoser> then, the snapshot can qcow2 off of that.. the issue wth that is that ew kind of need snapshot to be persistent
<smoser> so maybe when we snapshot it should "realize" the image.. rather than snapshost-qcow2 -> image->qcow2 -> /srv
<smoser> as the image thing can be deleted.
<powersj> Snapshot today calls platforms/nocloudkvm.py which creates a qcow backed by the /srv/ file and you are saying what needs to change is image, which should create a a qcow
 * powersj goes and gets food
<smoser> https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+ref/fix/cii-kvmimage-preserve-original
<powersj> smoser: looking
<smoser> that was what i had started a while ago
<smoser> powersj: rad the commit message and the 'also have to modify'...
<smoser> i think it'd be sufficient if we just di da copy (shutil.copy)
<smoser> that way the snapshto woudl be backed by the imagein /srv/
<powersj> smoser: that looks good
<powersj> going to propose it?
<powersj> oh yeah gotta change snapshot
<smoser> powersj: i can see if i can do that quick
<smoser> (this was a while ago... so my brain wsant too fresh)
<smoser> ug.
<smoser> htis is why i gave up. it gets ugly there.
<smoser> powersj: https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/334147
<smoser> i'm running tox here... it did run a quick test of nocloud kvm for me.
<smoser> but not throughogh testing
<powersj> smoser: ok I'll play with it and let you know by eod
<powersj> thanks!
<smoser> well, k'm about to eod.
<powersj> really appreciate it!
<smoser> but yeah, give some feedback. thanks.
<smoser> powersj: merged yoru cloud-init bionic so we'll hopefully get results of that tonight.
<smoser> thanks. and with that, i do go afk
<powersj> awesome
<powersj> o/ cya
#cloud-init 2017-11-23
<icee> So, I'm usin' ubuntu.  on 16.04 cloud images i could do -smbios "type=1,serial=ds=nocloud-net;s=https://raw.githubusercontent.com/mlyle/mlyle/master/cloud-metadata/linuxtst/
<icee> to kvm and have all the magic happen
<icee> but it seems name resolution/resolv.conf is broken on aardvark at the point of cloud init
<icee> is there anything i can do?
<icee> manually starting /lib/systemd/systemd-resolved in single user makes DNS work
<icee> adjusting the image to have /etc/resolv.conf of nameserver 4.2.2.1    initially makes it all work too
<icee> i am starting to think there is not a way to have working dns in nocloud-net with the stock images
#cloud-init 2017-11-24
<smoser> icee: hm... dns should work at that point.
<smoser> i'm not aware of that bug.
<smoser> but artful transitioned over to systemd-resolve and somet hings have been less than perfect.
<icee> smoser: systemd-resolved is definitely not running, and /etc/resolv.conf is a symlink to the systemd-resolved thing that is not there
<smoser> hm.
<icee> the rescue shell opens and i'm able to open it and poke around.. and if i remove that symlink and add a 'nameserver 4.2.2.1' file to the cloud image it all works
<smoser> rescue shell ?
<smoser> hm.. i can try to reproduce.
<icee> yah, on console it tells me to hit ctrl-d to continue to boot, or enter to get a shell
<smoser> could you open a bug with what you're doing ?
<icee> so i hit enter and poke around in the quasi-single-user-mode
<icee> i opened a bug
<smoser> link ?
<icee> uhhh heh let me try and figure that out sec
<icee> https://bugs.launchpad.net/cloud-init/+bug/1734167
<ubot5> Launchpad bug 1734167 in cloud-init "DNS doesn't work in no-cloud as launched by ubuntu" [Undecided,New]
<icee> just added a command line with the entire command lines
<icee> er
<icee> comment with the entire command lines :P
<icee> sorry i'm half awake :P
<icee> so i guess it's of note that i'm bypassing initrd, too
<icee> smoser: interesting.  it's also broken without my own kernel.  But i don't get the rescue shell to troubleshoot :P
<smoser> icee: yeah. verified.
<icee> smoser: cool, thanks :D
<icee> i don't understand the rescue shell thing
<icee> where exactly it comes from, etc... it's really, really nice :P
<smoser> icee: i suspect you're missing something in kernel that is required and seeing the fallout that way.
<smoser> but, dont know.
<smoser> anyway, i put some more information there and pinged our systemd people.
<icee> well, with correct dns in the image i get a login prompt and all works :P
<icee> so there's something baffling/magical-- the default failure behavior is different :P
<smoser> well i verified fail on artful and bionic
<smoser> not tried xenial
<smoser> i'm kind of holding out hope on that
<smoser> but yeah, broken by default :-(
<icee> xenial works
<icee> i've been using it
<icee> systemd makes me cranky.  i tried to jiggle the dependencies around myself to make this work and just couldn't in 20 minutes of fooling around
<smoser> "systemd makes me cranky."
<smoser> +21
<smoser> (the 2 there was a typo, but it isnt really wrong)
<icee> haha
<icee> smoser: thanks for all your help with getting this properly reported and marshalled
<smoser> thanks for reporting
<smoser> curious why you are using artful
<smoser> (perfectly fine to do that, just most people pick the lts and thus things like this dont get seen)
<icee> smoser: so i'm doing automated testing of kernel functionality, and i'd like to do it with non-ancient userlands too
<icee> not that 17.10 is really so bleeding edge, but..
