#cloud-init 2013-11-25
<smoser> harmw, i'd be open to that.
<smoser> i suspect that you'd run accross some linux-isms.
<harmw> yea
<harmw> python-jsonpatch is missing from freebsd10 repo, but the others are there (beeing cheetah, prettytable and some more)
<harmw> there are some tiny differences though, like files from /proc
<harmw> like, if I run cloud-init it isn't capable of reading system uptime
<harmw> or showing assigned ip addresses
<harmw> and ofcourse a proper module to handle configuration is needed :)
<harmw> (distro)
<smoser> harmw, yeah. i'm sure there are things to fix. python-jsonpatch can probably be made optional, and just fail/warn if the user tries to use it.
<harlowja> cheetah is a problem that we need to work out anyway harmw 
<harlowja> unsure what is the best way forward there (cheetah and python 3 still not behaving nicely afaik)
<harmw> ah ok, well ive tried cloud-init with python27 on freebsd 10 beta3
<harmw> as far as python's concerned, it looks to be ok
<harlowja> cool
<harlowja> the other area u'll hit i think is utils.py
<harmw> probably
<harlowja> unsure how much there will need to be adjusted for freebsd
<harmw> not sure on that, I merely deployed cloud-init to see if it would run - which it did
<harmw> now all I need is some time to dig further :)
<harlowja> :)
<harlowja> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/util.py 
<harlowja> for example, http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/util.py#L1009
<harlowja> and other ones that might be mostly linux specific
<harmw> smoser: any chance this cirros patch can be included in trunk? https://bugs.launchpad.net/cirros/+bug/1190372
<harmw> (and the fix from https://bugs.launchpad.net/cirros/+bug/1132686 aswell)
<smoser> harmw, whoot. thats awesome.
<smoser> thank you.
<harmw> :)
<smoser> other than i'd bikeshed on your shell.
<smoser> but i'd just never looked into it.
<smoser> what is the content of a 'staticroutes' ?
<harmw> let me check on that
<smoser> oh. i see . it looks like it must be pairs of (<netaddr>, <router>)
<harmw> 169.254.169.254/32 10.65.0.128
<harmw> there
<harmw> thats in $staticroutes
<harmw> took a while to create a new instance, my cloud isn't that fast :)
<smoser> harmw, could you test default.script at
<smoser> http://paste.ubuntu.com/6475373/
<smoser> that should cut down on forks for parsing output (echo $.. | cut)
<harmw> ah yea
<smoser> i know i over-do things
<harmw> nice function
<smoser> but i want cirros to boot as fast as possible
<smoser> and forks *do* have a real affect on that.
<harmw> hehe
<harmw> let me check it
<harmw> jeez, downloading plain requires a login
<smoser> yeah, silly.
<smoser> here, harmw http://paste.ubuntu.com/6475388/
<smoser> that is how to avoid logging in to ubuntu pastebin
<harmw> Lease of 10.65.0.129 obtained, lease time 120
<harmw> deleting routers
<harmw> adding dns 208.67.222.222
<harmw> adding net 169.254.169.254/32 with router 10.65.0.128
<harmw> looking just fine
<smoser> cool. i'll apply that then. thanks!.
<smoser> all, 
<smoser>  https://ch.tbe.taleo.net/CH03/ats/careers/requisition.jsp?org=CANONICAL&cws=1&rid=727
<smoser>  https://ch.tbe.taleo.net/CH03/ats/careers/requisition.jsp?org=CANONICAL&cws=1&rid=727
<smoser> first is SW Engineer, second is QA Engineer (both at Canonical).
<harmw> North America... kinda far, far away :)
<harmw> nice work @r284
<harmw> could you check up on the other bug aswell smoser? If time permits, ofc
<smoser> harmw, probably not a requirement
<smoser> (the location)
<smoser> what other bug ?
<harmw> https://bugs.launchpad.net/cirros/+bug/1132686
<smoser> harmw, can you verify
<smoser> http://paste.ubuntu.com/6475825/
<harmw> sure
<harmw> excellent
<harmw> interesting, my freebsd instance only configured the additional host route - not the gateway
<smoser> wait, what ?
<harmw> no, nothing cloudinit just yet
<harmw> merely booting an image, nothing fancy :)
<smoser> AH. K.
<smoser> did the cirros change work ?
<smoser> i'll push that to trunk.
<harmw> yea
<smoser> thats what you get for not testing
<smoser> s/you/i/
<harmw> it looks fine here
<harmw> nice @r285
<smoser> harmw, thanks for your help.
<harmw> np
#cloud-init 2013-11-30
<mushtar> does cloud-init work with yum?
<harmw> sure, it uses yum to install additional packages
#cloud-init 2014-11-24
<spandhe> mhayden: hey! I made a few changes to the IPv6 commit.. saw some issues earlier with a scenario with multiple NICs.. fixed that and updated the unittests.. http://bazaar.launchpad.net/~shraddha-pandhe/cloud-init/cloud-init-ipv6-support/revision/1036 and http://bazaar.launchpad.net/~shraddha-pandhe/cloud-init/cloud-init-ipv6-support/revision/1037
<harlowja> more easily reviewable  @ 
<harlowja> https://code.launchpad.net/~shraddha-pandhe/cloud-init/cloud-init-ipv6-support/+merge/242547
 * mhayden ganders
<harlowja> damn ubuntu format!
<harlowja> lol
<harlowja> someday we'll get off that 
 * mhayden crosses fingers for github one day ;)
<mhayden> spandhe: i can't see anything problematic in those commits
<mhayden> thanks for working on those
<spandhe> mhayden: in rev 1035, on lines 87 and 88, there was an issue
<spandhe> dev_name was used outside of the loop.. and that loop iterates over the interfaces.. 
<spandhe> actually, not outside of the loop, but it wasnât initialized.. 
<mhayden> ah okay
#cloud-init 2014-11-25
<harlowja> spandhe mhayden alright, merged that branch
<harlowja> my guess is the other non-rhel, non-ubuntu distros that also use that parsed format will need updating if they want to have ipv6 support
<spandhe> harlowja: awesome! thanks!
<harlowja> np
<harlowja> btw spandhe  if u feel interseted, and u want to add comments to some code let me know
<harlowja> i started adding http://cloudinit.readthedocs.org/en/latest/topics/modules.html#debug and 
<harlowja> http://cloudinit.readthedocs.org/en/latest/topics/modules.html#chef
<harlowja> but need to fill out the others
<harlowja> if people want to help, feel free ;)
<harlowja> format @ http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/config/cc_chef.py#L21
<harlowja> which then will turn into that web format when parsed
<harlowja> i know people loving writing comments/docs
<harlowja> lol
<spandhe> harlowja: haha
<harlowja> i did 3, if everyone does 3
<spandhe> will do..
<harlowja> then we will have most of them documented :-P
<JayF> Honestly moving to github or something git-like would enable me to make more smaller commits and I'd be likely to help with something like that
<JayF> as it is having to wrangle bzr and lp to get a change in makes it unappealing
 * goodwill waves at ceren 
<goodwill> folks ... Ubuntu 12.04's cloud image's cloud-init doesn't seem to have opennebula contextualization support ... or am I blind?
<harlowja> what is ' opennebula contextualization support '
<harlowja> 12.04 is old
<harlowja> so i'm not sure whats even there
<harlowja> an old version of cloud-init :)
<harlowja> JayF ya, i unerstand the pain with bzr; i've gotten used to it, but its still not fun having to know it 
<JayF> It's more that I spend most of my time working on a stack of other things
<JayF> they all use git
<harlowja> ya
<ceren> it's old, but both me and goodwill have to support it as a guest. literally 85% of my guests will be on it. :/
<goodwill> is there a way to rebuild it using latest 12.04.5 with latest cloud-init ?
<harlowja> i'm not sure, the ubuntu folks will have to answer for that
<harlowja> i don't think they do that kind of stuff though
<ceren> I'm utterly new to this, and just trying to work out how to... I don't know, cram in a new version of cloud-init to the old 12.04.5 cloud image.
<goodwill> harlowja: who built the existing image?
<ceren> yeah, this seems to fall sort of between the cracks. have been reading http://lists.opennebula.org/pipermail/users-opennebula.org/2014-February/043988.html and trying to parse it.
<harlowja> i think someone that smoser  (who works at canoncial) knows
<ceren> goodwill: canonical, I'm sure, is responsible for it.
<harlowja> i don't work at canonical though
<harlowja> some people here do, ha
<ceren> like they don't have enough on their plate. okay. just was hoping there was some obvious bits I was not understanding yet.
<ceren> thx :)
<harlowja> np
<harlowja> if u need to support a custom image; then shoving a newer cloud-init in that has the support u want is probably the way to go
<harlowja> but if u are expecting people to use the old 12.04 that comes with whatever cloud provider, then u likely are stuck with whatver is in there (which i don't believe the ubuntu folks upgrade unless its a security fix)
<JayF> ceren: I have a present for you
<JayF> ceren: https://github.com/racker/cloud-init-docker-build is what Rackspace is using to build modern cloud-init w/our patches
<ceren> that's fair enough. okay, is there any suggested "how to try to cram a newer cloud-init backwards into an old image" page
<ceren> ooh.
<JayF> ceren: that references https://github.com/racker/cloud-init-debian-pkg 
<JayF> which you're welcome to fork and take our patch out of :)
<JayF> ceren: ^ Solved Problems++
<goodwill> JayF: what are "our patches"
<JayF> goodwill: one patch to support some vendor_data that's locally patched into our cloud
<JayF> goodwill: I'll be upstreaming it "eventually" when the OpenStack spec to get the data into the normal OpenStack metadata gets closer
<JayF> tl;dr: get network configs via JSON rather than via an interfaces file template
<JayF> nothing you're interested in unless you're running the same patch we are in your cloud, but coming soon to Upstream by Kilo release :P
<goodwill> hmmm
<JayF> goodwill: this, specifically, is the patch. It's also in the cloud-init-debian-pkg repo I linked above (as a .patch file) -> https://github.com/racker/cloud-init/pull/1
<ceren> this is two birds with one stone, goodwill and I don't even work together. so the help is extra useful!
<JayF> this is the openstack cooperation side of it (we downstreamed a patch doing this and put it in vendor_data) https://review.openstack.org/#/c/85673/
<JayF> Hey it's no problem at all
<JayF> I built those docker builders for this because I have to support all those images
<JayF> for Rackspace OnMetal, and we exclusively use cloud-init for system configuration
<JayF> so if you wanna thank me; use Rackspace Cloud (and especially OnMetal)
<ceren> once we have these running, that's entirely possible. currently one of our teams bursts capacity to aws, but ... *shrug* we're not emotional about our providers. :P
<JayF> OnMetal is sweet because it uses OpenStack Ironic to provision *bare metal* instead of vms
<ceren> Does what it says on the tin, then.
<JayF> aka https://c34a6498d4802e89941e-16c214a4a8ca35317ce45a32e60db84b.ssl.cf1.rackcdn.com/47d4f2af-654f-4f50-a78b-c3beec555461.jpeg
<harlowja> lol, is that the offical logo now?
<ceren> official ENOUGH
<harlowja> :)
<JayF> The "Bare Metal Bear" came off a t-shirt
<JayF> and I may have funnelled it into an internal meme generator last week
<JayF> lol
<JayF> our official logo is boring so we have a couple of unofficial ones; that one and the toothy cloud 
<JayF> aka https://c34a6498d4802e89941e-16c214a4a8ca35317ce45a32e60db84b.ssl.cf1.rackcdn.com/3bbef025-1f3f-459e-be14-5749bc95f5d4.jpeg
<harlowja> ya, i'm liking the bear vs the toothy one
<JayF> the original codename for OnMetal at rackspace was "teeth", hence the cloud
<harlowja> ah
<JayF> ceren: goodwill: Good luck; if you guys have any problems feel free to ask and I'll help, but the reason for the docker builders is it's pretty dang hard to do it wrong if your docker works
 * JayF leaving for the day
<ceren> appreciate it! we'll bang on this with some sharp rocks for a while tonight :)
<JayF> ceren: I also suspect you'll need to s/jayofdoom/racker/ in some of those dockerfiles, as they don't appear to have been updated since we moved the repos ... but github 301s on the internet so maybe they redirect it for git too
<ceren> ha, that's fine. appreciate the heads-up.
<JayF> np, good luck
<ceren> right now I'm mostly reading the patch, goodwill's going to try building the current cloudinit in 12.04. 
<ceren> also laughing at https://c34a6498d4802e89941e-16c214a4a8ca35317ce45a32e60db84b.ssl.cf1.rackcdn.com/3bbef025-1f3f-459e-be14-5749bc95f5d4.jpeg 
<pfreund> Hello, can cloud-init clean itself at the end of the boot ? I'm trying to create images on OpenStack. I would like to do some "rm -rf /var/lib/cloud/instances" at the end of the boot. Any idea ?
#cloud-init 2014-11-26
<mgagne> who should I poke about having cloud-init 0.7.5 in epel6 ?
<gholms> mgagne: pixelbeat
<gholms> I recommend filing a bug to ask for an update, personally.
<gholms> Then your request won't be lost in the abyss.
<mgagne> who should I ask if we wish to have cloud-init 0.7.5 in epel6 ?
<JayF> mgagne: probably someone who works on CentOS 6/ RHEL 6. 
<JayF> mgagne: I do have a builder that works for CentOS 6 if you desire to build your own...
<mgagne> JayF: who this someone could be?
<JayF> mgagne: IDK, but likely not in this channel tbh
<JayF> mgagne: that might be better targetted for a mailing list if CentOS, and your support reps if RHEL
<mgagne> JayF: my wish isn't to have to maintain my own, this would be the last option tbh.
<JayF> OK :) Good luck getting someone to build that for ya then
<JayF> if you give up, I'll gladly link you to the builders I use 
<mgagne> JayF: I only do it for ubuntu precise and it isn't the greatest thing or highest priority in my backlog. os if it could go in someone else backlog, that would be great ;)
<JayF> lol
<JayF> I build cloud-init + some downstream patches for Ubuntu 12.04 + 14.04, CentOS 6+7, Fedora 20
<JayF> and I have builders that work for debian, but I don't use those packages today
<mgagne> JayF: are you maintaining a local mirror for it?
<JayF> No, but I have build scripts that work great for it
<JayF> we explicitly didn't wanna be in the business of providing cloud-init packages to the world :)
<mgagne> JayF: I use launchpad+ppa for ubuntu for what I don't like about this solution is that it doesn't support rpm based distros :-/
<mgagne> JayF: yha, same thing here
<JayF> https://github.com/racker/cloud-init-docker-build are the builders I use today
<mgagne> JayF: my poor ppa: https://launchpad.net/~iweb-openstack/+archive/ubuntu/cloud-init =)
<JayF> as a note; these will build with our downstream patch, which shouldn't hurt anything, but could be easily adapted to build stock
<mgagne> JayF: thanks, I'll take a look
<jroll> mgagne: to be clear, we build them only for our images, we install the package into the images we distribute, we don't even publish them internally
<mgagne> jroll: how are updates handled? that's the part I wonder
<jroll> mgagne: we build a new image
<gholms> mgagne: I told you yesterday:  file a bug to request an update.
<JayF> mgagne: given we only care about cloud-init on first boot; we don't do rolling upgrades in-image 
<jroll> cloud-init is only useful to us on first boot afaik
<mgagne> jroll: right, what happens if the guy runs apt-get upgrade?
<gholms> EPEL's cloud-init maintainer is pixelbeat.
<jroll> mgagne: something something apt pinning maybe, I don't know the details
<JayF> something something apt pinning sounds correctish :P
<mgagne> JayF, jroll: right. it *should* be the same here (firstboot only) but who knows if it could be useful later
<gholms> Heh
<jroll> ha, I knew something \o/
<mgagne> jroll, JayF: due to its usefulness in an openstack cloud, I kind of wish cloud-init was updated more often. Now as operators, we are kind of forced to reinvent our own packaging system for it. to less extend than openstack itself but still, it's an other thing to maintain.
<JayF> I agree that image management is a big part of running a cloud :)
<JayF> I'm lucky to work for a big company that has a group that assembles the image (even if I have to build the cloud-init packages)
<jroll> mgagne: updated in distro repositories? that's the story for all packages, not just cloud-init
<mgagne> jroll: I have tunnel vision, I only care about cloud-init at this particular moment. I will complain about an other package tomorrow =)
<jroll> mgagne: sure, I complain about this all the time, but it isn't cloud-init's fault :)
<gholms> Seriously, just file a bug.  The EPEL maintainer is a good guy.  ;)
<mgagne> gholms: where can I file that bug? bugzilla?
<gholms> Yeah
<gholms> bugzilla.redhat.com
#cloud-init 2014-11-27
<mgagne> for what it's worth: bz #1168444 => https://bugzilla.redhat.com/show_bug.cgi?id=1168444
<kodokuu> hi, Is it possible to execute local script with cloud-init ? 
#cloud-init 2014-11-30
<harmw> hiren_: you ever installed the cloudinit package on fbsd?
#cloud-init 2015-11-23
<usrenmae> Hello. What is the order of cloud-init directives execution? I'm running packer against AWS instances providing it with user-data as cloud-init file, which has "write-files" directives as the last line. I was hoping to only allow stopping the instance once the last directive is done, but it seems like the file is written, but the earliest directives do not manage to finish by the time the "write_files" is done.
<usrenmae> So I am left with packages not installed and repos not included in my resulting os image. What's the right way to approach the problem?
<manuq> can I have a cloud-config YAML script AND a bash script running both at boot?
#cloud-init 2015-11-26
<ftsf_> hi, is it possible to make an instance shutdown/terminate itself if any cloudinit steps fail?
<themadcanudist> hey guys. I was wondering. Is it possible to have a cloudstack datasource configured w/ appropriate cloud_*_modules and have it only work when it detects itself on a cloudstack environment? Currently, it takes like 2 minutes for cloud-init to timeout if the template is booted elsewhere
<themadcanudist> what are some best-practices for dealing with this. My CI builds images for all sorts of things and I configure cloud-init quite early in the process for a whole range of optiosn.
<themadcanudist> Even shortening the timeout from 2 minutes to something more reasonable woudl help
<themadcanudist> anyone around?
#cloud-init 2016-11-28
<smoser> vans163, i see something back from the 25th from you. fwiw, cloud-init will look for the cdrom seed every time.
<smoser> i had mentioned this to  you at https://irclogs.ubuntu.com/2016/11/16/%23cloud-init.html#t15:00
<smoser> not pointing to that to be rude and say "i already told you this" but rather so you can look at it as the answer is still relevant.
<vans163> smoser:  ah  "	if you want to detach that disk, then you will need to tell cloud-init "manual_cache_clean: True".  And you have to write that (probably via user-data) into /etc/cloud.cfg.d or the next reboot, it will go  looking for a datasource again."   ?
<vans163> smoser: Totally did not remember that appreciate the refresher.
#cloud-init 2016-11-29
<crab> hi.
<crab> am i wasting my time trying to change the default username for an instance from user-data via a #cloud-config section?
<crab> also, i want to enable the EPEL repository before installing a bunch of packages. the example in the documentation isn't terribly useful: where do the files like /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL come from?
<crab> so i'm trying to runcmd "rpm -Uvh http://â¦epel-release-rpmâ¦", but that gets executed AFTER the packages are installed, so the package installation fails.
<crab> should i be using a bootcmd?
<crab> bootcmd worked.
<crab> but changing the default username didn't.
<crab> so i might as well just go back to a shell script that does everything.
#cloud-init 2016-11-30
<crab> yeah. i'm better off with a shell script.
<rharper> smoser: do we ever write out maas user-data/meta-data to file ?
<smoser> rharper, user-data is in /var/lib/cloud/instance/user-data.txt
<smoser> and vendor-data in vendor-data, but meta-data is not written out.
<rharper> hrm
<rharper> smoser: what would have the apt_source config ? user-data right ?
<smoser> cloud-config.txt has the combined cloud-config
<rharper> I've a log which appears to have apt-source config sent via maas, the config module fails , but I find no apt source config
<smoser> and is yaml parsable
<rharper> it's empty
<rharper> there is a user_data.sh which is just a power-off script
<rharper> https://bugs.launchpad.net/maas/+bug/1635560
<rharper> I've got to run to an appt;   but something is amiss with their maas, I think but not sure why we don't have more in cloud-config.txt
<smoser> rharper, they dont send cloud-config
<smoser> they just send a runscript (#!)
<smoser> cpaelzer, https://bugs.launchpad.net/maas/+bug/1635560 is showing
<smoser> ValueError: Old and New apt format defined with unequal values True vs False @ apt_preserve_sources_list
<cpaelzer> hi smoser
<smoser> just really commenting
<cpaelzer> ok, read - yeah I think it is good we added the log message
<cpaelzer> but the issue is about more issues in his user_data anyway
<smoser> yeah.
<rharper> smoser: so I tested out systemd from proposed (https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1636912 ) ; installed nplan, added netplan yaml, modified cloud-init.service to add After=systemd-networkd-wait-online.service ; blew away my cloud data, reboot.  It works fine.  \o/  That said, I'm not sure what to do about the IPV6 RA delay;  it just plain adds 10 seconds to "bring network online" when you don't have
<rharper> v6 available.
<smoser> becauase it jsut tries
<smoser> right? as in you're just trying dhcpv6 ?
<smoser> i know we've discussed this, just forgot
<rharper> smoser:  it's the default configuration for networkd and dhcp of netplan
<rharper> smoser: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1636912/comments/27
<rharper> pitti's comment
<rharper> also checking out the newer kernel with +hv_* drivers ... I'll post up a new base image for testing out in azure
<rharper> first going to check the kernel snap and see what hv_ drivers show up
<smoser> cool
<rharper> smoser: so, specifically we need the hv_ drivers *inside* the initramfs, right?
<smoser> yes
<smoser> oter wise can't find root
<rharper> =(
<rharper> no dice
<rharper> replied
<rharper> they're in the kernel snap, but not in the initramfs
<Synthead> is there a cloud-init docker image I could use?
#cloud-init 2016-12-01
<qwluhbear> Hi, anyone around to troubleshoot a password bootstrapping issue with VMWare?
<doda> hi folks
<doda> it seems like cloud-init deletes the ssh server keys of an instance, but if it can't find a metadata service or config drive it doesn't generate them
<jgrimm> smoser, around?
#cloud-init 2016-12-02
<smoser> jgrimm, here now
<MrBleu> Hello all !
<MrBleu> Is there any way to "rsync" in cloud-init ?
<larsks> Hey folks, I'm seeing cloud-init-local crash on boot when it tries to read /sys/class/net/eth0/carrier.  It gets an EINVAL and blows up.  It looks like reads of this file will always return EINVAL when the link is down, and since cloud-init-local runs *before* networking, the link is always going to be in that state.
<larsks> Has anyone else seen this behavior?  Simply running 'ip link set eth0 down' followed by 'cat /sys/class/net/eth0/carrier' will reproduce the problem.
<larsks> On that topic: https://bugs.launchpad.net/cloud-init/+bug/1646919
<smoser> larsks, https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/305882
<smoser> i think
<larsks> Looks likely.
<smoser> i marked yours a dupe. un-dupe if you like.
<smoser> if you want to give harlowja's branch a kick, go for it.
<harlowja> supp
<harlowja> lol
<smoser> i've just been kind of weary of pulling it as it seems like it could possibly go awry
<harlowja> my code is perfect
<harlowja> lol
<smoser> that, and harlowja has not been responsive ;)
<harlowja> oops
<larsks> smoser, agreed, looks like a dupe.
<larsks> harlowja, kick.
<harlowja> kick
<harlowja> lol
<harlowja> soccer time
<harlowja> larsks ya, i'll fix it
<larsks> \o/
<harlowja> @smoser do u recall https://gist.github.com/harlowja/eb6f6688b3d6cefcaf302e588e034d9f
<harlowja> switching to building in a docker container
<harlowja> and hit that
<harlowja> on 0.7.7
<harlowja> let me see if bumping up the cloud-init tag works/helps
<smoser> that is fixed i think
<harlowja> k
<smoser> harlowja, 9972d246947f1a6ec102b978b99b26acc43133ec
<harlowja> k
<harlowja> switching up to 0.7.8
<harlowja> https://gist.github.com/harlowja/fa0ee4aa83acf8d7a35bf3b3742330e6 poop
<harlowja> lol
<smoser> suck
<smoser> harlowja, well, that is also fixed.
<harlowja> ya, prob, ha
<smoser> seems like maybe we need an 0.7.9 for you, eh?
<harlowja> https://gist.github.com/harlowja/6c403b87249c84b914e3d60dc4d388a8 (my jenkinsfile that is running all this)
<harlowja> can we get a 0.7.9 release with all those fixes :-P
<smoser> http://paste.ubuntu.com/23569017/
<harlowja> woah
<harlowja> u the man
<harlowja> lol
<harlowja> let me see if master works, ha
<smoser> please do
<harlowja> https://gist.github.com/harlowja/ed6dc735af22a491edf021690a43a49f
<harlowja> poopie
<harlowja> lol
<harlowja> two instances of new-style syntax :-P
<smoser> do you create gists manually ?
<harlowja> ya
<harlowja> this is me copying from a jenkins console file
<harlowja> to a gist, lol
<smoser> :-(
<smoser> ok.
<smoser> i can fix that for you quick. or you can and i'll pull
<harlowja> i can
<smoser> i'd really like the other branch you have to come in
<harlowja> i got it covered boss
<smoser> it scares me though..
<harlowja> lol
<smoser> the io errors one that larsks wants
<harlowja> its xmas, not halloween man
<harlowja> lol
<harlowja> don't be scared
<harlowja> be all santa like
<harlowja> lol
 * smoser is blamed for that set notation
<harlowja> :-p
<smoser> i didnt know
<smoser> oh. thats 2.6 even. wow.
<harlowja> ya
<smoser> :-(
<smoser> harlowja, i'm fix and push the py26
<harlowja> oh ok
<smoser> http://paste.ubuntu.com/23569084/
<smoser> right ?
<harlowja> i was nearly there, ha
<smoser> that passes in my cent6 container now
<harlowja> except i just got `fatal: remote error: DNS lookup failed: address 'xmlrpc.lp.internal' not found: [Errno -2] Name or service not known.`
<smoser> is there something more ?
<harlowja> when git pushing
<harlowja> which was weird, ha
<smoser> you work on the other one.
<smoser> yeah, that is wierd.
<smoser> :-(
<harlowja> ha
<smoser> if you want to 'git show'
<smoser> and paste
<smoser> i can grab your credit too
<harlowja> my credit card is in there
<harlowja> gittttttt
<harlowja> lol
<smoser> git config user.routing_number 482959929943
<smoser> bah.
<smoser> did i type that in irc ?
<harlowja> whats the security code
<harlowja> lol
<larsks> harlowja, in your sys-io-errors branch, should read_sys_net just have a on_einval parameter (since EINVAL is expected for down interfaces)?
<harlowja> larsks i'm gonna go with yes
<harlowja> :-P
<larsks> ...because that would simplify a lot :)
<harlowja> smoser paste looks ok to me
<harlowja> u can do https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/312390 also
<harlowja> back way when i didn't know those links expired, ha
<harlowja> oh well
<smoser> oh. they do ? i didnt know that.
<smoser> :-(
<harlowja> well or amazon changed it
<harlowja> so its dead anyway, lol
<harlowja> http://bit.ly/TyoUQs ---> nowhere now
<smoser> ill hold off on thatone. ast here are other bit.ly broken links
<harlowja> k
<smoser> i think its all just the actual links being dead
<smoser> not bit.lky
<harlowja> ya, amazon
<harlowja> amazon fault
<harlowja> ok larsks smoser https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/305882 updated
<larsks> harlowja, lgtm.
<harlowja> larsks u are the person who could possibly say update the package for cloud-init on epel right :-P
<harlowja> i'm sure mdorman_ would like that :-P
<harlowja> *epel 7 and 6 :)
<harlowja> wink wink
<larsks> harlowja, sort of.  I'm working on an updated package in RHEL.
<larsks> That should work it's way into CentOS.  The one in EPEL should probably just go away.
<larsks> But I will look into it.
<harlowja> u will get a cookie
<larsks> harlowja, oops, your patch has a bug (line 239)
<larsks> ...which shows up when you run the tests :)
<harlowja> ah
<harlowja> good catch, was running the test_net test, guess another one hit it or something
<smoser> you might as well fix larsks's comment about the super set too.
<smoser> names.extend(sorted(potential_interfaces))
<smoser>  ...
<smoser>  for name in names:
<larsks> Now I'm looking at a unicodedecodeerror exception.
<smoser>    if name not in potential_interfaces:
<smoser>    ...
<harlowja> k
<harlowja> done
<smoser> harlowja, open to disagreement
<smoser> you use _on_excp_false once
<smoser> well, 3 times, one function
<smoser> any reason not to have it namespaced inside that function
<harlowja> ??
<harlowja> ah
<harlowja> i can namespace
<smoser> http://paste.ubuntu.com/23569503/
<harlowja> ya
<harlowja> updated with that
<smoser> powersj, magicalChicken http://paste.ubuntu.com/23569508
<smoser> maybe because i'm in a container ?
<smoser> or because i'm root ?
<powersj> smoser: I think so
<powersj> you can run as non-root
<powersj> I do everything as my own user or jenkins user
<smoser> well, i was in a container
<smoser> so i was non-[real]root
<powersj> ah yeah
<smoser> what branch should i try ?
<powersj> start with mine as it is what I am going to push to merge https://git.launchpad.net/~powersj/cloud-init/?h=integration-testing
<smoser> k
<larsks> smoser, https://code.launchpad.net/~larsks/cloud-init/+git/cloud-init/+merge/312399 fixes a unicodedecodeerror I was hitting on centos 7.
<smoser> powersj, http://paste.ubuntu.com/23569531
<powersj> container is already running... that's a new one
<powersj> still doing this in a container?
<smoser> larsks, is there a specific reason for utf-8 there ?
<smoser> rather than just encode()
<larsks> No specific reason, other than "make everyting utf-8".  What does .encode() with no encoding do?
<larsks> I see, uses the default encoding.
<larsks> Sounds good to me.  I will update.
<smoser> powersj, no, that was outside now
<smoser> harlowja, if theres a bug, try to remember to
<smoser>  LP: #XXXX
<smoser> in the commit
<harlowja> ah
<larsks> smoser, does LP: do something magical?  Or is that just informative?
<smoser> and larsks for you, not Closes-bug
<smoser> but LP: #
<smoser> launchpad does read that now.
<larsks> Ah, okay.
<powersj> smoser: https://paste.ubuntu.com/23569555/
<smoser> https://help.launchpad.net/Code/Git <-- "Linking to bugs"
<smoser> larsks, and also, i've got stuff that scrapes that info for ubuntu changelog entries
<larsks> I've updated the merge request w/ LP: and .encode() with no explicit encoding.
<smoser> powersj, i'm on 16.04 on that system.
<smoser> lxc 2.05
<smoser> is that relevant ?
<powersj> smoser: might be, torkoal is running 2.6.2 and I'm running 2.4.1
<powersj> I didn't consider that
<powersj> actually that is the output of lxc --version... the package version on torkoal is 2.0.5....
<smoser> powersj, from ?
<smoser> hm... odd
<smoser> well, that'd probably stricktly be the client
<smoser> maybe
<smoser> i dont know
<smoser> larsks, maybe SysConf should just convert to a string ?
<smoser> wait. mayb not
<smoser> i'm just trying to think of fallout . somone getting a string now when they expected bytes.
<smoser> hm.. now im' confused.
<larsks> smoser, I'm pretty sure this is a safe change.
<smoser> we're calling load_file with no 'decode'
<smoser> so it should get the decode=True
<larsks> Ah, that sounds nicer.
<larsks> I have to take off, but I will look at that later this evening.
<smoser> which  means it should already return a string
<smoser> unless, larsks 0439d8a17d181a2546f2f7cb2d71a04bbb13b186 already fixed it for you ?
<smoser> and yeah, i'm turning into a pumpkin now too
<smoser> i'll leaev that in the MP
<smoser> man python2 is confusing.
<smoser> i'm confused.
#cloud-init 2017-11-27
<smoser> blackboxsw: fyi, log2dch for the hackio was
<smoser> https://hastebin.com/owogejakes
<smoser> hackio of the in-proposed SRU is
<smoser>  https://hackmd.io/BwUwrAhgRg7ADFAtMCBOJAWCDHQGwyIBmcAJqiBnhiAExRRA
<blackboxsw> good deal smoser thanks !
<blackboxsw> smoser, my ec2 istances still have ipv6 access issues with apt for some reason. in my outbound security group rules I do have ::/0 defined, so I'd have expected that to work. but apt times out on ipv6 to security.ubuntu.com
<blackboxsw> ok misconfiguration on my side. sorting the ipv6 vpc config on aws now with proper egress-only gateway
<smoser> blackboxsw: whew
<blackboxsw> ok still trying to sort ipv6 conectivity on aws
<blackboxsw> but in the meantime, just rejected https://code.launchpad.net/~johnguthrie/cloud-init/+git/cloud-init/+merge/331905 as the jinja-templates should take care of that approach
<blackboxsw> provided a bit of context to John,. https://code.launchpad.net/~johnguthrie/cloud-init/+git/cloud-init/+merge/331905/comments/875483
<smoser> blackboxsw: thank you
<blackboxsw> btw: "Jinja2 neither allows you to put arbitrary Python code into templates nor does it allow all Python expressions. The operators are limited to the most common ones and more advanced expressions such as list comprehensions and generator expressions are not supported. This keeps the template engine easier to maintain and templates more readable."
<blackboxsw> as per  side conversation we were having
<blackboxsw> ok validated in ec2 if I use the "vpc wizard" to create an dual stack ipv4/6 network, apt continues to work on my instances
<blackboxsw> I'll see what the wizard set up for me to understand what I missed when doing this manually
#cloud-init 2017-11-28
<exarr> Hi all. Please prepare for some n00b questions.
<exarr> So, I am planning to use openstack with rhel 6.9 and need to learn about cloud-init a bit. But the docs seem to go from "here's a file structure" straight into a here's a load of config.
<exarr> So, for an ice-breaker here, how can I set the /etc/hostname from the metadata passed form openstack?
<exarr> I believe this is a built-in function
<exarr> nvm, I'm gonna youtube and stop pestering you guys :-)
<smoser> bug 1734939
<ubot5> bug 1734939 in cloud-init (Ubuntu) "#include fails silently." [High,Confirmed] https://launchpad.net/bugs/1734939
<smoser> sigh
<blackboxsw> hrm did I break that? checking the bug
<smoser> blackboxsw: no
<smoser> you did not.
<smoser> no idea how long its been broken :-(
<rharper> smoser: isn't that bionic no-dns issue? did you try that on xenial instead ?
<rharper> ah, this is a reporting of the failure
<smoser> rharper: yeah, it is that issue. and will become harder to reproduce if that were fixed :)
<smoser> (probably we can use 404)
<smoser> blackboxsw: what am i missing at https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/333513
<blackboxsw> smoser: was just looking that over. I'm not sure what the prob is.   The new subcommand is 'cloud-init status' not systemctl status
<blackboxsw> I didn't wire that into systemctl cmd entrypoints
<blackboxsw> maybe it's just a disconnect about what we really want for 'cloud-init status' I was expecting scripts could call into our cli to 'cloud-init status --wait'  if their services needed to hold until cloud-init was done.
<blackboxsw> smoser: should we instead wire that subcommand into systemctl status so that custom params can be provided?
<smoser> blackboxsw: i must not have been clear.
<smoser> you are using the presense of systemctl binary to indicate "this is systemd"
<smoser> at least i think
<blackboxsw> smoser: we mentioned in comment "moving distro.uses_systemd" where should that live? It seems specific to distro-related attributes and be hesitant to further bloat cloudinit.util
<blackboxsw> maybe it could just be a separate public function in cloudinit.distro instead of a class attribute of Distro
<blackboxsw> cloudinit.distro.__init__
<blackboxsw> though cloud.distro.uses_systemd() method is called from cc_mounts, ntp, fan and rsyslog modules (as well as rhel/opensuse distros). So, we'd have to explicitly import the public function there and use that instead  (or have Distro.uses_systemd be a wrapper that just calls the separate function uses_systemd).
<blackboxsw> maybe I'll just do that. I'm in hangout if you want to chat through other options
<smoser> joining
<blackboxsw> I could be swayed to moving uses_systemd it to cloudinit.util too. We'll have to take a pass extracting functions out of cloudinit.util anyway at some point to make it more modular and focused
<rharper> smoser: surely urllib returns something on DNS resolution failure vs. 404
<smoser> rharper: right. but shoudl also fail and exit failure on 404
<smoser> and that is easier to create a test for
<smoser> than to break dns
<rharper> for sure
<rharper> I see, you were referring to how to reproduce/test
<paulmey> Hi all, what's the process for adding features to a datasource?
<paulmey> Azure is adding a feature where they are going to keep VM's 'warm' and hold it at boot till a customer needs it...
<paulmey> here's the MP with the code they have been working on https://code.launchpad.net/~dojordan/cloud-init/+git/cloud-init/+merge/334341/+index?ss=1
<paulmey> do we need an LP bug?
<smoser> paulmey: you dont strictly need a bug, but for a feature like that i guess i'd like one. gives a place to "hold"
<smoser> i'm not bent on it, but its not like it is hard to file a bug either.
<paulmey> ok, you want a bug... ð
<paulmey> https://bugs.launchpad.net/cloud-init/+bug/1734991
<ubot5> Launchpad bug 1734991 in cloud-init "cloud-init doesn't yet support azure vm preprovisioning" [Undecided,New]
<blackboxsw> hrm smoser rharper systemctl status cloud-init.target isn't conclusive. Even when cloud-init is pending (but runcmd is camping on a sleep)
<blackboxsw> then ActiveState remains 'inactive' and is-enabled remains 'static' until the magical moment when cloud-init is done and transitions to 'enabled'
<blackboxsw> or 'active'
<blackboxsw> I'm looking for another mechanism to detemine 'cloud-init is pending in systemd'
<blackboxsw> it really feels like cloud-init's  manipulation of systemd's $gen_d/${target_name}.wants/${CLOUD_TARGET_NAME} link is our best source of knowing if cloud-init really is going to run
<blackboxsw> or systemctl list-dependencies
<blackboxsw> feels like systemctl list-dependencies | grep cloud-init.target is probably the best bet
<blackboxsw> as tracking details of the link file names would be fragile if that link path changes in future releases.
<rharper> blackboxsw: you can query the list of jobs to see if it's pending
<rharper> I didn't think we needed that state though
<rharper> the tri state was (disabled, never going to run), (enabled and is pending), (enabled is done)
<rharper> is there something missing there?
<blackboxsw> rharper: yeah double checking today on xenial.... we have is-enabled returns 'static' if cloud-init.target is active,  and if it is disabled via kernel cmdline or /etc/cloud/cloud-init.disabled
<blackboxsw> so I can't inspect disabled if we've disabled our systemd generator via /etc/cloud/cloud-init.disabled or kernel cmdline
<blackboxsw> s/inspect/introspec/
<rharper> I didn't think that was the case
 * rharper reads the unit file states again
<blackboxsw> I tried adding a #cloud-config\nruncmd: ['sleep 60']  and I also don't get to see an enabled status untl     http://pastebin.ubuntu.com/26066805/
<blackboxsw> until cloud-init.target completes
<rharper> static == off (never going to run)
<blackboxsw> on a system currenty in runcmd final stage http://pastebin.ubuntu.com/26066812/
<rharper> on xenial the default is (it runs less it's told not to)
<rharper> ie, ds-identify is report-only
<blackboxsw> static is on a system where cloud-init is enabled
<smoser> blackboxsw: https://hastebin.com/raw/qetelayimi
<smoser> that is what i put together to collect info.
<smoser> i ran it for xenial, but yeah, didnt see anything that said "expected to be run"
<rharper> "static"	The unit file is not enabled, and has no provisions for enabling in the "[Install]" unit file section.
<blackboxsw> rharper: then I guess systemd is lying or how cloud-init links or doesn't link the wants file is breaking systemd?
<rharper> not following
<rharper> cloud-init.target gets dynamically installed
<rharper> so, an at-rest system is going to say static
<rharper> early, we generator run, and on xenial, the default is to enable unless we disable via file or cmdline
<blackboxsw> our cloud-init generator only dynamically links for wants file if we don't have /etc/cloud/cloud-init.disabled or the cloud-init=disabled in the kernel cmdline
<rharper> correct
<rharper> but you can't run status before the generator runs
<rharper> a cloud-init user cannot
<blackboxsw> but systemd is-enabled must be checking something else, because it doesn't report 'enabled' on a system that has those links
<rharper> it enabled"	Enabled via .wants/, .requires/ or Alias= symlinks (permanently in /etc/systemd/system/, or transiently in /run/systemd/system/).
<rharper> so, once generator adds it to a .wants
<rharper> then it will show enabled (that's post generator)
<rharper> that means cloud-init status should show *pending* until you have a file that indicates cloud-init is done (result.json)
<rharper> disabled;  or pending -> done
<blackboxsw> rharper: here's a system that has already run all of cloud-init note is-enabled still shows 'static'
<blackboxsw> http://pastebin.ubuntu.com/26066859/
<rharper> and you have a result.json ?
<blackboxsw> yeah with no errors
<rharper> the output of systemctl isn only relevant when we don't know
<rharper> we only need to know, disabled (never going to run, this is 'static') or we have a result file ; if not those cases, then pending
<rharper> recall, that is-enabled refers to the *unit* file
<rharper> not the active job
<rharper> systemd builds a tree of units, for each unit, it spawns a job
<rharper> the job runs and is removed from the job queue
<smoser> well, cloud-init remain Active when they're done.
<rharper> the units, yes
<rharper> but really the status command is meant to refer to the runtime element, (ie, the systemd job)
<smoser> we're interestd in asking systemd "are you expecting to reach the cloud-init target." thats what we need to know.
<smoser> we're not using status now.
<smoser> we're using show.
<smoser> if you're suggesting we look at the unit instead of a target, i dont know if that would hcange anything or not.
<rharper> a target is just a special unit AFAIK
<rharper> even if we ignore the jobs;  I think we have enough, given the result file an disabled; if we're not disabled and we don't have a result file, then we're pending
<blackboxsw> I'm still lacking how we determine that we are disabled. as I don't see anything from systemd cli that properly reports whether we are disabled
<smoser> right.
<smoser> there is nothing there.
<smoser> its obnoxious
<rharper> static means disabled
<blackboxsw> we can determine ourselves by looking at cloud-init.disabled OR kernel cmdline
<blackboxsw> static also means enabled
<blackboxsw> and not yet run
<rharper> no it does not
<rharper> well, yes, but you can't ask that
<rharper> before the generator runs
<smoser> static does not mean disabled
<blackboxsw> rharper: even after generator runs
<blackboxsw> I still see 'static'
<smoser> right.
<rharper> well, modulo bugs
<rharper> systemd says that once it's added to a .wants it's enabled (or enabled-runtime)
<rharper> so, maybe we have bug in xenial systemd w.r.t state
<rharper> notice in artful/bionic
<rharper> it says enabled-runtime
<rharper> which is the correct answer
<rharper> in xenial
<blackboxsw> right, artful/bionic look sane. <= zesty behave differently
<rharper> https://www.freedesktop.org/software/systemd/man/systemctl.html#
<rharper> well, they have newer systemd
<rharper> the man page is pretty clear on this
<rharper> "enabled"	Enabled via .wants/, .requires/ or Alias= symlinks (permanently in /etc/systemd/system/, or transiently in /run/systemd/system/).
<rharper> so, it should be 'static'; then when generator runs, if enabled, it will transition to enabled (or enabled-runtime) due to a symlnk in /run/systemd/system/multi.user.target.wants/cloud-init.target
<rharper> I don;t think I'm misreading that man page
<smoser> the results differ.
<rharper> aka, bug in xenial systemd's systemctl status ; are you referring to something else?
<rharper> s/status/is-enabled
<blackboxsw> http://pastebin.ubuntu.com/26066998/
<blackboxsw> referring to a bug in is-enabled
<blackboxsw> the above is on zesty
<blackboxsw> running in an enabled world first, then disabled 2nd
<rharper> not the units, the target
<rharper> we only put cloud-init.target in .wants in /run
<blackboxsw> I ran against cloud-init, cloud-init.target and cloud-init.service
<rharper> so querying the unit files (cloud-init-local , cloud-init, cloud-config, cloud-final) isn't helpful
<rharper> blackboxsw: right, but do target only on artful/bionic
<rharper> does it not report enabled-* vs. static ?
<blackboxsw> rharper: right we are good on artful ++ http://pastebin.ubuntu.com/26067021/
<blackboxsw> just broken on xenial & zesty (from our point of view)
<rharper> ok, so we file a bug
<rharper> so we can get proper is-enabled status for cloud-init.target
<rharper> I don't think any other trickery is going to help w.r.t systemd commands
 * rharper relocates
<blackboxsw> smoser/rharper: Ok so, systemctl list-dependencies does give us an idea about 'pending' cloud-init wants target, and 'systemctl is-enabled' works on artful and above.    But,  filing a bug aside, it's probably just best for us to only look at the /etc/cloud/cloud-init.disabled file and kernel cmdline for determining if we are disabled for the time being, if not disabled... assume pending until result.json
<blackboxsw> exists right?
<blackboxsw> the other gap in that logic (checking disabled file or cmdline) is ds-identify disabling us
<blackboxsw> maybe the 'fix' is for ds-dentify to precipitate a /etc/cloud/cloud-init.disabled file with 'disabled by ds-identify' content.
<blackboxsw> then we've covered that gap without relying on older systemd tooling which doesn't currently work
<smoser> blackboxsw: writing something. almost done.
<smoser> http://paste.ubuntu.com/26067095/
<smoser> blackboxsw: ^
<smoser> basically, forget about asking systemd. it seems useless to me
<blackboxsw> reading
<blackboxsw> yeah agreed on systemd
<blackboxsw> not enough info there yet (but we'll file a bug)
<blackboxsw> ahh didn't realize the ds-identify writes an enabled file
<blackboxsw> ok so yeah we can look for presence of (disabled file or kernel cmdline disable) or absence of /run/cloud-init/enabled
<blackboxsw> that makes sense (and works today)
<blackboxsw> ok, smoser I'm not going to move around the uses_systemd function/method then as we don't need it
<smoser> well, blackboxsw when not running on systemd
<blackboxsw> smoser why does that matter?
<blackboxsw> as sysvinit is always on?
<blackboxsw> because there is no generator?
<smoser> right
<blackboxsw> roger
<smoser> the generator writes /run/cloud-init/enabled
<blackboxsw> ahh right times two
<blackboxsw> ok keeping uses_systemd :/
<smoser> on systemd we get that benefit. -- easy and complete knowledge of "disabled".
<blackboxsw> hahah
<smoser> well, assuming we do not run before the generator
<smoser> but i'm willing to acept that for now.
<rharper> we can check whether the cloud-init generator has run via the /run/cloud-init dir presence
<rharper> that can catch the inb4-cloud-init-generator case
<blackboxsw> sure ds-identify creates that dir if it doesn't exist
<smoser> rharper: well, but the generator creates that
<smoser> no?
<rharper> yes
<rharper> so, if the dir doesn't exist
<blackboxsw> I see it in ds-identify
<rharper> then it hasn't run
<rharper> that's for the case of testing status before the generator runs
<rharper> if we wanted to support status that early, we could say something like (too early) or whatever
<rharper> ask-later
<blackboxsw> looks like both generator and ds-identify try to create that dir if !present
<smoser> oh. well we can't determine what the status is anyway then.  we can know we're in that state i guess.
<rharper> right, similar to systemd-analyze saying 'boot not finished yet'
<rharper> not particulary helpful but it is some sort of non-answer-answer
<rharper> I'm perfectly fine with not handling the case as that's super early
<smoser> in my tests with my script above, we're in at "1.0" seconds per lxc's /proc/uptime.
<smoser> and it exists already.
<smoser> there isnt a *lot* of opportunity to run 'cloud-init wait' earlier than that
<smoser> ie:
<smoser>  lxc start name;  lxc exec name -- cloud-init wait
<blackboxsw> finally wrapped it up and overhauled cloud-init status to use cloudinit.helpers.Paths object https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/333513
#cloud-init 2017-11-29
<Raboo> hi
<Raboo> what usually comes after the ssh key generation?
<Raboo> have a node that is stuck after the ssh key generation
<dpb1> Raboo: physical system?  cloud?
<dpb1> could be a RNG issue.
<Raboo> dpb1 physical
<Raboo> it gets stuck when running trusty, but when i tested xenial it worked..
<Raboo> but i would like it to work on trusty again
<dpb1> Raboo: through maas I assume?
<Raboo> dpb1 nope, I use The Foreman
<Raboo> https://theforeman.org/
<dpb1> Raboo: your images are up to date for Ubuntu?
<Raboo> yes, i build it daily
<smoser> Raboo: most likely, trusty is just not going to work easily
<smoser> it has no dynanmic network configuration
<Raboo> and i even triggered a manual build today when it failed
<smoser> meaning that the images have 'dhcp eth0'
<smoser> and if you dont have a 'eth0' or its not dhcp..
<smoser> but maybe thats not your issue.
<Raboo> the network came up
<smoser> yeah, then i assume it is that you dont have a eth0 ?
<Raboo> it printed ip before it generated ssh keys
<smoser> or eth0 is not plugged in and some other.
<smoser> oh.
<Raboo> it's a bond0 actually
<smoser> hm...  ... i'm not really sure where youd' be hung then.
<Raboo> and i could ping the node
<Raboo> the most annoying thing is that it got stuck. It didn't fail and drop to a console so I could read logs
<Raboo> so very hard to debug
<Raboo> i think it got stuck at the end of "init" phase
<dpb1> Raboo: what cloud init version are you using in your trusty images?
<dpb1> Raboo: is it the one in the trusty-updates pocket?
<Raboo> dpb1 hmm let me try to find out, i think it was 17 something
<Raboo> is there a binary or a config file that contains a version string?
<Raboo> could be 0.7.5.
<Raboo> it's the same as in https://cloud-images.ubuntu.com/trusty/current/
<Raboo> it contains this file ./usr/lib/python2.7/dist-packages/cloud_init-0.7.5.egg-info
<Raboo> so i guess it's 0.7.5
<Raboo> ./usr/share/doc/cloud-init/changelog.Debian.gz says at the top of the file "cloud-init (0.7.5-0ubuntu1.22) trusty; urgency=medium"
<dpb1> Raboo: ok
<Raboo> this used to work before, don't know if ubuntu has bricked something since I update my images daily
<smoser> :-(
<smoser> Raboo: you can open a bug. just please provide as much information as you can.
<smoser> you can just "backdoor" the image too
<smoser> then you can go in with the backdoor to get other info
<smoser> http://bazaar.launchpad.net/~smoser/+junk/backdoor-image/view/head:/backdoor-image
<Raboo> smoser what does the backdoor do in short?
<Raboo> cause i have a local user already
<Raboo> problem was it doesn't even drop to the console where i can login
<smoser> you said it set up network
<smoser> no ?
<smoser> just ssh in
<Raboo> smoser i don't think sshd is started
<Raboo> i'm going to reinstall another node tomorrow, see if i can replicate the issue on another node.
<dpb1> Raboo: from the sounds of it, that could be what is stalling, actually (sshd startup)
<smoser> rharper: https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/334147
<smoser> can you ack that ?
<rharper> acked
<blackboxsw> ok I'm out of the way on the two branches (no more changes from me on these): https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/333513 https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330115
<blackboxsw> just pushed latest changes/rebase and fixes
<blackboxsw> hitting review queue
<powersj> smoser: seems you pulled your KVM fix, so I am going to enable the KVM tests in jenkins
<blackboxsw> smoser: rharper http://pastebin.ubuntu.com/26075320/ shouldn't our v1 instance-data.json actually surface public-hostname instead of local-hostname  from metadata? Right now the instance-data.json sources cloud.get_hostname() which grabs local-hostname from metadata if present
<blackboxsw> checkout the ec2 section of that pastebin
<rharper> blackboxsw: what does hostname on the instance say now ?
<rharper> I suspect it retains the .internal hostname
<blackboxsw>   "public-hostname": "ip-10-0-20-27",
<rharper> an external/public hostname is for facing outward IIUC
<rharper> no, run 'hostname' on your instance
<blackboxsw> which is actually grabs from hostname on the cmdline
<rharper> ie, what actually ended up in the Linux hostname value
<blackboxsw> ubuntu@ip-10-0-20-27:~$ hostname
<blackboxsw> ip-10-0-20-27
<rharper> ok
<blackboxsw> but for ec2 instances, you can't really get ahold of that local 'hostname'
<rharper> then I would think that should match in the data (which  is what you're pointing out, yes?)
<rharper> can you not resolve that now ?
<rharper> the .internal ?
<rharper> what resolver is configured on the instance ?
<blackboxsw> I can from within amazon, but not from home
<rharper> right
<rharper> which is fine
<rharper> the instnace.json is from within the instance
<rharper> but, I think what's happening is that the hostname public value is not yet when dump the instance data
<rharper> but it's a valid hostname on the instance
<rharper> so, there's a little fuzziness w.r.t what that value should reflect
<rharper> should it be the hostname at the time we process the metadata (which is what I think it is doing now)
<blackboxsw> right it's a valid hostname on the instance, what would you call the public-hostname which you can reference external to a given machine?
<rharper> or should it reflect what the hostname value will be eventually (after cc_hostname runs)
<rharper> public-hostname
<rharper> I think
<rharper> we might need other cloud examples to see what makes the most sense
<blackboxsw> @ the time we process metadata, we could query md['network']['public-hostname'] and it'd give us ec2-18-216-195-116.us-east-2.compute.amazonaws.com
<rharper> I think I'd be fine with tagging hostnames with public or private
<rharper> and let instance users pick
<rharper> and skip having a instance_json['hostname']  value
<blackboxsw> yeah +1
<rharper> or have it include a dictionary of public/private values
<blackboxsw> I have ['v1'][
<rharper> right, I see that
<blackboxsw> I have ['v1']['public-hostname'] which I think is a misnomer (as it's actually the local/private hostname)
<blackboxsw> so I'll surface public/private hostname keys which will allow us to differentiate
<rharper> "public-hostname": "ip-10-0-20-27",
<rharper> yes, that seems wrong
<rharper> that's the .internal one
<blackboxsw> yeah definitely feeling like that
<rharper> but you're saying it has a hostname that matches
<blackboxsw> even though that's what our cloud.get_hostname() logic defaults to
<blackboxsw> I even believe cloud.get_hostname() should probably default to the public hostname where available instead of private
<rharper> I wouldn't change what that does
<blackboxsw> but, it currently doesn't (and that behavior is surfaced in multiple cc modules already.
<blackboxsw> yeah
<rharper> but in our rendering of the data into json, we could pick something else (or call it differently)
<blackboxsw> ok will tweak that. as public/private is more explicit
<rharper> depending on the scripts folks may want the local/private name, for configuring services, etc
<rharper> but if they're injecting values to configure a service to point to another host, those would be the public ones
<blackboxsw> right
<rharper> so I think we need both values
<blackboxsw> +1
<blackboxsw> vote 'private-hostname' vs 'local-hostname'?
<rharper> hrm; I dunno
<rharper> I kinda like public/private symmetry
<rharper> but I honestly have no strong feelings
#cloud-init 2017-11-30
<smoser> blackboxsw: i honestly dont know.
<smoser> iti varies widely on clouds. as to what 'local-hostname' or 'public-hostname' would mean.
<smoser> 'local' seems to imply much less.
<smoser> we dont need to do it in the first round, but i'd really like to provide ip addresses if cloud-init configured them. even ideally if it used dhcp to do so.
<blackboxsw> hrm, unhappy sru validation zesty ec2
<blackboxsw> https://bugs.launchpad.net/cloud-init/+bug/1735331
<ubot5> Launchpad bug 1735331 in cloud-init "ec2: temp dhclient.pid file issues" [Undecided,New]
<blackboxsw> will improve the summary
<Raboo> dpb1 perhaps, that is what comes directly after generating the sshd keys?
<smoser> Raboo: you can change the image, right?
<smoser> and you can see the console i think
<smoser> and this is 14.04.
<smoser> in /etc/cloud/cloud.cfg.d/05_logging.cfg looks like
<smoser>  https://hastebin.com/odahevojix
<smoser> change line 33 . level=WARNING to level=DEBUG
<smoser> that will give debug info to the console
<Raboo> smoser ok, will change to DEBUG
<catmando> hey all
<rharper> howdy
<catmando> question: i am running cloud-init (17.1) within ubuntu 16.04 running on a powervc host
<catmando> ssh-authorized-keys for a default user i want to create does not seem to be populating the file
<catmando> the user is created, but .ssh/authorized_keys is blank
<rharper> can you pastebin your user-data ?
<rharper> and or your /var/log/cloud-init.log;  those bits will let us look at what's going on
<rharper> are you using the default ubuntu user or are you adding a different user ?
<catmando> cfg's user lines:
<catmando> https://pastebin.com/yEwDjHP6
<catmando> 2017-11-30 14:43:47,070 - util.py[DEBUG]: Writing to /home/bcc/.ssh/authorized_keys - wb: [600] 0 byte
<smoser> catmando: can you paste /var/log/cloud-init.log ?
<smoser> also, fyi, 'pastebinit' is installed in Ubuntu by default. so you dont have to use pastebin.com
<smoser> (pastebinit /var/log/cloud-init.log
<smoser> or
<smoser> cat /var/log/cloud-init | pastebinit )
<catmando> https://pastebin.com/uGFN9BzR
<rharper>  if 'ssh_authorized_keys' in kwargs:
<rharper> try with _ instead of - in your user-data ;  I know in some places cloud-init accepts both - and _ interchangably but this might not be one
<rharper> ssh_authorized_keys   instead of ssh-authorized-keys
<smoser> rharper: yes. i verified that locally.
<smoser> :-(
<rharper> yay
<rharper> more bugs!
<smoser> catmando: use ssh_authorized_keys instead of ssh-authorized-keys and it does work.
<catmando> Hey lost internett
<catmando> Cool
<rharper> it appears that the docs show use of underbar; but
<rharper> http://cloudinit.readthedocs.io/en/latest/topics/examples.html#configure-instances-ssh-keys
<catmando> That's what I used
<catmando> Many thanks!
<rharper> bug
<rharper> err, but
<rharper> the add users-groups shows dash
<rharper> http://cloudinit.readthedocs.io/en/latest/topics/examples.html#including-users-and-groups
<rharper> so cut-n-paste-modify will fail there
<smoser> wow
<dojordan> hey @smoser, my merge build failed with an authorization error on jenkins. Any thoughts? https://code.launchpad.net/~dojordan/cloud-init/+git/cloud-init/+merge/334341
<dojordan> Second question, how do I trigger a rebuild?
<rharper> powersj: do you know about the jenkins auth error dojordan is seeing?
 * powersj looks
<powersj> Repository '~dojordan' not found.
<powersj> yet it is clearly there... hmm
<dojordan> weird that the code.launchpad.net works but git.launchpad.net does not
<powersj> oh smoser didn't start it right
<powersj> blame that guy
<smoser> ueaj/
<powersj> running here https://jenkins.ubuntu.com/server/job/cloud-init-ci/562/console
<rharper> powersj: thanks!
<smoser> that was 'yeah', offset a couple keys to the right.
<blackboxsw> whew, sorry, pulling my head out of the ground
<blackboxsw> SRU fix up for review https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/334543
<blackboxsw> testing on ec2 now
<blackboxsw> ok validated works greate
<smoser> blackboxsw: comments on that are there for you now
<blackboxsw> thx about to push status-and-clean subcommand changes
<blackboxsw> pushed review fixes and tests for clean-and-status. checking SRU bug comments
<blackboxsw> ok pushed review fixes for the SRU bug too
#cloud-init 2017-12-01
<dojordan> would someone be able to help me retrigger a build again? https://code.launchpad.net/~dojordan/cloud-init/+git/cloud-init/+merge/334341
<powersj> dojordan: added you to the CI team users group
<dojordan> thank you
<powersj> next trigger job will auto launch for you
<powersj> so in ~15mins
<powersj> or like right now since I pushed the big red button
<dojordan> even better!
<blackboxsw> thx smoser for the SRU bug merge
<catmando> hey all
<catmando> does anyone have experience with cloud-init on powervc/powervm hosts?
<catmando> specifically, with config drive datasources
<smoser> blackboxsw: in your mp for cloud-init... i think they'll want you to put a new changelog entry in
<smoser> i could just e reading the email-diff wrong but it looks like youjust updated teh original?
<blackboxsw> smoser: just pushed this, is this what you mean? https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/334593
 * smoser opens
<smoser> yeah.
<blackboxsw> I didn't pull the process bug up into the latest changelog. is that ok?
<blackboxsw> because that would be changing 'history'
<blackboxsw> smoser: resubmitting artful one sec
<blackboxsw> forgot the ~17.04.1
<blackboxsw> I mean ~17.10.1
<blackboxsw> https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/334594 https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/334595 https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/334597
<blackboxsw> ok all pushed, awaiting CI
<blackboxsw> getting a coffee while magic happen
<blackboxsw> s
<smoser> blackboxsw: given 'build-and-push' as http://paste.ubuntu.com/26090392/
<smoser> currently running
<smoser>  for r in artful zesty xenial; do git checkout $r && ../build-and-push || break; done
<smoser> with your merges into my local branches.
<blackboxsw> nice little script there.
<blackboxsw> trying to walk through it now
<blackboxsw> to make sure I understand.
<blackboxsw> what's sbuild-it alias?
<smoser> oh. sbuild-it is http://smoser.brickies.net/git/?p=tildabin.git;a=blob;f=sbuild-it;h=ab48dc16cfbb3c2153c91ba4a14243b627ff2568;hb=HEAD
<blackboxsw> dpkg-buildpackage -us -uc or something?
<smoser> it will build a dsc or a changes as intput. and reads the file to figure out what the release is. and then calls sbuild with correct --dist=
<blackboxsw> good deal. nice to streamline some of those steps
<blackboxsw> bookmarking
<smoser> looks like sbuild-it will also try to download the original source if it does not have a corresponding .orig.tar.gz
<blackboxsw> think I can ping   tjaalton now in #ubuntu-release?
<smoser> yes, blackboxsw
<blackboxsw> ok pinged there, updated the SRU process bug #1733653 and put up the placeholder updated SRU log sheet https://github.com/cloud-init/ubuntu-sru/tree/master/20171121
<ubot5> bug 1733653 in cloud-init (Ubuntu Artful) "sru cloud-init (17.1-27-geb292c18) update to 17.1-41-g76243487-0ubuntu1" [Medium,Fix committed] https://launchpad.net/bugs/1733653
<smoser> blackboxsw: updated https://hackmd.io/s/rJdKR9q3b
<blackboxsw> excellent smoser thanks
<dojordan_> hey guys, I believe I was added to the CI group yet when i log into jenkins I don't see an option to kick off a new CI build. Would someone be able to kick one off for me again?
<dojordan_> ignore me, one just got launched. I assume its time based or something?
<rharper> dojordan: one you push an update, a new job is queue
<rharper> queued, the backend does lots of ci, so it may take a bit before it runs your job
<rharper> on jenkins, you can see the queued jobs
<dojordan> awesome, thanks !rharper!
<dojordan> @rharper *
<blackboxsw> hrm trying to fire up kvm with  OVF per doc/sources/ovf/README by downloading cloudimages... Failed to initialize module: /usr/lib/x86_64-linux-gnu/qemu/block-iscsi.so
<blackboxsw> I've tried both i386 and amd64 images
<rharper> blackboxsw: can you paste your cmdline ?
<rharper> and dpkg --list | grep qemu
<rharper> you might need the block-extras package
<rharper> yeah, sudo apt install qemu-block-extra
<rharper> blackboxsw: ^^
<rharper> % dpkg -S /usr/lib/x86_64-linux-gnu/qemu/block-iscsi.so
<rharper> qemu-block-extra:amd64: /usr/lib/x86_64-linux-gnu/qemu/block-iscsi.so
<smoser> hm..
<smoser> blackboxsw: what are you doing ?
<smoser> how did you try to use block-iscsi ?
<blackboxsw> http://pastebin.ubuntu.com
<blackboxsw> following the readme in doc/sources/ovf
<blackboxsw> trying to validate against master before trying msaika's branch
<smoser> blackboxsw: yes, thats a nice pastebin :)
<rharper> blackboxsw: need longer url
<blackboxsw> hahaha
<blackboxsw> http://pastebin.ubuntu.com/26091298/
<blackboxsw> can't you guess the #?
<rharper> lol
<rharper> smoser: it's possible the newer qemu is linked against it; i think it's only a suggests though; cpaelzer might know more about that
<rharper> I thought on 2.5+ newer systems it was automatically installed though
<smoser> its wierd.
<smoser> i am missing something
<smoser> qemu-system-x86 'Suggests' qemu-block-extra
<smoser> qemu-system-common
<rharper> I am to; it should work without those
<smoser> Depends on it
<rharper> unless you specify a drive that uses that backend
<rharper> like rbd
<smoser> well, that makes sense in theory.
<smoser> but at least in bionic, you can't install qemu-system-x86 without getting it
<rharper> yeah, I don't recall why it's required  now
<rharper> it was originally broken out to keep the deps out unless you needed it (like librbd etc)
<smoser> qemu-system-x86 -> Depends qemu-system-common -> Depends qemu-block-extra
<smoser> right
<rharper> I have those, I move them out of the way
<rharper> and kvm still runs
<rharper> blackboxsw: I think we need to see more
<smoser> http://paste.ubuntu.com/26091332/
<smoser> it blames to some guy named Ryan Harper
<smoser> :)
<rharper> https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1495895
<ubot5> Launchpad bug 1495895 in qemu (Ubuntu) "Unable to attach rados block device to instances" [High,Fix released]
<smoser> hm..
<smoser> so that went into 2.3 something
<smoser> and blackboxsw  had 2.5
<smoser> so...
<smoser> confused.
<rharper> it's been around for some time
<blackboxsw> yeah I'll go through the steps again (I'm on xenial but will switch to artful to re-test)
<rharper> I'm confused as to why the message is present witout a disk asking for iscsi
<rharper> for example, in the original bug, they used a disk with the rbd driver
<rharper> qemu when launched will dynamically load those .so files if present and then add support for that block driver
<smoser> what is the error you saw ?
<smoser> blackboxsw:  ?
<smoser> you didnt show an error
<rharper> so, your command like of a -qcow2 virtio disk and cdrom shouldn't trip iscsi
<smoser> unless i just missed it.
<rharper> <-- ckonstanski has quit (Ping timeout: 248 seconds)
<rharper> <blackboxsw> hrm trying to fire up kvm with  OVF per doc/sources/ovf/README by downloading cloudimages... Failed to initialize module: /usr/lib/x86_64-linux-gnu/qemu/block-iscsi.so
<rharper> <blackboxsw> I've tried both i386 and amd64 images
<rharper> but the kvm command like pasted doesn't show any iscsi block devices ...
<ckonstanski> Someone needed me? Got new cable installed today -- underwent an outage.
<smoser> yeah i dont know. that is wierd.
<blackboxsw> ohh sorry will grab again. yes, I'm getting Failed to initlializew module etc http://pastebin.ubuntu.com/26091348/
<rharper> ckonstanski: sorry, pasted too much
<ckonstanski> oh
<smoser> blackboxsw: i'm not sure how it happened
<smoser> but the error message is pretty clear
<smoser> Note: only modules from the same build can be loaded.
<rharper> blackboxsw: can you qemu-img info on that qcow2
<smoser> qemu-block-extra:amd64                      1:2.5+dfsg-5ubuntu10.11
<smoser> qemu-kvm                                    1:2.5+dfsg-5ubuntu10.14
<smoser> its just dlopening stuff in the directory
<blackboxsw> not sure what build# I'm at or what the mismatch was but checking
<smoser> and it failed to open those things
<smoser> because tehy're wrong.
<blackboxsw> checking qemu-img
<smoser> see above.
<smoser> 10.11 and 10.14
<rharper> smoser is right, you have mixed packages
<blackboxsw> ahh right
<blackboxsw> bah
<smoser> you must have had a failed apt-upgrad e?
<rharper> you really want 2.5+dfsg-5ubuntu10.16 ;
<rharper> on xenial that is
<rharper> that's the latest
<rharper> apt-get -f install should clean that up after an update I'd think
<blackboxsw> strange yeah I  must've had a partial upgrade or something
<rharper> indeed
<blackboxsw> smoser: do you understand what slangasek is saying in ubuntu-release about building -v?
<smoser> oh. yes. sorry.
<smoser> blackboxsw: that is just annoying.
<smoser> it goes t hrough to dpkg-genchanges
<smoser> and by default it is the previous entry in debian/changelog
<smoser> there is kind of no way to do the right thing.
<blackboxsw> hrm, so what's the action here?
<smoser> i uploaded
<smoser> i was just giving background
<smoser> basically i did
<smoser>  git checkout ubuntu/xenial
<smoser>   build-package -- -v17.1-27-geb292c18-0ubuntu1~16.04.1 -S -d
<smoser> then did the same for artful and zesty
<smoser> with different ~XX.YY values
<blackboxsw> ahh I see, ok. right specify how far back you want changes documented for this release
<blackboxsw> ok kvm testing of ovf is fine. configuration error on my side thx smoser rharper
<rharper> np
<blackboxsw> installing updated kvm packages (and discovered via kvm-ok that I had also disabled kvm in testing that new virt snap)
<rharper> eww
<smoser> my http://paste.ubuntu.com/26091620/
<smoser> just sticking that here in case my system were to die.
<smoser> thats current 'launch-softlayer' as I make changes to launch-ec2 copy.
<smoser> and /me is out
<smoser> later
<blackboxsw> sweet !
<dojordan> hey guys, can someone take a look at this PR for azure when you get a chance? Many thanks https://code.launchpad.net/~dojordan/cloud-init/+git/cloud-init/+merge/334341
#cloud-init 2017-12-02
<blkadder> Hi allâ¦ Getting the following when attempting to partition a drive: https://paste.ubuntu.com/26092295/
<blackboxsw> dojordan: initial comments. I also see DatasourceOVF trying to do some marker file logic. I'll look it over to see if there are commonalities we can leverage (though unlikely needed for your branch)
