#cloud-init 2014-04-28
<sauce_> whats the best CentOS 6 AMI with cloud-init?
<sauce_> i am using Bashton's CentOS 6.4 AMI with cloud-init.  i am installing the puppetlabs yum repo via "yum_repo module," for use with installing puppet via the "puppet module".  unfortunately the yum repo doesn't get added, so the puppet install fails. any ideas?
<sauce_> i got the ubuntu version of my cloud-config working (installs puppetlabs apt repo, then installs puppet)
<sauce_> also, how do i test cloud-init manually? right now i keep on spinning up new instances to test
<smithmb> Hi folks -- I'm trying to configure chef using cloud init, and it's giving me complaints about apt-get in a stacktrace. But I'm using install_type: "omnibus" ... which shouldn't do an apt-get. Is omnibus still a valid type?
<smithmb> ah, apparently 12.04 has an earlier version of cloud-init that doesn't support omnibus
 * sauce_ patiently waits for smoser to get back
<smoser> sauce_, easiest way to do it is with lxc.
<smoser> second easiest is with kvm
<smoser> both of those are basically just faster ways than ec2 to create a "new instance"
<sauce_> smoser i think you're reading yesterday's question
<sauce_> my question today has to do with yum_repos module not kicking in
<smoser> i dont know about the yum_repo. 
<smoser> if i had to geuss, i'd guess the version of cloud-init youhave is old and doesn't support that.
<smoser> if that was wrong, i'd ask harlowja . 
<harlowja> ya, depends on the version and such
<harlowja> all smoser  fault!
<harlowja> lol
<sauce_> how about the one that ships with amazon linux AMI?
<harlowja> not sure, which one is that
<harlowja> never quite know what amazon runs :-P
<sauce_> ok let me ask you this
<sauce_> hmmm not sure how you can help me without getting annoyed, cause i am using old cloud-init
<sauce_> let me play around a bit then i'll get back to you guys
<sauce_> basically i need to set up my software repo before puppet module kicks in
<sauce_> ok next question:
<sauce_> i got the repo installed via: runcmd:
<sauce_> - rpm -ivh https://yum.puppetlabs.com/el/6/products/x86_64/puppetlabs-release-6-7.noarch.rpm
<sauce_> but unfortunately, the puppet module launched BEFORE that.  how can i set up dependencies with cloud-init?
<harlowja> u have to alter the cloud.cfg that is used, which has the sections of what is ran, bb
<sauce_> gotcha
<smoser> sauce_, don't use runcmd
<smoser> probaably your cloud-init supports bootcmd
<smoser> which happens much earlier.
<smoser> and probably you can insert the repo then
<sauce_> gotcha
#cloud-init 2015-04-27
<plathrop> Can anyone here help me understand what is broken about my cloud-init config? It is super simple: https://gist.github.com/plathrop/813c82018d5fde99959e
<plathrop> When I include this, the instance I spin up is completely inaccessible, even with the ec2 keypair
#cloud-init 2015-04-28
<jeffgus_> hi all,
<jeffgus_> is anyone else having issues with cloud-init on rhel7.1?
<jeffgus_> it doesn't seem to be processing the template correctly
<jeffgus_> as in the chef template
<jeffgus_> when i downgrade the version it works
<jeffgus_> 0.7.5-6 works
<jeffgus_> 0.7.6-2 doesn't work
<smoser> claudiupopa, what was the c-i you were showoing me earlier?
<smoser> link to github?
<claudiupopa> smoser: https://github.com/PCManticore/argus-ci
<smoser> thansk
<claudiupopa> develop branch is where the work goes atm.
<claudiupopa> np.
<smoser> JayF, on a OnMetal the smallest, what is the disk there? 
<smoser> where did you come up with a 32GB disk drive
<JayF> it's a 32GB SATADOM
<JayF> basically an SD card with a SATA port attached is the easiest way to describe it
<nrezinorn> hello, is this a good channel to ask for some quick help?  I wrote a cloud-config file that works on cent6, doesnt work on cent7 - and currently the cent7 partially works, but fails to run the 'runcmd:' section :(  
<plathrop> Is anyone here successfully using cloud-config for creating initial users on Ubuntu Trusty?
<harlowja_> nrezinorn can u describe what u are seeing, and maybe get a /var/log/cloud-init.log (that doesn't contain sensitive info)
<harlowja_> another option, get full debug log that we can use also
<harlowja_> if u put the following in your yaml u should force on debugging
<harlowja_> # Print debug output to console so that it is accessible from 'nova console-log'.
<harlowja_> log_cfgs: []
<harlowja_> something must be up :-P
<nrezinorn> sure i can do that, let me upload all my configs to GHE or something :)
<nrezinorn> i dont see anything logged after spinning things up (to verify cloud-init)
<harlowja_> hmmm
<harlowja_> ya, thats weird then :-/
<harlowja_> if u see nothing, thats odd :-/
<nrezinorn> the wierd thing to me, is it loads some parts of the config file
<nrezinorn> it drops the SSH keys, etc
<harlowja_> odd
<nrezinorn> but simply fails to run the runcmd section and its failing to run the packages section
<nrezinorn> it only fails on centos7, centos6 works perfectly!
<harlowja_> weird
<harlowja_> using that logging trick might help
<nrezinorn> give me a few to do the logging trick, then i will share what i have so you can see
<harlowja_> k
<nrezinorn> your log_cfgs has given me a ton of debug info
<nrezinorn> ill just look through it all , thanks  :)
<nrezinorn> it doesnt even try to load the modue for runcmd  , it does try bootcmd though
<harlowja_> so that makes me wondering if your /etc/cloud/cloud.cfg isn't listing runcmd
<harlowja_> if thats so, then there's your problem :-P
<nrezinorn> oh its in there :)  like is why i said im confused because my centos6 cloud-config file *should* also work on centos7 
<harlowja_> k, odd
<harlowja_> hopefully all the debug junk has something useful :)
<nrezinorn> all i can glean from it is "it didnt try to parse runcmd" 
<harlowja_> hmmm, no idea then
<nrezinorn> welcome to my last few days ;)
<harlowja_> ha
<nrezinorn> harlowja_: https://github.com/Nrezinorn/cloud-init  maybe you can spot some glaring syntax error between c6 and c7 cloud-config?
<harlowja_> kk
<harlowja_> soo the thing i do notice is that none of those logs are for the 'config' stage
<harlowja_> but only for init and init-local stage
<harlowja_> *there are 3 -4 stages of cloud-init
<harlowja_> 1) init/init-local
<harlowja_> 2) config
<harlowja_> 3) final
<harlowja_> runcmd is under http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/config/cloud.cfg#L42 (section 2)
<harlowja_> stage/section
<nrezinorn> those stages are set in  /etc/cloud/cloud.cfg for defaults?
<harlowja_> ya, but i'm wondering why u don't have logs for those other stages
<harlowja_> all of your log files end with  cloud-init mode 'init'
<nrezinorn> cloud init was intsall via rpm, default settings
<harlowja_> or running 'init-local' 
<harlowja_> ya, so for some reason yours maybe isn't running the later stages, not sure why
<harlowja_> who build that rpm :-P
<nrezinorn> let me check my KS config to see if something is messing with those cfg files after install (i dont think they are, so its a stock rpm install)
<harlowja_> redhat?
<nrezinorn> https://github.com/Nrezinorn/cloud-init/blob/master/cloud-init.txt i knew you'd ask!
<nrezinorn> i can ask the centos mail list 
<harlowja_> ya, so centos folks i guess
<harlowja_> ya, some reason its not really running the rest of the stages (it appears)
<harlowja_> which may mean its misconfiguered (systemd issues?)
<nrezinorn> this rpm was built last sept and i know our openstack has c7 images, and all the stuff works there
<harlowja_> i think http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/files/head:/systemd/ (those are the systemd scripts, but idk if they used that or something else)
<harlowja_> ya, so maybe u need to investigate if its really running those stages (systemd i think has this info, not sure)
<nrezinorn> is its nots runing the stages, that is an RPM issue or i need to fix it during kickstart
<nrezinorn> i would think an rpm install would take care of all that is needed :P
<harlowja_> ha
<harlowja_> if only :-P
<nrezinorn> im going to blame systemd here ;)  that is what people do right?  
<harlowja_> :)
 * nrezinorn stumbles though using systemd  
<nrezinorn> i really should learn this stuff.  i use arch @ home, but i never log into my system at home, pretty much ever!
<nrezinorn> loos like cloud-config service is dead.  
<harlowja_> hmm, that might be the issue :-P
<smoser> blaming systemd is always safe.
<smoser> if runcmd didn't run, i'd suspect that cloud-final didnt run
<smoser> (dont knwo if we saw that above or not).
<nrezinorn> so i looked into this more and it looks like it is a self-inflicted wound :(
<devicenull> so, before I go about re-inventing the wheel... I work for a cloud provider, and we're currently using cloud-init to handle all the provisioning required to handle machines off to customers
<devicenull> however, some of our customers want to be able to provide their own user-data scripts, which is what we're currently using to configure instances
<devicenull> is there a simple way to run multiple cloud-config scripts?
<devicenull> atm it feels like I need to move all of 'our' provisioning outside of cloud-init, so I can leave it available for the users
<smoser> devicenull, 2 options
<smoser> a.) you could consume the users's user-data and stuff it into a mime-multipart message with your parts at the beginning or the end (later trumps earlier)
<smoser> b.) vendor-data
<smoser> vendor-data was designed exactly for "cloud provider" providing stuff to cloud-init, and not interfering with user-data.
<smoser> vendor-data is supported in openstack.
<smoser> for other datasources we may have to modify cloud-init, but if you're a public cloud and willing to work a bit, that can be achieved even back into stable ubuntu.
<devicenull> ah
<devicenull> we use the ec2 metadata source, but the openstack one looks fairly compatible
<kwadronaut> it is 
<devicenull> that looks like it's a pretty good solution then
<kwadronaut> (or rather openstack offers ec2-compatibility, haven't run in corner cases in quite a while)
#cloud-init 2015-04-29
<bbradley> hello
<bbradley> i'm having trouble with cloud config apt_sources working. anyone spare some help?
<bbradley> i am using debian's cloud-init 0.7.6.
<bbradley> i'm not going anywhere so just ping me.
<bbradley> expect to bed but back after
<bbradley> it seems i'm experiencing this bug: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1355343
<smoser> bbradley, can you give exmaple of cloud-config ?
<bbradley> smoser: indeed
<bbradley> http://pastebin.com/4uDjfG77
<bbradley> and, everything else in the cloud config file works except for apt_sources.
<bbradley> i may not be experienced that exact bug. but my sources.list is not being created by cloud-init and my sources.list.d is empty.
<bbradley> experiencing
<bbradley> i am going to attempt to try using a new version and see if that helps. but it seems hard at best since the only way i see to use cloud-init is install by package.
<bbradley> meaning i have to create a package from the codebase to try.
#cloud-init 2015-04-30
<bbradley> i found a message in cloud-init-dev about building a package for cloud-init. i'll post a bug to the debian package if it works.
<smoser> bbradley, should be able to (with dependencies installed) ./packages/bddeb
<bbradley> smoser: thanks. if i have anymore trouble after testing a new version, i'm going to post to the mailing list.
<bbradley> oh my
<bbradley> i think it's because i'm quoting the value of source in apt_sources.
<bbradley> that's not it
<bbradley> but something is causing my apt_sources block to quit parsing. if i move my apt_sources block above other statements, the rest of the file is not parsed.
<bbradley> very strange
<smoser> can you show a file that is working and one that is wrong ?
<smoser> pastebin
<bbradley> http://pastebin.com/4uDjfG77
<bbradley> this one has the apt_sources statement at the bottom.
<bbradley> if i moved that statement above other statements, apt_sources & all below it do not execute.
<bbradley> move
<bbradley> smoser ^
<bbradley> it works if i remove the apt_sources statement.
<smoser> bbradley, /var/log/cloud-init-output.log ?
<bbradley> one moment
<bbradley> http://pastebin.com/xvdKJAxy
<smoser> use paste.ubuntu.com . much less obnoxious
<bbradley> sure no worries
<smoser> (also, for your info, you can 'apt-get install pastebinit; pastebinit /var/log/cloud-init.log')
<bbradley> sweet
<bbradley> http://paste.ubuntu.com/10952975/ cloud config
<smoser> so what is it that you think failed ?
<bbradley> apt_sources does not add anything to the sources.list or sources.list.d
<bbradley> nor are packages installed using using this source.
<bbradley> -using
<smoser>  paste /var/log/cloud-init.log ?
<bbradley> oih i'm on debian so pastebinit made this.
<bbradley> http://paste.debian.net/170226/
<smoser> fine with me.
<smoser> still much less obnoxious than pastebin.com . 
<bbradley> cool
<bbradley> please observe lines 708 and 709
<smoser> line 708 has warning. yeah.
<smoser> it is failing. you will need to declare the mirror for 'security' which you can just define to ftp.debian.org. or maybe None
<smoser> let me see how you could do that
<bbradley> oh ok
<smoser> ideally your debian image would have done that for you.
<bbradley> i can do that with my setup.
<bbradley> i'm using xen-tools to build images and it copies sources.list from the host machine.
<bbradley> i suppose cloud-init uses unattended-upgrades packages and needs a security package repo. is that the problem?
<bbradley> hmm
<bbradley> i have security in my sources file.
<bbradley> so you mean in the cloud config.
<smoser> http://paste.ubuntu.com/10953096/
<smoser> try adding that stanza
<bbradley> i noticed the 'security' in the warning immediately but was not sure of the context until your help just now.
<bbradley> done
<bbradley> will be a few minutes until i know if it works.
<bbradley> what do you mean by 'debian image would have done that for you'?
<bbradley> ah
<bbradley> my cloud.cfg (unedited from official package install) only has a failsafe for the primary.
<bbradley> smoser: that worked.
<bbradley> thanks a bunch
<bbradley> will file a bug with debian cloud-init.
<bbradley> the default cloud.cfg should have a failsafe security mirror in it.
<bbradley> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=783847
<nosleep77> hi there, can I inject a cloud init file into an existing running openstack instance?
<nosleep77> and then have it run of course
#cloud-init 2015-05-01
<`mx> I am having issues installing pip using the runcmd: in cloud-init. It will install and only allow the root user to execute pip. It sets the permisions on the site-packages for pip to 750. If I install it as root on the cli it will set it to 755. 
#cloud-init 2016-05-03
<mgagne> I'm trying to run cloud-init on Xenial and I'm always getting "Applying network configuration from fallback (dhcp)". On first boot, static IP is properly configured from configdrive but when rebooted, it falls back to dhcp and fails to assign an IP. (we only use configdrive, no dhcp)
<mgagne> How can I debug?
<mgagne> so it looks like network config is applied but later in stages.py, it thinks there is no datasource and fallback to dhcp ...
<mgagne> so I commented this line and now it works fine: http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/stages.py#L604
<mgagne> but it doesn't use network_data.json, it copies the network template instead
<mgagne> part of my logs: http://paste.openstack.org/show/496013/
<mgagne> so I managed to force dsmode to be 'net' instead of 'local'. Now it seems to persist the right network config but gateway is missing
<smoser> mgagne, can you give the confi drive ?
<mgagne> smoser: sure, how can I share it with you?
<smoser> so the issue is only on reboot ?
<mgagne> smoser: well, I'm concerned with the overall sanity of the process.
<mgagne> smoser: logs show it's fallbacking to dhcp and I fear other things might not work that well in the end
<mgagne> I'm uploading a new image and booting a new instance to get config drive
<smoser> https://bugs.launchpad.net/cloud-init/+bug/1575055
<smoser> that is why i'm asking about reboot
<mgagne> smoser: yea, I fixed that one already
<smoser> and your config drive has the network confi g?
<mgagne> yes
<mgagne> hold on, will find a way to share it
<smoser> https://bugs.launchpad.net/nova/+bug/1513267
<mgagne> I'm the author of the bug
<mgagne> and it's fixed already on our side
<mgagne> gimme a minute to boot an instance
<smoser> :)
<smoser> mgagne, please open a bug and i'll take al ook later.
<smoser> (put bug number here for me too please)
<mgagne> ok, what kind of info do you need? whole cloud-init.log ?
<smoser> sure thats ifne.
<smoser> and the config drive would be most useful.
<mgagne> ok, will package all that info and open bug
<mgagne> smoser: bug #1577982
<harlowja> sooo for https://raw.githubusercontent.com/mgagne/cloud-init-fedora-pkg/epel7/cloud-init-0.7.5-network-info-support.patch mgagne where can i get some sample data files for this thing :-P
<harlowja> do u have any i can use
<harlowja> or do i have to mess around with openstack to make that happen, lol
<mgagne> harlowja: I have one in my bug report BUT it's an internal implementation of the spec since upstream was taking too much time.
<harlowja> :-/
<mgagne> harlowja: so hopefully, it doesn't deviate too much from upstream
<mgagne> hold on
<harlowja> live examplessssss???
<mgagne> the one in bug report is from a real VM, not unit test
<mgagne> https://review.openstack.org/gitweb?p=openstack-infra/glean.git;a=blob;f=glean/tests/fixtures/liberty/mnt/config/openstack/latest/network_info.json;h=33f9af4a5bab5ee7b5bfb6d171681016d135958a;hb=9d493d941bbe4c36d46479c3138fd3602fc5a78f
<harlowja> glean, hmmm
<harlowja> whats that, lol
<mgagne> a project hosting fixtures for cloud-init unit tests :D
<harlowja> ah, nice, lol
<harlowja> looks sort of hacky :-P
<mgagne> what's hacky? json?
<harlowja> now, was just looking over more of glean, ha
<harlowja> *no
<mgagne> we are not here to judge =)
<harlowja> lol
<harlowja> ya, i'm just trolling :)
<harlowja> who does the judging?
<harlowja> lol
<mgagne> :D
#cloud-init 2016-05-04
<harlowja> smoser the conversion of that json into network config, looking into that, is that network config yaml made by juju somewhere at some point? and thats why the translation?
<harlowja> or maas
<harlowja> seems like  https://raw.githubusercontent.com/mgagne/cloud-init-fedora-pkg/epel7/cloud-init-0.7.5-network-info-support.patch isn't really needed (or a large part of it goes away) due to that conversion and stuffs
 * harlowja wonders why the network json stuff wasn't == this network yaml stuff ?
<mgagne> harlowja: I would like to make clear that I'm not the original author of this patch. The original version can be found here: https://github.com/jayofdoom/cloud-init-fedora-pkg/blob/master/cloud-init-0.7.5-onmetal-configdrive.patch I adapted it to support the "real" network_data.json location instead of loading it from vendor_data.json
<harlowja> jayofdoom, lol
<harlowja> nice
<harlowja> mgagne  np, there exists some kind of ubuntu network yaml stuff that the current code in cloud-init is converting network json into
<harlowja> and i'm more or less trying to figure out what that yaml thing is, ha
<mgagne> never heard of such yaml
<harlowja> cause it appears to convert network_data.json ---> in memory format closer to network yaml format --> then this gets turned into a yaml file (that then gets processed by?)
<harlowja> aka http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/net/network_state.py (this thing)
<harlowja> the end result of that thing seems to be a yaml file that is written out (which is different from the other yaml file?) that something then processes
<harlowja> just not quite sure that that thing is, ha
<harlowja> (and maybe that thing is already in rhel/fedora?)
<harlowja> or maybe its just a ubuntu thing
<harlowja> but i'm guessing smoser knows
<mgagne> I'm not sure why YAML is needed. I think it's only for debug purposes.
<harlowja> possibly
<mgagne> commit message doesn't tell much about it which is unfortunate
<harlowja> ya, so there is a conversion into a in-memory format, which then gets processed @ http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/stages.py#L596
<harlowja> which then activates a bunch of debian(?) specific output
<mgagne> it doesn't use the yaml
<mgagne> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/net/network_state.py#L120
<mgagne> debug
<harlowja> right, so maybe just for debug
<mgagne> and test http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/net/network_state.py#L391
<harlowja> so from the looks of it, http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/net/__init__.py#L590 then does all the 'rendering'
<harlowja> but it sort of looks like debian network style rendering
<harlowja> which is oddly not in the debian.py distro file, lol
<mgagne> I guess you are looking for rhel support?
<harlowja> well https://raw.githubusercontent.com/mgagne/cloud-init-fedora-pkg/epel7/cloud-init-0.7.5-network-info-support.patch has some of that :-P
<harlowja> just there is a bunch of work that exists to do stuff with this format for ubuntu, just need to figure out where the rhel stuff goes in
<mgagne> yea, but actually implementation changed a lot in upstream and looks to have made it much more debian specific than needed.
<harlowja> yup
<harlowja> i blame smoser
<harlowja> lol
<mgagne> now something hacky looks much more appealing ;)
<harlowja> lol
<harlowja> or refactor time
<harlowja> i'll bug smoser when he gets back
 * harlowja unsure how much of http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/net/__init__.py is really debian specific
<harlowja> seems like a larger part of it
<harlowja> why can't the distros just agree on a network format, lol
<mgagne> harlowja: I'm not sure why *anything* distro specific is in that file. should be in distro
<dmsimard> harlowja: I can get you a Mitaka config drive if you'd like (for rhel networking support)
<harlowja> dmsimard that'd be sweet
<harlowja> i can prob get one, but gotta check if the openstack we have is making that network json file
<harlowja> more samples i have the better :)
<dmsimard> harlowja: https://dmsimard.com/disk.config
<harlowja> thx
<harlowja> i gotta start ripping apart the cloud-init net stuff soon
<harlowja> (cause it seems ubuntu specific)
<harlowja> i mean debian
<smoser> harlowja, it does not have a rh renderer
<smoser> but we want one
<harlowja> smoser ya :-P
<smoser> and i have some other changes tat are voing to be necessary
<harlowja> just unsure what is debian specific and what isn't, ha
<smoser> relly need to change some things more to do this right.
<harlowja> ??
<smoser> the datasource searching needs to chnage some really.
<harlowja> :)
<smoser> here. i'll point you at a branch in a bit with some docs on what i want to change.
<harlowja> cool
<harlowja> smoser but ya, datasources are ummm, interesting
<harlowja> ha
<harlowja> smoser  do u want parallel searching, multiple datasource merging, something else :-P
<harlowja> or just redo the whole thing, lol
<harlowja> smoser whats up with the net stuff having references to 'Curtin' ?
<harlowja> whats 'Curtin'?
<smoser> curtin is the curt installer
<smoser> harlowja,
<harlowja> is curt a person?
<harlowja> lol
<harlowja> curt cobain
<smoser> curtin (the curt installer) is a "fast path" installer designed to install Ubuntu quickly.
<smoser> It is blunt, brief, snappish, snippety and unceremonious.
<harlowja> are u in marketing now?
<harlowja> lol
<smoser> harlowja, the goal is to move all that network stuff out into its own library
<harlowja> that'd be nice
<smoser> but under time pressure we just copied from curtin to cloud-init
<harlowja> has time pressure been fixed?
<harlowja> like if i do some rhel stuff, should i just put it in curtin?
<harlowja> (or maybe i can find someone from RH, lol)
<smoser> harlowja, heres a crappy dump of my thoughts http://paste.ubuntu.com/16222645/
<mgagne> smoser: FYI, I opened the bug: https://bugs.launchpad.net/cloud-init/+bug/1577982
<smoser> mgagne, yea, i saw. i reproduced .
<mgagne> smoser: let me know if you need access to a test instance
<mgagne> I can find a way to make that happen
<smoser> harlowja, if you do rh stuff do it in cloud-init
<smoser> and then we can move it there.
<harlowja> k
<smoser> did you read that paste above ?
<harlowja> ya, looked over a little :-P
<smoser> what do you think ?
<smoser> i think the consolidation of dsmode (basically making it go away) is a big simplification
<smoser> and also then a network datasource (such as openstack metadata) can look locally and say "yes, i'm the datasource".
<smoser> (by looking at something in dmi information)
<smoser> even though it does not have access to the network net.
<harlowja> ya, seems fair to me
<dmsimard> oh, and when you do that "rh stuff", let me know and I can test it :p
#cloud-init 2016-05-05
<harlowja> smoser the whole net stuff doesn't really have any tests or sample files and sample outputs :(
<harlowja> seems like i'll have to rewrite alot of it for RH support, but no tests makes that ya, ummm, painful
<harlowja> dmsimard mgagne smoser sooo first thing i am hitting (in trying to add some basic tests for the conversion process); is that openstack (or at least the example i got from dmsimard has a network type of bridge, but the cloud-init code doesn't handle any bridge type)
<dmsimard> that's probably insight from the compute node, there are no bridge to create on the VM
<dmsimard> in the specific case I sent you, it's a linuxbridge config -- so it goes something like "eth0" -> br-something -> tap-something
<dmsimard> tap-something is the interface of the VM
<harlowja> right, so that example gets processed via http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/sources/DataSourceConfigDrive.py#L369
<harlowja> into an internal format that is different from the network_data.json one, but has similar info
<harlowja> guess i gotta figure out what to do with bridges
<dmsimard> can you paste the specific config you're looking at ?
<harlowja> i'll do better, ha
<harlowja> http://bazaar.launchpad.net/~harlowja/cloud-init/cloud-init-net-refactor/revision/1216
<harlowja> is the stuff i'm working on, cause there are no tests really of the network format in and expected out
<harlowja> http://bazaar.launchpad.net/~harlowja/cloud-init/cloud-init-net-refactor/revision/1216
<harlowja> oops look @ test_configdrive.py  there
<harlowja> so thats my step 1, adding some basic tests of this conversion process, so i can then go figure out what the heck to do for rhel in the other net files (which also lack tests afaik)
#cloud-init 2016-05-06
<harlowja> smoser https://code.launchpad.net/~harlowja/cloud-init/cloud-init-net-refactor/+merge/293957
<harlowja> some refactors for today, ha and tests
<harlowja> another one of wtf is josh refactoring  merges :-P
#cloud-init 2017-05-01
<blackboxsw> Looks like somebody else has an openbuild cloud-init up with some stale package references. https://build.opensuse.org/package/show/Cloud:Tools/cloud-init
<rharper> interesting
<powersj> bah launchpad failed to create a diff...
<blackboxsw> smoser, this bug is linked as sru https://bugs.launchpad.net/cloud-init/+bug/1644064 but the related branch is "Needs Fixing" what should it be
<ubot5`> Ubuntu bug 1644064 in cloud-init (Ubuntu) "sshd_config file permission changed to 644 if ssh_pwauth value is true or false" [Undecided,New]
<blackboxsw> looks like all review comments were addressed
<blackboxsw> just not merged
<powersj> rharper: thanks for the 2nd review
<blackboxsw> powersj, sure, anyday (including today works well for me)
<blackboxsw> powersj, sure. Anyday .including today, works well for me.
<blackboxsw> geez I give up on English
<powersj> blackboxsw: Today works for me as well
<powersj> lol
<powersj> just tell me where
<blackboxsw> ok I'm just going through sru bugs to create a template.
<blackboxsw> and trying to reproduce failing tests
<blackboxsw> powersj, how's 3:30 sound, someone might drop a delivery by the house sometime between now and 3:30.
<blackboxsw> I can go anywhere (your choosing as last time we didn't have internet0
<powersj> blackboxsw: 330pm dazbog harmony/timberline
<blackboxsw> done
<smoser> blackboxsw, i suspect its fix-released in artful
<smoser> looking. i just marekd it fix-committed in cloud-init
<blackboxsw> smoser, thanks, does this bug need to be SRU'd? https://bugs.launchpad.net/cloud-init/+bug/1636531  it's just unit tests?
<ubot5`> Ubuntu bug 1636531 in cloud-init (Ubuntu) "unittests blkid command fails on slave s390x" [Undecided,New]
<blackboxsw> which aren't included in the deb anyway
<nacc> blackboxsw: probably we want the unit tests to pass on older releases?
<nacc> blackboxsw: unclear, i guess
<nacc> blackboxsw: although the 'fix' is a jenkins change?
<blackboxsw> well it's a unit test not leaking blkid calls change which could fail on a variety of systems
<blackboxsw> though I just found other leaks in testing the fix
<blackboxsw> but not really part of this bug.
<nacc> blackboxsw: ok, yeah, i guess it's not clear to me, i was just looking at powersj's response
<blackboxsw> yeah I'm with you nacc, I'm not quite certain why that makes an SRU cut. unless we have ungoing jenkins runs against proposed/released cloud-init too
<blackboxsw> which I guess is a goal
<powersj> if I recall smoser has SRU'ed test fixes before
<nacc> powersj: yep, we have
<powersj> as I remember doing SRU versification on them
<nacc> powersj: i'm just not sure here if the fix is in curtin but really a jenkins change?
<blackboxsw> powersj, how'd you verify unit tests as they aren't delivered in the deb produced. did you have to git clone the repo and attempt to run the tests again?
<powersj> There were two fixes, 1) the jenkins path change, so no action here, and 2) smoser made a new branch with some other changes related to this issue to prevent it in the future
<nacc> powersj: ah i see
<blackboxsw> yeah the part 2) is what I was wondering about. as it is just unit test mocks
<powersj> blackboxsw: pull-lp-source cloud-init <release>; cd cloud-init-*; run tests
<blackboxsw> heh nice
<blackboxsw> TIL ^
<nacc> powersj: didn't you have a test that did this specifically in jenkins?
<nacc> powersj: that is, ran the unit tests (or vmtests?) for a given release from the src for that release?
<powersj> blackboxsw: of course if you do it that way make sure to specify the version so they know it lines up and make sure to use release-proposed in the command
<powersj> nacc: yeah we have the proposed tests that run when cloud-init is there.
<powersj> it does the same thing
<nacc> powersj: yep, ok
<powersj> nacc: to be clear though, the jenkins job runs the integration tests
<nacc> powersj: we have so many new curtin/cloud-init jenkins jobs, i often don't know what does which :)
<nacc> powersj: ah ok
<powersj> so not unit tests, but maybe I should add those anyway
<powersj> nacc: I know :( I'm not sure how to clean them up either
<nacc> powersj: unit tests seem like a self-consistency check that's worthwhile
<powersj> I'm glad we have expanded coverage of things, but it isn't always clear
<smoser> blackjid, regarding "how'd you verify unit tests..."
<smoser> there is no ubuntu deb if the unit tests didn't run
<smoser> as they're run as part of build
<powersj> blackboxsw: https://paste.ubuntu.com/24495122/
<blackboxsw> thanks didn't realize disconnected from freenode
<blackboxsw> ok thanks, yeah I see unit tests being run in the build recipe before the package build. ok https://launchpadlibrarian.net/317312745/buildlog_ubuntu-xenial-amd64.cloud-init_0.7.9-1495-g2796dab-0ubuntu1+1267~trunk~ubuntu16.04.1_BUILDING.txt.gz
<smoser> blackboxsw, yeah..i was going to find al ink to that
<smoser> but for some reason... launchpad.net isn't resolving for me
<smoser> fun
<powersj> smoser: can you add artful to https://launchpad.net/~cloud-init-dev/+archive/ubuntu/test-archive
<powersj> and https://launchpad.net/~curtin-dev/+archive/ubuntu/test-archive
<smoser> not until i make launchpad.net resolv :)
<powersj> hahaha
<powersj> sorry just read that
<smoser> http://paste.ubuntu.com/24495160/
<smoser> someone really doesnt want me to et an ip for launchpad.
<smoser> (iv'e restarted network manaer, and killed the dnsmasq process that it is running)
<smoser> but 'host launchpad.net 8.8.8.8' should be asking 8.8.8.8 directly
<smoser> powersj, ok.. so added launchpad.net to /etc/hosts
<smoser> i can do that, but i
<smoser> i am pretty sure you could have too
<smoser> "Package details"
<smoser> "copy packages"
<powersj> for curtin maybe, I have no rights on cloud-init
<powersj> ah ha! ok I see you already did it for curtin
<smoser> the key thing to remember is that when you do it..
<rharper> smoser: if you're at a sprint, sometimes you need to push 8.8.8.8 infront of your caching nameserver; I've had to do that if DNSSEC was enabled, if youre on zesty or artful, systemd-resolvd tries to use DNSSEC
<smoser> *copy existin binaries*
<powersj> ah so no rebuild
<smoser> rharper, see my paste
<smoser> i'm on xenial here.
<smoser> but even directly asking (host launchpad.net 8.8.8.8)
<smoser> didn't work
<smoser> i'mi pretty sure that is directly asking 8.8.8.8 for launchpad.net
<smoser> so something (i think) is udp intercepting
<rharper> yes
<rharper> they intercept dns
<smoser> and has a negative cache for that.
<rharper> locally
<rharper> for example, they deny phone updates so there's no phone storm of updates on the network
<rharper> there are a list of other DNS hijacks via the sprints AP
<rharper> I used to run DNSSEC enabled local cache and it *always* broke at the sprint
<rharper> not quite sure why launchpad.net would be busted
<smoser> oh. i'm not on canoncial
<smoser> on mariott
<rharper> oh well, they could do the same thing at the hotel
<smoser> yeah. but seems still busted to cache a negative result
<rharper> ie, hijacking dns for shit n giggles
<rharper> if you bring up vpn, it works right ?
<smoser> well, i think i have vpn sending only vpn traffic
<rharper> you can flip the bit to send it all
<smoser> yeah
<rharper> which would direct dns through vpn (at least temp)
<smoser> but i assume this problem will go away at some point
<smoser> and i just put /etc/hosts entry in
<rharper> well, it does, until you forgot you put in there and then at some point in the future you freak out again
 * rharper did that with diglett, bastion and other hosts
<rharper> why u no connect! ?!
<nacc> rharper: i think DNSSEC has been deisabled again, right?
<rharper> in systemd-resolvd, I think so
<smoser> right.
<rharper> or resolved
<nacc> rharper: yeah, i got a prompt at some point last week
<rharper> flippin spelling
 * rharper is playing with pihole 
<rharper> putting that rpi2 to work
<blackboxsw> generally speaking are these the typical things we ask for when users submit bugs against cloud-init http://pastebin.ubuntu.com/24495391/?
<smoser> blackboxsw, all of /var/lib/cloud/ is good too
<smoser> and /var/log/cloud-init-output.log
<smoser> and /me goes away
<blackboxsw> +1 smoser
<powersj> blackboxsw: https://paste.ubuntu.com/24495434/ is what our integration tests pull after each test
<blackboxsw> thx ppo
<blackboxsw> thx powersj
<powersj> smoser: thanks for the ppa updates
<blackboxsw> ok updated bug filing request https://bugs.launchpad.net/cloud-init/+filebug
<blackboxsw> just to request the info we care about when someone files bugs
 * blackboxsw relocates home after picking up some food 
#cloud-init 2017-05-02
<blackboxsw_bbl> smoser: I was trying to think of the best way to test/validate a DigitalOcean fix for https://bugs.launchpad.net/cloud-init/+bug/1676908 . WDYT about testing an lxc w/ overwritten /sys/class/dmi/id/system_vendor == "DigitalOcean"?
<ubot5`> Ubuntu bug 1676908 in cloud-init (Ubuntu) "DigitalOcean network improvements" [Undecided,Fix committed]
<smoser> blackboxsw_bbl, no easy way to do that. you'd have to mount bind over /sys to do it i think
<smoser> i'd just launch on digital ocean is the easiest thing. (i still need to get the ds-identify unit tests in)
<blackboxsw_bbl> trying to think how better to test (other than spinning up a DO droplet).
<blackboxsw_bbl> yeah
<blackboxsw_bbl> ok
<smoser> that is non-trivial for sure.
<smoser> we could/should bother utlemming to help
<smoser> and ask for him to give template and such
<blackboxsw_bbl> thx again. yeah /me â¤ï¸'s nontrivial and trying to generate the failure case currently to prove it works once we see the fix
 * blackboxsw_bbl needs to head out for the night. see you all tomorrow 
<blackboxsw> hi smoser: https://bugs.launchpad.net/cloud-init/+bug/1673637 has some updated customer comments that point to this sru bug not quite resolving things for them. How should we proceed for this SRU-related bug?
<ubot5`> Ubuntu bug 1673637 in cloud-init "cloud-init - Hosts in softlayer receiving warning" [High,Confirmed]
<ragechin_> From the "It's probably staring me in the face" department - is there a public changelog available?
<powersj> ragechin_: is there a particular version of cloud_init in Ubuntu you are looking for or are you after the git log?
<ragechin_> 0.7.9.
<ragechin_> It's been injested into RHEL and, consequently, overrides ifcfg-eth0;
<ragechin_> I'm trying to debug that.
<ragechin_> If not, that's fine. I'll just dig directly into the code.
<powersj> so 0.7.9 is the version that is under development, so you can look at the master git log here https://git.launchpad.net/cloud-init/log/?h=master
<powersj> If I recall there is the yum-changelog command you can use as on RHEL, but it has been a while...
<ragechin_> the topic suggests otherwise.
<powersj> ah you are right, getting my numbers mixed up
<powersj> here was the announcement for 0.7.9 https://lists.launchpad.net/cloud-init/msg00057.html
<ragechin_> No worries.
<ragechin_> Thanks. I found the section of code that I'm concerned with.
<ragechin> Anyone around that's intimately familiar with the config options?
<ragechin> There's an undocumented key under 'system_info', and I'm trying to determine what options are available for it.
<ragechin> ('network', specifically)
<rharper> ragechin: yes, working on getting the docs landed
<rharper> lemme get the PR link
<ragechin> Thanks.
<rharper> https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/321397
<rharper> that's not the easiest to read, but the network_config.rst in there is pretty comprehensive
<rharper> feel free to leave any feedback or ask questions on what's not clear in here or in the PR
<ragechin> I'm skimming, so I may have missed it..
<ragechin> I don't see an option to disable cloud-init's network config stuff.
<ragechin> Found it
<ragechin> To be clear, tha'
<ragechin> That's still under system_info? So system_info/network/whatever, correct?
<rharper> ragechin: network: disabled
<rharper> sorry, network: {config: disabled}
<rharper> there's a section, Dsiabling Network Configuration
<rharper> can be done via kernel command line or config,  echo "network: {config: disabled}" > /etc/cloud/cloud.cfg.d/10-disable-networking.cfg
<ragechin> rharper: Yeah, found it, thanks
<smoser> blackboxsw, readin
<smoser> blackboxsw, reading
<smoser> blackboxsw, so i should probably open another bug
<smoser> as i think what is happening is it is fixed, but the warnings do not stop showing on upgrade.
<blackboxsw> ahasenack: when you validated https://bugs.launchpad.net/nova-lxd/+bug/1673411 for previous sru did you end up using make-configdrive-dir script that was attached?
<ubot5`> Ubuntu bug 1673411 in nova-lxd (Ubuntu Yakkety) "config-drive support is broken" [Medium,Triaged]
<ahasenack> blackboxsw: no, I used another one from a tools or sru branch, let me find it
<blackboxsw> I was trying to glean the intent behind https://launchpadlibrarian.net/314573877/make-configdrive-dir but that attached file doesn't seem to be rendered properly
<blackboxsw> ahh
<blackboxsw> gotit ahasenack
<ahasenack> blackboxsw: you found it?
<blackboxsw> /chad.smith@git.launchpad.net/~smoser/cloud-init/+git/sru-info
<blackboxsw> yeah in the tool directory.
<ahasenack> I found the script, but I lost the branch
<ahasenack> yeah
<blackboxsw> ok thanks
<smoser> blackboxsw, did you get what you needed before ?
<ahasenack> blackboxsw: right, that one
<smoser> basically i think that bug is fixed.
<smoser> but we never clear up the .warning files
<blackboxsw> smoser thanks for the message before on the earlier bug. Yeah, I'm not quite certain how we know what was broken on the followup comment as it sounded like it was a fresh install at a later date. let me dig up that comment
<blackboxsw> smoser: https://bugs.launchpad.net/cloud-init/+bug/1673637/comments/7
<ubot5`> Ubuntu bug 1673637 in cloud-init "cloud-init - Hosts in softlayer receiving warning" [High,Confirmed]
<blackboxsw> so what about that comment tells you it's an upgrade which hadn't cleaned up a previous warning?
<smoser> the version he lists does not have this fix
<smoser> ight ?
 * blackboxsw needs to check my reading comprehension
<blackboxsw> ahh
<blackboxsw> ok me goes back to the git logs to match fixed version versus bug report
<smoser> new version is 0.7.9-113-g513e99e0-0ubuntu1~16.04.1
<smoser> old version is 0.7.9-90-g61eb03fe-0ubuntu1~16.04.1
<blackboxsw> +1 thanks smoser
<blackboxsw> smoser: trivial for get-proposed-image for zesty? http://paste.ubuntu.com/24500525/
<blackboxsw> strike that
<blackboxsw> just a sec
<blackboxsw> smoser this rather: http://paste.ubuntu.com/24500540/    zesty no longer publishes img files named *-disk1.img
<blackboxsw> http://cloud-images.ubuntu.com/daily/server/zesty/current/
<smoser> yeah. blackboxsw it already knew that... just didn't have an entry for zesty
<smoser> see line 6
<smoser> so it already *should* work with zesty, youjust gave a better error message and broke arful
<smoser> (i htink)
<blackboxsw> oops true
<blackboxsw> hmm why did that break me. I must've typo'd
<blackboxsw> hmm smoser  powersj rharper what's the best way for me to create a nova-lxd image containing proposed cloud-init content for testing?
<blackboxsw> I was going down the route of something like http://pastebin.ubuntu.com/24501031/
<blackboxsw> but I'm not sure that's correct
<smoser> blackboxsw, you could use the lxd proposed script there
<smoser> and then export it
<rharper> lxc init ubuntu-daily:<release> test;  mount-image-callback --system-mounts lxd:test chroot _MOUNTPOINT_ /bin/bash -c 'script to upgrade to proposed'
<smoser> and that should be importable/uploadable to nova-lxd
<smoser> https://git.launchpad.net/~smoser/cloud-init/+git/sru-info/tree/bin/lxc-proposed-snapshot
<smoser> rharper, ^ fyi, that is what you do.
<smoser> what you said... that just does it.
<rharper> smoser: thanks
<rharper> I think you could also add -proposed, update cloud-init; rm -rf /var/log/cloud-init* /var/lib/cloud/{data,instance*,scripts,sem}; and reboot the instance
<blackboxsw> ok will try loading that lxc-proposed-snapshot up as an lxd image on serverstack.
<blackboxsw> rharper: ahh good thoughts as well, so I really wouldn't have to actually use an openstack cloud for this , just lxcs locally?
<rharper> well
<rharper> no, you want nova-lxd I think since we need to see that it exports something into the ENV
<blackboxsw> I'd prefer the former (loading them up in openstack and checking there)
<blackboxsw> yeah in /proc/1/environ
<rharper> blackboxsw: I was manually telling you what smoser already has a script for
 * rharper relocates back hom
<blackboxsw> gotit thx
<blackboxsw> hmm didn't work the instance hasn't come up on serverstack. let me get a trace of what I did so you can tell me what went wrong.
<blackboxsw> http://pastebin.ubuntu.com/24501137/  rharper smoser I was able to export the zesty proposed image I created  via 'lxc image export' and i attempted to upload it into our openstack cloud as --disk-format raw
<smoser> raw is wrong file type for nova-lxd
<blackboxsw> the main difference I see vs a working lxd image is that disk-format is reported by the working image as root-tar
<blackboxsw> http://pastebin.ubuntu.com/24501149/
<smoser> but i dont know what the right format is.
<smoser> nova-lxd also does support squashfs
<smoser> oh shoot.
<smoser> shoot shoot.
 * blackboxsw is trying to figure out how to get that root-tar type
<smoser> so lxd export is going to give you a tarball of a root.tar and then also the metadata
<smoser> and you only want the root.tar to go into nova-lxd
<smoser> (as it adds the metadata bits)
<smoser> i think
<smoser> so take the export output of lxd
<blackboxsw> ahh so I might need to extract/prune then load
<smoser> and extract it. i think you'll see 2 files
<smoser> or what ever.
<smoser> and one is root.tar.xz
<smoser> i think.
<smoser> yeah.
 * smoser has to run
<blackboxsw> have a good one
<blackboxsw> that worked, thanks smoser. The following snippet worked http://pastebin.ubuntu.com/24501524/
<blackboxsw> ok three bugs left to template #1682871 #1673637 #1676908
#cloud-init 2017-05-03
<paulmey> https://code.launchpad.net/~paul-meyer/cloud-init/+git/cloud-init/+merge/323532
<paulmey> found and fixed a bug...
<paulmey> bug # 1687712
<blackboxsw> thx for that paulmey. I'm end of day now. but I'll get you a thorough review on this tomorrow if it isn't reviewed before then.
<blackboxsw> forgot to mention powersj  I found an existing project registered over in opensuse build service https://build.opensuse.org/package/show/Cloud:Tools/cloud-init
<blackboxsw> for when we start looking at trying to auto-gen RPMs
<powersj> ah yes I did see this the other day. Looks like someone setup cloud-init builds for suse
<blackboxsw> not sure how we'd coordinate w/ that (or avoid stepping on toes). But yeah it looked like it was about ~2 months old
<powersj> +18 patches
<blackboxsw> yeah lots of patch love
<blackboxsw> some of those I checked against our master and saw fixes already landed
<smoser> blackboxsw, powersj i signed up for fedora account yesterday (actually had one already, as i'm cool like that)
<smoser> signed up for copr
<smoser> i think copr is right for centos/fedora/rhel rpms
<smoser> and then suse build for suse
<smoser> thoughts?
<blackboxsw> smoser: yeah that sounds good to me. I signed up for one end of last week as I'm not cool like that
<blackboxsw> it's doable.
<blackboxsw> both looked viable
<blackboxsw> and copr is probably best in class for RHEL centos anyway (though it didn't seem to have a *ton* of projects registered if I recall)
<blackboxsw> smoser, I've setup a digital ocean account and am spinning up a couple test instances to validate failed behavior of https://bugs.launchpad.net/cloud-init/+bug/1676908.   Will wrap up these SRU templates today I hope
<ubot5`> Ubuntu bug 1676908 in cloud-init (Ubuntu Zesty) "DigitalOcean network improvements" [Medium,Confirmed]
<smoser> blackboxsw, i can just bother utlemming and ask him for help on that
<smoser> and i'm perfectly fine to have him veriy
<smoser> this is quite reasonable. he opened bug, he mostly did work, his company benefit..
<blackboxsw> smoser: +1 on bothering BHoward :)
 * smoser pings utlemming
<utlemming> smoser: here....I thought I was hanging out, but was disconnected :)
<smoser> utlemming, thanks
<smoser> blackboxsw, bother utlemming
<blackboxsw> hi ben, time to bother you about https://bugs.launchpad.net/cloud-init/+bug/1676908
<ubot5`> Ubuntu bug 1676908 in cloud-init (Ubuntu Zesty) "DigitalOcean network improvements" [Medium,Confirmed]
<utlemming> yup
<blackboxsw> I was going to try setting up a test config droplet in DO that'd reproduce the problem you fixed with your branch
<smoser> utlemming, we'd like for you to do the sru template and verify fix
<smoser> as your knowledge of digital ocean surpasses even blackboxsw's (who signed up more than 8 hours ago)
<blackboxsw> but wanted to get some guidance on a general test config/setup that'd expose this original proble
<utlemming> I'd be happy too...
<utlemming> smoser: any chance that https://code.launchpad.net/~utlemming/cloud-init/+git/cloud-init-1/+merge/323273 could be included?
<utlemming> that is the root cause on another bug....let me fetch that for you
<utlemming> smoser: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1681531
<ubot5`> Ubuntu bug 1681531 in cloud-init (Ubuntu) "networking.service fails on ifup if networking configured via cloud-init" [Undecided,Incomplete]
<blackboxsw> yep, I've created my first droplet and it looks like your initial changes from #1676908 are already present http://paste.ubuntu.com/24505957/
<blackboxsw> so DO pulls from xenial-updates
<blackboxsw> wow spelling auto-correction by irccloud. hmm don't think I like that
<blackboxsw> DO droplets pull from xenial-updates and I see part of ben's changes already present.
<utlemming> blackboxsw: we're using the official images, but that is a dpkg-divert because...welp, broken
<smoser> blackboxsw, of course they pull from -updates
<smoser> ah.
<smoser> hm..
<blackboxsw> AMheh.
 * blackboxsw steps out for an early lunch
<ragechin> rharper: Do you have that documentation PR from yesterday handy? I seem to have lost it.
<rharper> i do
<rharper> https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/321397
<rharper> ragechin: feel free to nudge smoser to land it; then it will get published on cloud-init's read-the-docs
<ragechin> THanks.
<smoser> utlemming, why wouldn't you fix your metadata on digital ocean?
<smoser> essentially  https://code.launchpad.net/~utlemming/cloud-init/+git/cloud-init-1/+merge/323273 looks like the MD is provding a gateway that you're saying doesnt work.
<utlemming> smoser: checking, on that
<utlemming> but I would still like to ignore the second nics
<utlemming> for the purpouses of a gateway
<smoser> :)
<smoser> utlemming, do you agree that the information provided to cloud-init is kind of broken ?
<smoser> why would you have gateways on two nics?
<ragechin> DHCP assigned addresses on both NICs
<ragechin> see: AWS
<utlemming> smoer: yup, I'm getting the history on that now
<ragechin> hey smoser, any chance you can push this along please?
<ragechin> https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/321397
<paulmey> blackboxsw, smoser, I updated https://code.launchpad.net/~paul-meyer/cloud-init/+git/cloud-init/+merge/323532
<utlemming> smoser: so yeah, its wrong, but we're going to have use a different version of the json in order to fix that
<paulmey> cleaned up tests, more specific assertions and added warnings for unused options
<smoser> paulmey, thanks.
<smoser> utlemming, ok, can you open a bug and provide info, then we comment in the MP on what we're doing
<smoser> because otherwise i'm going to wonder WTH IS GOING ON in 6 months when i look at that again.
<smoser> rharper, on https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/321397
<smoser> hmm.
<blackboxsw> paulmey: looking now, thanks
<smoser> i'll just make some commits and sugest them, rharper
<rharper> smoser: ok, more since the last time
<rharper> I pulled your PR and fixed what you suggested
<smoser> yeah. just some stuff i thin is confusing. its ard to type
<rharper> sure
<utlemming> smoser: updated the commit message to refence the bug, and updated https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1681531
<ubot5`> Ubuntu bug 1681531 in cloud-init (Ubuntu) "DigitalOcean DS defines mutliple gateways via meta-data" [Undecided,Incomplete]
<blackboxsw> utlemming: are you generating an SRU template for https://bugs.launchpad.net/cloud-init/+bug/1676908  too ?
<ubot5`> Ubuntu bug 1676908 in cloud-init (Ubuntu Zesty) "DigitalOcean network improvements" [Medium,Confirmed]
<blackboxsw> smoser: I'm not really sure what the intent of extra_opts is in fs_setup. I see no docs about it in https://cloudinit.readthedocs.io/en/latest/topics/examples.html#disk-setup and the behavior of the code seems to lead me to think any functionality could have been handled by setting fs_setup: cmd: to include everything someone wants to do.
<utlemming> blackboxsw: fwiw, with out my other patch requested, the SRU will fail.
<utlemming> blackboxsw: both have been SRU'ified
<utlemming> although I seem to have been kicked out of all my LP groups, so I can't even nominate the bugs
<blackboxsw> excellent utlemming
<blackboxsw> or should I saw darkmuggle
<blackboxsw> say*
<utlemming> either :)
<smoser> rharper, http://paste.ubuntu.com/24506946/
<smoser> see HELPME comments ...
<rharper> ok
<smoser> rharper, does that stuff make sense ?
<smoser> (the patch there)
<smoser> utlemming, thanks.
<smoser> paulmey, htanks
<rharper> smoser: yes;  extra docs for helping folks
<rharper> working them now
<blackboxsw> paulmey: minor comments added to your proposal. thanks
<paulmey> blackboxsw: implemented all your suggestions. Thanks for reviewing!
<blackboxsw> np paulmey
<rharper> smoser: merged and re-pushed to branch;
<anticw> i have vm images i want to use on openstack, ec2 and locally ... when using them locally there is nothing for cloud-init to talk to (which is fine)
<anticw> however, i can't seem to make cloud-init time-out promptly or reliably
<anticw> seeing a datasource and seting timeouts, etc i still see it retrying for 120s (default which i thought i changed)
<rharper> anticw: you may want to use the NoCloud datasource locally instead of modifying the timeouts;
<rharper> anticw: if you want to debug the timeout, getting /var/log/cloud-init.log along with the cloud-config you seeded in the image will help us debug
<anticw> rharper: how does cloud-init know to use NoCloud on boot?
<rharper> fileysstem label
<rharper> http://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html
<rharper> cidata is the volume label
<anticw> cloud-drive?
<rharper> yeah
<anticw> but i 100% don't want that
<anticw> why can't it just give up mucking about after 5s or so?
<rharper> or, if you want to embed it directly in the image, you write files to /var/lib/cloud/seed/nocloud/{user-data,meta-data}
<anticw> in ec2 it will easily get a response before then
<anticw> no, i want the same images in all 3 envs
<anticw> the exact same bits
<rharper> but if you're not on ec2, the cloud-config won't apply
<anticw> but then it delays usable boot times for 120s or more
<anticw> VMs locally boot in about 2s or so
<anticw> with it ... well, it's 125s+
<anticw> (boot meaning to the point i can login)
<rharper> right, if you use NoCloud or ConfigDrive, with the same user-data, then cloud-init will find those and run them;
<anticw> and if it can't find them ... can i not have it timeout?
<rharper> if you supply the same user data to an ec2 instance, it will find the ec2 metadata service and consume the same user-data and run the same paths w.r.t cloud-config
<anticw> i mean, i could also have an init process that looks to see if it's still there after 10s and shoots it
<anticw> i must be missing something simple sorry
<rharper> cloud-init is attempting to find a source of data;  if you don't provide one, it currently assumes you *might* be on an ec2 instance;  historically it;s had a long timeout for various reasons;
<rharper> I'm suggesting that you keep the same image, but provide the user-data via a config drive, so cloud-init can find it when you boot locally
<anticw> and if i don't want config drive but would rather a timeout?
<rharper> that will provide the same results w.r.t applying user-data (ie, set this password, import this key, etc)  as it would if you booted the image in ec2
<rharper> the default timeout is long, as you know; changing that requires modifying the image
<rharper> embedding some cloud-config in /etc/cloud/cloud.cfg.d/  for the EC2 datasource
<rharper> there's a max_wait: value; I'd need to read the code a bit more to write up the details
<rharper> even if your time out, no user-data will be applied
<rharper> anticw: do you want the image to not run cloud-init locally ?
<anticw> i tried setting a max_wait without success; googling i find others struggling with this too
<anticw> rharper: locally i do NOT want cloud-init right
<rharper> if so, you can pass a kernel parameter to disable cloud-init altogether
<anticw> but kernel parms come from the boot loader ...
<rharper> yes, I see your concern here
<anticw> if i can't reduce the 120s via config i can just hack the code i guess
<rharper> well, I'll file a bug on ec2 timeout docs;  there isn't much there
<anticw> i could probably put a metadata server at 169.254.169.254 if there was a suitable reply that meant 'go away' but i'm not sure there is
<anticw> rharper: there are docs/examples there but i can't get them working, likely it's me
<rharper> anticw: in trunk cloud-init, the ds-identify feature will not run cloud-init unless it detects a local datasource; but on ec2 it woudl detect it's ec2 and enable itself
<rharper> this is on it;s way back to previous releases (in zesty, yakkety, and eventually xenial)
<anticw> this is ub1604 current upstream so likely not in that
<rharper> not yet
<rharper> you can modify it to use the strict policy now, if you like though
<anticw> i could pattch and make a new pkg
<rharper> you only need a new config file
<anticw> patch even
<anticw> oh?
<rharper> % cat etc/cloud/ds-identify.cfg
<rharper> policy: search,found=first,maybe=all,notfound=disabled
<rharper> ci.datasource.ec2.strict_id=true
<rharper> for 1604 with cloud-init version: 0.7.9-90-g61eb03fe-0ubuntu1~16.04.1
<rharper> which is in xenial-updates
<rharper> that policy file will force cloud-init to ensure it finds a datasource or it won't run
<rharper> this means for a local boot, without config drive, it will skip cloud-init running altogether
<rharper> but the same image with config drive, or booted in ec2, will run
<rharper> does that get you what you need?
<anticw> i think it might, waiting for it to time-out to test
<rharper> you might be interested in mount-image-callback tool which let's you mount and modify the image without booting it
<rharper> in the cloud-image-utils
<anticw> i can modify images just fine
<anticw> the goal though is to have a base images that is *not* modified
<anticw> i've had to mount the zvol more than a couple of times locally to unfark stupid things i did
<rharper> hehe
<anticw> rharper: i created ds-identify.cfg as you showed above but it cloud-init still runs
<anticw> let me check i didn't goof
<rharper> are you running the cloud-init version I pasted above ?
<anticw> oh, i see ... i actually assumed i was but didn't check, sorry
<anticw> that's a git version id in there so not release tag, my guess is not
<rharper> ok, if you apt update && apt install cloud-init, it'll upgrade to the newer version which has ds-identify available for use
<anticw> it's in the apt-repo very recently?
<rharper> for a few weeks or more
<anticw> i do have it
<rharper> ok, then we can look at /run/cloud-init/ds-identify.log
<rharper> the default behavior without the ds-identify.cfg is to always run (as it has in the past); the policy file should direct it to not run unless it detects a local datasource (or a some other cloud identifier, for say ec2)
<anticw> https://pastebin.com/5xdMeUE1 && https://pastebin.com/S5HMyVTW
<anticw> maybe for ec2
<anticw> "1 datasources returned maybe: Ec2"
<rharper> ack, looking
<rharper> anticw: do you have any Ec2 cloud-config in any of the /etc/cloud/* subtree ?
<anticw> no
<rharper> hrm, I have a plain 1604 image with that same policy loaded, but it doesn't return maybe for ec2 ... trying to figure out why yours is...
<anticw> i did alter cloud.cfg to try and get things going and also to take out the apt modules, because with those in place source.list gets wrecked
<anticw> i can [re]test with the stock cloud.cfg likely
<rharper> that wouldn't affect it
<rharper> if you're on your instance now;, you can remove /run/cloud-init/.ds-identify.result and then  sh -x /usr/lib/cloud-init/ds-identify &> ds.log;  that'll re-run ds-identify with execution tracing, and I can see how it got a maybe
<anticw> --force?
<anticw> otherwise cached
<rharper> I think the dot file is the cache test
<rharper> but you can also pass --force, won't hurt
<anticw> so this time i got: 2 datasources returned maybe: OVF Ec2
<anticw> pastebin ds.log for you?
<rharper> please
<rharper> =/
<rharper> I think maybe OVF is expected ...   and I've been down this path before;  /me triple checks the contents of ds-identify.cfg
<anticw> https://pastebin.com/Uz7hPdjZ
<rharper> you need either the --force or rm /run/cloud-init/.ds-identify.result
<rharper> the paste was from that cached run
<anticw> oh sorry, i did it 2x and forgot to remove it the second time
<anticw> rharper: https://pastebin.com/8jTnqpXX
<rharper> k
<blackboxsw> smoser: last SRU bug template for softlayer. For testing, shall I  make-configdrive-dir, but remove the dated directory from the lxd  leaving only the latest directory. Then I could run ds-identify and make sure it still finds the content?
<blackboxsw> the bug I meant to reference: https://bugs.launchpad.net/cloud-init/+bug/1673637
<ubot5`> Ubuntu bug 1673637 in cloud-init "cloud-init - Hosts in softlayer receiving warning" [High,Confirmed]
#cloud-init 2017-05-04
<smoser> blackboxsw, i think that sounds fine.
<smoser> anticw, "in ec2 it will easily get a response before" is not really true. the annoying timeouts are there for good reason (although possibly historical)
<smoser> anticw, so right now if you take a stock image of 16.04, the behavior you are seeing is what is expected.
<anticw> smoser: it would be useful to perhaps put timing detais into the logs so we can get some census on how long it takes in ec2
<anticw> but IME of late, vm's from launch (create, api call) to login are often up in 20s (to login)
<blackboxsw> rharper: thanks for the review on the make deb changes. I've copied your comments over into https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/323088 as they are appropriate to that proposal instead. Per your suggestions it sounds like our approach really should be an recommends package for cloud-init on libyaml-0-2 in debian/control
<blackboxsw> and our try: import CSafeLoader  except ImportError:  import SafeLoader as a fallback
<rharper> blackboxsw: bah, sorry I put it on the wrong one
<blackboxsw> then we can have unit tests which exercise both options if available
<blackboxsw> yeah it's ok, I got the drift ;)  and thanks
<rharper> hehe, sure
<rharper> I think a recommends makes sense; but I'll defer to smoser when he's had time here;  I think that manual installation would still pick up the recommands unless users do a --no-install-recommends
<rharper> I suppose if it's just that runtime library, that might be ok;
<rharper> ideally we'd get the standard image to include it if it's a big win on yaml parsing perf
<powersj> rharper: do we care about fedora builds? or just epel for now?
<rharper> powersj: both
<powersj> rharper: ok trying a build on those right now.
<powersj> The build system assumes you have a spec file in your repo, so going to need to figure that one out. Still trying to see if we can generate it on the fly so we have less to change.
<rharper> not sure what to do about guessing (testing) which builds should use systemd flag;
<powersj> yeah that too...
<rharper> so they just read the specfile as is? no "run this cmd" first ?
<powersj> correct, I give git repo + branch (optional) + path to spec file
<rharper> hrm
<powersj> that's the "mock scm"  method.
<powersj> there is another method that uses "tito", but our repo obviously is not setup for that
<powersj> can also upload a srpm or give a URL to srpm for builds
<powersj> then check the release + arches and press go
<rharper> yeah, I wonder if we need a cloud-init-spec file repo
<rharper> well, maybe srpm uploads would be fine
<rharper> we'd do a package/brpm  from master
<rharper> and then upload the .src rpm
<powersj> yeah doing the git way would be awesome because webhooks ftw
<powersj> https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init/build/547115/
<rharper> fancy sauce
<blackboxsw> thanks again for the review rharper  for tomorrow I've addressed your comments on https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/323088
<blackboxsw> I'm off for today. dinner beckons
#cloud-init 2017-05-05
<blackboxsw> hi rharper I noticed most of cloud-init/config/cc_*py contain semi-structured content describing the config feature and that some of the help docs are out of date. Have we previously discussed trying to generate the example or schema docs from these docstrings to keep docs from gettting dated?
<blackboxsw> s/dated/out of sync/
<powersj> rharper, smoser: does this list look right: https://paste.ubuntu.com/24517862/
<smoser> is centos6 sysvinit ?
<smoser> not upstart ?
<powersj> it has /etc/init.d.. so I assumed sysvinit
<powersj> that is looking at a lxc
<powersj> [root@c6 init.d]# stat /proc/1/exe
<powersj>   File: `/proc/1/exe' -> `/sbin/init'
<powersj> rharper: can you bit flip https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/323351 if you are good with it?
<rharper> powersj: will review asap
<rharper> blackboxsw: we've not; I think the first step was getting in-file docs to turn into rtd info;  it's certainly reasonable to move that around from the header to something else;  not entirely convinced that we can capture everything as method docstrings versus how it's done today;
<blackboxsw> rharper it just feels like there is a lot of duplication in the module docstrings that is almost, but not quite the RTD doc examples
<blackboxsw> nothing I'll spend time on at the moment, but figured it warranted a bit of discussion at some point.
<rharper> blackboxsw: ack, the rtd examples are not at their best
<powersj> rharper: thx!
#cloud-init 2018-04-30
<blackboxsw> oops well I had the date wrong in the topic for this... but here goes
* blackboxsw changed the topic of #cloud-init to: Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting: Monday 4/30 16:00 UTC | cloud-init 18.2 released (03/28/2018)
<blackboxsw> #startmeeting Cloud-init bi-weekly status meeting
<meetingology> Meeting started Mon Apr 30 16:04:15 2018 UTC.  The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
<meetingology> Available commands: action commands idea info link nick
<blackboxsw> hi folks, sorry for the mis-representation of when our cloud-init status meeting date. It's time for another episode/update of the happenings in cloud-init.
<blackboxsw> Next meeting will be in two weeks: May 7th
<blackboxsw> at 16:00 UTC
<blackboxsw> The last couple weeks on the upstream side of the house has been a big push to get testing and stability into master for the Ubuntu Bionic release freeze
<blackboxsw> ... I'd better start with the topic
<blackboxsw> #topic Recent Changes
<blackboxsw> The last couple weeks on the upstream side of the house has been a big push to get testing and stability into master for the Ubuntu Bionic release freeze.
<robjo> May 7th would be 1 week from today that should be May 14th
<blackboxsw> robjo: gah, I did it again. Thank you... glad someone's listening.    Next cloud-init status meeting Monday May 14th 16:00 UTC
<blackboxsw> #topic #cloud-init Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting: Monday 5/14 16:00 UTC | cloud-init 18.2 released (03/28/2018)
<blackboxsw> ok topic agrees in channel now, so I don't botch it at the end of meeting
<blackboxsw> Along with a blitz for stability in Bionic the following changes have been shepherded into tip of master
<blackboxsw>     - Add reporting events and log_time around early source of blocking time
<blackboxsw>       [Ryan Harper]
<blackboxsw>     - IBMCloud: recognize provisioning environment during debug boots.
<blackboxsw>       (LP: #1767166)
<blackboxsw>     - net: detect unstable network names and trigger a settle if needed
<blackboxsw>       [Ryan Harper] (LP: #1766287)
<blackboxsw>     - IBMCloud: improve documentation in datasource.
<ubot5> Launchpad bug 1767166 in cloud-init (Ubuntu) "IBMCloud datasource does not recognize provisioning in debug mode." [Medium,Confirmed] https://launchpad.net/bugs/1767166
<blackboxsw>     - sysconfig: dhcp6 subnet type should not imply dhcpv4 [Vitaly Kuznetsov]
<blackboxsw>     - packages/debian/control.in: add missing dependency on iproute2.
<ubot5> Launchpad bug 1766287 in cloud-init (Ubuntu) "18.04 minimal images on GCE intermittently fail to set up networking " [Undecided,In progress] https://launchpad.net/bugs/1766287
<blackboxsw>       (LP: #1766711)
<blackboxsw>     - DataSourceSmartOS: add locking of serial device.
<blackboxsw>       [Mike Gerdts] (LP: #1746605)
<blackboxsw>     - DataSourceSmartOS: sdc:hostname is ignored [Mike Gerdts] (LP: #1765085)
<ubot5> Launchpad bug 1766711 in cloud-init (Ubuntu Bionic) "cloud-init missing dependency on iproute2" [Medium,Fix committed] https://launchpad.net/bugs/1766711
<blackboxsw>     - DataSourceSmartOS: list() should always return a list
<blackboxsw>       [Mike Gerdts] (LP: #1763480)
<ubot5> Launchpad bug 1746605 in cloud-init "DataSourceSmartOS needs locking" [Medium,Fix committed] https://launchpad.net/bugs/1746605
<blackboxsw>     - schema: in validation, raise ImportError if strict but no jsonschema.
<blackboxsw>     - set_passwords: Add newline to end of sshd config, only restart if
<blackboxsw>       updated. (LP: #1677205)
<ubot5> Launchpad bug 1765085 in cloud-init "DataSourceSmartOS ignores sdc:hostname" [Medium,Fix committed] https://launchpad.net/bugs/1765085
<blackboxsw>     - pylint: pay attention to unused variable warnings.
<blackboxsw>     - doc: Add documentation for AliYun datasource. [Junjie Wang]
<blackboxsw>     - Schema: do not warn on duplicate items in commands. (LP: #1764264)
<ubot5> Launchpad bug 1763480 in cloud-init "DataSourceSmartOS list() should always return a list" [Medium,Fix committed] https://launchpad.net/bugs/1763480
<ubot5> Launchpad bug 1677205 in cloud-init "cloud-init eats final EOL of sshd_config" [Medium,Fix committed] https://launchpad.net/bugs/1677205
<ubot5> Launchpad bug 1764264 in juju 2.3 "bionic cloud-init 18.2 WARNING Juju's 'runcmd' stanza" [High,Triaged] https://launchpad.net/bugs/1764264
<blackboxsw> the general theme has been: new IBMCloud datasource support for cloud-init, SmartOS datasource work by mgerdts, and some json schema improvements
<blackboxsw> so background on IBM, is that their support used to be ConfigDrive based datasource only, but there is now some additional support for different IBM boot/provisioning  stages, hence a new datasource that can support different boot modew
<blackboxsw> *boot modes
<blackboxsw> over the last two weeks we've landed an SRU into xenial and artful: 18.2-4-g05926e48-0ubuntu1~16.04.1  and   bionic sits at 18.2-14-g6d48d265-0ubuntu1
<mgerdts> On the SmartOS side, my changes are driven by our adoption of bhyve (moving away from kvm/qemu).  qemu provides a dhcp server VMs could fall back to if could-init was missing or misbehaving.  bhyve doesn't have that, so I've been working on getting cloud-init to be more stable with the bhyve serial metadata service.
<blackboxsw> Also, to our continuous integration on jenkins we now have an additional test for proposed packages in ubuntu for the bionic release to make sure ubuntu doesn't break across pending upgrades
<blackboxsw> #link https://jenkins.ubuntu.com/server/job/cloud-init-integration-proposed-b/
<blackboxsw> that integration tests hits the suite of platforms lxd, kvm and ec2
<blackboxsw> excellent mgerdts, and thanks for the blitz on these branches
<blackboxsw> looks like there are a few still in our review queue that we'll be able to get through once the dust settles on the bionic release  (which should be this week)
<blackboxsw> #link https://code.launchpad.net/~cloud-init-dev/cloud-init/+git/cloud-init/+ref/master/+activereviews
<mgerdts> Is now the right time to discuss bug 1765801, or is that later?
<ubot5> bug 1765801 in cloud-init "network should be optionally reconfigured on every boot" [Undecided,Confirmed] https://launchpad.net/bugs/1765801
<blackboxsw> I think over the last 2 weeks there have been a couple of requests in channel for how someone goes about getting newer cloud init into RHEL7, if anyone on the line today knows the contact point or process for that it'd be helpful. larsks doesn't seem to be around
<blackboxsw> mgerdts: probably in about 10 mins. thanks for brining it up
<blackboxsw> hopefully less.
<blackboxsw> ok I think that's it for recent changes, next topic (in-progress dev, then office hours (and bug discussion))
<blackboxsw> #topic In-progresss Development
<blackboxsw> We'll make this one short:
<blackboxsw> for ubuntu : bionic just went feature freeze last week, our team has a couple of IBM-related cheanges that we are pulling together for a quick SRU into xenial/artful to handle upgrade path from configdrive -> IBMCloud that we are working on the beginning of this week
<blackboxsw> we are also trying to wrap up validation of a Bionic SRU per the following bug
<blackboxsw> #link https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1767412
<ubot5> Launchpad bug 1767412 in cloud-init (Ubuntu Bionic) "SRU cloud-init 18.2-27-g6ef92c98-0ubuntu1" [Medium,Fix committed]
<blackboxsw> which grabs a number of the updates I listed in the last topic
<blackboxsw> since Ubuntu tends to sync all changes from tip into each release stream
<mgerdts> Is there any chance the SmartOS changes can piggy back on that IBM SRU
<mgerdts> asked too soon - I see they are mentioned in that bug.
<blackboxsw> mgerdts: no worries. good ask. probably not for this IBM SRU into xenial/artful which is going to be an exception to our update rule and only be a single cherry pick, but planning a folllowup SRU in about 2 weeks which will pull all changes from tip into artful/xenial/bionic/chunky releases
<mgerdts> ok
<blackboxsw> the cherry pick is to fasttrack it for IBM into xenial with minimal risk.
<blackboxsw> and we want to pull in all your changes if we can (and perform additional validation)
<blackboxsw> so the next SRU is our target
<blackboxsw> Also inprogress is some more Azure work on pre-provisioning that should land shortly:
<blackboxsw> #link https://code.launchpad.net/~jocha/cloud-init/+git/cloud-init/+merge/344192
<blackboxsw> as well as some builddeb fixes and network configuration printout fixes from smoser
<blackboxsw> smoser and rharper also worked out  some issues on specific google regions where cloud-init was getting hit by a race condition. Cloud-init started up before the kernel/udev was able to rename network devices to stable names like ens4 etc, so cloud-init's network configuration written ended up breaking because it represented devices like eth0 etc.
<blackboxsw> there are a couple of branches in flight to fix this issue:
<blackboxsw> #link https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/344181
<blackboxsw> #link https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/344198
<blackboxsw> ok I think that's it for in-progress work. So we'll head to office hours so we can chat bugs, branches reviews etc
<blackboxsw> #topic Office Hours (next ~30 mins)
<blackboxsw>  We'll be hanging out here for anyone who wants more eyes on a review, feature discussions or bug triage....
<blackboxsw> well, some of us will be :) a couple of us are at a feature planning conference for the week.
<mgerdts> In https://code.launchpad.net/~mgerdts/cloud-init/+git/cloud-init/+merge/343712 smoser said that he was concerned about how this would interact with eventual network hotplug
<mgerdts> There doesn't seem to be a timeline for network hotplug and the lack of network autoreconfig on reboot is has popped up a couple times in the past week.  This is just with a couple early adopters and internal users.
<blackboxsw> #link https://code.launchpad.net/~mgerdts/cloud-init/+git/cloud-init/+merge/343712
<blackboxsw> just to track it in the meeting
<mgerdts> So coming up with some mechanism to make this work soon is pretty important to us.
<mgerdts> gotcha, will be sure to do that in the future.
 * blackboxsw reads up on that link 
<blackboxsw> no worries, I'm pedantic :)
<mgerdts> That's how you got chosen to run the meeting, I suppose.  :)
<blackboxsw> yeah network hotplug will have a long tail as far as feature develpment (agreed). I believe it's on our charter for this next quarter. but that's what is being discussed this week
<blackboxsw> heh on meeting comment ;) too true
<blackboxsw> so mgerdts your branch allows metadata to set maintain_network to allow cloud-init to control network configuration each reboot with a True value
<blackboxsw> ?
<mgerdts> yes
<mgerdts> if it's not set to true in our metadata, the traditional behavior stays.
<mgerdts> That is, in the default path, any customization that someone does in the guest will not get whacked.
<robjo> cloud-netconfig handles hotplug https://github.com/SUSE/Enceladus/tree/master/cloud-netconfig contributions for other distros welcome
<blackboxsw> nice reference robjo
<blackboxsw> #link https://github.com/SUSE/Enceladus/tree/master/cloud-netconfig
<robjo> We currently have no GCE specific information but that is easy enough to add. The GCE guest environment handles this and we use the GCE guest environment code in our images in GCE
<blackboxsw> mgerdts: so can a user turn off that feature on an instance once they've already deployed, or is it create-time only
<mgerdts> It can be flipped at any time, in the current implementation.
<mgerdts> current implementation is only in a development branch
<blackboxsw> mgerdts: the only things I can see being an issue with the maintain network in cloud-init is that we are adding the cost of another function call  && metdata dict parse to look for a signal about maintaining the network. I agree that cloud-init having granularity between is_new_instance vs just re-do network, is something that cloud-init should have.
<blackboxsw> we probably need to discuss this too with rharper about what short-term vision we can get to while we await our network hotplug support in cloud-init proper
<blackboxsw> I'd tend to agree that waiting on fully baked hotplug solution is probably too long in this case
<blackboxsw> as that runway will be at least 2 months I'd think
<blackboxsw> ok, I'll take an action item to resolve this if we can by next meeting
<mgerdts> Not only that, but support for it will likely require changes in the host as well.  We tend not to do host updates very often, so it could be a year or more after the feature is available in images before it will be useful.
<blackboxsw> #action blackboxsw to have discussions w/ team on datasource maintaining network on each reboot per  https://code.launchpad.net/~mgerdts/cloud-init/+git/cloud-init/+merge/343712
<meetingology> ACTION: blackboxsw to have discussions w/ team on datasource maintaining network on each reboot per  https://code.launchpad.net/~mgerdts/cloud-init/+git/cloud-init/+merge/343712
<mgerdts> thanks
<blackboxsw> good topic.
<mgerdts> Is there another place that is good to catch up with larsks or other people that can offer guidance on for redhat/centos?
<blackboxsw> let's see, anything else folks want to chat about? stagnant reviews, bugs of interest etc?
 * blackboxsw looks at the last cloud-init community summit attendees list to see if rhel folks have another contacts that was supposed to replace larsks
<stanguturi> Chad, Is it possible that someone from cloud-init team can take a look at https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1766538
<ubot5> Launchpad bug 1766538 in cloud-init (Ubuntu) "network customization with cloud-init does not work on Ubuntu18.04 Beta2 Server" [Undecided,Confirmed]
<blackboxsw> mgerts, ryan mccabe is a potential contact too, looks like he's not here either today.
<mgerdts> ok, thanks
<blackboxsw> hrm, yeah not certain what mechanism is used to get cloud-init updated into RedHat mgerdts. Maybe filing a redhat bug about the request
<blackboxsw> mgerdts: https://bugzilla.redhat.com/ maybe
<blackboxsw> stanguturi: yes we can, we are trying to sort and understand any bugs against Bionic that we can
<mgerdts> ok, I can try that.
<blackboxsw> #link https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1766538
<ubot5> Launchpad bug 1766538 in cloud-init (Ubuntu) "network customization with cloud-init does not work on Ubuntu18.04 Beta2 Server" [Undecided,Confirmed]
<stanguturi> blackboxsw: Thanks
<blackboxsw> stanguturi: ok, so this is netplan + cloud-init related right?
<stanguturi> blackboxsw: Yes.
<akik> what does network hotplug mean in cloud-init context?
 * blackboxsw tries to remember what vmware datasource does, (like writing files direct to network /etc/network/interfaces.d)
<blackboxsw> akik: https://hackmd.io/M1Tae41PQBC7a9qMsurTJw?both  is a shared document for comment on hotplug in cloud-init
<blackboxsw> #link https://hackmd.io/M1Tae41PQBC7a9qMsurTJw?both
 * blackboxsw looks to see if there was a better doc hrm
<stanguturi> blackboxsw: Oh. But in the case of netplan, why does cloud-init remembers?
<akik> blackboxsw: does it mean that cloud-init stays running, waiting for new network interfaces to appear?
<blackboxsw> akik: right, it would mean that you wouldn't have to reboot cloud-init if devices get added at a later time (post-boot)
<blackboxsw> cloud-init would listen to some sort of event channel and react, re-write, and apply network config to add new devices
<akik> would it do the same thing as you could do with ansible or puppet? sorry i'm trying to understand why you would do it with cloud-init
<blackboxsw> akik: you would try to do it with cloud-init if you didn't want to rely on additional configuration management solutions if the only thing you needed was network config to reflect reality (not full system configuration and system automation)
 * blackboxsw has more puppet/chef background than ansible.
<blackboxsw> cloud-init does currently detect and write network configuration based on what the user/cloud-metadata tell us is the proper config for the instance
<akik> i only thought of cloud-init to do the initial configuration
<blackboxsw> so it would follow that if the metadata could dynamically tell the instance that network config has changed, cloud-init should probably try to react to that to fix the config to match the updated network configuration
<blackboxsw> akik: correct. cloud-init current only handle initial boot config and leaves the rest up whatever mechanism someone uses to update detailed config after that boot
<akik> ok thanks
<blackboxsw> akik: and we'd make that feature configurable (handle hotplug:True/False) so if users have other services handling hotplug cloud-init wouldn't collide
<blackboxsw> ok I think we're hitting the end of office hours. please feel free to continue discussion, we all poke around here throughout the day as our primary means of communication
<blackboxsw> thanks robjo akik stanguturi and mgerdts for the lively discussion
<blackboxsw> stanguturi: I'll dig up more info on that bug today
<mgerdts> thank you
<blackboxsw> as always notes will be here
<blackboxsw> #link https://cloud-init.github.io/
<blackboxsw> #endmeeting
<meetingology> Meeting ended Mon Apr 30 17:14:19 2018 UTC.
<meetingology> Minutes:        http://ubottu.com/meetingology/logs/cloud-init/2018/cloud-init.2018-04-30-16.04.moin.txt
<stanguturi> Thanks blackboxsw
<smoser> blackboxsw: after quite too long... just updated
<smoser> https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/344784
<smoser> there is one behavioral change there.
<blackboxsw> smoser: checking now
<smoser> just rebaed and fixed the pylint herror
<smoser> blackboxsw: could you  mark https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/342334 as approved ? or just make sure you'lre ok with that ? and then i'll queue it into ubuntu/devel
<blackboxsw> smoser: ok so you drop the is_ibm_cloud == not_found  detection from nocloud in ds-identify now right?
<blackboxsw> minor unittest change and +1 on it. I'm testing the updated deb in ibm w/ a nocloud setup  to be sure.
<smoser> correct. i dropped that (previously)
<smoser> the new change is to drop the ibm_cloud detection for the config-drive seed case.
<blackboxsw>  ok I must've missed that earlier, but I wanted a specifically exercise tha path (as I didn't look at it formerly)
<blackboxsw> do we want to queue your merge 344784 too?
<blackboxsw>  https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/342334 is approved, but  doesn't it need a rebase now to fix versioning in debian/chanelog
<smoser> blackboxsw: yeah... so thats kind of an interesting topic i talked iwth rharper some last week.
<smoser> last time i proposed two MPs for merge int ubuntu/* branches (the addition of the isc-dhcp-client and iproute2)
<smoser> i did not upate the debian/changelog
<smoser> largely because of that.
<smoser> but then... debian/changelog didn't get updated...
<smoser> i caught those later before the SRU upload.
<smoser> i'm not really sure how to handle this on the commits to ubuntu/ branches
<blackboxsw> so a commit to debian/changelog with an out of order version won't matter will if we are following up with something >= during our next 'C' package release right?
<smoser> blackboxsw: you are corect. we do not want to merge that out of order like that.
<blackboxsw> Just validated nocloud and configdrive behavior on IBM: https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/344784 can land with minor unittest tweak
<smoser> blackboxsw: the line go shorter then original
<smoser> ie.. at first i neded the paren
<smoser> i'll drop . thanks.
<blackboxsw> guessed as much
<smoser> pushed update
<blackboxsw> smoser: landed in tip
<blackboxsw> as far as the dropping of new-upstream-snapshot. it has current merge conflicts with ubuntu/devel, so I'm not sure how to resolve it other than rebasing and rebuilding debian/changelog with a new entry, or manually resolving the conflict and injecting this debian/changelog item where you think best in the most recent changelog section.
<blackboxsw> I guess it's easiest in the future if we don't let changes against ubuntu/devel age, and try to resolve them before we pull in any other new-upstream-snapshots against that branch.
<smoser> blackboxsw: cat a minute ?
<smoser> chat even
<blackboxsw> chat with a cat on my way
<blackboxsw> https://hackmd.io/jlq3C4qbSgurZ_DZ5GTiuw
<ahasenack> did you guys see #1768118? fwiw, i tried such an upgrade and it worked just fine
<blackboxsw> bug #1768118
<ubot5> bug 1768118 in cloud-init (Ubuntu) "/etc/netplan/50-cloud-init.yaml fails silently with no dhcp address assigned" [Undecided,New] https://launchpad.net/bugs/1768118
<ahasenack> not sure why it was filed against cloud-init, it's lacking many details
<blackboxsw> thanks ahasenack looking now.
<blackboxsw> smoser's on it ahasenack thanks
<blackboxsw> https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1766538 here's another network related concern to peek at
<ubot5> Launchpad bug 1766538 in cloud-init (Ubuntu) "network customization with cloud-init does not work on Ubuntu18.04 Beta2 Server" [Undecided,Confirmed]
<rharper> blackboxsw: I don't suspect that's a cloud-init issue, 18.04 desktop nor server are going to have ifupdown installed;  I don't think we're getting the whoe picture there
<blackboxsw> rharper: yeah we are talking about this now. we're gonna ask for cloud-init collect-logs
<smoser> rharper: yeah.
<rharper> well, logs won't matter
<blackboxsw> and go to bed ;)
<rharper> 18.04 has never had ifupdown in it
<rharper> so someone's not telling the whole story
<rharper> it's also vmware
<blackboxsw> yeah agreed
<smoser> it absolutely rendered witih netplan
<rharper> which has quite a few odd bugs filed w.r.t cloud-init and no vmware datasource, etc
<smoser> the logs are clear on that.
<rharper> so I suspect testing issue
<rharper> hehe
<blackboxsw> and multiple boots in that log listed too
<blackboxsw> might have started as xenial or something who knows
<rharper> yes
<rharper> so, a dist-upgrade to 18.04 would make sense
<rharper> but the scenario is far from clear in the bug
<smoser> mgerdts: https://pythonhosted.org/smartdc/
<smoser> is that something you would recommend ? have experience with ?
<mgerdts> I don't have experience with it, but I can ask around internally to see what people say.  There's also #smartos that is pretty active.
<mgerdts> version history that ends about 5 years ago makes me worry.
<blackboxsw> rharper: marked incomplete
<smoser> mgerdts: fair. i'm basically looking for something to launch an instance with python. none of our tools are node , so i dont want that at the moment.
<blackboxsw> smoser: for xenial https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/344866
<smoser> blackboxsw: i think you cherry-ipcked the wrong thing
<blackboxsw> wow typo, pulling again sorry
<smoser> 11172924a48a47a7231d19d9cefe628dfddda8bf
<mgerdts> smoser: one engineer reported having success a couple years back, but others remarked that the examples seemed to use obsolete arguments.  It doesn't seem as though there's a better maintained alternative.  :(
<smoser> mgerdts: thanks.
<mgerdts> In particular: "the createmachine example there uses "dataset" for the image argument... using URNs instead of UUIDs."
<smoser> blackboxsw: i'll go ahead and grab that
<smoser> but it loks like there are some differences between yours and mine
<smoser> http://paste.ubuntu.com/p/5NBPqmpBCS/
<smoser> i think that is due to
<smoser> a.) ~/.quiltrc http://paste.ubuntu.com/p/M3v6cJDfHg/
<smoser> b.) i probably have newer git and it is picking 8 hash by default
<blackboxsw> I repushed, but go4it smoser
<blackboxsw> checking
<smoser> probably some changes needed to debian/cherry-pick to specify both of those things
<blackboxsw> probably right, I did this from a xenial env
<blackboxsw> if I did it from my bionic env it probably would have been equal
<smoser> well, the 'ab' style comes from ~/.quiltrc
<smoser> maybe i can look at making it do the rigiht thing all the time tomorrow
<smoser> blackboxsw: uploading. i did take my version, but with your name.
<smoser> as soon as build completes it will upload. oh. fudge. failed build.
<smoser> fudge.
<smoser> namedtuple usage
<smoser> http://paste.ubuntu.com/p/99RW7NrSTQ/
<smoser> that commit isit stand alone.
<smoser> it needs 6ef92c98c3d2b127b05d6708337efc8a81e00071
<smoser> blackboxsw: i'll be b ack in in 3 hours or so. sorry
<blackboxsw> gah, ok
<blackboxsw> sounds good
<blackboxsw> strange as tox completed for me locally
<blackboxsw> ahh can reproduce w/ make deb
<smoser> blackboxsw: i think maybe just cherry pick the one you had and the other
<smoser> they're both ibm related
<blackboxsw> wow 3 hours was fast
<smoser> and for ibm internal test, they might want the debug thing.
<blackboxsw> sorry pushing that
<smoser> yeah.
<smoser> i just thought that
<smoser> and i have to go afk now.
<smoser> that make sense thouigh? 6ef92c98c3d2b127b05d6708337efc8a81e00071
<smoser> is what you need to make the test ru
<smoser> run
<smoser> i think
#cloud-init 2018-05-01
<smoser> blackboxsw: still there?
<smoser> i just uploaded.
<smoser> only difference from yours is the order of the cherry picks (and teh quilt/git diffs mentioen dpreviouisly)
<smoser> http://paste.ubuntu.com/p/8cMxzs2yjK/ <-- blackboxsw that is what i think we need to debian/cherry-pick
<smoser> and i think we should probably move that out of debian/ and next to new-upstream-snapshot
<blackboxsw> smoser: back from kiddo bedtime routine.
<blackboxsw> smoser: yeah let's move it out into qa-scripts instead of trying to fixup each series
<smoser> i'm here for  abit.
<blackboxsw> I'll repost then
<blackboxsw> with 8 char comitish
<blackboxsw> or smoser looks like you merged it, or your version of it into ubuntu/xenial
<blackboxsw> so we need SRU templates on those two bugs right for the xenial update
<smoser> ah. yeah, i did merge and uploaded.
<smoser> so yeah.. we do need sru templates.
<blackboxsw> ok I'll add them
<smoser> yeah, i just skipped the round trip with you
<blackboxsw> yeah, since I hadn't yet fixed my env up.
<blackboxsw> sorry for that
<blackboxsw> *wasted time
<smoser> blackboxsw: you did not write/start any sru templates did you ? for
<smoser> bug 1766401
<ubot5> bug 1766401 in cloud-init (Ubuntu Xenial) "full config file wiped after apt-upgrade issued" [High,In progress] https://launchpad.net/bugs/1766401
<smoser> or
<smoser> bug 1767166
<ubot5> bug 1767166 in cloud-init (Ubuntu Xenial) "IBMCloud datasource does not recognize provisioning in debug mode." [Medium,In progress] https://launchpad.net/bugs/1767166
<smoser> just wondering. i will start them, but didn't want to duplicate work.
<smoser> ah. i see you did. good work.
<blackboxsw> started, but didn't get through many
<blackboxsw> artful SRU up for review https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/344901
<smoser> blackboxsw: the quitl refresh didn't seem to stick
<smoser> stick/take
<smoser> i wanted a/<path> and b/<path> in the patches/
<smoser> not cloud-inti.orig/ and cloud-init/
<smoser> i expected QUILT_REFRESH_PASTCH_ARGS to do that
 * blackboxsw checks the patch to make sure I applied it . this was from running the script I hoisted over into qa-scripts (with the patch  I think, because commitish is now 8 chars). I'll try peeking at quilt args on my env to see 
<smoser> er.. QUILT_REFRESH_ARGS=
<smoser> i spelled it correctly in the patch i think
<blackboxsw> seems to match the man page
<blackboxsw> trying on the cli
<smoser> well... QUILT_REFRESH_ARGS
<smoser> is not actually documented in man page itself
<smoser> only by reference
<blackboxsw> ahh right, and the man page seems to state adding those params to both QUILT_REFRESH_ARGS. and QUILT_DIFF_ARGS
<blackboxsw> I'll re-run with both exported to see if if behaves differently
<blackboxsw> nope, no change here  with both exports
<smoser> you have a ~/.quiltrc ?
<blackboxsw> ENOEXIST
<smoser> can you try this:
<smoser>  https://hastebin.com/raw/uxunixiyad
<blackboxsw> trying
<smoser> c-i is finally happy with me at https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/344894
<blackboxsw> --- a/cloudinit/sources/DataSourceConfigDrive.py
<blackboxsw> +++ b/cloudinit/sources/DataSourceConfigDrive.py
<blackboxsw> looks good smoser
<blackboxsw> will pull that into qa-scripts
<blackboxsw> smoser: re-pushed artful branch
<blackboxsw> https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/344901 is updated and qa-scripts docs are updates for cherry-pick script
<blackboxsw> grabbing collect-logs branch
<smoser> blackboxsw: neither here nor there... but other times when i'd copied one of the cherry-picks across, i'd cherry-picked from the first branch
<smoser> ie, you could have:
<smoser>  cherry-pick 44a44ae18e7ee51322110dc66107d2a58f5ff304
<smoser> err..
<smoser> git cherry-pick -x 44a44ae18e7ee51322110dc66107d2a58f5ff304
<blackboxsw> ahh I see, so I could have pulled the ubuntu/xenial commit into ubuntu/artful too
<blackboxsw> shall I do that instead and push again
<smoser> this is fine. i just pushed
<smoser> the value in git cherry-pick is the '-x'
<smoser> then the commit message gives a reference to the original
<smoser> but only realy in text form
<blackboxsw> smoser: couple comments https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/344894
<blackboxsw> can you update the description to describe how this fixes (avoids) the traceback in chrooted envs that don't have a /run directory
<smoser> oh. i didn'teven thikn of that. i was just tyring to run it on my desktop :)
<blackboxsw> fixes the world without a 2nd thought
<blackboxsw> I added the bug to your descr, just didn't update the text beyond that
<smoser> blackboxsw: updated... thanks. updated description and pushed a "drop which" commit.
<blackboxsw> sorry was just invalidating another collect-logs bug
<blackboxsw> landed
<jocha> blackboxsw: raharper: Hey, I just added a new iteration on this merge request: https://code.launchpad.net/~jocha/cloud-init/+git/cloud-init/+merge/344192 :)
<blackboxsw> ahh blah jocha, on it :)
<blackboxsw> thanks
<jocha> no problem!
<smoser> blackboxsw: https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/343846 updated
<blackboxsw> jocha:thanks for working the unit test isolation stuff in the azure branch.  minor unit test update request posted to your branch, if there is a way to tease apart some of those unit tests into separate focused test classes, we won't have to append *args to each test case to ignore unneeded mocks
<blackboxsw> adding comment now per something like this http://paste.ubuntu.com/p/gXNTdj4rq5/
<blackboxsw> I'd like to see us separate a lot of those test cases so we don't have superfluous mocks being setup (and subsequently ignored) by specific unit tests
<blackboxsw> so if there are some easy cases where your branch can do that excellent.
<blackboxsw> if it's too much work, you can say that in the branch too. I just dont want to add more tests cases that have to pass and ignore *args if we can avoid it
<jocha> alright sure, let me take a look into this!
<smoser> cosmic
<smoser> https://launchpad.net/ubuntu/cosmic
<blackboxsw> sure enough! long live cosmic
<blackboxsw> minor comment here https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/344544
<jocha> blackboxsw: I've updated the unit tests: https://code.launchpad.net/~jocha/cloud-init/+git/cloud-init/+merge/344192 :)
#cloud-init 2018-05-02
<Gianlu> hello everyone. Where can I open a bug (or check is an open bug already exists)? I have a problem with the ca_certs module
<rharper> Gianlu: bugs are at http://bugs.launchpad.net/cloud-init
<Gianlu> ok, thanks
<jocha> blackboxsw: hey Chad, just wanted to let you know I've refactored the unit tests, if you'd kindly take a look: https://code.launchpad.net/~jocha/cloud-init/+git/cloud-init/+merge/344192 :)
<blackboxsw> +1 jocha I was thinking I was going to land this today. I like the changes good work
 * blackboxsw wants to spin up azure again to run a quick test
 * powersj launched CI on it and it was good
<jocha> blackboxsw: thanks! appreciate it! :)
<blackboxsw> powersj: https://github.com/canonical-server/jenkins-jobs/pull/38
<blackboxsw> if you have a chance, I'd like to fix the trigger
<blackboxsw> good 'ole 1 character diffs
<powersj> blackboxsw: when did that get added? I don't see it in your last commit?
<blackboxsw> my PR? I just added that a couple seconds ago
<powersj> the '--'
<blackboxsw> ohh you mean the --
<powersj> trying to understand why it was hit
<blackboxsw> yeah lemme see. git blame show point an ugly finger
<powersj> and also why the URL was 404, as I think it is more than the --
<powersj> ac77cc9f (Joshua Powers
<powersj> what a noob
<powersj> was this even working since january?!
<blackboxsw> shows how often we kicked the trigger ;)
<powersj> because this means it will try to launch the job $pre$t
<powersj> $pre == cloud-init-integration-proposed-
<powersj> then $t == triggerwhat == cloud-init-integration-proposed-a
<powersj> so that would combine to be cloud-init-integration-proposed-cloud-init-integration-proposed-a
<powersj> which ain't going to work
<blackboxsw> ugh bad patch, what a noob
<powersj> we need to get rid of pre
<powersj> and just use the $t a la triggerwhat
<powersj> agreed?
<blackboxsw> right we just don't  need pre actually read the code now ;(
<powersj> yeah
<blackboxsw> checking traceback in jenkins again to be sure, but yeah I think so
<blackboxsw> roblem accessing /server/job/cloud-init-integration-proposed--cloud-init-integration-proposed-b   yep
<blackboxsw> duplicated as you said
<blackboxsw> no prefix. pushing now
<blackboxsw> force pushed now
<powersj> blackboxsw: want to check on https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/344976 too
<powersj> I put a comment there
<powersj> once travis +1s I'll pull
<blackboxsw> so the slop continues
<blackboxsw> I mistakenly pulled the dep version from the wrong tox env
<blackboxsw> for flake8
<blackboxsw> fixing/testing and pushing
<powersj> pulled your jenkins job fix and jobs redeployed
<blackboxsw> ok inconsistencies with 3.5.0. will sort it
<blackboxsw> thx powersj
<powersj> see I can actually do reviews when not at a hockey game or on the beach ;)
<blackboxsw> heh:     Flake8's dependencies explicitly state "< 2.4.0"   https://github.com/PyCQA/pycodestyle/issues/741
<blackboxsw> so might not be able to bump both flake8 and pycodestyle to tip :/
<powersj> need a new flake8 on pypi
<powersj> 3.5 since oct
<smoser> maybe time to drop flake8 then i guess
<smoser> and just run pycodestyle and pyflakes
<blackboxsw> smoser: you think a new tox env pycodestyle with a pinned version 2.4.0 to supplement tip-pycodestyle tox env?
<blackboxsw> and we have jenkins run that in ci
#cloud-init 2018-05-03
<jocha> Hi all, so during a regular ubuntu 17.10 azure deployment I'm seeing a 15 second delay between two lines of cloudinit log, wondering if I could tap into some networking experts here who could tell me if something looks fishy here, logs here, https://paste.ubuntu.com/p/83MQQ8qQYr/
<jocha> just realized I used here 3 times, :p
<blackboxsw> jocha: you've been good and patient, I'm currently thrashing a bit on your branch trying to apply a patch to rework logic a bit more
<blackboxsw> checking your comment
<blackboxsw> your logic in the bug fix is good, I had wanted to avoid leaking the MARKER file outside scope of poll_imds  as it's not needed above that level, unfortunately I'm going through unit test changes now as a result
<jocha> mmm ic fosure!
 * blackboxsw wonders about bug: #1739672
<ubot5> bug 1739672 in systemd (Ubuntu Artful) "Regression in getaddrinfo(): calls block for much longer on Bionic (compared to Xenial), please disable LLMNR" [Undecided,New] https://launchpad.net/bugs/1739672
<blackboxsw> I know there's either a dhcp6 or ipv6 router advertisement bug that adds about 10s + to boot times. trying to fish for that bug #
<blackboxsw> jocha: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1765173
<ubot5> Ubuntu bug 1765173 in systemd (Ubuntu) "networkd waits 10 seconds for ipv6 network discovery by default" [Undecided,Fix released]
<jocha> sweet I'll take a look! thanks!
<blackboxsw> jocha: I'm done on your branch for sure. testing it now on azure
<blackboxsw> http://paste.ubuntu.com/p/MsVK5PZZ98/ is a suggestion to limit leaking REPORTED_READY_MARKER_FILE knowledge outside of poll_imds since nothing at outside that scope cares about the report_ready flag/file
<jocha> @blackboxsw: thanks! also the bug you sent me looks exactly like what we're seeing, and it looks like the fix has been released, will look to try it out!
<blackboxsw> tested your bug branch + the minor fix patch and things look good on my end . updated it for flakes http://paste.ubuntu.com/p/DtSmYqbgMV/
<jocha> blackboxsw: Looks good to me! thanks for going through with it for me! :)
<blackboxsw> good deal jocha, if you can apply that patch and git push, then I'll land it
<jocha> cool will do!
<jocha> git applied and git pushed :)
<blackboxsw> cheers jocha
<blackboxsw> oops fixing your commit message so each line is  < 74 chars long
<blackboxsw> jocha: do you approve the updated commit message I added to your branch? https://code.launchpad.net/~jocha/cloud-init/+git/cloud-init/+merge/344192
<blackboxsw> just prefaced with azure: and reformatted line length
<jocha> approved!
<blackboxsw> landed jocha thanks for the contribution
<blackboxsw> it'll show in our next SRU to bionic
<jocha> nice, ill be on the lookout
<blackboxsw> we'll update the bug #1765214 when it officially publishes to (bionic|xenial|artful)-updates
<ubot5> bug 1765214 in cloud-init "Multiple success messages sent to Azure Fabric if reboot occurs during pre-provisioning" [Medium,Fix committed] https://launchpad.net/bugs/1765214
<paulmey> rharper: I updated https://code.launchpad.net/~paul-meyer/cloud-init/+git/cloud-init/+merge/344538 and added your suggested checks, ptal
#cloud-init 2018-05-04
<blackboxsw> forgot to mention smoser https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/344976 switches to pyflakes
<smoser> blackboxsw: reviewed... suggesetd a change in your comit message.
<blackboxsw> great thx
<blackboxsw> smoser: confirmed launch-softlayer os:xenial 'cloud-init clean --reboot' continues to detect ConfigDrive
<blackboxsw> testing template 'launch-softlayer -i xenial' now
<blackboxsw> and confirmed configDrive is re-dedected across clean reboots on xenial
<blackboxsw> re-detected rather
<blackboxsw> but again that's with the ds-id warning banner we know about saying NoCloud initially detected but cloud-init properly applies the mounted ConfigDrive
<smoser> blackboxsw: ok. in hangout again
<blackboxsw> jcoming
<blackboxsw> smoser: I'm there
<blackboxsw> ok smoser I'm back
<blackboxsw> I'm in hangout, things have settled.
<smoser> blackboxsw: well, i've convinced myself that the path for error is template with user-data
<smoser> so i think focus on that if you're looking to reproduce something.
#cloud-init 2020-04-27
<IanTPrice> I'm having an issue with the `write_files` module in cloud-init
<IanTPrice> For the .yaml file I'm using:
<IanTPrice> `#cloud-configwrite_files:  - encoding: ""    path: /home/ubuntu/test.sh    owner: ubuntu:ubuntu    permissions: '0755'    append: true    content: |      #!/usr/bin/env bash      ls -lsa /home/ubuntu`
<IanTPrice> and I'm running that using
<IanTPrice> `multipass launch --name k5 -c 1 -d 3G -m 1G --cloud-init cloud-init/k3s-master.yaml daily:20.04`
<IanTPrice> However, the code wipes out all files in the `/home/ubuntu`  directory with the exception of the `.ssh` directory
<IanTPrice> In addition the `owner: ubuntu:ubuntu` line has never worked
<IanTPrice> Anyone got any ideas or links to show where I'm going wrong?
<Saviq> IanTPrice: it's very difficult to see what your YAML actually is, can you please https://pastebin.ubuntu.com/ it ?
<IanTPrice> I've tried many, many combinations of the `.yaml` file (BTW, multipass does a yaml validation check so I know it's not incorrect yaml
<Saviq> IanTPrice: it doesn't have to be incorrect YAML for cloud-init to do something that you didn't expect
<IanTPrice> saviq: will do - anyone know know how to paste multiline code in IRC?
<Saviq> IanTPrice: your client would split it into multiple messages
<Saviq> would/should
<IanTPrice> #cloud-configwrite_files:  - encoding: ""    path: /home/ubuntu/test.sh    owner: ubuntu:ubuntu    permissions: '0755'    append: true    content: |      #!/usr/bin/env bash      ls -lsa /home/ubuntu
<Saviq> nope
<IanTPrice> https://pastebin.ubuntu.com/p/rWM2ntXdKR/plain/
<IanTPrice> link including the run command: https://pastebin.ubuntu.com/p/m3mW7RcCx2/
<Odd_Bloke> LongLiveCHIEF: o/ Are you still having your Pi problem?
<Odd_Bloke> IanTPrice: I've just tested that user-data (though not in multipass) and AFAICT it's working as expected (an executable test.sh is present in /home/ubuntu, with the specified contents) with the exception of setting the owner (which I will find a bug for in a moment).  Why would you expect there to be any files in /home/ubuntu other than .ssh?
<Odd_Bloke> IanTPrice: As for the owner issue, you're running into https://bugs.launchpad.net/cloud-init/+bug/1486113.  write_files runs before cloud-init creates users/groups (so that the written files can affect how users/groups are created), which causes problems in your case.  The easiest workaround is to add a runcmd which sets the permissions/ownership as you would like (as runcmd happens after both write_files
<ubot5> Ubuntu bug 1486113 in cloud-init "write_files runs before users/groups, renders "owner" useless" [Medium,Triaged]
<Odd_Bloke> and users/groups).
<IanTPrice> Odd_Bloke Thanks thanks for the bug report - hadn't come across that but should be able to code around it
<Odd_Bloke> Yeah, it's an annoying one, for sure, but the workarounds are pretty simple once you know about it.
<Odd_Bloke> Happy I could help. :)
<IanTPrice> Odd_Bloke If I don't write the file, or if I write it to /test.sh then there are the standard files in the users home directory, namely ~/.bashrc among others
<IanTPrice> all those files disappear if I write the file to /home/ubuntu/test.sh with the exception of the .ssh directory... ???
<IanTPrice> Just tried it on AWS EC2 user data - it does the same thing  that helps with some elimination - it's not multipass or linux distro
<Saviq> IanTPrice, Odd_Bloke, won't that actually be a result of the same problem?
<Saviq> write_files creates /home/ubuntu, so adduser then doesn't copy the skeleton?
<Odd_Bloke> Yeah, that probably is the problem, I see what you mean now.
<Odd_Bloke> That's not the bug I linked though, that's specifically to do with `owner` not working as expected.
<Odd_Bloke> And a search ("write_files skel") doesn't turn up any existing bugs for it.
<IanTPrice> but you're right - it is a timing issue - I created the file in /home/newuser/test.sh and `write_files` creates the directory structure automatically
<IanTPrice> so, as you say, `adduser` is not adding the skeleton becuase the user directory has already been created
<IanTPrice> I could see the wood for the trees there.  many thanks for giving me the right steer - pair programming works wonders!
<IanTPrice> *could* *couldN'T*
<IanTPrice> The smoking gun `The home directory `/home/newuser' already exists.  Not copying from '/etc/skel'`
<IanTPrice> Priceless: `This was first reported in 2013 in bug #1231541` =$
<ubot5> bug 1486113 in cloud-init "duplicate for #1231541 write_files runs before users/groups, renders "owner" useless" [Medium,Triaged] https://launchpad.net/bugs/1486113
<Odd_Bloke> Yeah, it's been open for a long time because changing the behaviour is backwards-incompatible.  (The trivial case to consider: a write_files to /etc/skel to ensure that all created users have the same .wibblerc in their home directories.)
<Odd_Bloke> s/behaviour/the ordering of the modules/
<Odd_Bloke> blackboxsw: We didn't assign PRs in stand-up: I'll take the older one, and I'll assign you to the newer one.
<Odd_Bloke> blackboxsw: I don't know why CI on your PR didn't catch this, but it looks like your schema changes introduced a pylint error: https://github.com/canonical/cloud-init/pull/332
<blackboxsw> thanks Odd_Bloke just out of mtgs. will grab it.
<blackboxsw> Odd_Bloke: typo in the comment and +1 on https://github.com/canonical/cloud-init/pull/332/files
<IanTPrice> Odd_Bloke The suggestion of adding a `write_files_late` seems eminently suitable and should not break backwards compatability
#cloud-init 2020-04-28
<socket-> Hey all, any idea why my cloud-init config is not creating the ssh_host keys for me in this CFT? https://imgur.com/a/5t86kS7
<socket-> instead it does the normal random generation at the start up... Generating public/private ecdsa key pair. Your identification has been saved in /etc/ssh/ssh_host_ecdsa_key. Your public key has been saved in /etc/ssh/ssh_host_ecdsa_key.pub. The key fingerprint is: SHA256:sILmAwj+6f1qIJTvw0R1QGegrX75uY+q9bc5yLIniRI root@ip-10-0-1-129
<socket-> https://apaste.info/X8x0
<rharper> socket-: the cc_ssh module will write host keys if you provide them in user data,   you'll need to lay out your config, like so https://cloudinit.readthedocs.io/en/latest/topics/modules.html#host-keys
<rharper> socket-: if are are including the host keys in the expected form and it's not writing them out, please file a bug and include the tarball output from cloud-init collect-logs
<socket-> Hmm, so i think the problem is that I am trying to use cloud-init code in 'AWS::CloudFormation::Init:'. I think i need to figure out where is this AMI the cloud-init cofig is stored and add the ssh stuff there
<socket-> AWS::CloudFormation::Init only supports packages, groups, users, sources, files, commands and services
<rharper> huh
<socket-> im saying cloud-init is different from cloudformation init.  I thought one just called the other, but it only has a subset of the commands
<socket-> So it looks like I can pass a cloud-config in the EC2 instances userdata like you said.  I currently have shell code being passed to the user data. Do you know if they can be mixed, or if i have to change all the shell code to be cloud-config runcmds?
<socket-> rharper: do you know if i can do something like this? https://apaste.info/EteD
<Odd_Bloke> socket-: You would either need to convert your shell script to a runcmd stanza (which can be strings, so that's not impossible by any means), or you'd need to look at multipart MIME user-data.  (Let me look up references for those options now.)
<Odd_Bloke> runcmd documentation is https://cloudinit.readthedocs.io/en/latest/topics/modules.html#runcmd, MIME multipart is https://cloudinit.readthedocs.io/en/latest/topics/format.html#mime-multi-part-archive.
<rharper> Odd_Bloke: socket-  cloud-init/tools/make-mime.py is quite useful here for constructing a multi-part mime
<Odd_Bloke> rharper: Hmm, seems like we should probably point to that instead of the inline helper script.
<Odd_Bloke> I'll fix that now.
<socket-> so iv learned some more. AWS user data allows only one content type. You can make this content type be multipart/mixed though.  So iv been testing out a simple mix https://apaste.info/c4HT
<socket-> I havent got it working yet, but so far i can confirm that cloud-init-output says it ran the final_modules, and also the shell code at the bottom was run, and the file was created
<socket-> but none of the packages are being installed, its like cloud-init is skipping that section
<Odd_Bloke> socket-: Can you pastebin what your latest config looks like?
<socket-> Odd_Bloke: lol thanks just saw your message
<socket-> https://apaste.info/c4HT
<socket-> thats what im sending to aws userdata
<socket-> https://apaste.info/WtST
<socket-> ^ is the cloud-config.txt attachment aws uploads
<Odd_Bloke> socket-: What does `cloud-init query -a` give you?
<socket-> lots of meta data about the instance
<socket-> do you wanna see the whole thing?
<socket-> Odd_Bloke: https://apaste.info/GYGV
<socket-> i removed the crypto but the rest is intact
<Odd_Bloke> OK, I was hoping that would display the user-data after we'd unpacked it, so that didn't help. :p
<Odd_Bloke> socket-: Do you have any WARN lines in /var/log/cloud-init.log?
<socket-> i think it just writes 2 files to disk, the cloud-init.txt and userdata.txt
<socket-> so i have access to both of those
<socket-> checking
<socket-> nothing for grep -i warn /var/log/cloud-init.log
<socket-> Odd_Bloke: if i take out cloud_final_modules ... then it installs the packages defined
<socket-> but it doesn't run the shell script
<Odd_Bloke> socket-: Could you try using make-mime.py to create your input file?  That b64 encodes stuff, which might work if there's something strange about the input that's breaking things?
<socket-> havent tried that yet, but in my latest test I noticed..
<socket-> 2020-04-28 19:22:20,990 - __init__.py[WARNING]: Unhandled unknown content-type (text/plain) userdata: 'b'MIME-Version: 1.0'...'
<socket-> when using https://apaste.info/Ji5n
<socket->  python mime.py -a user.txt ... ValueError: not enough values to unpack (expected 2, got 1)
<socket-> whats the syntax
<Odd_Bloke> socket-: filename:mimetype (where mimetype will be cloud-config for one and x-shellscript for the other)
<Odd_Bloke> Something like `./tools/make-mime.py -a config.yaml:cloud-config -a script.sh:x-shellscript`
<Odd_Bloke> blackboxsw: Would really appreciate your review on https://github.com/canonical/cloud-init/pull/335, I put that together in the background while I was lurking in that meeting earlier.
<Odd_Bloke> (Obviously has a couple of TODOs, but would like to know how that looks as a technical schema as far as you're concerned.)
<Odd_Bloke> (And I'd like to know what testing you generally do, too. :)
<blackboxsw> will do Odd_Bloke
<blackboxsw> haha nice Odd_Bloke
<blackboxsw> I thought I heard a bit of typing then :)
<blackboxsw> Odd_Bloke: rharper rick_h just drew up an initial spec for review for cloud-init a json schema artifact
<rharper> blackboxsw: nice
<blackboxsw> Odd_Bloke: couple of minor comments on https://github.com/canonical/cloud-init/pull/335/files
<blackboxsw> rharper: Odd_Bloke I think we have to think a bit about how we handle values that come in from /etc/cloud/cloud.cfg https://github.com/canonical/cloud-init/pull/335/files#r416940582
<rharper> blackboxsw: I don't think so, args is not set via user-data , no ?
<rharper> if you wrote your own etc/cloud/cloud.cfg.d/XXX file *and* you override the modules_stage list, *and* you pass args on that
<blackboxsw> rharper: I think args is only set via /etc/cloud/cloud.cfg.d/XXX by overriding the modules_list and providing - <config_name> <frequency>   <n-args>
<rharper> right
<rharper> you can provide user-data that overrides the list, but it's really awkward to use
<rharper> I'd rather log deprecated and avoid exposing it more
<blackboxsw> yeah it feels gross/wrong.
<blackboxsw> and undocumented
<blackboxsw> rharper: Odd_Bloke here's the undocumented image-based module config  I was talking about https://pastebin.ubuntu.com/p/qjkVYVq5Vg/
<blackboxsw> provide a list for the config module and you'd have to know the specifics of how that module reacts to additional args
<blackboxsw> so yeah we can chalk up an item to deprecate that
#cloud-init 2020-04-29
<blackboxsw> also rharper, and Odd_Bloke I believe the args passed into the handle function is also supported via 'cloud-init single --name cc_ssh_import_id --frequency always ubuntu chad.smith'   so if we are deprecating use of providing 'args' directly to cloud-init config modules, we would be impacting that CLI call path as well.
<blackboxsw> yes just confirmed on a container
<socket-> Hey all, im tryingo to set my hostname to the instanceId, and i put this in my config.  hostname: {{ v1.instance_id }}  however, it says Invalid format at line 4 column 11: "while constructing a mapping in "<unicode string>" any ideas what Im doing wrong?
<socket-> https://apaste.info/VwYP
<socket-> ^ config and this is the error
<socket-> https://apaste.info/Rjxn
<amansi26> I have a question. Is multi NIC deployment is possible with cloud init. If so which part of code is responsible to make Multi NIC IPs up?
<Odd_Bloke> socket-: You need `## template: jinja` as your first line for the substitution to happen: https://cloudinit.readthedocs.io/en/latest/topics/instancedata.html#using-instance-data
<sackrebutz> Hi. I use runcmd to do a git clone upon init. however, it seems that git is not using the private SSH key (placed in ~/.ssh/id_rsa), because when I log into the consle (TTY) and run the command, everythingâs fine. Is there a âproper wayâ to run a git pull with cloud-init ?
<sackrebutz> ah - I get âHost verification failedâ during cloud-init.
<rharper> sackrebutz: cloud-init runcmd runs as root, so you'll need to specify a path to the key
<sackrebutz> well the key is placed in /root/.ssh/id_rsa
<sackrebutz> which Iâd expect to be used when root is running `git â¦`
<rharper> I would as well;
<sackrebutz> how would i specify the key explicitly?
<rharper> man git says, GIT_SSH or GIT_SSH_COMMAND can override the default git ssh command it uses
<rharper> if nothing else, you could use ssh -vv  to see what's happening
<sackrebutz> well it works if I connect through TTY (as root) and execute the git clone. it does not work when i execute that very command through runcmd.
<sackrebutz> I have verified that the files are all in place before the git command is executed
<sackrebutz> as to the env variables, it looks like theyâre not being set , as a runcmd `set > /tmp/set` file shows ..
<amansi26> I have a question. Is multi NIC deployment is possible with cloud init. If so which part of code is responsible to make Multi NIC IPs up? Second thing is is the route files being cleaned up by cloud init itself?
<amansi26> Third is: PER_INSTANCE basically means it will run just once even if we reboot or does it mean the file have frequency PER_INSTANCE means it will run on every boot
<rharper> sackrebutz: right, we don't pass and env, you can wrap them like sh -c 'SSH_CMD=ssh -vv git clone https:....'
<sackrebutz> ah, good hint, Iâll try that!
<Odd_Bloke> amansi26: We don't have support for user-provided network configuration in general.  What cloud/data source are you using?
<rharper> amansi26: yes, https://cloudinit.readthedocs.io/en/latest/topics/network-config.html, the code for networking is cloudinit.net;     you say route files, so that sounds like sysconfig related, and cloud-init does not "clean-up sysconfig files" as good as it should, so re-using the same image in a new instance could have issues;
<rharper> amansi26: per-instance means first time image has booted on a particular platform;  subsequent reboot of the same image will not re-run certain parts of cloud-init;  especially for network configuration, this depends on the platform,  on Azure for example, network configuration is rendered on every boot, not just the first time;
<rharper> sackrebutz: https://stackoverflow.com/questions/4565700/how-to-specify-the-private-ssh-key-to-use-when-executing-shell-command-on-git
<rharper> this looks relevant
<amansi26> so the issue I see is the ip was not coming up initially to I made a custom module and wrote code to make IP up and set frequency=PER_INSTANCE. but I see my module running evertime I reboot
<rharper> amansi26: look in your cloud-init.log, you should see 'Calling handler <handler name> ... with frequency <once per-instance, always, ...>
<sackrebutz> @rharper the solution was to do `ssh-keyscan github.com >> /root/.ssh/known_hosts`
<sackrebutz> issue was bcs of HOST key not PRIVATE key - my bad.
<rharper> sackrebutz: nice!
<rharper> TIL ssh-keyscan
<sackrebutz> me too :-D
<rharper> hehe
<sackrebutz> thanks for you support!
<rharper> sure!
<Odd_Bloke> sackrebutz: As an aside, you might want to consider specifying the exact key you want in .ssh/known_hosts (using write_files).  Fetching the key at runtime means that an attacker could claim to be github.com without you realising, whereas specifying the key means that your git clones would fail in such an attack.
<Odd_Bloke> (A caveat: I don't know how often GH rotate their host keys, so there may be an unwelcome maintenance burden there.)
<sackrebutz> Odd_Bloke: Thanks - I guess I take that risk as I donât plan to deploy often, too.
<amansi26> rharper: As you mention on Azure for example, network configuration is rendered on every boot, not just the first time; is this the only case with Azure or with all other distros
<amansi26> Second thing is once the vopt get removed, the n/w conf will set to dhcp (as designed) . so is there a way where we can make sure the n/w conf set only once(apart from network: disabled)
<rharper> amansi26: unrelated to the distro, property of the Datasource, DataSourceAzure.py configures this.
<amansi26> I am using ConfigDrive as Datasource
<rharper> amansi26: I don't understand 'vopt get removed'
<rharper> ConfigDrive does not render every boot;  the default for datasources is once per instance;   Azure, EC2 (classic), SmartOS, and maybe one more Scaleway i think, are per-boot ...
<amansi26> So what I mean is if the datasource is set to None and the system is rebooted , DHCP n/w will get configured right?
* blackboxsw changed the topic of #cloud-init to: pull-requests https://git.io/JeVed | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting May 5 16:15 UTC | 20.1 (Feb 18) | 20.2 (Apr 28) | https://bugs.launchpad.net/cloud-init/+filebug
<rharper> if you don't have a datasource, cloud-init won't run
<rharper> if you're previously booted  the default network config is to DHCP on one of the interfaces; that config will still be present on new boots;
<amansi26> so is there a way where we can make sure the n/w conf set only once(apart from network: disabled)
<rharper> for configdrive, it's only once
<rharper> but generally we don't have a way to say, please generate network-config once and only once no matter what
<Odd_Bloke> blackboxsw: So I've reviewed the process doc parts of https://github.com/canonical/uss-tableflip/pull/45, and I've just approved https://github.com/canonical/cloud-init/pull/312.  I'm going to go ahead and push your branch to ubuntu/daily/devel to save you from doing it as you're doing release work currently.
<blackboxsw> thanks Odd_Bloke please do. I've got the 20.2 release branch up for review rharper Odd_Bloke https://github.com/canonical/cloud-init/pull/337
<blackboxsw> I'm currently curating the 'hightlights for 20.1 -> 20.2'
<rharper> Odd_Bloke: nice!
<rharper> blackboxsw: excellent
<Odd_Bloke> Cool, merged, synced to Launchpad and daily build kicked off.
<Odd_Bloke> Bit of a backlog for builds, so it'll be a few before we can confirm it's fixed.
<blackboxsw> good deal
<blackboxsw> SRU process bug created with 20.2 upstream release highlights  https://bugs.launchpad.net/cloud-init/+bug/1875951
<ubot5> Ubuntu bug 1875951 in cloud-init "Release 20.2" [Undecided,New]
<blackboxsw> Odd_Bloke: or rharper if either of you can review the 20.2 upstream release, I can send out email and discourse post about it
<blackboxsw> https://github.com/canonical/cloud-init/pull/337
<blackboxsw> and I can close out bugs, we can have any post-release bug management/milestone discussions separately if we need to tweak the process a bit
<rharper> blackboxsw: hrm, do you have local changes to tableflip scripts?  I used the upstream-release script and my CHangelog is different than yours
<blackboxsw> rharper: checking. nope I'm on fresh master of uss-tableflip
<blackboxsw> I did have to enter the process bug number 1875951
<blackboxsw> to the prompt
<rharper> lemme try again
<blackboxsw> but I ran cd /tmp; git clone cloud-init; cd cloud-inoit; upstream-release 20.2 20.1
<blackboxsw> I'll try re-running as well to make sure I didn't do some other stuff.
<rharper> blackboxsw: what i"m seeing is all of the cherry-pick/update changelog entries , did you manually redact those ?
<blackboxsw> rharper: makes me think you are working from non-master branch when you ran upstream-release
<blackboxsw> we need to run that script from master to get the right changelog
<blackboxsw> like you were in ubuntu/devel or daily/devel branch when upstream-release was run maybe?
<rharper> oh
<rharper> yes
<rharper> yes i iwas
<rharper> thank you
<rharper> there we go
<blackboxsw> schweet. I'll add that to the script so it warns
<blackboxsw> PR for uss-tableflip incoming
<rharper> cool
<blackboxsw> rharper: https://github.com/canonical/uss-tableflip/pull/46
<blackboxsw> a quick pre-flight check on the branch we have checked out
<rharper> k
<blackboxsw> s/ upstream-snapshot/upstream-release/ on the commit message and PR
<blackboxsw> done and force-pushed
<rharper> blackboxsw: can we get some comments added to the upstream_release doc mentioning that ubuntu/devel and release branches are going to produce incorrect output...
<blackboxsw> rharper: +1 will add that to the PR. and will also move the git pull outside of the if "master" check (as we always want to make sure master is at latest
<rharper> blackboxsw: +1
<rharper> blackboxsw: looks good
<blackboxsw> thanks rharper
<blackboxsw> done pushing to uss-tableflip PR as well
<blackboxsw> thanks
 * blackboxsw needs to setup my gorilla sbuild  enbv
<blackboxsw> env
<blackboxsw> Another one for tomorrow for releasing 20.2 to gorilla https://github.com/canonical/cloud-init/pull/339
#cloud-init 2020-04-30
<amansi26> Hi, I have a doubt, the network-details when we pass through openstack, how is that get handled by cloud init?
<amansi26> Like where is that information get processed. I am using v19.1. Not able to figure out the exact place where it get processed
<sackrebutz> hey again - i have yum_repos enabled but it seems itâs silently ignored by cloud-init when run on the server. Config: https://dpaste.org/xCEg // Log: https://dpaste.org/MafQ
<sackrebutz> Also no file is created within /etc/yum.repos.d/
<felfert_> sackrebutz: Have a look in /var/lib/cloud/instance/
<felfert_> In there are several files which represent your user-data and how it is recognized by cloud-init
<felfert_> Your config looks goot to me but there might be yaml errors somewhere else and if there are errors, then /var/lib/cloud/instance/user-data.txt.i and /var/lib/cloud/instance/user-data.txt show the difference of the raw data and how cloud-init has interpreted it
<sackrebutz> felfert_: Thanks - i can see that in both files, there is the yum_repos config and itâs just as i have put it into the config
<sackrebutz> is there any other log that would tell me what itâs doing ?
<sackrebutz> at some point, cloud-init has do make a decision whether it takes the yum_repos config into account or not i guess?
<felfert_> It does not specifically mention that in the logs
<sackrebutz> I now copy-pasted the example from https://cloudinit.readthedocs.io/en/latest/topics/examples.html?highlight=yum#adding-a-yum-repository and itâs not workiing either - might be a bug ?
<felfert_> Well, I use a much older cloud-init (that comes preinstalled with CentOS-7 stock cloud images and *that* one works)
<felfert_> Is yours a stock CentOS-8 cloud image? Or did you install cloud-init yourself?
<sackrebutz> itâs the centos8 stock
<sackrebutz> v18.5
<felfert_> The only other difference: My yaml files use 2-space indentation, yours use 4-spaces. Oh perhaps check, if you have indented with tabs accidentially.
<felfert_> Do you use multiple yaml files as userdata or just a single yaml?
<sackrebutz> I already converted to 2-spaces for the sake of testing, but dodnât make a difference
<sackrebutz> only 1 yml file
<felfert_> Then to me this looks like a bug.
<rharper> sackrebutz: can you check if  'yum-add-repo' is present in your /etc/cloud/cloud.cfg  ; do you have the complete cloud-init.log from your test ?
<sackrebutz> rharper: I stumbled over the same on https://bugzilla.redhat.com/show_bug.cgi?id=1027406 - testing it right now
<ubot5> bugzilla.redhat.com bug 1027406 in cloud-init "cloud-init should probably have yum_add_repo module enabled" [High,Closed: currentrelease]
<rharper> sackrebutz: ah, ok
<sackrebutz> rharper: it is presend below âcloud_config_modulesâ is that corret ?
<sackrebutz> present*
<rharper> yes
<sackrebutz> hm
<felfert_> But 1027406 is ancient (and fixed in 0.7.5-1.el6)
<rharper> ok, so next, let's look if it ran in cloud-init.log
<sackrebutz> yep, one sec
<sackrebutz> https://dpaste.org/WctE
<rharper> Skipping modules 'yum-add-repo' because they are not verified on distro 'centos'.  To run anyway, add them to 'unverified_modules' in config.
<rharper> nasty
<rharper> distros = ['fedora', 'rhel']
<sackrebutz> ahhh
<rharper> sackrebutz: so, you can add to your config,  unverified_modules: [yum-add-repos]   ;  let me confirm that
<sackrebutz> yum-add-repo it would be, no?
<rharper> yeah, a list with the module name
<rharper> yes
<rharper> I suspect that's worth an upstream patch
<sackrebutz> yes! now itâs adding it correctly
<rharper> \o/
<sackrebutz> i was already doubting on my whitespace indentation skills even after 20years of python
<rharper> =(
<sackrebutz> once again, thank you very much rharper  felfert_
<rharper> I generally use something like python3 -c 'import sys, yaml; print(yaml.dump(yaml.load(open(sys.argv[1]))))' my.cfg
<rharper> sackrebutz: sure
<rharper> https://github.com/canonical/cloud-init/pull/340
<potoftea> Hi I'm trying to run single module "runcmd" with command: "cloud-init --file /root/cloud-init.yaml single --name runcmd --frequency=always". From logs I see that it commands are being written (Shellified 7 commands) but not execute.  Could this be a bug?
<rharper> potoftea: it's a two step process, runcmd writes it's output to a file and later in final, runparts is called on it, let me get the final handler name
<rharper> cloud-init single --name scripts_user --frequency=always
<rharper> potoftea: ^
<potoftea> ohh ok, thank you for quick response and yeah it helped.  rharper Thank you
<rharper> sure
<amansi26> rharper: Hi, I have a doubt, the network-details when we pass through openstack, how is that get handled by cloud init?  Like where is that information get processed. I am using v19.1. Not able to figure out the exact place where it get processed
<rharper> amansi26: in cloudinit/sources/DataSourceOpenStack.py the  'network_config' property  uses cloudinit/sources/helpers/openstack.py:convert_net_json() which converts network_data.json from openstack into network-config v1 format
<amansi26> rharper: but I am using datasource as configDrive
<rharper> ConfigDrive has it's own network-config property, it has two network config formats it reads
<amansi26> you mean version1 and version2. Right?
<rharper> one is network-config which is a debian etc/network/interfaces file format, we convert that into v1 config, via  cloudinit.net.eni.convert_eni_data()  if network_data.json is present in the config drive, then we use the same openstack.convert_net_json
<rharper> amansi26: no
<amansi26> rharper: The issue I am facing is when I try deploying dhcp network. it shows type as static in netcfg (stages.py) http://paste.openstack.org/show/792946/
<rharper> what config files are inside your config drive ?
<amansi26> didn't get you rharper
<rharper> amansi26: inside the configdrive, which is an iso attacted to your instance, it contains files that cloud-init reads, in particular, there may be a file 'network_config' or 'networkdata'  ;  they wil define what your network config will be
<rharper> amansi26:  like so  https://paste.ubuntu.com/p/BGJ7fbM37p/
<amansi26> So if I need to change something in the netcfg details it has to be done from openstack commands (like we need to pass a new parameter)
<rharper> amansi26: typically yes, your tenent networks define this,  is it a dhcp subnet, or a static ip allocation, etc
<amansi26> rharper: Thanks, cleared a lot of underline concepts
<rharper> amansi26: great
<Odd_Bloke> blackboxsw: Do we have any automated testing that validates our schemas?
<Odd_Bloke> (e.g. ensures that the examples are valid under the schema.)
<Odd_Bloke> blackboxsw: https://github.com/canonical/cloud-init/pull/335 <-- ready for your review, I believe
<mutantturkey> guten morgan
<blackboxsw> good mornin :)
<blackboxsw> Odd_Bloke: approved yet needs rebase https://github.com/canonical/cloud-init/pull/341
<blackboxsw> Odd_Bloke: unittests CI runs perform schema validation against all our existing rtd examples to make sure they adhere to the schema we have just defined.
<blackboxsw> Odd_Bloke: I think it would be nice to extend that unittest run to also validate each of the schema['examples'] defined in the config modules too
<blackboxsw> to validate our docs
<blackboxsw> and the schema we claim will work
<Odd_Bloke> It sounds like you're drawing a distinction between "our existing rtd examples" and the schema examples: don't the schema examples end up in our generated docs?
<Odd_Bloke> (Just making sure I understand. :)
<bpo> I wonder if someone here might point me in the right direction to analyze my cloud-init setup. I tried to get started with `cloud-init analyze blame` but that was empty - and `cloud-init analyze dump` returns `[]`. Any hints on how I should proceed? `systemd-analyze blame` shows `cloud-config.service` taking 8.5s, and `cloud-init.service` taking
<bpo> 1.092s. This system is running Amazon Linux 2.
<Odd_Bloke> bpo: Is /var/log/cloud-init.log readable by the user you're running these commands as?
<Odd_Bloke> (And what version of cloud-init?)
<Odd_Bloke> (I'm about to be AFK for a while, so either someone else will answer or you'll have to be patient. ;)
<bpo> @Odd_Bloke Yes, /var/log/cloud-init.log is readable (running with sudo). `/usr/bin/cloud-init 19.3-2.amzn2`
<bpo> No rush
<bpo> (and the log has entries)
<blackboxsw> Odd_Bloke: right, I was trying to draw the distinction. But I was misremembering our unittests actually only validate schema against our integration tests examples in tests/cloud_tests/testcases https://github.com/canonical/cloud-init/blob/master/tests/unittests/test_handler/test_schema.py#L400-L4012
<blackboxsw> But I think it would be helpful if we actually extended unit tests to validate both the rtd examples in doc/examples as well as each cc_*.py module's schema["examples"] as well
<blackboxsw> as I believe we have found a couple doc errors in the past
<blackboxsw> ... or been informed of
<blackboxsw> it'd help vet our new schema changes as well because example cloud-config  with top-level keys that are not covered by existing schema, don't error (as we aren't strict at that level).
<blackboxsw> let's try proper grammar
<blackboxsw> Having unittests attempt shema validation on doc/examples cloud-config as well as any config module schema["examples"] will help validate schema in CI as we add it to each config module.
<blackboxsw> Performing schema validation against cloud-config from doc/examples that contains config keys not yet in schema will not warn about unknown top-level keys.
<LongLiveCHIEF> is there a way to control the order of module execution during the final 2 boot stages?
<blackboxsw> paride: for tomorrow I think we can re-enable unittests on centos in your branch https://github.com/canonical/cloud-init/pull/231  ; a git rebase with those changes and it can land
<blackboxsw> LongLiveCHIEF: if you created your own image you can edit the order of modules named in /etc/cloud/cloud.cfg
<LongLiveCHIEF> thanks!
<blackboxsw> LongLiveCHIEF: or more generally add an additional config in/etc/cloud/cloud.cfg.d with the full cloud_config_modules:  in whatever order you wanted and it would override the defaults in /etc/cloud/cloud.cfg
<LongLiveCHIEF> what if I put them in a specific order in user-data of the datasource?
<LongLiveCHIEF> i'm actually doing this with NoCloud dsmode local
<blackboxsw> LongLiveCHIEF: something like this https://paste.ubuntu.com/p/77pqCKjrBB/
<blackboxsw> so providing user-data that re-orders modules is trickier
<LongLiveCHIEF> makes sense
<blackboxsw> hrm, you could use #cloud-config's write_files  to emit that additional config file to /etc/cloud/cloud.cfg.d
<blackboxsw> https://cloudinit.readthedocs.io/en/latest/topics/modules.html#write-files
<LongLiveCHIEF> i'm working on raspberry pi booting with ubuntu 20. Got it working pretty well, but only if i disable and mask systemd-networkd-wait-online, and then run netplan apply
<blackboxsw> write-files is run the init phase which would write that config out before you got to the later stages
<LongLiveCHIEF> which means I can't use user-data modules for easy things and instead have to put them in a script called by runcmd module
<blackboxsw> nice on rasppi work. you mentioning running netplan apply cloud-init should be doing that for you pre-network bringup if you provide network configuration to it
 * blackboxsw looks up nocloud again to see the mechanism
<LongLiveCHIEF> yeah, that's the trick i use for writing out and then running netplan
<LongLiveCHIEF> i do, but it never resolves. I have a power_state module usage at the end that reboots, and the second boot is when the network finally resolves
<LongLiveCHIEF> (wifis obviously... ethernet works fine)
<LongLiveCHIEF> but if I do it the workaround way, i can do it purely with wifi and it works prior to cloud-init finishing
<LongLiveCHIEF> so I'm hoping to plug back in for other modules in the final stage
<blackboxsw> LongLiveCHIEF: hrm, so your seed directory or seed url contains a network-config file in it?
<LongLiveCHIEF> s=/boot/firmware/cloud-init
<LongLiveCHIEF> i'm creating the cloud-init directory on the boot partition, which gets mounted to /boot/firmware during initial boot
<LongLiveCHIEF> and that /cloud-init directory has just user-data and meta-data
<blackboxsw> +1 on that and /boot/firmwware/cloud-init/ contains user-data and meta-data and maybe a network-config?
<blackboxsw> ok I *think* you cold provide your netplan configuration in the network-config file as network config v2 to cloud-init to represent your entire network config on the rasp https://cloudinit.readthedocs.io/en/latest/topics/network-config-format-v2.html#network-config-v2
<LongLiveCHIEF> /boot/firmware contains network-config... which winds up merging into 50-cloud-init.conf
<blackboxsw> rharper: or Odd_Bloke could correct me if I'm wrong there
<blackboxsw> LongLiveCHIEF: right I believe so
<LongLiveCHIEF> yes, confirmed, i CAN
<LongLiveCHIEF> BUT
<blackboxsw> when network config v2 is passed to cloud-init, it should be passed directly though to /etc/netplan/50-cloud-init.yaml
<LongLiveCHIEF> if I do, then network won't fully resolve until after reboot, which means all the package modules in the final cloud-init stage will fail
<blackboxsw> hrm, that part lost me. why wouldn't it resolve if you define dhcp on eth0 and whatever wireless config you have?
<LongLiveCHIEF> but if I run netplan apply using the runcmd module during the final stage, the  network connection gets established
<LongLiveCHIEF> no wait, i don't have eth0
<LongLiveCHIEF> it's the whole wifis with cloud-init thing
<blackboxsw> I was wondering about something like this https://paste.ubuntu.com/p/YrmftwVD9G/
<blackboxsw> but I think I'm missing your problem
<LongLiveCHIEF> <blackboxsw "hrm, that part lost me. why woul"> @blackbox that's the part I can't figure out either.  I admit, i haven't spent enough time digging through the logs yet though.
<LongLiveCHIEF> i'm doing something like that, only i've removed eth0 entirely
<LongLiveCHIEF> if i leave it in, then cloud init hangs forever
<LongLiveCHIEF> if i take it out, then it finishes, but doesn't establish network connection until reboot
<LongLiveCHIEF> which means I can't use modules that run in the final stage that require connection
<LongLiveCHIEF> and the funny thing is, canonical has these instructions verbatim in their new 20.04 docs for IoT, despite literally everyone you see on youtube stating wifi networking will fail if you do this: https://ubuntu.com/tutorials/how-to-install-ubuntu-on-your-raspberry-pi#3-wifi-or-ethernet
<blackboxsw> LongLiveCHIEF: this reminds me of this bug https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1870346
<ubot5> Ubuntu bug 1870346 in cloud-init (Ubuntu) "Wifi configuration" [Medium,Triaged]
<blackboxsw> is this also what you are experiencing?
<blackboxsw> I think it could be.... or it'll be what you hit next if you provide your network-config file.
 * blackboxsw has to jump away for a bit. will check later 
<LongLiveCHIEF> https://github.com/HackerHappyHour/bootcc/tree/setup-config-options/system-boot
<LongLiveCHIEF> yes, that's what I'm experiencing.  As the bug report states, it's not an inconvenience to reboot... however, that means that you can't use many of the cloud-init modules in the final stage that require a network connection
<LongLiveCHIEF> i've got it working with the project i linked above so network will work even on first boot, but I want to make sure I can control the order of the final stage modules to ensure I can still use them, instead of having to run another script in the runcmd stage that will install packages and do other configs that cloud-init already has modules for
<LongLiveCHIEF> *working so that even wifi networks will work on first boot
<LongLiveCHIEF> It would be cool if the docs for the modules included what stage each module was run in.  Some of the modules specify, but most don't
<LongLiveCHIEF> https://bugs.launchpad.net/ubuntu/+source/netplan.io/+bug/1874377
<ubot5> Ubuntu bug 1874377 in netplan.io (Ubuntu Focal) "Netplan does not connect to Wireless after `sudo netplan apply` until reboot" [High,Fix released]
<LongLiveCHIEF> I think i know how to fix this. I'll work on it. thanks for the help blackboxsw_
<LongLiveCHIEF> scratch that, there's already a fix merged! https://github.com/CanonicalLtd/netplan/pull/133
<rharper> blackboxsw: we don't netplan apply specifically because we run before networkd starts, so once we netplan generate (which creates the required systemd-networkd files) and naturally systemd-networkd finds what it needs.
<rharper> LongLiveCHIEF: w.r.t netplan and wifi; and the linked PR; it's not yet clear to me that the fixed version is actually fixed, as in the bug blackboxsw mentioned,   (and in your case) we have a valid wifi config, and the "fixed" version of netplan. yet not seeing wifi come up;  so something is still missing in netplan;  that is, the apply fix is too late;  whatever needs to happen during apply should happen on the initial boot;  so maybe this clean up
<rharper> of other networkd services mentioned needs to be figured out without a netplan apply command
<LongLiveCHIEF> yeah, i'm seeing the same thing now that i've gone through the PR
<LongLiveCHIEF> my end goal is the same as it would be on a digital-ocean droplet really. To setup a specified user, and packages, on first boot/provision
<LongLiveCHIEF> using wifi only
<rharper> yeah, AFAICT, the only issue is netplan itself;
<LongLiveCHIEF> and I can actually do that now using cloud-init, it's just a hackier way of doing it than i prefer
<LongLiveCHIEF> just annoying because I need that netplan apply prior to the start of the config stage of cloud-init, (and it seems it would definitely work by that time), but I can't really hook into it until the final stage
<rharper> well, you shouldn't
<LongLiveCHIEF> haha, why, there's a reason network stage comes before config stage :p
<rharper> right
<rharper> when we generate, everything should be fine ... I'm going to see what gets written out on generate;  are you in a focal image ?
<LongLiveCHIEF> yessir
<rharper> k
<LongLiveCHIEF> i can confirm that generate creates everything correctly
<rharper> that's confusing then ...
<LongLiveCHIEF> no kidding!! haha
<rharper> we're still missing something if apply fixes things
<LongLiveCHIEF> yep
<rharper> ok, so 0.99-ubuntu1 writes a /run/netplan/wpa-wlan0.conf (and stuff)
<rharper> upgrading to ubuntu2;
<rharper> and it still writes it
<LongLiveCHIEF> not sure which build i was on, downloaded it early in the day on the 24th, which is the day 0.99 was released
<rharper> ah, that;'s the config
<LongLiveCHIEF> so i may have just missed it
<rharper> could you try the daily image from today ?
<LongLiveCHIEF> point me to the dl?
<rharper> https://cloud-images.ubuntu.com/daily/server/server/focal/current/
<LongLiveCHIEF> they don't make dailies for the preinstalled arm64
<LongLiveCHIEF> not sure how to work with the arm64 cloud root image
<rharper> ah, that's a bummer
<rharper> well, you can mount up the image, upgrade the package, and try with that
<rharper> you can use cloud-init --clean --logs
<rharper> to clear out the previous run (mostly, some things like package instakks and user-adds will remain) but you can test the networking bit
<LongLiveCHIEF> yeah, that's what I was going to do
<LongLiveCHIEF> i have a hunch that I want to test first
<LongLiveCHIEF> i'm telling netplan what IP to assign.  I wonder what will happen if I just let an IP get assigned by DHCP instead
<LongLiveCHIEF> might be a bug with managing resolve when using manually assigned IP's
<Odd_Bloke> LongLiveCHIEF: (I'm catching up on backlog so apologies if this isn't useful info, but) there are known issues with wifi on Pi that I believe are being worked on.  I'll see if I can find a bug reference.
<LongLiveCHIEF> no worries.
<LongLiveCHIEF> no rush. I'm just waiting to put everything together for docs for github.com/octoprint/docker for a recommended setup guide for our less tech savvy users
<Odd_Bloke> bpo: Hmm, that's strange.  I'm not sure what's going; what does `cloud-init analyze blame -i /var/log/cloud-init.log` give you?  (The same, I expect.)
<bpo> Yep, same:
<bpo> -- Boot Record 01 --1 boot records analyzed
<LongLiveCHIEF> i get 4
<LongLiveCHIEF> ah n/m, i've booted that particular one a few times
<Odd_Bloke> rharper: The netplan fix only landed 2 days ago, it's a better version of the pre-release wifi fix.  Have we definitely seen runs with that new version (that's only in groovy-proposed currently)?
<Odd_Bloke> bpo: Would you be able to pastebin your cloud-init.log?  (Or, alternatively, could you try copying that file onto an updated Ubuntu server, xenial or later, and running `cloud-init analyze blame -i that_file`?)
<LongLiveCHIEF> I'm guessing if I change the cloud.cfg.g final_module order using write_files, it wouldn't take the new config in until service restart
<LongLiveCHIEF>  * I'm guessing if I change the cloud.cfg.d final_module order using write_files, it wouldn't take the new config in until service restart
<Odd_Bloke> LongLiveCHIEF: cloud-init actually executes from scratch for each phase, so I believe later phases would pick up the new configuration.
<Odd_Bloke> rharper: (^ this is perhaps something we need to consider for the daemon plans, in terms of reloading configuration?)
<LongLiveCHIEF> write_files happens in the final stage though, correct?
<LongLiveCHIEF> so it would be too late
<Odd_Bloke> LongLiveCHIEF: You can see the order by looking at /etc/cloud/cloud.cfg; write_files runs early in the init phase (i.e. the first one).
<Odd_Bloke> (At least in the configuration we ship upstream and in Ubuntu by default. :)
<LongLiveCHIEF> cool
<LongLiveCHIEF> i have a workaround method that works for me atm, so I'm going to run with that for the next week, but lmk if you guys want me to test anything/share logs with you in the meantime
<LongLiveCHIEF> i did test out my dhcp hunch, and it was a no-go. Still failed to resolve when enabling dhcp (both 4 and 6)
<LongLiveCHIEF> i've learned a ton about cloud-init as a result of all this though, so at the very least, I'll likely be submitting a few docs contributions over the next 2 weeks
<Odd_Bloke> \o/
<Odd_Bloke> (That's why we leave the bugs in. ;)
<LongLiveCHIEF> i'm also considering writing a cloud-init module that would allow you to do things like set the default pin states and such for pi
<Odd_Bloke> Oh, that would be cool!
<LongLiveCHIEF> you fix wifi, i'll add gpio. deal?
<LongLiveCHIEF> ð
<LongLiveCHIEF> my pipedream goal however, is to enable raspberry-pi imager to be a datasource
<Odd_Bloke> LongLiveCHIEF: I'm not familiar with the RPi ecosystem, I'd be interested to hear a little more about how you would see that working.
<LongLiveCHIEF> raspberry pi imager is an electron based application that burns bootable usb's.  What that means though is that it could also use local network details to lookup and generate network-config and user-data at an endpoint accessible to your own internal network, and it could easily add that network endpoint to the ds seed in cmdline.txt
<LongLiveCHIEF> so imagine the rpi imager having similar configuration screen as digital-ocean, and in one step it downloads, burns, and configures a bootable microsd for your pi that will install whatever packages you desire
<LongLiveCHIEF> many of the options I'm talking about i've organized into a project here: https://github.com/HackerHappyHour/bootcc/projects/1
<LongLiveCHIEF> i'm going to be hacking on a prototype for the next 4 days, and then see if it's something that makes sense to contribute to the https://github.com/raspberrypi/rpi-imager project
<LongLiveCHIEF> annonced here: https://www.raspberrypi.org/blog/raspberry-pi-imager-imaging-utility/
<blackboxsw> good point Odd_Bloke on daemon config reload. that'd be a gap vs current implementation
<LongLiveCHIEF> i'll ping you guys when i have a simple demo ready. Probably about this time tomorrow
<blackboxsw> good deal LongLiveCHIEF.
<Odd_Bloke> LongLiveCHIEF: Cool!
<bpo> Odd_Bloke: https://pastebin.com/uhJNysgm
<bpo> Must be an issue with the distro, that's from a fresh AWS Linux 2 AMI with no customisations - same behavior.
<Odd_Bloke> bpo: That doesn't give me any output with cloud-init master so Something Strange is happening; I'll see if I can figure it out.
<bpo> Odd_Bloke: thank you! I'll be curious to hear what you find. I will check back in periodically but will probably be slow to respond.
<LongLiveCHIEF> bpo: you guys talking about the wifi/network issue or something else?
<Odd_Bloke> Something else. :)
<Odd_Bloke> bpo: OK, the issue has something to do with the log format differing between upstream and Amazon Linux (meaning that the upstream code doesn't identify any of the cloud-init.log lines as being cloud-init log lines).
<Odd_Bloke> bpo: Could you run `grep _log -R /etc/cloud` on an instance to find any cloud-init logging configuration (and if you find some, pastebin it)?
<blackboxsw> Odd_Bloke: https://github.com/canonical/cloud-init/pull/335 is good (needs rebase)
<Odd_Bloke> blackboxsw: https://github.com/canonical/cloud-init/pull/329 is ready for your re-review.
<blackboxsw> hrm Odd_Bloke. so lxc images are all a different hash, how do we know if a hash is from a xenial image or a bionic. and if we've changed from xenial -> bionic across travis runs aren't we just removing all images and re-downloading the new series
<Odd_Bloke> blackboxsw: We only use one lxd image in our builds, a xenial one.
<Odd_Bloke> Let me add that to the explanatory comment.
<blackboxsw> ahh right --os-name xenial
<blackboxsw> ok I'm good with that then
<blackboxsw> but comment welcome for future me
<Odd_Bloke> blackboxsw: Pushed as a separate commit for ease of review: https://github.com/canonical/cloud-init/pull/329/commits/32b2ea23d28accf4c4c34f32865c8041712e2480
<bpo> Odd_Bloke: Here is /etc/cloud/cloud.cfg.d/05_logging.cfg: https://pastebin.com/x6NZ4NFJ
<Odd_Bloke> bpo: Thanks!  I'm just finishing up my day, so I'll take a look in the morning.
<bpo> Odd_Bloke: sounds good, thanks
#cloud-init 2020-05-01
<ms7821> hi, anyone know how cloud-init calculates metrics? It claims to base them on the adapter ID, but I've ended up with a broken AWS instance because eth0 had metric=200 and eth1 metric=100
<ms7821> editing netplan/50-cloud-init.yaml fixed this, and persists across reboots/runs of cloud-init
<ms7821> so it looks a lot like metric calculation isn't idempotent
<ms7821> this looks suspiciously like it's basing the metric on the MAC https://github.com/canonical/cloud-init/blob/6600c642af3817fe5e0170cb7b4eeac4be3c60eb/cloudinit/sources/DataSourceEc2.py#L766
<ms7821> ahh, https://github.com/canonical/cloud-init/pull/114 fixes it by the looks of it
<ms7821> which is odd given I have the latest version
<Odd_Bloke> ms7821: The "latest version" on which distro?
<ms7821> EC2's Ubuntu 20.04 LTS
<ms7821> $ cloud-init -v gives 20.1-10-g71af48df-0ubuntu5
<ms7821> which should include that PR
<Odd_Bloke> Yep, it should.  So perhaps this is a separate bug?  Could you file one at https://bugs.launchpad.net/cloud-init/+filebug and include the tarball that `cloud-init collect-logs` on a failing instance (if you can get access somehow)?
<ms7821> ohh wait maybe it doesn't
<Odd_Bloke> It does, it was included in https://launchpad.net/ubuntu/+source/cloud-init/20.1-10-g71af48df-0ubuntu3
<rharper> ms7821: it would be really good to have cloud-init collect-logs on your failing instance and the rendered netplan file;  prior to the PR you linked, we did not renderer route-metric on secondary nics;  rather we did not render config for any secondary nics in AWS;
<ms7821> ahh, perhaps I've misread the PR then. maybe it's *caused* the bug
<rharper> possibly, yes
<ms7821> I read "On Ec2, cloud-init will configure networking on all attached NICs, instead of just the primary NIC." as the current state of affairs
<rharper> yes,  and that was added in Focal only;  older releases do not yet do that by default;
<ms7821> so I probably can't share the collect-logs but I'm pretty sure sorting by mac is wrong
<rharper> we're not sorting by mac
<ms7821> it should use meta-data/network/interfaces/macs/*/device-number
<rharper> we rely on the order provided by the instance metadata
<Odd_Bloke> ms7821: Can't collect, or can't share for non-technical reasons?  (We can help work around the former, not so much the latter. ;)
<ms7821> I mean if I share it I need to rebuild this instance which has taken me a couple of days to troubleshoot already
<ms7821> should it be creating a default route for secondary interfaces at all?
<rharper> that's what the route metric is for; it's provided in the instance metadata
<ms7821> OK, looking at this PR, it deleted and reintroduces the line I linked above
<ms7821> deletes*
<rharper> ms7821: I think I see where the issue is w.r.t obtaiing the device-number
<ms7821> that line where it derives nic_idx from sorting by the macs is wrong
<rharper> the macs-to-nics list index isn't what we want at all;  it should have been the device-number set in metadta
<rharper> ms7821: indeed
<rharper> it's fine for us to enumerate in any order but the index value we assign should come from the instance metadata value
<Odd_Bloke> rharper: In your opinion, should we land a fix for this in master before kicking off SRUs?
<rharper> Odd_Bloke: yes
<rharper> I think our multi-nic testing should/would have found this
<Odd_Bloke> rharper: Shall I start looking into it, or are you on your way already?
<rharper> Odd_Bloke: please do
<rharper> nic_idx = int(nic_metadata['device-number']) + 1  # zero-based
<rharper> that's basically what we need to do instead
<rharper> so, device-number: 0  will have the lowest metric and all secondary nics+ are higher;
<Odd_Bloke> blackboxsw: FYI, ^ is a blocker on starting the SRU process for 20.2
<Odd_Bloke> rick_h: (FYI too. :)
<ms7821> looks like the macs are static and ordered in test_ec2.py, which is why it won't have been picked up
<ms7821> (on the plus side, this will usually be the case so I expect very few people will see it)
<Odd_Bloke> Yep, I just changed that test data to reproduce this particular issue.
<Odd_Bloke> ms7821: Would you like to file a bug for the issue (even without collect-logs)?  (If not, I will. :)
<ms7821> sure, can do
<Odd_Bloke> Thanks!
<ms7821> https://bugs.launchpad.net/cloud-init/+bug/1876312
<ubot5> Ubuntu bug 1876312 in cloud-init "route metric on multihomed ec2 instances is based on mac address instead of device-number" [Undecided,New]
<ms7821> thanks for picking this up so quickly!
<rharper> ms7821: thanks for reporting it in here
<rick_h> Odd_Bloke:  sorry was in a call. /me reads back
<rick_h> ah good find then and thanks ms7821
<Odd_Bloke> rick_h: No problem (and no action from you required), just keeping you in the loop.
<Odd_Bloke> rharper: https://github.com/canonical/cloud-init/pull/342
<bpo> (Hopefully) quick issue for someone who knows Sphinx/RST better than I do: the link for 'analyze' in the first sentence of this section should link to another page (the top-level 'analyze' label) but instead links to the 'cli_analyze' label within the CLI page: https://cloudinit.readthedocs.io/en/latest/topics/cli.html#analyze -- when building the
<bpo> docs there are several warnings about duplicate label names which may point to the issue.
<rharper> bpo: thanks
<LongLiveCHIEF> is it acceptable to submit documentation issues for cloud-init on launchpad?
<powersj> LongLiveCHIEF, please do
<blackboxsw> +1 LongLiveCHIEF
<LongLiveCHIEF> :thumbsup:
<LongLiveCHIEF> like this? https://bugs.launchpad.net/cloud-init/+bug/1876333
<ubot5> Error: Could not gather data from Ubuntu for bug #1876333 (https://launchpad.net/bugs/1876333). The error has been logged
<Odd_Bloke> bpo: FYI, I filed https://bugs.launchpad.net/cloud-init/+bug/1876323 for the analyze issue you were seeing yesterday.
<ubot5> Ubuntu bug 1876323 in cloud-init "`cloud-init analyze` fails to produce useful output on Amazon Linux 2 due to log format configuration" [Undecided,New]
<Odd_Bloke> bpo: Would you mind filing a bug for that doc issue you identified?
<bpo> Sure, happy to.
<Odd_Bloke> Thanks!
<bpo> Odd_Bloke: do you think this issue with the arg0Formatter's interaction with cloud-init should be raised separately with AWS? Or is it something the cloud-init project will address
<bpo> doc issue filed here: https://bugs.launchpad.net/cloud-init/+bug/1876341
<ubot5> Ubuntu bug 1876341 in cloud-init "Broken analyze link in CLI page" [Undecided,New]
<Odd_Bloke> Thanks for the bug!
<Odd_Bloke> bpo: The log format _is_ configurable, so we need to figure out a better way of handling this in general.  In the short term, I'll look into fixing analyze for this particular log format, but in the medium term we may want to prefer using journalctl (which allows us to specify the format we want, circumventing the problem of what's on disk entirely) or something else along those lines.
<bpo> Makes sense. Reading straight from journald would be great!
<Odd_Bloke> (Obviously with a fallback to the log file if we don't have journald, in case anyone reading along is worried. :)
<Odd_Bloke> blackboxsw: So I've attempted to run through the manual process AIUI, and it hasn't worked for me.  Can you review https://paste.ubuntu.com/p/HBbVmcf5BY/ and let me know if I did the wrong thing (or omitted steps)?
<Odd_Bloke> blackboxsw: (Just noticed I omitted it, but those commands were all run on the ubuntu/daily/devel branch.)
<blackboxsw> Odd_Bloke: will do
<blackboxsw> Odd_Bloke: ok I see the problem in the logic there. hrm.  ok might have to go back to the drawing board. I'm toying with a couple of scenarios to make sure it's viable for multiple cherry picks. I expected we'd have to do a manual push to u/daily/devel to get this in the right state. but adding more cherry-picks might always hit those conflicts related to a missing debian/patches/series file each time we merge a
<blackboxsw> new cherry pick/patch.
<Odd_Bloke> blackboxsw: I have a method that did work: https://paste.ubuntu.com/p/kWqny27sHm/
<blackboxsw> nice, though I'm trying to resolve that missing series file. I don't think this was the case if we were merging u/devel -> u/daily/devel. We shouldn't have to resolve those conflicts each time we uss-tableflip/cherry-pic
<blackboxsw> so Odd_Bloke what happened on your local ubuntu/devel then if you then ran uss-tableflip/scripts/cherry-pick 25698b144e3b6548ffc29ab14bed1882242b161a
<blackboxsw> I'd expect more errors on the ubuntu/daily/devel 'git cherry-pick` that gets called too
<blackboxsw> s/errors/conflicts
<Odd_Bloke> Well, ubuntu/daily/devel shouldn't have a series file, because we've reverted all the patches.
<Odd_Bloke> But ubuntu/devel will have one (if any cherry-picks have happened, of course).
<blackboxsw> right, which I think also means that u/daily/devel cannot git cherry-pick <a debian patch commitish u/devel> after we've already reverted the first cherry-pick
<blackboxsw> so every time our u/devel gets past 1 cherry-pick-debian-patch, then u/daily/devel can no longer git cherry-pick the 2nd debian patch into u/daily/devel because the debian/patches/series file is already absent
<rharper> doesn't daily/devel come from a previous series; don't we always have at least *one* patch in the series file ?
<Odd_Bloke> It's absent and that's different to ubuntu/devel (where it exists with N-1 lines, if this is the Nth cherry-pick).
<blackboxsw> rharper: not once we git revert that original cherry-pick from u/daily/devel. then series is gone
<rharper> can you have comments in series files ?
<rharper> can we have an empty file ?
<blackboxsw> yes, but still git conflicts because the diff won't match right
<Odd_Bloke> Either way the content will be different between the two branches.
<Odd_Bloke> So git is going to ask us how it should resolve that difference.
<Odd_Bloke> I don't think this is a _huge_ problem: the daily branch allows us to resolve these conflicts _once_ then push it.
<Odd_Bloke> rharper: (FYI, responded to your comments on https://github.com/canonical/cloud-init/pull/342)
<blackboxsw> hrm. Odd_Bloke I'm trying to get back to the  reason we care about only merging  u/daily/devel back into u/devel and why we don't want to just force push to u/daily/devel and recipe biulds build from there directly
<Odd_Bloke> blackboxsw: Force push what to u/daily/devel?
<blackboxsw> if we simplified things a bit, and just force pushed u/devel -> u/daily/devel and added a single commit to reverse patch all debian/patches/cpick* files or rm debian/patches/cpick*, then aren't we building what  we care about
<Odd_Bloke> We were trying to avoid having to update another branch during Ubuntu releases.
<Odd_Bloke> And if we're prepping debian/* changes in ubuntu/devel for the next upload, then we would need to remember to manually sync the daily branch./
<blackboxsw> so each time we uss-tableflip/cherry-pick the cherry-pick tool of ours would force push u/devel -> u/daily/devel for us and revert any debian/patches/cpicks
<Odd_Bloke> If we're only building from u/daily/devel, then we need to constantly keep u/devel pushed to u/daily/devel though.
<Odd_Bloke> Not just on cherry-pick.
<blackboxsw> hrm darn right
<blackboxsw> thanks for the refresh there. you're right.
<Odd_Bloke> No worries, there are a bunch of moving pieces, it is tricky.
<blackboxsw> Odd_Bloke: if uss-tableflips/cherry-pick kept a running ordered list of the commits it pushed/popped, it'd be able to re-apply them all and properly revert the latest (it was my earlier iteration on the cherry-pick PR)
<blackboxsw> so we could reapply all cpicks  to u/daily/devel then revert all in reverse order
<blackboxsw> then we wouldn't have conflcts
<blackboxsw> just a bit of extra noise on u/daily/devel branch with the re-apply all, revert all
<Odd_Bloke> I _think_ git wouldn't recognise those re-applied cherry-picks as being the same as the cherry-picks applied to u/devel.
<blackboxsw> I mean we'd track the local revert commitish on u/daily/devel an revert the revert kinda thing
<Odd_Bloke> blackboxsw: Instead of thinking about how to automate it, try runnig through your proposal manually and see what happens. :)
<blackboxsw> that sounds good will have a past
<blackboxsw> paste
<Odd_Bloke> Thanks!
<rharper> Odd_Bloke: lol, https://bugs.launchpad.net/bugs/1876363
<ubot5> Ubuntu bug 1876363 in cloud-init "network-config macaddress needs to be lower case" [Undecided,New]
<rharper> this showed up in the inbox *just* as I approved 342 =)
<Odd_Bloke> HAH
<Goneri> blackboxsw, I've refreshed my patch to use parameterize: https://github.com/canonical/cloud-init/pull/305
<blackboxsw> Odd_Bloke: finally got it
<blackboxsw> though too little to late I presume. paste coming
<blackboxsw> Odd_Bloke: rharper for monday/tuesday https://paste.ubuntu.com/p/jPxsJtQMmg/
<blackboxsw> it involves us reseting ubuntu/daily/<series> each time we cherry-pick a change, since this branch is only intended to revert cpicks for daily builds, I don't see the harm in us resetting and force pushing u/daily/<series> when we have a cherry-pick to perform. This would be the only time we'd update u/daily/<series> branches
<blackboxsw> it also avoids having to maintain two branches every commit that lands in ubuntu/<series> branches and it avoids merge conflicts as  u/daily/<series> would only get updates to exactly what ubuntu/<series> has exactly when a cherry-pick is added.
#cloud-init 2020-05-02
<LongLiveCHIEF> Odd_Bloke: idk if you or rharper had something to do with it, but there are now groovy builds for preinstalled image, and there wasn't 3 days ago
<LongLiveCHIEF> whoever did that... thank you
#cloud-init 2020-05-03
<LongLiveCHIEF> still no wifi connection from modifying the default network-config, however using `netplan apply` as the first script of `runcmd` in the final stage brings network online
<LongLiveCHIEF> (groovy gorilla, yesterdays build)
<LongLiveCHIEF> i have logs collected and can send wherever if anyone wants a look
