#cloud-init 2013-11-14
<harlowja> ok i'm back
<harlowja> yaaaaa
<harlowja> smoser getting tim the canoncial license stuff
<harlowja> have the do a little red-tape to make that possible
<harlowja> smoser tim still working through red tape, legal guy is busy today
<harlowja> if it takes to long, i'll just submit patch
#cloud-init 2013-11-15
<smoser> harlowja, cool. thanks.
<harlowja> smoser yt
<harlowja> since the legal crap might take a while, https://code.launchpad.net/~harlowja/cloud-init/mtab-fix/+merge/195453
<harlowja> tim might of pissed off the legal guy, lol
<harlowja> by saying something like 'why is this taking so long, wtf'
<harlowja> if i have to get rid of tim's name there let me know ;)
<harlowja> and i'll squish it away
#cloud-init 2013-11-17
<harmw> hi guys
<harmw> I was wondering if anyone could shed some lights on how a centos6 instance is to configure static networking (based on what it received through dhcp) with cloud-init
<harmw> does it require loading specific modules in /etc/cloud/cloud.cfg, or putting specific stuff in cloud-config?
#cloud-init 2014-11-10
<smoser> o.
#cloud-init 2014-11-11
<spandhe> Hi everyone!
<spandhe> qq.. does cloud-init support IPv6?
<smoser> spandhe, "sort of".
<smoser> if you're not feeding networking in via confdig-drive or something then really thats just a matter of the image config.
<spandhe> I am using config drive
<smoser> and inserting a /etc/network/interfaces ?
<smoser> it could work. there might be some issues with ubuntu images.
<smoser> but largely if /etc/network/interfaces is going to come up fine, then that should work.
<smoser> MAAS is using cloud-init in ipv6 only scenarios.
<spandhe> one odd thing I observed is, inside the disk. cat /etc/sysconfig/network doesnt say networking_ipv6=yes
<smoser> so it is possible.
<smoser> then that (sysconfig) probably needs some work for it.
<smoser> it is work that needs to be done "really soon now".
<spandhe> I see that /sysconfig/network-scripts/ifcfg-eth0 has the right info.. but probably because ipv6 is disab;ed in network file, network config is not happening correctly.. 
<spandhe> smoser: is that something to be done on cloudinit side or nova side?
<smoser> spandhe, well, that is probably cloud-init reading a /etc/network/interfaces style file provided to it by nova/neutron
<smoser> and rendering it.
<smoser> to sysconfig style
<smoser> the going-forward plan for this is to have a better declaration format for netwroking
<smoser> than "/etc/network/interfaces".
<smoser> https://review.openstack.org/#/c/85673/13/specs/juno/metadata-service-network-info.rst
<spandhe> smoser: can u point me to the code where this rendoring happens?
<smoser> probably in cloudinit/distros/rhel.py
<spandhe> smoser: thanks! :) I found https://github.com/number5/cloud-init/blob/master/cloudinit/distros/rhel.py#L94-L98
<spandhe> doesnt seem to do NETWORKING_IPV6=Yes
<spandhe> smoser: adding NETWORKING_IPV6=yes seems to help.. should I open a bug?
<smoser> spandhe, sure.
#cloud-init 2014-11-12
<spandhe> smoser: looking further, need a few more things.. 
<spandhe> smoser: the network-scripts file should have the following format: http://www.cyberciti.biz/faq/rhel-redhat-fedora-centos-ipv6-network-configuration/
<spandhe> Therers âIPV6â prefix, which is currently missing
#cloud-init 2014-11-13
<Wumpus42> question about /etc/cloud/cloud.cfg. How does One determine into which "section" (cloud_init_modules, cloud_config_modules or cloud_final_modules) to put a module? I wanted to try out the resolv-conf module, but it wasn't listed in the default /etc/cloud/cloud.cfg created by the yum install of cloud-init.
<Wumpus42> I tried it in cloud_config_modules:, then just before final-message in cloud_final_modules, which seemed to work. Is the intent that you can put any of the modules in any of the sections? Or are they intended to be part of just a particular section?
<Wumpus42> Also: Is order important? Does where you put the module in the .cfg file affect when it's "run"?
<Wumpus42> over :)
<Wumpus42> Umm... another also: The example at http://cloudinit.readthedocs.org/en/latest/topics/examples.html#configure-an-instances-resolv-conf seems to be wrong (for 0.7.5 anyway). It should be "manage_resolv_conf: true" not "manage-resolv-conf: true"
<smoser> Wumpus42, order is important.
<smoser> and some modules wont mnatter where you put them, but others would need to be run sooner or later.
<smoser> Wumpus42, you're probably right about that dock.
<smoser> :-(
<Wumpus42> but they do get "run" in the order in which they are in their section?
<Wumpus42> thanks for the feedback. is the doc error a bug report? I didn't see anything on the page itself that lent itself to feedback
#cloud-init 2014-11-14
<Odd_Bloke> smoser: I'm scoping out Python 3 changes; would we be talking about supporting Python 2 _and_ 3, or shifting to just Python 3?
<Odd_Bloke> smoser: If both, is just 2.7 acceptable?
<JayF> I definately use cloud-init on boxes with only py26 :(
<JayF> (CentOS 6)
<smoser> Odd_Bloke, 2 and 3
<smoser> Odd_Bloke, https://code.launchpad.net/~harlowja/cloud-init/py2-3/+merge/225240
<smoser> that is harlowja's start. 
<Odd_Bloke> smoser: Thanks for the pointer.
<harlowja> smoser ok back!
<harmw_> https://cloud-images.ubuntu.com/utopic/current/utopic-server-cloudimg-armhf-disk1.img 
<harmw_> smoser: can I just add that to glance?
<harmw_> you happen to know?
<harlowja> harmw u can try my tool @ https://github.com/stackforge/anvil/blob/master/tools/img-uploader.py 
<harlowja> if it still works, ha
<harlowja> https://github.com/stackforge/anvil/blob/master/anvil/components/helpers/glance.py is the code that it uses to extract image, upload it...
<harlowja> should still work i think (although it may expect an image archive with a kernel and ram disk as well)
<harmw> harlowja: could you decribe what it tries to do? :) looks like extracting initrd+kernel and adding that to glance, right?
<harlowja> ya
<harmw> (im unsure on how to exactly get ARM images in glance)
<harlowja> pretty much :-P
<harlowja> let me look at, forgot what it does, haha
 * harlowja wrote it to long ago
<harlowja> from what i remember, it calls into https://github.com/stackforge/anvil/blob/master/anvil/components/helpers/glance.py#L439
<harmw> ok, so, why :p since kernel+ramdisk are often just supplied :p
<harlowja> which then calls into to extract, register, upload @ https://github.com/stackforge/anvil/blob/master/anvil/components/helpers/glance.py#L391
<harlowja> harmw mainly u have to associate the ramdisks and crap with the raw image file, although if u just have a single image, i think u can just upload that directly
<harmw> glance image-create --name 'Ubuntu Utopic ARM' --disk-format raw --container-format bare --property architecture=armv7l --file utopic-server-cloudimg-armhf-disk1.img
<harlowja> ya, something like that should work also
<harmw> I'm hopeing it will :p
<harmw> libvirtError: internal error: no supported architecture for os type 'hvm'
<harmw> hm
<harlowja> hmmm, ya, not sure about that one
<harlowja> https://ask.openstack.org/en/question/7519/instance-error-caused-by-libvirterror-internal-error-no-supported-architecture-for-os-type-hvm/?answer=7520#post-id-7520 
<harlowja> maybe 'libvirt_type=qemu'
<harlowja> in nova.conf
<harmw> nah, I apparently didn't load kvm - trying to that now
<harmw> *krnel module
<harlowja> k
<harmw> ah, yes, I needed the -lpae kernel for that
<harmw> hmk, well, I can boot a vm from that image - but it doesn't show anything in the console.log
<harmw> or am I using the wrong hardware thingy here
<harlowja> unsure :-P
#cloud-init 2015-11-11
<natorious> smoser: is anyone meeting today?
<smoser> natorious, bah
<smoser> i'm not really supposed to be working
<smoser> Canonical gets national holidays off
<natorious> right on.  So, I've been thinking about the state of things.  Perhaps getting moderator access would be helpful for progress on 2.0
<smoser> moderator access? natorious ?
<natorious> so I can help contribute to others merge requests in gerrit.  I have some other merge requests to push up still too.  Did a bit of work around adding a subprocess check_output derivative and started on some network_config for linux
<natorious> I'd also broke out some distro namespaces on a branch to get that going
<natorious> I'm not sure their would be immediate gain but, at least I could add value to promoting dev help on the project
<natorious> also did a bit of work getting windows to detect config drive partitions using ctypes from stdlib
<natorious> all those more or less depend on the two merge requests that are already in so if there is something needed w/ them, do let me know
<manuq> hi, if a cloud-config yaml has runcmd and packages, are packages installed before or after the commands?
<manuq> I have packages: - git and runcmd: - git clone ...
<manuq> I better switch to cloud-init bash script?
<natorious> manuq: it depends on the order you have them defined in your /etc/cloud/cloud.cfg file
<natorious> the default upstream config has package-update-upgrade-install running before runcmd
<natorious> but, there is also bootcmd which runs before them as part of early boot
<natorious> I'd think you'd get better results from using the cloud-init modules than just passing in a bash script content file
<natorious> but if your not into leveraging all the other modules and only need to run simple commands, that might be a better option
<manuq> natorious, that's an excellent answer, thanks
<natorious> most of the distros should have the same upstream ordering but, its possible that the package creator could change it
<natorious> np
#cloud-init 2015-11-12
<smoser> Odd_Bloke, hm..
<smoser> about to merge this, but one question
<smoser> is data from dmi == data in shareconfig ?
<smoser> or will an updated instance think that it is now a new instance because it loks in a different place
<Odd_Bloke> Oh, that's an interesting point.
<Odd_Bloke> smoser: Just bringing up a fresh instance to test. :)
<Odd_Bloke> smoser: Oh, bah, no they aren't the same ID.
<Odd_Bloke> Blah, and that means we can't even get away with it in xenial, because older instances might be upgraded to xenial and get the new code.
<smoser> well, we can handle it in package upgrade
<Odd_Bloke> Ah, true.
<Odd_Bloke> It would be good to not have to maintain the old code paths in trunk.
<smoser> ideally we do something that doesn't require package upgrade
<smoser> erl... doesn't require package upgrade to fix the problem for us.
<smoser> ie, best if e can somehow dtrt.
<smoser> i absolutely want to be able to ditch the old code though.
<Odd_Bloke> smoser: I don't know that we can do anything without keeping the old code.
<Odd_Bloke> smoser: Do you have a specific case where package upgrade won't work?
<smoser> just as upstream
<smoser> would be better for upstream to handle it
<smoser> Odd_Bloke, but if you can do a good job in packaging, i think i'm ok to take it.
<cbolt> does runcmd happen before scripts-user?
#cloud-init 2015-11-13
<smoser> cbolt, "it depends"
<smoser> runcmd is executed via 'scripts-user'
<smoser> runcmd is executed via a script named 'runcmd' that gets written into the 'scripts/per-instance' dir (/var/lib/cloud/instance/scripts/per-instance)
<smoser> and then that dir is executed run-parts style, so C locale sorted.
<cbolt> ty smoser
<mwak> hello
<mwak> smoser any update about the PR? https://code.launchpad.net/~edouardb/cloud-init/scaleway-datasource/+merge/274861
<smoser> mwak, so you 're ok that it wont be enabled by default, right?
<niluje> smoser: by default, you mean we'll need to update the cloud-init configuration file to add the scaleway provider?
<smoser> yes.
<mwak> smoser: yes, no pb with that!
<niluje> so yep that's ok :)
<smoser> but really... i thikn you're the providers, right? look into exposing something in dmi. and then we can.
<smoser> ok. next thing.
<niluje> since we have no choice but doing a network resource, we'll make some documentation to ask to our users to do some configuration to use the provider
<smoser> i run 'tox' on my system ehre, and your tests fail.
<niluje> oO
 * niluje retries
<smoser> i think they end up timing out
<smoser> requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): 169.254.42.42
<smoser> cloudinit.sources.DataSourceScaleway: DEBUG: Trying to get user data (bind on port 1018)...
<niluje> give me a sec, I don't have the code in mind
<smoser> http://paste.ubuntu.com/13247624/
<niluje> indeed, it seems there's an issue. The test were running fine when I did them. I'm looking into it and will be you when fixed
<niluje> Oooook, of course they were, I was running nosetests and not tox, we'll fix the issue
<smoser> hm.
<smoser> nosetests woudl just use your system dependencies rather than those in the tox
<smoser> so possibly a dependency difference
<niluje> yep
 * niluje finds tox useful, but hates it
<smoser> niluje, i share those feelings.
<smoser> mine extend to pip
<niluje> mine extend to pip and setuptools
<smoser> but, they're useful for creating similar environments.
<smoser> when you have some spare time, you can create 'pip-distro-env' for me.
<smoser> thats what i really want
<smoser> pip-distro-env ubuntu-14.04 urllib3 pkg1 pkg3 ....
<niluje> https://github.com/spotify/dh-virtualenv
<smoser> and have that create me a venv with those things in it at the right versions.
<smoser> (for ubuntu-14.04)
<smoser> thats interesting. but not what i'm interested in.
<niluje> not sure to understand what you're looking for then
<smoser> i want to be able to easily test a venv of a set of python dependencies at the versionsx they are at for a bunch of different distros
<niluje> oh
<niluje> dox
<niluje> looked right for that
<niluje> idk if it's mature or usable though
<niluje> but having a tox-like with docker seemed fun
<niluje> https://pypi.python.org/pypi/dox/0.1
<smoser> but 'pip install . -r test-requirements.txt'
<smoser> what would that do ?
<smoser> just a pip install
<smoser> which is generally (ideally) selfl contained anyway.
<smoser> right?
<smoser> i guess it would help with getting the python at the correct version
<smoser> but then i want somethign to know all the package versions for those things too
<niluje> you mean you want to test your package with every version of its dependencies?
<Odd_Bloke> niluje: Just every set that you might find packaged for a distribution.
<smoser> not *every* version, but the right versions
<smoser> yeah.
<Odd_Bloke> smoser: Trying to do it with pip is probably impossible because of changes made in packaging.
<niluje> can't you make more tox environments, and pin dependencies in it?
<smoser> dox is similar in concept though. and actually using the *packaged* versions is even superior (outside of the additional weight)
<smoser> niluje, yeah, you can do that. but figuring out what package is at what version in what distro... just that is a PITA
<smoser> thats why i was trying to have *you* do it for me. and maintain that database :)
<niluje> ok :p
<smoser> but thanks for the pointer to dox. containers of each environ are definitely a way to do that.
<smoser> and as Odd_Bloke pointed out, that covers the depends packaged differences too.
<Odd_Bloke> Yeah, then hopefully all you would need is a mapping from pip name => package name for each distro.
<niluje> grml
<niluje> can't run tox in my environment
<niluje> I'm on osx ( :( ), cloud-init unittest require to be on linux, I'm trying to run them on a docker container, but python setup.py sdist tries to make a hardlink which fails because cloud-init is an exported folder
<Odd_Bloke> niluje: You're doing sdist, or tox is doing sdist?
<niluje> tox is doing it
<niluje> I'll clone cloud-init in the docker, that's just inconvenient but that'll work
<smoser> niluje, you want me to give you an instance somewhere ?
<smoser> ah. ok. you found a solution.
<smoser> i  just launch instances on "the cloud".  i wonder if you have a cloud provider you'd recommend :)
<niluje> ever heard about digital ocean? they're quite new but they look promising
<niluje> :ppp
<mwak> :D
<kwadronaut> they're cheap. Loads of webdevs have their unpatchd wordpresses and such over there ;-)
<niluje> kwadronaut: that was a joke, b/c I'm working for Scaleway, another cloud provider :p
<kwadronaut> niluje: ah so I can bug you for some free goodies? ;-)
<niluje> kwadronaut: poke mwak instead!
<smoser> niluje, they're 32 bit arm, right?
<smoser> no kvm
<niluje> yes
<niluje> yes, dedicated hardware
<smoser> right. but i cant run kvm there.
#cloud-init 2015-11-14
<spandhe> hi folks, is this the correct repo for latest cloudinit code? https://github.com/openstack/cloud-init
#cloud-init 2016-11-14
<Odd_Bloke> vans163: If your shell supports <(...), that might work?
<vans163> Odd_Bloke: I need to try but it needs user-data AND meta-data,  does it just cat the two?
<vans163> if it just needed one I would definetly try piping it in without thinking about it
<Odd_Bloke> I would expect: genisoimage ... <(cat user-data) <(cat meta-data)
<vans163> Odd_Bloke: let me try that thats a good idea
<smoser> vans163, you could just use cloud-localds too
<smoser> which cleans up stuff, and provides the meta-data if you dont need it.
<vans163> smoser: that is useful, so I can just pip the (cat user-data) into cloud-localds?  http://ubuntu-smoser.blogspot.ca/2013/02/using-ubuntu-cloud-images-without-cloud.html  this tutorial does not show how to account for the user-data supplied by user which can be a bash script
<smoser> oh. i dont know if it supports user-data from - or not
<smoser> let me see
<smoser> seems like it does
<vans163> smoser: where are you looking if you dont mind me asking, googling does not seem to reveal any documentation
<smoser> $ printf '#!/bin/sh\necho hi mom\n' | cloud-localds -v my.img -
<smoser> wrote my.img with filesystem=iso9660 and diskformat=raw
<smoser> vans163,  i looked at source code :) and source code I wrote even ;)
<smoser> http://bazaar.launchpad.net/~cloud-utils-dev/cloud-utils/trunk/view/head:/bin/cloud-localds
<smoser> http://bazaar.launchpad.net/~cloud-utils-dev/cloud-utils/trunk/view/head:/bin/cloud-localds#L206
<smoser> but as Odd_Bloke suggested above, bash with <() basically makes a temp file and then cleans it up i think.
<vans163> smoser: ty is that () really critical?  so in my case it would look like <("#cloud-config\npassword: ...") ?  does () not mean execute whats inside?
<smoser> vans163, see 'Process Substitution' in 'man bash'
<smoser> $ bash -c 'for i in "$@"; do echo -n $i:; cat "$i"; done' -- <(echo first) <(echo another)
<smoser> /dev/fd/63:first
<smoser> /dev/fd/62:another
<smoser> in your case, if you just need the one thing, then its sufficient to do like i did above with printf, or you could do a "here document"
<smoser> $ cloud-localds -v my.img -  <<END> #!/bin/sh
<smoser> > hi mom
<smoser> > END
<smoser> note, the '> ' are the prefix the shell added to m ulti line input.
<smoser> sell, shoot. but the '#!/bin/sh' was on a new line too' p
<smoser> better: http://paste.ubuntu.com/23476971/
<vans163> smoser: ah ty for that explain, i think if it works that way, geniso should do the trick,  I do not see the benefit of cloud-localds for simple usecase like mine
<powersj> smoser: why is there both a "set hostname (cc_set_hostname)" and an "update hostname (cc_update_hostname)" modules? They both use the same keys, so does that mean they both always get run when set?
<smoser> powersj, remind me again if i dont answer.
<smoser> i have to go now htough
<powersj> smoser: will do thx
#cloud-init 2016-11-15
<aisrael> I want to execute a post-initiation piece of user-data. I tried adding the data to /var/lib/cloud-scripts/per-boot but the script fails. What should that script look like?
<rharper> http://cloudinit.readthedocs.io/en/latest/topics/examples.html#run-commands-on-first-boot
<aisrael> rharper: I tried that, but seeing an error in cloud-init.log. Here's the script: http://pastebin.ubuntu.com/23480882/
<aisrael> and the relevant log: http://pastebin.ubuntu.com/23480880/
<rharper> aisrael: and the first paste is the contents of that 99-test ?
<aisrael> rharper: Correct
<smoser> well, it says:
<smoser>  ['/var/lib/cloud/scripts/per-boot/99-test']#012Exit code: -#012Reason: [Errno 8] Exec format error#012Stdout: ''#012Stderr: ''
<smoser> oh. yeah. so aisrael if you want to just execute something per boot
<smoser> the things in that dir need to be executable
<smoser> they're things to execute, not cloud-config
<rharper> aisrael: if you use runcmd, it's exec directly from cloud-init.   vs.  adding a file to that dir
<rharper> what I suggested is different than writing files to per-boot dir
<aisrael> smoser: Is there a way to run a bit of cloud-config on every boot? For background, I'm writing a cloud-init charm layer, where the underlying machine has been pre-allocated (think manual provider)
<smoser> you can change the runcmd to run every boot. the thing that stinks about doing that is that you have to re-define all of 'cloud_final_modules' (see /etc/cloud/cloud.cfg) in order to modify the frequency of scripts-user
<aisrael> smoser: ok, supposing I did that ([scripts-user, always]), would I then put that cloud-config into /etc/cloud/cloud.cfg.d?
<smoser> http://paste.ubuntu.com/23480925/
<smoser> you can put it in user-data, or you can modify the file in /etc/cloud/cloud.cfg or add it to a file in that dir.
<rharper> aisrael: fwiw, your examine with runcmd works fine on a new instance; I think your failure was related to the modifications of files in the local per-boot dir
<smoser> right.
<smoser> aisrael, you can also use write_files to put something into /var/lib/cloud-scripts/per-boot
<smoser> and that thing that you write there will run once per boot
<rharper> that seems best
<smoser> aisrael, http://cloudinit.readthedocs.io/en/latest/topics/modules.html#write-files
<aisrael> Ok, let me see if I can get this working via cloud.cfg. It's not writing files, per se, that I need but to inject custom cloud-config into an already bootstrapped instance.
<smoser> aisrael, oh. well, that will be more difficult.
<aisrael> smoser: rharper: Thanks for the info!  I'm not sure what my path forward on this will be, but I have a clearer idea now.
<smoser> aisrael, you can 'cloud-init single --frequency=always --name=<that-name>'
<smoser> but you have to know what that-name is. ie, which cloud-initmodules will need tob e run for your config.
<aisrael> smoser: ok, that makes sense
<smoser> rharper,
<smoser> http://pad.ubuntu.com/tMdgYGEvgL
<smoser> look at the bold
 * rharper looks
<smoser> i'm not sure if we want Before=basic.target there.
<smoser> versus before=sysinit.target
<rharper> hrm
<rharper> lemme look at my yakkety instance
<smoser> pitti reviewed that and acked it at https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/310547
<rharper> you have Before=basic.target twice
<rharper> line 24 and 26/27
<rharper> http://paste.ubuntu.com/23481950/
<rharper> smoser: I think looking at that, basic.target is too late
<smoser> rharper, its not twice. where do you see that ?
<rharper> I mentioned the lines
<rharper>  - add Before=basic.target (**wonder if this is right**)
<rharper> - add Before=basic.target
<rharper> Â Â Â Â Â  this is done to become part of "early boot" in order to to be able to affect mounts (http://pad.lv/1611074)
<smoser> oh. in the pad.
<rharper> yes
<rharper> sorry
<smoser> i thought you mean tin the .service
<smoser> adjusted the pad
<rharper> so, the -replace 'Wants'/After local-fs.target with 'systemd-remounte-fs.service' is much earlier
<rharper> if local-fs.target is good enough? why would we wait longer ?
<smoser> we dont wait longer.
<smoser> we are now running earlier (possibly)
<smoser> after local-fs.target would possibly include all the local systemd generated .mount
<smoser> which we have to run before
<smoser> i think its right.
<rharper> smoser: so, Xenial without the SRU is running *too late* for mount;  what did we put in Yakkety ? Before=local-fs.target ? and slangasek suggests systemd-remount-fs.service ?
<smoser> slangasek saw what he believed was a loop
<rharper> smoser: I'm confused between what we did in yakkety, vs. where Xenial is at (I'm ignoreing where Xenial is at, and interested in changes to cloud-init trunk to understand the changes; then we can look at the Xenial -> SRU changes
<smoser> but not one i'd seen in practice.
 * rharper looks a this cloud-init-loca.service now
<smoser> sure. thats fine. thats ultimately what we're doing.
<rharper> the Wants=local-fs.target with an After=local-fs.target
<rharper> is what we have in yakkety now
<rharper> for cloud-init-local.service
<rharper> that *functions* for early mount
<rharper> and slangasek suggests that makes a loop or something like that ?
<smoser> right.
<smoser> well, he did.
<rharper> Im not seeing it either
<smoser> which is why i dropped the local-fs stuff
<smoser> i'm not sure it is or not
<smoser> but essentially saying : After=local-fs.target
<smoser> *and*
<rharper> it seems we need a Before=sysinit.target *AND* a After=systemd-remount-fs.service
<rharper> maybe that RequiresMountsFor does the trick
<smoser> no.
<smoser> shoot.
<smoser> so... we drop the local-fs.target
<smoser> as we dont need that and because we're writing things to /etc/fstab, it seems like there could be a loop there.
<smoser> as we say that those mounts should not be processed untill after cloud-init.service
<rharper> ahh
<rharper> I see
<smoser> but we said cloud-init.service should be after local-fs.target (which would include them)
<smoser> it seems reasonable
<rharper> yes
<smoser> but i'd never seen it
<rharper> oddly neither have I
<rharper> and there's no complains
<smoser> no 'break's or complaints or anything
<rharper> complaints from journalctl
<rharper> right
<smoser> so i suspect that sytemd resolved it elsewhere
<smoser> probably by adjusting the .mount things
<rharper> well
<rharper> clearly from the chain, local-fs.target is *reached* prior to cloud-init-local running
<rharper> and trivially cloud-init.service
<smoser> i assume it changed the meaning of local-fs.target
<rharper> so it must not be a hard requirement w.r.t fstab writing
<smoser> so at the moment i think that Before=basic.target is ok,
<smoser> at least i dont have any fallout of that that i'm aware of
<rharper> it's clearly OK, as basic.target is much later
<smoser> i think i'll prepare sru with it as it is in zesty
<rharper> the question is whether we want to ensure we run even earlier
<smoser> much later than waht ?
<rharper> say Before=sysinit.target
<rharper> or local-fs.target
<smoser> yeah, so i suspect that cloud-init.services 'Before=basic.target' ends up not doing anything
<rharper> smoser: so, I think we have to be after local-fs.target in that, we can't assume we can write to /etc/fstab unless we have local-fs.target, right ?
<smoser> shoot
<smoser> yeah, so i suspect that cloud-init-local.service  'Before=basic.target' ends up not doing anything
<smoser> as cloud-init.service is Before=sysinit.target and After=cloud-init-local.service
<rharper> well, it's before it, but that's not really making it run "early"
<smoser> so cloud-init-local.service gets forced early due to cloud-init.service's Before=sysinit.target
<smoser> so i think this is fine.
<rharper> smoser: we might want to do an in HO with slangasek in a bit
<smoser> :)
<jgrimm> +1
<rharper> I think cloud-init-local.service should be explicit
<smoser> rharper, yes, we should change it , but it does not actually change anything
<smoser> added https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/310919
<rharper> smoser: yes
<powersj> smoser: ping re: set_hostname vs change_hostname
<powersj> err s/change/update/
<smoser> powersj, thinking
<smoser> ok. so
<smoser> update_hostname runs PER_ALWAYS (every boot)
<smoser> (more coming)
<smoser> powersj, http://pad.ubuntu.com/NtJAvEJEiJ
<powersj> smoser: thank you. Let me think about that and implications with testing them
<powersj> harlowja: around? question re: cc_ssh_authkey_fingerprints
<smoser> rharper, http://paste.ubuntu.com/23482707/
<smoser> does that seems right ?
<rharper> systemd-networkd.service is After dbus.target should say After=dbus.service
<rharper> and sysinit.target is before dbus.service; hence the loop
<rharper> dbus.service which runs after sysinit.target, but cloud-init.service is Before=sysinit.target
<rharper> smoser: and the last paragraph is spot-on
<smoser> rharper, fixing
 * rharper relocates home
<vans163> question. Does cloud-init automatically resize the root partition to the size of the image?  Say i download a cloud image that is 2gb max size, i do qemu-img resize img.img 50G, making it 50G now, when it first boots using the cloud-init iso will the root partition autoresize to 50G/
<powersj> vans163: I think so... take a look at https://cloudinit.readthedocs.io/en/latest/topics/modules.html#resizefs
<vans163> powersj: ty guessing default may not be true
<smoser> vans163, yes.
<smoser> was it not doing that ?
<smoser> you can tell it not to. but generally thats the desired behavior
#cloud-init 2016-11-16
<vans163> smoser: I want it to, i set it to true right now getting ready to test
<vans163> would genisoimage execute the meta-data (containg #cloud-config) and user-data in the order specified?
<vans163> so for exampl I should probably set the meta-data first, then the user-data (which can also be #cloud-config but supplied by user)
<vans163> genisoimage -output cloud-init.iso -volid cidata -joliet -rock meta-data user-data    ?
<vans163> https://cloudinit.readthedocs.io/en/latest/topics/modules.html#set-passwords  Also is that top level password by default the root password?
<vans163> For example even on ubuntu which seems to say the default user is ubuntu
<smoser> vans163, not sure what you mean 'execute'
<smoser> password is for default user.
<smoser> meta-data does not get executed. and order of arguments to genisoimage does not matter. you're mastering a filesystem.
<vans163> smoser: i see okay, I think i need to use chpasswd then to change root password
<niluje> j/close
<vans163> is the correct way to mount the cloud-init for qemu to set it to -hda ?
<vans163> also do we need to remove it before restarting the vm? or would cloud-init gaurantee it wont execute twice?
<cpaelzer> vans163: I don't know what config you have - but e.g. user-scripts usually only run once
<cpaelzer> vans163: on those you find a bit more, maybe the frist slightly outdated http://stackoverflow.com/questions/6475374/how-do-i-make-cloud-init-startup-scripts-run-every-time-my-ec2-instance-boots http://cloudinit.readthedocs.io/en/latest/topics/examples.html#run-commands-on-first-boot
<cpaelzer> and I must admit I should know better but I don't
<cpaelzer> gut feeling tells me to remember that the actual config is only done once, but I'm not feeling confident enough to tell any hard facts
<cpaelzer> highlighting smoser for some real insight for you
<cpaelzer> smoser: ^^
<smoser> vans163, most user-ata is handled once "per instance"
<smoser> meaning per-instance-id. if you want to run it again, you'd need to change thte instance id that is on the config disk.
<smoser> mostly , it shoudl "do the right thing".
<smoser> if you want to detach that disk, then you will need to tell cloud-init "manual_cache_clean: True".  And you have to write that (probably via user-data) into /etc/cloud.cfg.d or the next reboot, it will go  looking for a datasource again.
<smoser> -hda doesn't matter.
<smoser> any block device will suffice.  cloud-init finds disks by the disk label, so as long as the guest sees the block device its fine.
<rharper> vans163: alternatively, if you're just re-testing that same user-data; I usually blow away the instance data (sudo rm -rf /var/lib/cloud/* /var/log/cloud-init* ) and reboot;
<smoser> i would nto suggest using '-hda' generally as virtio is superior in just about everyr way
<rharper> other than having a nice shorthand cli name
<vans163> smoser: im thinking if I dynamically can add disks and things. if then it would be good to assign that cloud-init disk to something that would not get in the way.
<vans163> for example one instance may need cloudinit.iso mycustomiso.iso  disk.qcow2 disk2.qcow2
<vans163> where should cloudinit.iso so it does not get in the way
<smoser> i dont know what "get in the way means".
<vans163> because afaik using -cdrom it automatically assigns a hdx
<rharper> there's only one cdrom
<smoser> it doesnt have to be a cdrom
<smoser> any block device.
<rharper> vans163: yeah, if you use virtio to host the iso file, that works fine too
<smoser> pretty sure you *could* attaach multiple cdroms to a guest if you wanted.
<rharper> the IDE devices (hd[abcd]) supports only 4 devices, but virtio-blk, or scsi support many more
<smoser> but i'd just do it as a virtio disk
<smoser> surely you could with virtio cdrom
<rharper> smoser: only two, IIRC
<rharper> well, virtio block
<rharper> with an ISO payload
<vans163> ah so  -drive id=disk0,file=/cloud/cloudinit/100.iso,if=virtio   for example?
<smoser> ther eis a virtio cdrom i think, no?
<rharper> that's not quite the same as a cdrom device
<rharper> vans163: yeah
<vans163> then disk1 could be the qcow2?
<rharper> smoser: no, just virtio-blk, or virtio-scsi (which can have cdrom properties)
<smoser> ah.  sure.
<rharper> vans163: sure;  Ubuntu cloud-images find the root devices via label
<rharper> and cloud-init finds the datasource disk via label as well
<vans163> disk1=...disk1.qcow2   disk2=...disk2.qcow2   disk5=...Another.iso
<rharper> root=cloudimg-rootfs  userdata=cidata
<smoser> vans163, cloud-localds --help shows:
<smoser>     * kvm -net nic -net user,hostfwd=tcp::2222-:22 \
<smoser>          -drive file=disk1.img,if=virtio -drive file=my-seed.img,if=virtio
<smoser> that works fine.
<vans163> ah so i can drop the id then, im not using it elsewere in the cmdline
<smoser> drop the id ?
<vans163> smoser: -drive id=disk0,..
<smoser> oh. well, that is a more verbose way of doing things. you do need the moer verbose way in some cases.
<smoser> qemu is not fun ot interact with from a command line.
<smoser> i use xkvm. http://bazaar.launchpad.net/~curtin-dev/curtin/trunk/view/head:/tools/xkvm
<smoser> which generates those id= and such.
<smoser> its not perfect, but makes at least network devices and block devices easier
<smoser> xkvm --disk=/tmp/asdf --dry-run --netdev=user
<smoser>  qemu-system-x86_64 -enable-kvm -device virtio-scsi-pci,id=virtio-scsi-xkvm -device virtio-net-pci,netdev=net00 -netdev type=user,id=net00 -drive file=/tmp/asdf,id=disk00,if=none,index=0 -device virtio-blk,drive=disk00,serial=asdf
<smoser> http://bazaar.launchpad.net/~curtin-dev/curtin/trunk/view/head:/tools/xkvm
<vans163> smoser: that looks sweet ty
<vans163> gonna check it out
<vans163> so im just seeing a black screen with blinking cursor
<vans163> i tried to use a live iso and it works fine, when i use the cloud-init iso i made with genisoimage
<vans163> black screen, blinking cursor at bottom left
<vans163> I used genisoimage -output 100.iso -volid cidata -joliet -rock user-data meta-data    user-data was an empty file, meta-data had the #cloud-config
<vans163> catting the 100.iso i can see the #cloud-config is htere
<vans163> could this mean my #cloud-config is incorrect, or could it mean the iso is not correctly generated?
<powersj> vans163: if you suspect issues with your cloud-config you could use lxd to test it if the config is not too complex.
<powersj> if the config is bad you can view the /var/log/cloud-init-output.log and it will have a message about failing to read the user data
<vans163> i think its to do with qemu and the image i am using, i cant figure this out atm
<vans163> I download xenial-server-cloudimg-amd64-disk1.img and set that as first disk via -disk
<vans163> but even that does not boot. afaik without cloud init the cloud image should boot no?
<vans163> i can boot using a live iso just fine though, say if i insert a Fedora_24_workstation.iso
<vans163> Qemu just says booting harddisk.. failed to boot, not a bootable disk.  I set virtio off
<vans163> gave the .img 777 even
<vans163> file on it reveals xenial.img: QEMU QCOW Image (v2), 2361393152 bytes
<powersj> Are you using qemu via the cli? or via someting like virt-manager?
<vans163> cli for now i can paste my config
<powersj> if you could
<vans163> so 2 cases here i tried both. 1 with that -cdrom 1 without.  With cdrom errorcode 0004, without cdrom hangs indefinately on Reading harddrive (i forgot to change format=raw to format=qcow2)  https://gist.github.com/anonymous/5049a93256c69f4a2a3dd93bc163c826
<vans163> "Could not read from CDROM (errorcode 0004)
<vans163> 100.iso is the genisoimage with the #cloud-config
<vans163> my thoughts are even if the #cloud-config is wrong, the CDROM should still boot/read from and give an error
<vans163> or maybe not..
<vans163> ahhh missing a \ at end of the -cdrom line. added it and its stuck indefinately at booting from harddrive
<vans163> iotop reveals 0 disk activity
<vans163> http://imgur.com/a/FrBJV
<powersj> looking - nice catch on the \
<vans163> powersj: ty note if I change that -cdrom to -cdrom /root/iso/fedora_24.iso  i boot immediately to the live installer
<vans163> my best guess right now is not proper genisoimage ?
<powersj> well so your qemu command line says boot order=d which means to boot the cdrom first
<powersj> order=c will boot the drive first
<powersj> when using -drive I think you have to use order=c as well, but I'm not certai
<powersj> certain, rather
<powersj> qemu-system-x86_64 -enable-kvm -boot order=c -drive format=qcow2,file=ubuntu-16.04-server-cloudimg-amd64-disk1.img
<powersj> is what I used as a very simple example to get drive going. Otherwise may chime in with better suggestions
<vans163> let me try
<vans163> it just boots from harddrive first now and stucks indefinetaly. I think a good question would be can a ubuntu cloud image (qcow2 format) boot itself?
<vans163> without a cloud-init iso supplie
<smoser> vans163, you have to boot the cloud image.
<smoser> i'm confused as to what you were trying to boot.
<vans163> does not matter what, if I boot the CDROM it fails and boots the harddrive, if i boot the harddrive it skips booting cdrom
<vans163> in both cases the problem is the same, infinite stuck while booting harddrive   seems there are issues https://bugs.launchpad.net/cloud-images/+bug/1573095
<vans163> going to try another cloud image
<smoser> vans163, can you just show what you're doing ?
<smoser> you're using qemu (or kvm) right ?
<vans163> this is qemu.  and i think the problem is im not using the right clud image
<vans163> it seems most cloud images are for openstack or ec2
<vans163> going to try the generic centos image
<vans163> 1 moment
<vans163> okay so the centos generic cloud image boots to grub atleast
<vans163> then the infinite hang, guessing this is where cloud-init comes into play
<vans163> so after grub it needs to do cloud-init?
<vans163> *after grub boots kernel
<smoser> http://paste.ubuntu.com/23486774/
<smoser> ^ that works. i dont know about centos images, or what they have.
<smoser> but quite possible you can substitue the centos cloud image for this.
<vans163> smoser: wow let me try that!
<vans163> smoser: ty so much it works!
<vans163> smoser: 1 key point that i missed was the xenial image is compressed. had to qemu-img convert -O qcow2 to uncomrpessed
<smoser> you dont have to do that.
<vans163> strange
<smoser> you can, but you dont have to
<vans163> let me try again
<vans163> first time i tried it hanged at Reading harddrive..
<smoser> if you try with compressed, it will be slower.
<smoser> as you're doing cpu decompression on every read
<smoser> but it will work
<smoser> i did exactly above and it worked.
<vans163> yoyur right
<vans163> it works in both cases
<vans163> strange i think im just confused as i have liek 10 of these configs
<vans163> must of confused something, but compressed and not compressed both work
<vans163> going to try that same seed with the openstack debian image
<vans163> works
<vans163> I found the problem. I was using -cpu host
<vans163> when I add that to your config, i get the hang on boot
<vans163> just like with my config
<vans163> but as far as im aware -cpu host gives you better performance?
<vans163> it seems it might not be the case
<rharper> vans163:  -cpu host just exports host cpu features into the guest but the default qemu64 cpu is usually sufficient unless you have a highly tuned application looking for specific recent cpuflag features and the app would otherwise turn things off
<rharper> for example, ffmpeg does per-cpu tuning of ussing sse2 vs sse4;  there are ways to expose just those features instead of -cpu host which can cause issues for migration (if that's a use-case)  , you can do -cpu qemu64,+sse4  etc.  or use the cpu classes like Nehalem, etc.
<smoser> vans163, where are you running this ?
<smoser> adding -cpu host doesn't change anything for me.
<smoser> you probably need -m 512 (or more) . default memory is 128, and with that i saw an OOM on boot. i'd just not go there.
<vans163> smoser: i get  warning: host doesn't support requested feature: CPUID.0DH:EAX [bit 0]  (bit 0 1 2) 3 of these errors
<smoser> yeah, i see that.
<smoser> just noise
<vans163> Maybe -cpu host is passing all features of the cpu but this particular cpu is buggy?
<vans163> hum
<vans163> rharper: good to know that -cpu qemu64,+sse4  I will note that, as I woudl like the guest to take advantage of things like that and AES-NI
<vans163> also note that i never had an issue using -cpu host before, but here its refusing to boot the qcow2 image
<vans163> and on this node i can boot a qcow2 image with fedora that i installed from a iso with no problems. and win10 works
<rharper> vans163: if you have the bootlog from the failed scenario, I can look at see if something obvious comes up
<vans163> i built qemu from source hence its version 2.7.5
<vans163> maybe something do to with that?
<vans163> any idea how to fetch that?
<rharper> you can use -serial stdio
<rharper> and it will spit serial console data to stdout
<vans163> sec
<rharper> Ubuntu cloud images boot with console=ttyS0 by default, so boot messages will show up there
<vans163> nothing out of the ordinary, those same warning msgs
<vans163> then vnc shows Booting from hard disk... MBR   and stuck
<vans163> remove -cpu host,  and it shows the same thing except continues
<vans163> then boots
<vans163> maybe theres a way to get more logs?
<vans163> it does not even get to boot  http://imgur.com/a/FrBJV  this screen is where it gets stuck on .  Except it also prints MBR now
<vans163> maybe.. its in some kind of eufi mode and looks for a GPT?
<vans163> It also prints SYSLINUX 6.03 EDD 20150820 Copyright (C) 1994-2014 H. Peter Anvin et al  after the MBR on a new line. and thats it
<vans163> cpu usage is 100% io seems 0
<rharper> what host cpu do you have?
<rharper> I wonder if the 16-bit emulation in-kernel kvm is failing
<vans163> Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz   stepping 7, microcode 0x710
<rharper> newer intel cpus have a feature to provide support for 16-bit "big real mode" instead of having kvm do software 16-bit emulation
<vans163> ohh i enabled 1 thing new, on the motherboard i enabled power saving mode
<vans163> before it had all powersaving and C states disabled, and i passed to the kernel to disable it
<rharper> power states shouldn't matter
<vans163> okay so thats off the table
<rharper> can you show your full commandline ? with the -cpu host ?
<vans163> i could run it in valgrind and print a report
<rharper> it's in-guest
<rharper> so valgrind won't show yuou anything
<vans163> ah
<rharper> it's stuck in bios
<vans163> sec
<rharper> which is early bootstrap of the vm, running 16-bit asm of SeaBIOS
<vans163> https://gist.github.com/anonymous/5cd8e1110161adba6a597b1494ea458d   remove that -cpu host everything works
<rharper> this may help see the difference of bios boot between with and without -cpu host;  -chardev stdio,id=seabios -device isa-debugcon,iobase=0x402,chardev=seabios
<rharper> and just the debian image? or all images?
<vans163> all
<rharper> ok
<vans163> (ubuntu deb fedora)
<vans163> i could boot non-cloud images but I did not test once i enabled power saving
<vans163> let me try..
<vans163> so its not an oversight
<vans163> gonna try your extra param first
<rharper> ok, on Yakkety qemu 2.6.1, that boots fine with a Xenial VM
<rharper> on my Intel NUC (i5)
<rharper> can you try a different qemu as well?  I can't quite see why -cpu would affect the bios boot
<rharper> if you drop -enable-kvm
<rharper> and it boots, that might suggest some emulation if 16-bit boot code being an issue ... what host kernel ?
<vans163> https://www.diffchecker.com/JOdZ8vQk  left fails right works. (no diff)
<rharper> very strange
<rharper> what about leaving -cpu host, vut droppping the -smp ?
<vans163> Linux x7 4.7.6-200.fc24.x86_64
<rharper> I'm just poking at things;  there's noething obviously wrong with your command, and I see no reason why host cpu flags would affect it;  you could try to extract the kernel/initrd from the image, and boot those directly with -kernel  and -initrd;  that would take the bios bits out of the picture
<rharper> basically we need to hunt and peck to find out which code path is failing;  something is sensitive to the cpu flags; just not sure what or why; nothing obvious
<vans163> drop smp still stuck.  drop kvm gives error (host cpu model require kvm).  Do you think I should try the QEMU that is in fedora 24 (the current host system)
<rharper> yeah, that's another data point
<vans163> it would be 2.6.2
<vans163> 2.6.2 works
<vans163> o.o
<vans163> does not get stuck
<vans163> Also 2.6.2 does not print the warnings that CPU extension bits are missing
<vans163> maybe it means 2.7.5 got better support for other cpu extensions, and host is enabling them but really the CPU dont have them, and QEMU gets stuck trying to use them?
<rharper> lemme see what that's about
<rharper> it did sorta stickout
<vans163> i tried googling for those errors but there is close to 0 details
<vans163> *warnings
<rharper> https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg07278.html
<rharper> that looks relevant
<rharper> the xsave cpu flag, it seems
<vans163> rharper: let me check if that landed
<vans163> IT does not seem so . https://github.com/qemu/qemu/blob/83c83f9a5266ff113060f887f106a47920fa6974/target-i386/cpu.c#L2264
<vans163> but i386 arnt I using x86_64?
<vans163> or is i386 still used?
<vans163> in an emulated way to something
<rharper> it's not i386 specific
<rharper> it's just how the cpu targets are organized
<rharper> the target-i386 code produces both a 32-bit and 64-bit cpu model
<vans163> they have 2.8 out now, let me try to patch that manually first, then stash and try 2.8
<vans163> Im not sure if I even need 2.7.5 i just wanted -device virtio-input-host-pci support
<vans163> funny the latest commit on 2.7.5 is sept 28. so 2.8 might fix things
<vans163> commit 453ac8835b002263a6 is very nice, its a doc update on pcie passthrough best practises
<powersj> rharper: the snappy module is primarily for snap based system I take it? If for example I wanted it to install packages from the store on a cloud image could I use it for that?
<powersj> I was hoping https://paste.ubuntu.com/23487411/ would install those snaps, not just install snap
<rharper> are those snaps in stable channel ?
<powersj> yes
<rharper> which cloud-init are you testing?
<rharper> those certainly should get installed
<rharper> I'm testing with 'hello' snap
<rharper> but I can switch to those
<rharper> powersj: it will work on any system that has snapd installed (so classic, or all snap)
<powersj> rharper: using lxd cloud image of xenial with the cloud config linked above
<rharper> powersj: snap has to be installed, which it is in xenial +
<powersj> snap list after boot works, but shows nothing installed
<rharper> which cloud-init are you running ?
<rharper> we're just now sru
<rharper> 'snap' command into xenial cloud-init; it's not there yet
<rharper> so, yakkety  would work
<rharper> on default lxc, snap install fails
<powersj> 0.7.8-1-g3705bb5-0ubuntu1~16.04.3
<rharper> so there may be some sort of lxd update or config to enable snap install to work
<rharper> # snap install hello
<rharper> error: cannot perform the following tasks:
<rharper> - Mount snap "ubuntu-core" (423) ([start snap-ubuntu\x2dcore-423.mount] failed with exit status 1: Job for snap-ubuntu\x2dcore-423.mount failed.
<rharper> maybe error like that in cloud-init-output.log
<rharper> or an error in cloud-init.log around installing snaps
<rharper> might pop in #lxd to check about snap installing
<powersj> ok I'll mark this down as something to circle back to then
<powersj> "modules-final/config-snappy: SUCCESS: config-snappy ran successfully"
<rharper> thrm
<rharper> there should be some subp on the command in the log
<rharper> if not, we definitely need a task for util.subp to create events so they're trackable via cloudinit-analyze
<rharper> magicalChicken: smoser ^^ item for roadmap
#cloud-init 2016-11-17
<Odd_Bloke> smoser: Are you going to be able to look at __builtin__ (i.e. https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/310432) this week, or shall I pick up work on it?
<smoser> Odd_Bloke, if you can make tests pass, then i'm good with it in trunk
<smoser> but, no. i wont get there. right now i'm workingon
<smoser> https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1611074
<Odd_Bloke> smoser: OK, cool, I'll look at the tests.
<smoser> and i'll hope to have some review from you on that...
<smoser> its kind of a pita
<Odd_Bloke> smoser: Ugh, that bug.
<smoser> the code that was there has always been racy
<smoser> i do think that i have a reasonable plan
<Odd_Bloke> smoser: FYI, I'm out tomorrow, Monday and Tuesday.
<smoser> you're on UTC hours or are you on this continent now.
<Odd_Bloke> smoser: UTC at the moment, flying to that continent tomorrow and back on Monday.
<smoser> fun weekend.
<Odd_Bloke> Yeah, I'm having to do a quick trip to activate my permanent residency.
<smoser> Odd_Bloke, that doesnt really make sense.
<smoser> :)
<Odd_Bloke> :)
<Odd_Bloke> Being with my Canadian wife (anywhere in the world) counts as "resident".
<Odd_Bloke> And my medical expires in December, so I'm having to make a trip before that.
<Odd_Bloke> smoser: Oops, you'll just have got a huge email from LP because of me; proposed merging my tests to your branch having forgotten that I rebased on to master. Â¬.Â¬
<Odd_Bloke> smoser: https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/311164 is your commit with an additional commit fixing the tests.
<smoser> Odd_Bloke, nice. thank you!
<rharper> Odd_Bloke: smoser: the "ephemeral0" disk in azure is mapped how? /dev/disk/by-label? by-id ?  ie, what sort of kpath would such a device have ?
<Odd_Bloke> rharper: cloud-init ships udev rules that make it appear at /dev/disk/cloud/azure_resource
<rharper> ok, lemme read those rules
<Odd_Bloke> rharper: That's only post-trusty, before xenial walinuxagent ships rules which name them slightly differently.
<Odd_Bloke> (Well, I think walinuxagent still ships those rules)
<rharper> Odd_Bloke: ok, would you be able to get me access to a trusty and xenial instance in azure?  I'm poking at a disk formatting and mounting bug there, https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1642383
<Odd_Bloke> rharper: On it.
<rharper> thanks
<Odd_Bloke> rharper: ubuntu@xenial-161117-1647.cloudapp.net
<Odd_Bloke> rharper: And ubuntu@trusty-161117-1648.cloudapp.net
<Odd_Bloke> I'll leave you to work out which one is which suite. ;)
<rharper> Odd_Bloke: thanks!
<Odd_Bloke> rharper: I'm heading off for vacation, so could you shutdown the machines when you're done with them?
<rharper> Odd_Bloke: yes!
<Odd_Bloke> Danke. :)
<rharper> smoser: looks like we'll need this https://code.launchpad.net/~daniel-thewatkins/cloud-init/lp1460715  to fix disk_setup partitioning under xenial (sfdisk changes in 16.04)
<smoser> rharper, yeah, there are other issues... i hope/expect to fix that bug with what i have for the other
<smoser> part of my motivation to have you look at that one was so you'd understand my changes coming here.
<smoser> and why they should work for both
<smoser> but yea... using that dynamic path is part of this.
<rharper> k
<smoser> as well as making sure we wait for the device to appear
<rharper> right, that's a 16.04 ism w.r.t kernel partition awareness under systemd
<rharper> that sounded like the kpartx thingy
<rharper> smoser: would really like to replace the cc_mounts/cc_disk_setup with curtin backends  but that's not going to happen right now
<smoser> rharper, yeah. that is absolutely true
<smoser> and additionally we need to be able to take disk config in curtin like format.
<rharper> y
<rharper> another thing for the roadmap =)
<rharper> the 33, 66, 88 thingy isn't non-parseable unless you read the help documentation, so moving to a more helpful format would be nice
<smoser> rharper,
<smoser> https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/311205
<smoser> Odd_Bloke, ^ also
<rharper> +            if os.path.isfile(devpath + suff + "2"):
<rharper> that seems off, what if there are 3 ?
<rharper> also, I think it's possible to have un-ordered partitions (gpt) for example
<smoser> well, it woudl fail if there was a partition 1 and partition 3 but *not* a partition 2
<rharper> which is possible
<smoser> (you can do that with mbr also)
<smoser> yeah, it is hueristic for sure.
<smoser> i could probably use a function in disk_setup
<smoser> but this was what i did initially
<rharper> we have a sysfs device partition counting in curtin
<smoser> yeah, but that wont work on non-linux
<smoser> the sfdisk stuff in disk_ probably would
<rharper> not with -L Linux
<smoser> so yeah, thats not the greatest bit of code, but it is better than what *was* there.
<rharper> oh sure; I just don't know how far we want to go here
<smoser> -L linux isn't "linux specific stuff"
<rharper> hehe
<smoser> its just "dont complaina bout stuff linux doesnt care about"
<rharper> well, it's deprecated anyhow
<smoser> the code that was there before didn't even check for a second partition at all
<rharper> but, it *is* linux specific
<smoser> !
<smoser> ?
<smoser> i suspect that sfdisk -L works on freebsd
<rharper> Stderr: "sfdisk: --Linux option is unnecessary and deprecated\nsfdisk: unsupported unit 'M'\n"
<rharper> in 16.04
<rharper> upstream util-linux changed
<rharper> in 2014 or so
<smoser> ah. right
<rharper> https://bugs.launchpad.net/cloud-init/+bug/1460715  is still needed for partitioning on xenial
<rharper> the can_dev_be_reformatted is specifically there to help reprevent dataloss if someone "used" the disk ...
<rharper> certainly it's possible to populate it the ntfs existing partition though?  we can't protect against that?
<smoser> rharper, if there are files on it, we leave it alone
<smoser> (other tahn the DATALOSS_ file)
<rharper> ok, I'm posting comments, the msg on the return True, is confusing
<smoser> rharper, addressed that.. you're right. i
<smoser> it should have said no important files
<rharper> I thought so after reading the rest of the code
<rharper> and I switch to util.log_time should be easy and announce the wait on the log
<smoser> i think i  might be racing systemd and mount now.
<rharper> I've had several folks ask about making sure cloud-init announces when it's blocking ( which I think it does most of the time) but we need to do that here too
<smoser> rharper, ubuntu@40.78.152.32
<smoser> er... smoser@
<rharper> ok
<smoser> byobu or /var/log/cloud-init.log
<smoser> mount -a failed as already mounted.
<smoser> which should not happen until
<smoser> defaults,nofail,x-systemd.requires=cloud-init.service,comment=cloudconfig
<rharper> urg
<smoser> maybe in new systemd world, a mount-a is going to fail in that scenario... i'm not sure, but i'm pretty sure it should *not* fail with already mounted
<rharper> Nov 17 20:11:02 repro3 systemd[1]: Mounting /mnt...
<rharper> Nov 17 20:11:02 repro3 systemd[1]: Mounted /mnt.
<rharper> Warning: mnt.mount changed on disk. Run 'systemctl daemon-reload' to reload units.
<rharper> $ journalctl -o short-precise --unit mnt.mount
<rharper> -- Logs begin at Thu 2016-11-17 20:10:56 UTC, end at Thu 2016-11-17 20:22:01 UTC. --
<rharper> Nov 17 20:11:01.513996 repro3 systemd[1]: Unmounted /mnt.
<rharper> Nov 17 20:11:02.345932 repro3 systemd[1]: Mounting /mnt...
<rharper> Nov 17 20:11:02.820894 repro3 systemd[1]: Mounted /mnt.
<rharper> check the timestamps with cloud-init log
<smoser> well, the timestamps suck
<rharper> Nov 17 20:11:02.868532 repro3 cloud-init[1037]: 2016-11-17 20:11:01,686 - util.py[WARNING]: Activating mounts via 'mount -a' failed
<rharper> journalctl -o short-precise --unit cloud-init.service
<rharper> will get you real timestamps
<rharper> so, we were too late
<smoser> you want short-monotonic here
<rharper> as you said
<rharper> sure
<rharper> that too
<smoser> as clock runs all over :)
<rharper> hrm
<rharper> that shows something different then
<rharper> ~$ journalctl -o short-monotonic --unit cloud-init.service | grep mount
<rharper> [   15.757407] repro3 cloud-init[1037]: 2016-11-17 20:11:01,686 - util.py[WARNING]: Activating mounts via 'mount -a' failed
<rharper> smoser@repro3:~$ journalctl -o short-monotonic --unit mnt.mount
<rharper> -- Logs begin at Thu 2016-11-17 20:10:56 UTC, end at Thu 2016-11-17 20:22:01 UTC. --
<rharper> [   14.406073] repro3 systemd[1]: Unmounted /mnt.
<rharper> [   19.680401] repro3 systemd[1]: Mounting /mnt...
<rharper> [   19.750265] repro3 systemd[1]: Mounted /mnt.
<rharper> something is lieing
<rharper> but it does seem like if we update fstab, we may need to systemctl daemon-reload
<rharper> so generators trigger
<rharper> which would do the fstab generator
<rharper> and do the mount for us
<rharper> IIUC
<smoser> right. thats possible.
<smoser> but both the old and the new fstab entry had not before cloud-init.service
<smoser> so it really shuld not be doing anything until that is finished.
<smoser> holy moley
<smoser> the monotonic is not in order
<smoser> journalctl -o short-monotonic
<smoser> needs to sort
<rharper> ew
<rharper> that seems *wrong*
<smoser> yeah. its probably because its just timestampping the different threads as the come in
<rharper> so,  something umounts /mnt right after cloud-init.service starts ( that's expected, right?)
<rharper> [   14.713559] repro3 cloud-init[1037]: Cloud-init v. 0.7.8 running 'init' at Thu,
<rharper> [   14.406073] repro3 systemd[1]: Unmounted /mnt.
<rharper> actually before
<rharper> not sure why
<smoser> well, cloud-init *does* do that. or could
<rharper> -local ?
<smoser> if it was ntfs, then cloud-init mounts, reads, unmounts
<rharper> well, this is the mnt.service saying
<rharper> so, that happens after fstab generator runs
<rharper> otherwise we wouldn't have a mnt.mount unit
<smoser> probably not -local (looking for the datasource disk) but it coudl be
<rharper> bbiab, picking up the kiddos, then back to help debug
<smoser> yeah, everything happens after the fstab generator runs
<rharper> smoser: back
<smoser> i poked slangasek, he might be looking too
<smoser> its kind of interesting/annoying that 'mount -a' failed
<smoser> as i've never seen that before
<smoser> "everything is already mounted" normally exits 0
<rharper> that seems new
<rharper> if we run mount -a now, do we see ?
<rharper> no
<rharper> it returns 0
<smoser> right.
<rharper> so, something is very strange
<rharper> what's the RC?
<smoser> so it could just be a really unlucky race
<rharper> well, the return code should give us a hint
<smoser> 32
<rharper> maybe it's ntfs return code
<rharper> 32
<rharper> mount failure
<rharper> that's so helpful
<rharper> =(
<rharper> https://linux.die.net/man/8/ntfs-3g.probe doesn't list 32
<rharper> so unlikely ntfs error
<rharper> ov 17 20:11:02.868532 repro3 cloud-init[1037]: 2016-11-17 20:11:01,686 - util.py[WARNING]: Activating mounts via 'mount -a' failed
<rharper> Nov 17 20:11:02.868575 repro3 kernel: EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null)
<rharper> smoser: that does look like a race , mount -a got the OK to mount it, but something mounted it before it could ?
<rharper> that's what 50 ms?
<smoser> thats way shorter than that
<rharper> 6 places isn't nano, ?
 * rharper sucks at subsecond placement 
<smoser> oh. but you're looking at the timestamp of the message that it failed.
<rharper> sure, but I still think it's related
<smoser> 20:11:01,652
<smoser> is 'Running'
<rharper> [   14.425648] repro3 systemd[1]: Stopped File System Check on /dev/disk/cloud/azure_resource-part1.
<rharper> [   14.484319] repro3 systemd[1]: Starting File System Check on /dev/disk/cloud/azure_resource-part1...
<rharper> [   14.560400] repro3 systemd-fsck[1140]: /dev/sdb1: clean, 11/25688 files, 8896/102400 blocks
<rharper> [   15.757407] repro3 cloud-init[1037]: 2016-11-17 20:11:01,686 - util.py[WARNING]: Activating mounts via 'mount -a' failed
<smoser> yeah, that just seems completely bogus
<rharper> [   15.440692] repro3 kernel: EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null)
<rharper> something was "using" sdb1 (systemd-wise)
<rharper> when cloud-init attempted to mount it
<rharper> if fsck was still running, it would be seen as busy
<rharper> ie, an open on the device
<rharper> systemd-fsck
<smoser> :-(
<smoser> yeah
<rharper> These services are started at boot if passno in /etc/fstab for the file system is set to a value greater than zero.
<rharper> whichi s set to 2
<rharper> which triggers the service
<rharper> we could try not setting the passno in the fstab entry and see if things just work
<rharper> if so then we may need to mask systemd-fsck service prior to the mount (or as you see) it gets mounted automatically
<rharper> ie, if we're system_is_systemd
<rharper> we could skip the mount -a
<smoser> right, on ubuntu we do not in theory need the mount -a now.
<smoser> mask systemd-fsck service ?
<smoser> :-(
<smoser> another instance, i see:
<smoser> 2016-11-17 21:20:04,125 - DataSourceAzure.py[DEBUG]: Azure ephemeral disk: All files appeared after ['/dev/disk/cloud/azure_resource'] seconds: 0
<smoser> 2016-11-17 21:20:04,125 - DataSourceAzure.py[DEBUG]: reformattable=False: device /dev/disk/cloud/azure_resource is not a file
<smoser> which is very odd.
<rharper> it's alink
<rharper> right ?
<smoser> right. but the check is os.exists
<rharper> and it's /dev/disk/azure/resource
<rharper> not azure_resource
<smoser> err.. isfile
<smoser> which isvalid
<smoser> no its both
<rharper> your instance says no
<smoser> and cloud-init is the /dev/disk/cloud/azure_resource one. the other is from walinuxagent
<rharper> ah
<rharper> two paths
<rharper> is suspect that's udev racing with symlink creation
<smoser> but it shoudlnt ever stop existing
<smoser> after it exists
<smoser> i have to go... to teacher conferences
<rharper> k
<smoser> i'm using http://paste.ubuntu.com/23492436/
<smoser> to make it think its a resize
<smoser> (just partitions it as 1 parittion and mkfs ntsf)
<rharper> and then reboot ?
<smoser> and then reboot
<smoser> yeah
<rharper> ok
<smoser> here, i'll give you another instance
<smoser> smoser@smoser1117x.cloudapp.net
<smoser> shoot
<smoser> isfile is just wrong
<smoser> fix that
<smoser> os.path.isfile("/dev/disk/cloud/azure_resource")
<smoser> False
<rharper> heh
<rharper> islink
<rharper> will work
<rharper> >>> os.path.islink('/dev/disk/cloud/azure_resource')
<rharper> True
<smoser> well i dont want to know if its a link
<smoser> cause that says it can be danling
<rharper> you need readlink -f
<rharper> I'll get a fix
<smoser> i just changed to exists
<smoser> and pushed
<smoser> so that should be fine really
<rharper> well, yeah
<smoser> going afk
<rharper> k
<rharper> >>> os.path.realpath('/dev/disk/cloud/azure_resource')
<rharper> '/dev/sdb'
<rharper> >>> os.path.exists(os.path.realpath('/dev/disk/cloud/azure_resource'))
<rharper> True
 * rharper gives it a go 
<rharper> smoser: getting further;  with the realpath and exists going, now we're looking to find sdb1 .. it's not immediately present when we're detecting if the devices can be reformatted ..
<rharper> smoser: currently, sdb1 isn;t being detected as ntfs  via the blkid util.find_devs_with(); debugging that
<rharper> ah , looks like it's working, but we tried to mount /dev/sdb, instead of /dev/sdb1 (real_partpath)
<rharper> and, wrong name of the semaphor file, config_disk_setup, not config_disk_config
<rharper> almost there
#cloud-init 2016-11-18
<rharper> smoser@smoser1117x:~$ sudo blkid
<rharper> /dev/sda1: LABEL="cloudimg-rootfs" UUID="e990f8b3-1d6b-4615-8280-8ead4ed2fe7c" TYPE="ext4" PARTUUID="2865a230-01"
<rharper> /dev/sdb1: UUID="c2108ffd-a113-4c1f-add4-4d6c92c7bef9" TYPE="ext4" PARTUUID="9b4d08ab-01"
<rharper> \o/
<rharper> smoser: here's my changes to get it working: http://paste.ubuntu.com/23493164/
<smoser> rharper, still there ?
<rharper> y
<smoser> realpath will work on the devpath even if its a link
<smoser> exists does the right thing on a link or dangling link
<rharper> you mean if it's not a link ?
<smoser> right?
<rharper> but we *want* to be sure it's a real device ?
<rharper> I'm confused why you ask ?
<rharper> if the link exists but the device doesn't then we can't do anything
<smoser> ln -s /does-not-exist /tmp/foo
<smoser> $ python3 -c 'import os; print(os.path.exists("/tmp/foo"))'
<smoser> False
<smoser> for this case, just checking if somethign exists, exists is enough even on a link.
<smoser> the only rason we convert it to realpath was because that is what find_devs_with would respond with
<rharper> realpath only resolves a symlink if it has a valid target
<smoser> right.
<smoser> but it has a valid target
<rharper> sure
<smoser> or os.path.exists("this symllink") will say no
<rharper> it doesnt check if it exists (the target)
<smoser> it does
<smoser> see above
<rharper> only if it's pointing to other links
<smoser> no. i doubt it
<smoser> stupid me and isfile
<rharper> we went throught this in curtin;  don't we want to know if the target kernel device exists? (/dev/sdb1 ) ?
<smoser> i just assumed "in unix everythign is a file"
<smoser> we do, but it is sufficient to say "os.path.exists()"
<smoser> as os.path.exists("dangling symlink") == false
<smoser> and
<smoser> os.path.exists("non existant file") == false
<rharper> (foudres) tmp % ls -al invalid
<rharper> lrwxrwxrwx 1 root root 8 Nov 17 19:00 invalid -> /dev/sdz
<rharper> (foudres) tmp % test -e /dev/sdz; echo $?
<rharper> 1
<rharper> (foudres) tmp % python3 -c 'import os; print(os.path.realpath("invalid"))'
<rharper> /dev/sdz
<rharper> I don't have an sdz, but realpath tells me that it's resolved any relative paths
<smoser> ok.
<smoser> http://paste.ubuntu.com/23493164/
<smoser> in line 9 there, you dont have to do that....
<smoser> you could use your line 14 ahead of it
<rharper> (foudres) tmp % ls -al invalid crazylink
<rharper> lrwxrwxrwx 1 root root 7 Nov 17 19:05 crazylink -> invalid
<rharper> lrwxrwxrwx 1 root root 8 Nov 17 19:00 invalid -> /dev/sdz
<rharper> (foudres) tmp % python3 -c 'import os; print(os.path.realpath("crazylink"))'
<rharper> /dev/sdz
<smoser> sure.
<smoser> but print(os.path.exists("crazylink")
<smoser> it will be false
<rharper> yes
<rharper> which proves that realpath does not check for existance
<smoser> :)
<rharper> just resolvs symlinks
<smoser> i agree
<rharper> =)
<smoser> but we dont care
<rharper> why not?
<smoser> in your pasete
<rharper> how can you know you can reformat if the device doesn't exist ?
<smoser> if you moved line 14 up
<rharper> that's the part I'm confused about
<smoser> then "device does not exist"
<smoser> before we ever try to resolve the devpath
<rharper> hrm
<rharper> then why did you fail before ?
<rharper> indeed, exists does a realpath (I suppose)
<smoser> because we used os.path.isfile
<rharper> ah
<rharper> right
<smoser> which is not true for a block device
<rharper> and there is no path.isdevice
<smoser> but /me just assumed it was
<smoser> right
<rharper> well, it's pretty crappy to not have that but I suppose that's a posix thingy (or even linux ism)
<smoser> yeah. curtin probably has one
<smoser> just stat and check its doable. but this is sufficient. we're reasonably sure that //dev/disk/cloud/azure_resource is not goign to point to a character device or a directory
<smoser> or even a file
<smoser> and if it does, then other crap is gonna fail
<rharper> y
<smoser> thank you for your help, rharper!
<rharper> np
<smoser> rharper, did you find evidence of /dev/sdb1 existing
<smoser> when the /dev/disk/cloud/azure_resource did not ?
<smoser> er... /dev/disk/cloud/azure_resource-part1 did not
<smoser> i did consider it possible htat we could find existance of /dev/disk/cloud/azure_resource before kernel had probed the partitiones
<rharper> smoser: no, but it's certainly possible w.r.t udev race with symlink
<rharper> I thought it was the case
<smoser> but a udevadm settle shoudl work
<smoser> right ?
<rharper> yes (empty the queue)
<smoser> or maybe not
<smoser> actually not though
<smoser> because i guess the device could exist
<smoser> but the kernel not have probed
<smoser> then we see it exist
<smoser> udevamd settle
<smoser> and *then* kernel probe
<smoser> possible ?
<smoser> so i dont know what to do there...
<rharper> well, the kernel will probe the device
<rharper> for us
<rharper> and reports the partition table (and emits two events)
<rharper> oen for the the disk and one for the partition
<smoser> right. but it might not have gotten around to it
<rharper> so, the only tiem would be between the two
<smoser> right.
<rharper> ok, I've gotta run and get my son, bb in 30
<smoser> ok.
<smoser> thankds.
<jgrimm> powersj, for awareness -> https://gist.github.com/smoser/29ea35a797c0df1fcb6ac875a024efa9
<jgrimm> consider as part of scenarios we'd like to make sure covered in integration tests
<powersj> jgrimm: thanks - I will add that to the docs of things we need to do
<jgrimm> thanks
<powersj> rharper: there was another document you put together that had like 8 steps for cloud-init that was a good test. I can't seem to find my bookmark for it.
<powersj> Do you recall where that is as well?
<rharper> yes
<rharper> <rharper> but, really it was just a list of things to check (didn't discuss how to do that): Networking, User Creation, Import ssh keys, ssh to instance, sudo privs, resizes disk, ârefreshes on bootâ, install snaps
<rharper> <powersj> I wanted to check to see if the tests I wrote either covered all that or I should write a custom case for that
<rharper> <rharper> nice
<rharper> <powersj> refreshes on boot == re-runs parts of cloud-init on a reboot?
<rharper> <rharper> the "refreshes on boot" and "install snaps" is snappy specific,  resizes disks happens both in cloud-image and UC16 image but in different places (UC16 initramfs does it vs. cloud-init)
<rharper> <rharper> no, rather snapd checks if there are new snaps (core, kernel) downloads, installs and schedules a reboot
<rharper> <rharper> that can be put on-the-side for now (UC16 specific test)
<rharper> <rharper> and unrelated to cloud-init other than, it should still work in UC16 + cloud-init (which it does, though testing that is hard since we don't control what's in the channel)
<rharper> <powersj> ok thank you!
<vans163> Fedora 24 cloud image. "calling 10.0.2.2/latest/meta-data/instance-id"  does this mean my cloud-init iso is invalid?
<vans163> so its defaulting to trying cloudinit over the network?
<vans163> is instance-id: a manditory field? i was missing it
<vans163> Alot of cloud images do not have root user inmind.  I want to set root password the same on all cloud images (vs using their default users)
<vans163> so I just did   chpasswd: list | root:12345678
<vans163> maybe this is wrong?
<rharper> vans163: if you provide a configdrive then you need an instance-id
<rharper> if you only provide user-data, it is going to search for an instance id
<rharper> vans163: you don't want a root password, really ;  cloud-init can configure sudo privs for a user (The ubuntu cloud-image does this for the default ubuntu user if you don't specify your own list)
<vans163> rharper: my problem is this is for veterans to beginners of linux, the vets can use their own custom cloud-config, but beginners will have a hrd time trying new distros if every distro has their own default user
<vans163> rhapher: let me try with adding instance-id
<vans163> (kind of how digital ocean and vultr do it)
<vans163> every flavor of linux has the same default user (root)
<rharper> =(
<rharper> sudo bash, gets you to root
<rharper> without setting root password
<rharper> you can define the name of the user the same across distros via the cloud-init user: config
<rharper> http://cloudinit.readthedocs.io/en/latest/topics/examples.html   the 'users'
<vans163> rharper: that is what I am confused about, root should automatically be defined so if I wanted to make it only SSH key accessible, I do users: - name: root ssh-authorized-keys: ...
<vans163> and it worked for me on a simple config, duplicate sshkeys were ignored
<rharper> root of course exists, but it doesn't have a password set
<rharper> to enable root ssh access, there may be another switch
<rharper> remote root via ssh is really bad too
<vans163> hum.. neither is working for me now. it prints "cloud init 0.7.7 running init-local".. and stucks. Then after a while starts trying URL, then boots the cloud image having not ran anything.  Going to try I guess to pick apart the cloud-config I have?
<vans163> and see what could be making it freeze like this
<rharper> http://cloudinit.readthedocs.io/en/latest/topics/modules.html#ssh
<rharper> it's not freezing, but likely looking for a network datasource which you don't have;  I suggest using cloud-localds  --help as a starting point for building a working local data source (configdrive)
<vans163> using a simple config taht smoser linked me, i had no problems eirler
<vans163> going to start from there i guess
<vans163> instance-id does this have to be string actually?
<vans163> i just put 1 for testing, maybe thats causing the issue?
<rharper> not sure what the min is
<rharper> but using `uuidgen` output is going to work fine
<rharper> the example is i-abcdefg
<vans163> rharper: currently trying this config http://paste.ubuntu.com/23486774/
<vans163> can that instance-id go into the user-data tho?
<rharper> no, instance-id is always part of meta-data
<rharper> so, the configdrive is an iso of a directory structure
<rharper> in there will be a metadata dir with the instance-id in it
<rharper> % cat m/meta-data
<rharper> instance-id 7c871218-4ba9-45ba-854a-2a0a41cb928f
<vans163> ah
<vans163> so using that links config, I can boot BUT the password for user fedora (the default fedora 24 user) is not passw0rd
<vans163> also i see the cloud init stuff go off
<vans163> it says I need to provide a http://cloudinit.readthedocs.io/en/latest/topics/datasources.html#no-cloud.
<vans163> no-cloud datasource?
<rharper> well, you did, if you created the iso file and passed it in
<vans163> yea i see.  let me go back to trying the ubuntu image again and confirm the password for ubuntu changed
<rharper> nocloud is embedded within the image, the configdrive is equivalent ;
<rharper> looking at the fedora distro code in cloud-init, need to see where the default username is set
<vans163> i jsut googled for fedora cloud image default username, seems to indicate fedora.
<rharper> right
<rharper> but cloud-init has a distro class which can set these values
<rharper> I'm not sure what's set at the moment in your cloud-init rpm
<vans163> ahh
<vans163> i have cloud init 0.7.7 not sure what ver of cloud-localds
<rharper> http://paste.ubuntu.com/23496463/
<rharper> I think that should get you a fedora user for now;  harlowja had a merge proposal which would have broken out the default distro and user stuff  better;    you can look at /etc/cloud/cloud.cfg in your instance to see what the system_info dict looks like
<vans163> tried ubuntu. I see the cloud init go off saying things like "resizing root " etc, but go to login screen and i cant login as ubuntu/passw0rd
<vans163> let me try that fedora config, back to fedora
<powersj> smoser: rharper: have time for int. test meeting?
<rharper> y
<jgrimm> powersj, interetsted too if you have a HO
<vans163> hum  hostfwd=tcp::2222-:22
<vans163> is this line critical?
<vans163> i took that line out
<vans163> dont see why the need to forward a port.. but cloud-localds shows that line in the example
<rharper> that just lets  you ssh into your VM if you're running local
<rharper> vs login via console
<rharper> it forwards host port 2222 into the guest 22
<vans163> yea i see, so not critical to cloudinit i tried to ssh to localhost 2222
<vans163> fedora and pass passw0rd
<vans163> Thisis what im doing  https://gist.github.com/anonymous/1406b7470736587554075f1f9be58734
<vans163> maybe the config is broken so it stops parsing it without error?
<rharper> vans163: not sure, if you cleanly shutdown, then you can mount it up and extract /var/log/cloud-init*.log so we can see what happened
<vans163> okay il do that, let me try 1 last thing.  notice how I passed paths to the file, but the example clearly say user-data and meta-data
<vans163> maybe the cloud-localds does not form the right paths?
<vans163> okay rharper: so if I follow the instruction to a T it works
<vans163> x.x
<vans163> but how can I add more flexibilty
<rharper> cloud-localds does the right thing
<vans163> like I cannot call the files meta-data as tehre could be a case when I spin up 2 at once
<rharper> not following, you can point to the same iso, it's read-only
<vans163> actually there maybe 1 thing I missed  the two EOF's
<rharper> if you want the same user-data into multiple instances, you can re-use the same iso
<vans163> or is that just for the terminal.. sec
<vans163> yea that EOF is just for the terminal
<vans163> (it does not change the file)
<vans163> oaky let me try something then
<vans163> fedora still does not work. Ubuntu worked, fedora I see my script executed (because sshd restarted)
<rharper> ok, there may be something not quite right with the default cloud.cfg in the fedora cloud-init;  is there a fedora cloud-image that is publicly available ?
<vans163> https://getfedora.org/en/cloud/download/  i got it from here the raw image and made it into a qcow2
<rharper> ok
<rharper> I'll see what I can figure out in a bit
<vans163> md5sum  672669a587d653ef4100bcd03cb4ce32  /cloud/image_mnt/Fedora-Cloud-Base-24-1.2.x86_64.qcow2
<vans163> thanks
<vans163> rharper: i fetched the cloud-init-output.log does not seem to have anything useful.  cloud-init.log is empty
<rharper> that's odd
<vans163> do you still need the *-output.log?
<rharper> not really, if cloud-init.log is empty, I don't see why we'd have anything in -output.log (that's stderr/stdout from cloud-init )
<vans163> ah I see a line like this:
<rharper> doe syslog have an error ?
<rharper> maybe like a python stacktrace ?
<vans163> ci-info: no authorized ssh keys fingerprints found for use fedora.
<vans163> cc_set_passwords.py [WARNING]: no default or designated user to change password work
<vans163> (that was the password: passw0rd line probably outside the scope of users)
<vans163> *work/for
<vans163> it seems as if it ignored the users: field
<vans163> or no thats not the case. otherwise it would not print that ci-info:  (the order of those 2 is reversed, first the cc_set_passwrd.py warn, then that other)
<rharper> I think the cloud.cfg inside the image is probably still set to ubuntu, rather than fedora;  need to check that out...
<vans163> where woudl the syslog be located?
<rharper> var/log/syslog
<vans163> its not there
<vans163> i can dump you the /etc/cloud/cloud.cfg?
<rharper> I'm opening it up now
<vans163> ah
<rharper> it's working for me here
<rharper> the default user is fedora
<rharper> so, you can leave the 'user:' key out
<vans163> does your config look the same as this http://paste.ubuntu.com/23486774/?
<vans163> let me try redownload the image again maybe i got the openstack one by accident, pretty certain i got raw tho.
<rharper> yes
<vans163> so you just downloaded the raw and did  xz Image.raw.xz  then used that?
<rharper> I used the qcow2 file
<rharper> it works fine
<rharper> but the 0.7.7 I think has a bug that's been fixed upstream but not in there, yet
<rharper> so, it's trying to load the user-data but failing
<rharper> which is why you're seeing anything besides fedora: passw0rd not working
<vans163> let me try.. also does that mean I wont be able to set a ssh key for a user other then fedora?
<rharper> I think so, but I need to find out a bit more
<rharper> I believe the cloud-init in the image is going to need a patch; but I need to find out the actual bug
<vans163> say once it gets patched (if it does) would I be able to simply mount the image and apply the patch?
<vans163> vs a more complex process
<rharper> cloud-utils includes a tool called mount-image-callback which would aide in injecting an updated package (or applying a patch)
<rharper> what does your meta-data look like ?
<rharper> https://gist.github.com/anonymous/1406b7470736587554075f1f9be58734
<vans163> hum wtf
<vans163> that raw image is 500mb while the decompressed openstaack qcow2 is 1gb
<rharper> that looks fine, you can drop the users dict;  then you can in-line a ssh-authorized key
<vans163> let me try that other image
<vans163> instance-id: i-abcdefg   is all thats inside meta-data
<rharper> that's fine; I had a typo in mine
<rharper> fixed that and the sample config works just fine
<rharper> I'm testing adding an ssh public key to fedora user
<vans163> going to brb in an hour or two
<rharper> sure
<rharper> vans163: so, things appear to be working fine for me:  http://paste.ubuntu.com/23496940/  is the template I'm using, I see a default password for user: fedora, and then import a ssh public key to them, I used 'users' to define a new user 'rharper' and import an ssh public key there.  That runs fine in the qcow2 image (I expect the raw to work just as well);  I can ssh in to either fedora or 'rharper' in that image after
<rharper>  cloud-init runs.
<vans163> rharper: I am back let me test
<seanbright> hi. about to head out for the weekend, does everything look ok with this merge request:
<seanbright> https://code.launchpad.net/~sbright/cloud-init/+git/cloud-init/+merge/311313
<seanbright> first time submitting a LP patch
<vans163> rharper: this is not working for me on ubuntu :( http://paste.ubuntu.com/23496940/
<vans163> i replaced <ssh key here> with  wtff abababab ee
<vans163> in both cases
<vans163> if I only leave the top 4 lines of that paste, it works
<vans163> otherwise even ubuntu users password is not passw0rd
<rharper> vans163: if you don't want the ssh key, take the 3 lines out;  5-7, and 13-15; don't make them gibberish
<vans163> the command works its starnge i just mounted the drive and the user has a home folder authorized keys are there
<vans163> but at the login prompt i cant login as the user
<vans163> (using vnc)
<vans163> so cloud init successfully created everything
<vans163> maybe some command is blocking..
<vans163> i cant even ssh to it
<vans163> as soon as i delete everything except the top 4 lines of that config file I can login as ubuntu/passw0rd
<vans163> when I add the line to create a new user (leaving the top 4 as well)
<vans163> the /home/jharper/.ssh/authorized_keys is there
<vans163> but i cannot login as jharper/passw0rd nor ubuntu/passw0rd
<vans163> also /var/log/cloud-init.log is quite populated
<vans163> now even ubuntu stopped working
<vans163> something is def wrong with my system i guess
<vans163> and not cloud init
<vans163> no nevermind i got confused, i just retried ubuntu with the top 4 lines of the config. everything works
<vans163> as soon as i make config more complex. nothing
<vans163> call it a bug or a feature but apparnetly passwords like passw0rd and toor dont work
<vans163> using wtf12345! for example works
<vans163> so maybe something inside cloud-init is checking the passwords complexity or something and silently failing
<vans163> and a silent failure here seems to make the entire config fail or locks the system from login or something
<vans163> fedora works finally!
<vans163> so conclusion. 99% of ppl use cloud-init in openstack
<vans163> and openstack generates the passwords by random
<vans163> for ppl using cloud-init and setting a password of hello, it silently fails
<vans163> thus this bug/feature was never reported
<vans163> maybe this conclusion is wrong and the underlying OS is limiting it but i doubt it as i can set a password of 12345 for excample on fedora 24 wiht no problems
<vans163> even for root
#cloud-init 2016-11-19
<vans163> yup configs are working now with no problems..
<vans163> Can anyone confirm if setting a weak passwords locks all users from being able to login?
<vans163> okay strange password 12345678 works too now
<vans163> anyways in conclusion i dont know wtf happen
<vans163> maybe a rogue space or linebreak
<vans163> broke the formatting i dont know
#cloud-init 2017-11-13
<redcavalier> blackboxsw, good evening. I was wondering if you guys had managed to look into that bug I submitted last month.
<cetex> smoser: Not sure if i said thanks for everything earlier: Thanks :)
<rharper> blackboxsw: urg, the c-i meeting is on top of our standup
<blackboxsw> urg
<smoser> o/
<blackboxsw> \o
<blackboxsw> it
<rharper> o/
<blackboxsw> it's that time again
<blackboxsw> #startmeeting Cloud-init bi-weekly  status
<meetingology> Meeting started Mon Nov 13 16:03:13 2017 UTC.  The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
<meetingology> Available commands: action commands idea info link nick
<powersj> o/
<blackboxsw> time change got to us
<blackboxsw> #meetingtopic Recent Changes
<blackboxsw> hey folks. thanks for joining just pulling together the content for the last couple weeks of work for the cloud-init project
<smoser> http://paste.ubuntu.com/25954862/
<smoser> $ git log a90a8b1cb3104ee3250ac79d6e25a9ff4f527baa.. | log2dch | sed 's,^    ,,' | pastebinit
<blackboxsw> most of the ubuntu-side of the house was involved in handling the SRU of 17.1 into ubuntu and handling any discovered regressions
<blackboxsw> Published cloud-init packages to Bionic Beaver release
<blackboxsw> Update Gentoo Linux support to "rc-service" scripts as "service" is deprecated, thanks to ckonstanski!
<blackboxsw> Detected and fixed a pre-release regression of resizefs when root path is specified by UUID on the kernel cmdline (LP: #1725067)
<ubot5> Launchpad bug 1725067 in cloud-init (Ubuntu Zesty) "cloud-init resizefs fails when booting with root=PARTUUID=" [Medium,Fix committed] https://launchpad.net/bugs/1725067
<blackboxsw> #link http://paste.ubuntu.com/25954862/
<blackboxsw> #info SRU queued for release today
<blackboxsw> Here's the cloud-init content we published for the last two weeks:
<blackboxsw> #link https://github.com/canonical-server/dev-summary/blob/master/doc/2017-10-31.md
<blackboxsw> #link https://github.com/canonical-server/dev-summary/blob/master/doc/2017-11-07.md
<blackboxsw> last week we handled an EC2 behavior regression for xenial, whereby we didn't want to change cloud-init to configure all nics based on ec2 metadata, we will only configure the primary nice
<blackboxsw> last week we handled an EC2 behavior regression for xenial, whereby we didn't want to change cloud-init to configure all nics based on ec2 metadata, we will only configure the primary NIC
<blackboxsw> with those SRU regresssions fixed and published to master, we expect cloud-init 17.1 updated in Xenial,Zesty and Artful today
<blackboxsw> #meetingtopic In-progress development
<blackboxsw> smoser: rharper anything here?
<smoser> #link http://bit.ly/ci-reviews
<smoser> robjo has done a couple fixes for SuSE and i've pulled a few of them.
<smoser> he has one up i saw yestderday for ntpSuSE
<smoser> others ther.e we've been delinquent due to some distractions recently.
<rharper> blackboxsw: nothing new for me at the moment
<smoser> and chad had one up for clean and status
<blackboxsw> btw thx robjo ckonstanski and Dave Mulford for the fixes over the last iteration. We also expect that a couple VMware branches for the OVF datasource will last this week or next
<smoser> which is nice.
<robjo> moving the meeting an hour foward while we are on Standard time or is this a one time occurance, did I miss an announcement?
<blackboxsw> #meetingtopic Office Hours (next 30 minutes)
<robjo> lp#1731619, chrony support, should that also be driven through ntp config or should there be a new config option?
<blackboxsw> so we'll hang out with eyes on this channel for any burning questions/bugs/questions
<smoser> robjo: well, the meeting is listed in UTC time
<smoser> that pays no attention to US legislation to change clocks at random points in the year :)
<robjo> oK, my fault when I added it to my calendar, eay enough to fix ;)
<smoser> but the humans here were also affected :)
<blackboxsw> heh, anyone opposed to shifting this meeting time +30 from now during the next few months?
<blackboxsw> as the meeting now collides w/ another meeting for us
<blackboxsw> :/
<blackboxsw> officially 16:30 UTC?
<robjo> Well, I'd prefer to either follow the "randomness" clock manipulation or not follow it
<robjo> meaning don't change the meeting time because there exists a conflict when standard time switches to daylight savings or vice versa, becaus if you do that you might as well follow the silliness of the government to begin with
<blackboxsw> fair point. ok let's keep the new time as is.
<blackboxsw> we've discussed side-channel, we can shift our meetings out of the way of this
<blackboxsw> so robjo +1
<blackboxsw> 16:00 UTC
<blackboxsw> also related to CI side, powersj and rharper spent quite a bit of time w/ our continuous integration infrastructure fixing/addressing memory & storage pressure issues to make sure we avoid intermittent false test failures due to timeouts or system resource contention
<blackboxsw> #link https://jenkins.ubuntu.com/server/view/cloud-init/
<via> is there a way to use metadata in the cloud-init file? specifically, if i want to use the aws-provided instance id in an attribute
<robjo> OK, back to my question about chrony:  lp#1731619, chrony support, should that also be driven through ntp config or should there be a new config option?
<via> like configuring the chef node name to have my instance id in it
<blackboxsw> #link https://bugs.launchpad.net/cloud-init/+bug/1731619
<ubot5> Launchpad bug 1731619 in cloud-init "Support chrony as a client for ntp" [Undecided,New]
<blackboxsw> it's a good bug, we've had a couple of discussions about ntpd versus timesyncd for different system environments
<blackboxsw> current implementation of cc_ntp module is to return False ('ntp' not installable) on certain known environments where we know we want systemd timesyncd to run instead by default
<smoser> via: i think what your asking is (i htink) covered in https://trello.com/c/AYaCdQyT
<via> well, i'm trying to do it in a yaml cloud-config file
<smoser> right. as it is right now, via you cann't reference anything from the metadata.
<via> does that mean i need to use #jinja and if so how does that play with #cloud-config ?
<via> oh
<via> bummer
<via> should i just switch to a shell script?
<smoser> but we'd hope to implement that.
<smoser> via: thats really the only way right now. and then in the shell scripty you'd have to query the metadata service yourself.
<via> okay, damn
<via> thanks
<blackboxsw> robjo: we think that's a good approach/feature suggestion. We could add chrony template files etc like the ntp templates, and we might be able to have the distro report what time sync daemon it wants to run
<smoser> basically... we realize what you're asking is quite helpful and reasonable but dont have a way to do it right now
<smoser> but we do plan on implementing it.
<via> no worries, i'm stuck on an ancient version anyway
<robjo> blackboxsw: That was my thinking, move the "service_name" setting to the distro as "time_service_name" and then drive cc_ntp based on that
<robjo> since with a third option the black/white decision being made today will no longer work
<blackboxsw> +1 robjo yeah. rharper was chatting about this potential approach as well
<robjo> look there is also grey ;)
<blackboxsw> heh yeah
<robjo> Next question.... network config.
<blackboxsw> yeah might have to 'grow' an override option in cc_ntp module eventually
<blackboxsw> as those grey use-cases come up (per bugs/requests ;) )
<robjo> A long timi ago the RHEL implementation was re-written to use sysconfig renderer, but RHEL sysconfig and SLE sysconfig are different, why wouldn't they be
<robjo> that also implies that the openSUSE/SLES implementation for network config rendering still uses the "old" implementation and thus produces a warning in the log file
 * blackboxsw is looking for the warning generated
<robjo> this would imply some refactoring is in order if we want to move openSUSE/SLES to using the newer API to render the network config
<robjo> blackboxsw: "apply_network_config is not currently implemented "
<robjo>                     "for distribution '%s'.  Attempting to use apply_network"
<blackboxsw> ahh. right-o
<robjo> the question from my point would be is, when I want to implement the SUSE bits am I also on the hook for the refactoring part or can I get some help with that? which of course will make my life easier ;)
<robjo> And yes, I realize a bug will need to be filed, but I haven't figured out how to formulate this nicely
<blackboxsw> robjo: I think we should be able to help out a bit with that refactor to make sure it's cleaner and easier to maintain.
<robjo> OK :)
<blackboxsw> there are a couplengeneric distro fixes which need to get designed (just like in the datasources) to make the common distro classes a bit easier to maintain as well as making classes a bit more modular and more easily tested.
<blackboxsw> we still haven't landed some of the common datasource changes we had talked about during the Summit because we've been avoiding risk during the 17.1 release. But, similar/minor architecture changes should start taking shape here for datasources and distros now that we see a light at the end of the tunnel on the release.
<blackboxsw> we'll keep our eyes open for discussions/suggestions from folks
<robjo> Speaking of data sources, for the SUSE Container As A Service Platform, we implemented a data source to read from local disk, is that something that would be of interest upstream? Yes, this might seem silly but in our use case it makes perfect sense ;)
<blackboxsw> robjo: I'm curious how different that datasource would be from nocloud datasource
<blackboxsw> http://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html?highlight=nocloud
<blackboxsw> #link http://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html?highlight=nocloud
<blackboxsw> which allows for providing local data instead of dealing with metadata
<blackboxsw> well network metadata
<robjo> I wasn't really involved, just accepted the patch to the package and have not done a comparison to nocloud, but I'll take a look
<blackboxsw> good deal.... think we are at the top of the hour... so I'll probably end meeting now
<blackboxsw> thanks via robjo rharper powersj & smoser. next meeting 2 weeks same early time
<blackboxsw> #endmeeting
<meetingology> Meeting ended Mon Nov 13 17:01:43 2017 UTC.
<meetingology> Minutes:        http://ubottu.com/meetingology/logs/cloud-init/2017/cloud-init.2017-11-13-16.03.moin.txt
<powersj> thanks blackboxsw for running
* blackboxsw changed the topic of #cloud-init to: is Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj |  Next status meeting: Monday 11/13 16:00 UTC | cloud-init 17.1 released
<blackboxsw> np
<via> smoser: will $(cloud-init query instance-id) work in said shell? it doesn't seem to evaluate
<smoser> via: no. cloud-init query doesn't really exist.
<smoser> :-(
<smoser> the goal is to have a good solution for you
<smoser> but there is not one at the time :-(
<via> so wait, there is actually no way to use cloud-init at all to get my instance id?
<via> i have to query the aws 169 address with curl or whatever?
<ajorg> via: if you don't mind hackiness you could resolve the /var/lib/cloud/instance symlink
<via> ok
<ajorg> instance_id=$(basename $(readlink -f /var/lib/cloud/instance))
<ajorg> (assuming 0.7+)
<ajorg> you mentioned you might be using something much older
<via> looks like ec2metadata is installed by default
<blackboxsw> via theres something in progress that will help    https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330115
<blackboxsw> I'm hoping we can land this soon. it'll put a /run/cloud-init/instance-data.json blob thatt will report that for you
<via> well, tbf, if ec2metadata is installed, i dunno why i'd use anything but it
<via> i didn't realizeit was
<blackboxsw> and we'd support it as a public interface that folks can consume
<blackboxsw> that branch referenced is the precursor to cloud-init query subcommand
<blackboxsw> precursor/prerequisite
 * ajorg changes his calendar entry for the meeting to UTC...
<blackboxsw> thanks ajorg. then it can surprise us again in spring
<ajorg> haha
<ajorg> smoser: https://code.launchpad.net/~ajorgens/cloud-init/+git/cloud-init/+merge/331660 < this is okay to merge? or did you have other concerns?
<smoser> ajorg: i'll look now.
<ajorg> thanks
<ajorg> I blame Ben Franklin for the meetings I've missed.
<blackboxsw> you should stop hanging out with that guy ajorg, he's a bad influence
<ajorg> totally. the guy likes to fly kites in the rain
<blackboxsw> ... ok I'm back, wrapping up review comments on the cloud-init clean/status branch
<blackboxsw> thx smoser for the review, yeah I had pulled out cloud-init status disabled as I had wanted to chat w/ you about all the ways it could be 'disabled'
<blackboxsw> rharper: smoser if ds-identify 'disables' cloud-init, we just expect /etc/cloud/cloud.cfg.d/90_dpkg.cfg to have an empty datasource list?
<blackboxsw> rharper: per your inotify comment, I tried not depending on python inotify libs as they aren't currently pulled in by cloud-init as a dependency. Shall we add that as a pkg dependency?
<rharper> no, /run/cloud-init/cloud.cfg will be empty
<blackboxsw> ahh ok rharper thx
<rharper> cloud-init reads the cloud.cfg that ds-identify writes
<smoser> blackboxsw: no. ds-identify exits wihtout putting the target in systemd's goals
<rharper> blackboxsw: bummer on not having inotify tools;
<blackboxsw> yeah even the base-pkg inotify-tools !present yet.
<rharper> I saw a python syscall binding "inotify-basic" package
<rharper> but it would be nice to just have that built-in
<smoser> rharper: blackboxsw fyi there are cloud-images for bionic now
<smoser> rharper: do you know why we get an ipv6 address in lxc on artful ?
<smoser> that seems wrong
<rharper> your lxd config enabled ipv6 dhcp
<rharper> --dhcp-range fd42:3cd1:a543:bb7::2,fd42:3cd1:a543:bb7:ffff:ffff:ffff:ffff,1h
<rharper> smoser: ^^
<smoser> https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1732002
<ubot5> Launchpad bug 1732002 in systemd (Ubuntu) "cloud images in lxc get ipv6 address" [Undecided,New]
<smoser> rharper: its a bug. they should not. they did not request it.
<rharper> doesnt ipv6 ra get you that ?
<smoser> xenial doesnt get it
<rharper> doesn't run networkd
<rharper> networkd does ipv6 ra by default
<smoser> that is a bug
<rharper> https://bugs.launchpad.net/ubuntu/+source/nplan/+bug/1717404
<ubot5> Launchpad bug 1717404 in nplan (Ubuntu Zesty) "IPv6 support regresses with nplan transition" [Undecided,Fix committed]
<rharper> https://bugs.launchpad.net/maas/+bug/1655440
<rharper> all seem related
<ubot5> Error: Could not gather data from Launchpad for bug #1655440 (https://launchpad.net/bugs/1655440). The error has been logged
<smoser> rharper: so 1655440 may imply that cloud-init needs to write 'accept-ra: no'
<smoser> ?
<rharper> I think 1) they do request it, the kernel has ipv6 enabled, which means it does RA  2) dnsmasq for lxd, can be configured to disable ipv6; your host does not have it disabled ;  if there is a bug it's related to dhclient6 not running by default in xenial
<smoser> no.
<rharper> it's hard to say where it should be written
<smoser> lxd and cloud-init config specifically say (via omission) not to have ipv6
<rharper> that's unclear
<smoser> so behavior that just enablels an ipv6 is wrong.
<rharper> look at the lxd bug that stgraber filed
<rharper> if you don't want ipv6, you have to pass kernel config
<rharper> it's certainly a change in behavior from xenial; I think that needs some discussion w.r.t what that means for artful/bionic
<smoser> stgraber's comment does not hmake sense.
<smoser> "ever since the switch to nplan by default in Ubuntu, we're not getting an IPv6 address anymore when a RA reaches the container.
<smoser> "
<smoser> as if we *were* getting ipv6 address by default previously
<smoser> but we are not
<rharper> IPv6AcceptRA=no
<rharper> nplan was disabling IPV6RA by default
<rharper> it's no longer doing that
<rharper> and you have ipv6 enabled on dnsmasq
<rharper> so you're now getting ipv6 addrs in your containers
<rharper> https://bugs.launchpad.net/ubuntu/+source/nplan/+bug/1717404/comments/3 ;  m-h-d suggests a 3 way switch in the config; but someone is going to be unhappy;  this needs discussion AFAICT
<ubot5> Launchpad bug 1717404 in nplan (Ubuntu Zesty) "IPv6 support regresses with nplan transition" [Undecided,Fix committed]
<smoser> rharper: im' missing something
<smoser> but i just made cloud-init render
<rharper> I suspect that we likely didn't have a dnsmasq that was ipv6 enabled, which means there were not IPV6 RAs going to our containers;
<smoser> http://paste.ubuntu.com/25955872/
<smoser> and rebooted
<smoser> and i still have the ipv6
<rharper> look at /etc/default/lxd-bridge.upgraded which shows we've got ipv6 enabled
<smoser> rharper: no. i had in the past explicilty enabled dnsmasq for ipv6
<smoser> (and i think lxc did that by default)
<rharper> it does by default
<smoser> i dnot follow.
<rharper> what does your /run/systemd/network-* files look like ?
<rharper> netplan *used* to disable ipv6
<rharper> so even if you had ipv6 for lxd enabled, it never worked
<rharper> ifyou used netplan
<rharper> they filed a bug to get netplan to not disable RA by default
<smoser> that is bad.
<smoser> its worse to open people to attack that were not expecting it
<rharper> it's complicated depending on who wants what "default" behavior
<rharper> you mean having your kernel enable ipv6 by default ?
<cyphermox> you have an accept-ra option in netplan if you really want avoid getting RAs.
<smoser> ok. so maybe that is "complicated" and i somehat agree with m-h-d =.
<smoser> but in my newly updated cloud-init i just rendered
<rharper> please check the run dir to see what got generated
<smoser> http://paste.ubuntu.com/25955901/
<rharper> no
<cyphermox> accept-ra is per-interface.
<smoser> http://paste.ubuntu.com/25955908/
<rharper> in any case, cat /run/systemd/network/10-netplan*
<smoser> ok. let me try
<rharper> cyphermox: should it fail to parse on that  ?
<rharper> configs present under tree that doesnt' parse them ?
<cyphermox> rharper: I fail to parse your question
<rharper> you said he put it in the wrong spot
<rharper> should be under eth0:
<rharper> and I'm saying should that fail to parse with the value in the wrong spot
<cyphermox> yep
<cyphermox> if it didn't fail, that's a bug
<rharper> yes
<rharper> agreed
<smoser> it did not complain.
<smoser> so you can consider that a bug.
<cyphermox> but I'm a little surprised, netplan usually is failing to parse pretty aggressively
<rharper> indeed
<smoser> and putting it in the right place does get the behavior i expected.
<smoser> so my feeling is that cloud-init should render 'accept-ra': no unless there is a dhcp6 stanza
<smoser> but this still imo seems broken
<rharper> s/cloud-init/netplan; and that's one of the discussion points;  I'm not sure we've reached consensus w.r.t the "right default";
<rharper> so in your lxd case were it has a dhcp6 enabled subnet; then should lxd emit DHCP6 config in addition to the v4 dhcp ?
<smoser> lxd should, yes.
<rharper> I suspect stragber has an opinion to that
<rharper> which is counter to what you're suggesting; hence the bug
<cyphermox> rharper: smoser: if I put your config on my system and run 'netplan generate', it does fail to parse
<smoser> network config should be deterministic
<rharper> what version of netplan is in your image ?
<rharper> smoser: ^
<cyphermox> smoser: except you may really want to no do DHCP for an IPv6 network and do SLAAC, which needs the RAs.
<smoser> not "well, you'll get config if someon (nefarious or not) was sending packets on your network)
<rharper> smoser: you're arguing with the wrong person; I suggest a comment in the lxd/nplan bug if you want to strike up a discussion
<smoser> netplan is bionic image today
<cyphermox> smoser: ergo, disabling RAs in MAAS might be what you want, since MAAS does DHCPv6 and is supposed to manage all, but it's very not the right default elsewhere.
<smoser> an oracle (lxd or maas) should declare.
<rharper> I think 0.30 doesn't have the accept-ra parsing
<cyphermox> rharper: it does, it's added in 0.28
<rharper> huh
<rharper> I see an error when I run generate as well; smoser do you have the cloud-init.log ?
<smoser> rharper: bionic does work if we make cloud-init interpret no 'dhcp6' as 'accept-ra: no'
<smoser> which i think we should
<smoser> oh. yes. we do get a 'generate' error
<rharper> wonder how network came up ?
<rharper> or did it ?
<rharper> the /run/systemd/network/ file shouldn't have been written
<smoser> no it didnt
<rharper> which means no networking
<smoser> yeah.
<rharper> oh, ok
<smoser> fail.
<smoser>  just looked at the ipv6
<smoser> and saw nothing
<smoser> so that seemeed right
<rharper> hehe
<rharper> ok
<smoser> http://paste.ubuntu.com/25955976/
<smoser> cyphermox: ^ that is what you were saying should work right ?
<cyphermox> no
<smoser> hows this for confusing
<smoser> $ python3 -c 'import yaml; print(yaml.load("false") is yaml.load("no"))'
<cyphermox> accept-ra is per-interface, it should be under eth0
<smoser> True
<cyphermox> how is that confusing?
<cyphermox> you're asking python if a boolean is X for a language with a pretty loose grammar, not unlike javascript
<smoser> javascript has a much more strict gramar.
<smoser> its confusing that 'no' is interperted as a boolean.
<smoser> the lack of quoting requirements in yaml makes it wierd
<rharper> the netplan parser does some additional string -> boolean conversion IIRC
<rharper> true == on == yes == y;  false == off == no == n ; for fields which are designated boolean : accept-ra uses the handle_netdef_bool which does the conversion
<blackboxsw> rharper: smoser addressed most comments on https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/333513. I
<rharper> blackboxsw: cool
<blackboxsw> am not quite sure what we should do about standardizing config paths etc. Right now, it feels a bit painful the way content is organized in /etc/cloud/cloud.cfg.d/05_logging.cfg and cloudinit.helpers.Paths and various cmd files. I think we may need to look over a way to centralize more of these config defitinitions in an official config file that can be sourced by other modules
<blackboxsw> s/defitinitions/definitions
<dpb1> blackboxsw: is that to help with schema?
<blackboxsw> dpb1: nope just to help with cloudinit modules knowing where the proper log_file_path or run_dir is etc.
<dpb1> k
<blackboxsw> there's a bit of  hardcoding of strings and some duplication that I've introduced that'd be nice to bring together in one place
<blackboxsw> hardcoding paths
<dpb1> I was just looking over your branch, makes sense now
<rharper> well, there is the confg paths; which consumes the system config into an object
<rharper> so I suspect we'll need to use the stage wrappers to ask for path to various keys
<rharper> which allows the system config to override
<blackboxsw> yeah, it feels like the Init() -> Init.read_cfg() -> cfg object is almost the right place. it just doesn't feel 'accessible' or parameterized quite enough
<blackboxsw> it feels like it's still a beast of an object to tear apart into the component you want to look at (as it still contains unparsed  yaml strings as the attributes)
<blackboxsw> or maybe I've just dug into the wrong hole on it
<rharper> oh, that's completely fair
<rharper> blackboxsw: I think you want the init object paths properties
<rharper> which gives you the cloudinit/helpers.py:Paths() object
<rharper> that has alot of the methods that combine the system config with "standard" paths (like instance_link,  run_dir, etc)
<blackboxsw> yeah, though it doesn't seem to surface log paths.
<blackboxsw> maybe we need to extend Paths to be our source of truth... no quite sure
<rharper> that sounds reasonable (extending paths)
#cloud-init 2017-11-14
<blackboxsw> hrm just reviewing our active queue today. https://code.launchpad.net/~johnguthrie/cloud-init/+git/cloud-init/+merge/331905 it feels like a nice thought (chef variable substitution. I feel like we could provide a variable substiturion mechanism for cloud-config modules based on the instance-data.json though instead which could be a bit more generic.
<blkadder> Pretty please?
<blkadder> Right now I am writing a wrapper script to do substitutions in my yaml files and writing another script to do substitutions in config files on the box as I am trying to see how far I can get without using chef/puppet/et al.
<blkadder> variable substitution within cloud-init would be great...
<blackboxsw> hehe :) yep. I need to get https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330115 landed so we can build the substitution framework on that standardized instance-data.json
<blackboxsw> I might take a quick stab at it today to see how tough this substitution framework would be.
<blkadder> I am sure a generic framework will be much more challenging than the very specific little tidbits I am doing. :-)
<blackboxsw> definitely, I'm trying to weigh how painful and whether it's better in the short term to grow our cc_* modules individual templating language vs. just swallowing the cost of generic templating
<blackboxsw> (I'm an unrealistic optimist and hope the generic templating isn't too bad)
<blkadder> Heh
<blackboxsw> thx robjo on the branch. couple comments for discussionhttps://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/333575
<robjo> OK, will take a look in about 10 minutes, thanks
<robjo> blackboxsw: fixed the comment and the substitution, for the other stuff, just let me know the desired direction, symlink or template name substitution
<robjo> I think there is a latent bug in cc_ntp, on line 127 rename_ntp_conf() is called, but if the timesync client is timesyncd the configuration file that may or may not be provided by the distro is never renamed as the file to be renamed alwasy falls back to NTP_CONF
<robjo> the call should really be rename_ntp_conf(confpath) IMHO
<robjo> smoser: blackboxsw ^^^^ thoughts?
<blackboxsw> checking
<blackboxsw> robjo: +1 on rename_ntp_conf(confpath) given that we could be dealing with a different service than actually NTP. and renaming NTP_CONF ->  NTP_CONF_FILE  too
<robjo> OK, starting with chrony support so this will be intermingled ;)
<jhogarth> rharper, ping ... heys ... it's James working on that net-tools deprecation :)
<rharper> jhogarth: hey
<jhogarth> I've made it a personal mission to "cleanse" Fedora of the dependancy ... not trivial though ...
<rharper> hehe
<jhogarth> just grabbed a NetBSD ISO so I can validate that ifconfig and route/netstat output and options against the net-tools version
<rharper> is sysconfig net-tools free?
<rharper> jhogarth: I think there are bsd images on azure as well, in case you just want to kick off one of those instead of the installer
<jhogarth> yes ... as of about F19 I think or a little before that ... EL7 is net-tools free for the network service so it must be around F15-F19 they finished up on that
<jhogarth> that means using azure ... I'm allergic to MS stuff :P
<jhogarth> in all seriousness though it's probably useful to have a BSD VM template on my laptop for cloneing and testing
<rharper> cool
<rharper> currently the sysconfig renderer checks for "ifup" and "ifdown" as well as the /etc/sysconfig/network-scripts/network-functions
<rharper> in F19/EL7 are there still ifup/ifdown tools provided outside of net-tools ?
<rharper> see cloudinit/net/sysconfig.py:def available()
<jhogarth> yes ifup and ifdown are owned by the initscripts package rather than the net-tools one and they either use nmcli if NM is running (and the interface isn't marked with NM_CONTROLLED=NO) or call the relevant network service funstions which use iproute2
<jhogarth> so far as I can tell the only things that cloud-init relies on net-tools for on Red Hat based systems is the info printouts
<jhogarth> so once netinfo.py is ported to have ip as a preference and ifconfig/netstat as a fallback then we *should* be good to drop the net-tools dependency in the rpm spec
<blackboxsw> cloudinit/sources/DataSourceAzure.py has some ifconfig uses
<blackboxsw> sorry, just eaves dropping
<rharper> jhogarth: yeah; that's what I recall as well
<blackboxsw> however that's spelled ;/
<rharper> blackboxsw: good find
<jhogarth> it's actually been quite amusing on this journey at times since at least a few packages already had iproute2 codepaths upstream but the maintainers just didn't realise they could drop the dependancy :)
<rharper> I suspect there are other ifconfig (I think in the bsd space as well)
<blackboxsw> don't thank me, thank fgrep -r :)
<rharper> so a grepping of the source for dropped binaries is worth it
<blackboxsw> rharper: true cloudinit/distros/freebsd.py
<rharper> mmm, legacy
<jhogarth>  grep -E '\b(ifconfig|netstat|arp|route)\b' -R * <-- you're friend ;)
<blackboxsw> heh
<jhogarth> well we dont' need to worry about porting the bsd stuff as that will never have ip ;)
<rharper> right, it would be non-bsd paths use of net-tools
<jhogarth> and looking at the code that azure adat source stuff is bsd only :)
<rharper> indeed
<blackboxsw> yeah though the stock  BOUNC_COMMAND.. isn't bsd-specific is it?
<blackboxsw> line 30
<jhogarth> there's some test stuff calling route etc ... but that's your debian/canonical packaging and testing :
<jhogarth> :p
<jhogarth> blackboxsw, that's calling ifup/ifdown which is safe though
<blackboxsw> ahh too true.
<rharper> hrm, I wonder how the Azure stuff on artful/bionic are getting on then;
<rharper> we've no ifup/ifdown in netplan only images
<blackboxsw> I'll spin one up now that we have images
<jhogarth> luck? heh
<rharper> well, there is some mystical things that happen under the agent which may or maynot need the bounce
<jhogarth> this is why CI setups are good at both unit and integration/functional levels :p
<rharper> =)
<jhogarth> so .... NetBSD ifconfig output is different from even the current net-tools ifconfig linux snapshot heh ... glad I checked ;)
<jhogarth> because everyone easily can read the netmask when it looks like: inet 192.168.124.96 netmask 0xffffff00 broadcast 192.168.124.255  :/ and hwaddr actually has a tok of address .... NetBSD output on cloud-init most have been nothing like the linux output :/
<jhogarth> still ... now I have sample output to add to the test cases :)
<jhogarth> do you guys just support NetBSD or should I confirm if FreeBSD has the same output?
<jhogarth> and of course BSD has to have completely different routing output with netstat -rn ... and now I doubel check the code paths and it's FreeBSD you support, not NetBSD ... although I could have sworn NetBSD was referenced in a bug ...
<blackboxsw> rharper: http://paste.ubuntu.com/25963614
<blackboxsw> bionic works on azure
<jhogarth> oh well ... enough of my verbiage ... it's bed time in the UK ... thanks for the feedback and discussion ... I'll keep on hacking this over the next week to hopefully get every case tested and something nice to merge with a bunch of bugs solved too :)
<blackboxsw> because of the magic :)
<blackboxsw> erm ,,,, heh n/,m
<blackboxsw> traceback on hostname bounce
<blackboxsw> due to failed ifdown/ifup
<blackboxsw> I'll file a bug
<jhogarth> i'll be about in the channel whilst working on the patch this week or so :) laters
<blackboxsw> smoser: already had it https://bugs.launchpad.net/cloud-init/+bug/1722668
<ubot5> Launchpad bug 1722668 in cloud-init (Ubuntu) "Azure: bouncing of network device/publishing of hostname fails on artful" [Critical,Confirmed]
#cloud-init 2017-11-15
<blackboxsw> sweet cloud-init 17.1.25  hit xenial
<blackboxsw> ok blog post rework this afternoon
<jdandrea> I see how to set http/s proxy and no_proxy for apt. Is there a way to set proxies for the default environment? Suspecting it's not wise to just blow away /etc/environment (and I can't append).
<smoser> jdandrea: there is not.
<smoser> you could use boothook to intelligently update /etc/environment (boothooks run every boot)
<smoser> but generally speaking its just a pita when you try to set http_proxy globally
<smoser> like that
<jdandrea> @smoser Ok. I ask because without that then the apt settings aren't as helpful for us on first boot.
<smoser> so i really don't recommend it.
<jdandrea> @smoser Hmm, ok. I wish this lab wasn't behind a proxy. :/
<smoser> so.. why isnt it helpful for you on first boot ?
<smoser> basically, static http_proxy and https_proxy fall apart as soon as you try to use an address that isnt proxy-able
<jdandrea> We also set no_proxy tho
<smoser> yes, but that is vastly insuffiicent
<jdandrea> Indeed it is. I wish I had a better option.
<smoser> gnu tools read it sanely, and allow you to use wildcard and even cidr entries
<jdandrea> In another lab they have proxyless gateways so it's a sinch.
<smoser> but python (and other languages) do not.
<jdandrea> s/sinch/cinch
<smoser> so they're uselss.
<smoser> adn then, you do something like launch a container, and it has its own network that wasnt in your http_proxy and you are foobarred again
<jdandrea> Aye, and in those use cases that's already taken care of with install. I'm really more concerned with bootstrapping things.
<smoser> here is what i do:
<smoser>  http://bazaar.launchpad.net/~smoser/+junk/sstack-proxy/view/head:/sstack-proxy
<smoser> so bootstrapping ? as in with maas ?
<jdandrea> Right, I get that. Presuming I have a solid handle on communicating proxy settings down to various bits, I'm left with trying to get things to a happy state during cloud-init. In this case it's just cloud-init stuffs that are kicked off after an OpenStack nova boot.
<jdandrea> Reading...
<jdandrea> This particular lab already has a proxy server, thus I suspect something like tinyproxy wouldn't fly (?).
<smoser> ah. ok. i see.
<jdandrea> Basically I wish they didn't have one, but ... hands tied.
<smoser> jdandrea: oh. i run the tiny proxy on the host
<smoser> and point http_proxy/https_proxy at *it*
<jdandrea> @smoser Ah
<smoser> then you dont ever have to change your http_proxy/https_proxy vars
<jdandrea> Ohhhh
<smoser> and you can just update tinyproxy and restart
 * jdandrea gets clue
<smoser> and then all things get updated
<smoser> the issue is its slower than no proxy
<jdandrea> Right, and in this case I'm betting folks won't like that. But hey, if I can get it to work sanely...
<jdandrea> So it requires at least one reboot, ya?
<jdandrea> (Not necessarily a bad thing.)
<smoser> well, a new session after /etc/environmetn is wrtten, yeah.
<smoser> and daemons restarted possibly.
<jdandrea> Ok
<jdandrea> I will study this. Thank you!
<smoser> actually , daemons do not read /etc/environment enerally speaking
<smoser> you can make similar change to systemd config...
<jdandrea> Right. This is there purely so that the one-time scripts that are running at boot time can do their thing, and then they will communicate proxy settings to whatever services need them from that point on.
<smoser> i dont think we added an  'environment' config to cloud-init
<smoser> but i have wanted to.
<jdandrea> I think I can replace /etc/environment but I suspect that is a Very Bad Idea(tm). :D
<smoser> ie, cloud-init would read your settings in 'environment:' and update its environ.
<jdandrea> mhm mhm that's what I was looking for, like for apt.
<smoser> jdandrea: i'm surprised at the amount of peole that write proxy data into e/tc/environment
<jdandrea> Well, honestly? I'd rather not do that.
<smoser> it seems without a solution like tinyproxy to be copmetely a broken idea to me.
<smoser> so yeah, i'd stay out of taht.
<jdandrea> But it *looks* to me (and I could be wrong)... like Ubuntu needs to know about proxies early on in cloud-init. I may be wrong.
<smoser> what is it that you ahve to do before then ?
<smoser> what is "ubuntu" ?
<smoser> in this case... what uses ?
<jdandrea> Because there are some things it's trying to get and it can't reach some servers. Ubuntu meaning Ubuntu 14.04 or 16.04.
<smoser> apt will do the right thing.
<jdandrea> Lemme try and make a minimal paste to show...
<jdandrea> I could also just be wrong. :)
<smoser> my 'sstack-proxy' came from usage of a cloud that was simliarly configured.
<smoser> in that you can't get to the interwebs without proxy
<smoser> but i only did things that *needed* access after cloud-init was up and hadn run that
<jdandrea> @smoser So this is early on in cloud-init. The IPv6 I'd ignore. But the pollinate stuff overall... connecting to entropy: https://hastebin.com/ucaviyajuq.rb
<smoser> http://paste.ubuntu.com/25969929/
<jdandrea> This is before any user scripts are run, methinks.
<smoser> thats my user-data that i run
<smoser> for use case of general system i want to go to.
<jdandrea> *nod*...
<smoser> you can probably ignore the pollinate.
<smoser> they shouldnt cause trouble generally.. failure shoudl not cause issue
<jdandrea> Ok, then in that case I don't need to touch environment.
<jdandrea> Whew.
<smoser> timeouts suck though
<jdandrea> Yeah. And y'know it may have absolutely nothing to do with proxies. Could also be noisy neighbors.
<jdandrea> (Now seeing 1 minute downloads take 45. Um...)
<smoser> you can also i think tell cloud-init to disable pollinate. if you dont like its warnings.
<jdandrea> TIL: can disable pollinate. Thanks! Good to know
<jdandrea> Hopefully we don't need the entropy. XD
<jdandrea> Appreciate the insight. I will look at both of these pastes.
<smoser> actually i'm wrong. you can't disable pollinate.
<smoser> ewll, you can, but not through cloud-init.
<smoser> (you could with a bootcmd:
<jdandrea> Hehe. But... it's not a total disaster if it doesn't do its thing.
<jdandrea> ?
<jdandrea> Ah, ok.
<smoser> ln -sf /bin/true /usr/bin/pollinate
<smoser> jdandrea: its supposed to be seeding random data.
<smoser> so it reads some random information off of a remote web service and writes it to /dev/random
<smoser> to help with entropy
<jdandrea> *nodnod*
<smoser> entropy quality in a vm is a complicated topic
<jdandrea> But if it can't reach it... "because proxy"... yeah.
<jdandrea> Oh yes, I can imagine.
<smoser> people suggest that not having enough entropy when your ssh host keys are generated could leave you at risk of attack
<jdandrea> Right. Threat models and all that. So... yeah. Something to ponder.
<smoser> oh.
<smoser> you're runnign 14.04 there
<smoser> 16.04 i dont think will spew error to console
<jdandrea> Yup. BUT... 16.04 in some cases.
<jdandrea> Ok.
<smoser> basically your complaint was "fixed"' by siliently failing :)
<jdandrea> Bwahahaha!
 * jdandrea applauds
<smoser> https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1554152
<ubot5> Launchpad bug 1554152 in pollinate (Ubuntu) "pollinate fails in many circumstances, cloud-init reports that failure, maas reports node failed deployment" [Critical,Fix released]
<jdandrea> Let me just turn down that baby monitor... :p
<smoser> actually, openstack
<smoser> will provid eyou with some blob of random data from the server
<smoser> in metadata
<smoser> (soo does azure)
<smoser> and cloud-init will use that
<smoser> so that provides locally to the cloud the value that pollinate would provide.
<jdandrea> Ok, then we're good.
<jdandrea> But really, beyond pollinate, I can set the proxies within a user script and it's fine.
<jdandrea> And it's temporary. After that the service/VNF that comes up does the right thing.
<jdandrea> I'm just glad I don't need to touch /etc/environment.
<smoser> yeah./bu13
<blkadder> Perhaps a silly question but why is touching /etc/environment considered such a bad thing? Reading through https://help.ubuntu.com/community/EnvironmentVariables there is nothing saying âOMG this is super bad donât touch.â
<blackboxsw> smoser: rharper just pushed a cut at sourcing logfile paths from the cloud-init cfg to https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/333513
<rharper> blkadder: I suspect in general, system wide variables can cause problems in places that may be unexpected;  I would think per-application settings are generally preferred; reading the wiki page; it sounds like that's the line on /etc/environment; which is (IMO) sane policy
<blkadder> Definitely preferable to do per app but if this indeed was intended to be have a global, persistent effect that would seem to be the place to do it.
<blkadder> Totally off-topic for here just found the earluer convo interestingâ¦
<blkadder> earlier...
<blkadder> Understood about side-effects though.
<rharper> blkadder: yeah, not that off topic for system initiazation
#cloud-init 2017-11-16
<SmokedCheese> hello! it's possible to set hostname with variables?
<SmokedCheese> or should i use something like bootcmd: - hostnamectl set-hostname "hostname-$(http://169.254.169.254/2009-04-04/meta-data/instance-id)" ?
<smoser> SmokedCheese: its not possible right now.  we have a card to do something like that.
<smoser> so yeah, your only option is what you have there.
<SmokedCheese> okay, thanks!
<smoser> SmokedCheese: https://trello.com/c/AYaCdQyT
<smoser> that is the card that describes what we'd be looking to do.
<smoser> and i just added a section for what i think you were asking for.
<SmokedCheese> sweet
<smoser> powersj: so... https://code.launchpad.net/~ajorgens/cloud-init/+git/cloud-init/+merge/331660
<smoser> ci didnt run. but didnt complaon
<smoser> and /me didn't see it cry
<smoser> because ajorgens isnt in happy list
<smoser> probably need some solution for that
<powersj> aah
<powersj> let me add him to the white list
<blackboxsw> +1 powersj can you make sure robjo is on that too
<blackboxsw> he might already be
<powersj> blackboxsw: robojo is rjschwei?
<blackboxsw> powersj: yep
<smoser> powersj: where is that list ?
<powersj> smoser: torkoal:~/.jlp/jlp.config
<powersj> and done
<robjo> my lp userid is rjschwei
<robjo> powersj: ^^^^
<powersj> robjo: thx
<blackboxsw> welcome to the wide world of automated CI votes on your branches robjo
<robjo> blackboxsw: thanks, something new to figure out ;)
<robjo> on that note, anyone have any time to provide some feedback on the chrony integration on the ML?
<smoser> blackboxsw: hm..
<smoser> hm..
<blackboxsw> hm..
<blackboxsw> ok cloud-init status systemctl checks is pushed
<blackboxsw> sorry smoser yeah that last paragraph of the blog post was WIP. will ping when done
<blackboxsw> ok done.
<robjo> smoser: blackboxsw was there a network rendering fix after 17.1 was released?
<robjo> reference is https://bugzilla.opensuse.org/show_bug.cgi?id=1064854
<ubot5> bugzilla.opensuse.org bug 1064854 in OpenStack "cloud-init ignores network settings from config-drive in external network" [Normal,New]
<robjo> it looks to me as if a V1 config is not properly transformed
<blackboxsw> robjo approve https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/333575 with comments
<blackboxsw> approved rather. nothing for you to fix that I can see.
<blackboxsw> probably will merge later today
<blackboxsw> checking the bug
<robjo> this appears to be tested in test_apply_network_config_eni_ub in test_netconfig
<robjo> and I fail to see where this transformation would be distro implementation dependent
<robjo> The way I see it the transformation should be as follows from V1 config to the debian/ubuntu network specification, which should then end up in the distro implementation which for openSUSE is still _write_network
<blackboxsw> robjo: looks there is a distro-dependent component for network renders. I see renderer_configs defined in cloudinit/distros/debian.py which direct what renders to use for debian/ubntu
<blackboxsw> checking how that's used
<robjo> I am not using the renderer concept yet, because the sysconfig renderer has the path hard coded and the paths between RH and SUSE do not match, that's where the refactoring comes into play that I mentioned in the meeting the other day
<robjo> for now I end up in network_state.parse_net_config_data
<robjo> In "regular" code flow somewhere would be a call to distro,apply_network_config(), which fails for openSUSE/SLES the exception is then caught and we end up in _apply_network_from_network_config
<robjo> in this method the information is supposed to be transformed to the "standard" debian/ubuntu interface configuration, i.e. it should look something like V1_NET_CFG_OUTPUT in the test
<robjo> is there a test that covers that specific transformation?
<rharper> robjo: so, network_data.json -> v1 config  happens, then, since you're not yet using the sysconfig renderer,  you're getting an already rendered eni config, but the opensuse write_network is re-reading it to convert it to sysconfig; it appears
<blackboxsw> generally there is ./tools/net-convert in our repository we use to validate rendering types given a provided network config and specifying whether your are rendering ['eni', 'netplan', 'sysconfig'] output files.
<rharper> I don't think we have a v1 -> net_util.translate_network() path
<rharper> which is what's happening here
<rharper> robjo: and there are no unit tests for net_util
<robjo> :(
<blackboxsw> meh the problem with 63% unittest coverage, is the other 37%
<rharper> whats the delta in rhel sysconfig paths and suse ones?
<rharper> that seems like a shorter path (using sysconfig renderer in the suse distro class)
<robjo> 'network' (SUSE) vs. 'network-scripts' (RHEL) , yes I also roll my eyes over stuff like this ;)
<rharper> man, that wants some temporary symlinking
<rharper> if that's it
<robjo> well "two ugly" do not really make "one pretty"
<rharper> nope
<rharper> but the instance boots with networking
<rharper> too bad we can't ask sysconfig what paths it's using
<robjo> but I suppose I could force the symlink in the cloud-init package, although I I remember correctly I tried to patch the path for 0.7.9 and that failed
<rharper> robjo: where duse SUSE put the /etc/sysconfig/network (global settings) file ?
<robjo> of course when 17.1 came around I had forgotten all of that and just moved forward since a good chunk of my out of tree code was now included
<rharper> on RH, that has NETWORKING=yes and NETWORKING_IPV6=yes ;
<robjo> the global settings file is /etc/sysconfig/network/config
<rharper> urg
<rharper> so both those paths
<robjo> I suppose in the rendered we could test the and then switch between 'network' and 'network-scripts' but that's not very pretty either
<robjo> s/rendered/renderer/
<rharper> well, I was thinking about that as well but; I think there may be other issues
<rharper> in that config file, it uses NETCONFIG_* namespace for some variables
<rharper> I suspect you'd need a subclass of sysconfig for some things, but others would be fine
<rharper> bbiab
<robjo> yes, there are NETCONFIG_* settings in that file
<robjo> I agree that the proper route is subclassing i.e. refactoring, hence my question during the meeting
<robjo> I think the shorter path for now may be to figure out a v1 -> net_util.translate_network()
<robjo> unfortunately I am getting lost in network_state.py in parse_config_v1, cannot quite wrap my head around what's happening there
<robjo> BTW at least in the code the path from v1 -> net_util.translate_network() at the high level is not too convoluted
<robjo> If I put he puzzle together properly it would go along these lines:
<robjo> apply_network_config -> this raises an exception which is caught and gets me to _apply_network_from_network_config here the config gets transformed vi network_state.parse_net_config_data() and eni.network_state_to_eni() then we end up in apply_network which finally calls _write_network() and the last method is implemented in the distro subclass and for openSUSE/SLES we end up caling net_util.translate_network()
<robjo> my thinking is that either in network_state.parse_net_config_data() or eni.network_state_to_eni() something goes wrong
<smoser> blackboxsw: were you looking at centos fail ?
<smoser> i verified it does fail here locally
<smoser>  ./tools/run-centos --unittest 7 --keep
<blackboxsw> smoser: yeah I had forgotten while fixing up disable and the blogpost. but reproduced on 6 and I'm working now
<blackboxsw> looks like the URLError text raised is different in centos than ubuntu: 403 Client Error: Forbidden for url: %s versus just "403 Client Error: Forbidden"
<ubot5> Error: Ubuntu bug 403 could not be found
<blackboxsw> thx ubot5
<blackboxsw> I'll update a patch to provide the bad_url if not present in the error message
<smoser> nacc: i think you told me of somethign that is like
<smoser>  git commit-tree && HEAD
<smoser> ie, head after commit-tree
<smoser> ?
<nacc> smoser: sorry, not followinng
<smoser> if i have changes in my working directory
<smoser> i wanted to refer to HEAD + changes-to-tracked files
<smoser> ithink like this https://stackoverflow.com/questions/2766600/git-archive-of-repository-with-uncommitted-changes
<nacc> smoser: you can write-tree
<nacc> smoser: and that gives you a treeish
<nacc> and git-archive can archive a treeish
<nacc> smoser: it depends on what you want to do ... HEAD points to a commit, so it can't point to an uncommited state
<smoser> and the yes
<nacc> smoser: is there any reason you ca't just do a wip commit?
<nacc> (which you might discard later)
<smoser> how do you you reset to state before the commit ?
<smoser> including 'git add' or other things.
<smoser> ie, if i
<smoser>  touch foo
<smoser>  git add foo
<smoser>  git commit -a -m "that wip commit"
<smoser>  git reset HEAD^
<nacc> smoser: you can alwasy reset --hard to a commit
<smoser> then i've lost the fact that 'foo' was added
<nacc> smoser: and potentially do a `git clean -fxd`
<nacc> smoser: no, it's still inn the commit you created
<smoser> yeah, but the state of having been added is lost
<nacc> smoser: i mean, i'm not sure i understand the use case, exactly
<smoser> except for in that commit
<nacc> smoser: oh you want to go back to an index state?
<smoser> yeah, that was the goal.
<smoser> basically i wanted to collect the tree up as it is right now.
<smoser> wierd, i looked at 'git stash create'
<smoser> this seems wrong:
<smoser>  echo "hi mom" > foo.txt
<smoser>  git add foo.txt
<smoser>  git stash create
<smoser> the stash does not contain the new 'foo.txt'
<smoser> oh wait. it does
<blackboxsw> powersj: smoser https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/333845 for the ci nightly errors
<rharper> robjo: re v1 -> translate; I think at least for the immediate concern, matching the post-up "default gw" -> gateway: <value>  ought to get that working
<nacc> smoser: i'm still trying to understad why you want the tree up to now? for a git-archive call?
<robjo> rharper: ?
<rharper> robjo: by my reading it goes config-drive -> openstack helper convert_network_data_json to v1 config -> v1 config gets rendered to eni, then the eni content get's passed to your class which calls net_util.translate()
<robjo> OK, then I am on a wild goose chase
<rharper> yes, the https://bugzilla.opensuse.org/attachment.cgi?id=749026
<rharper> shows the 2017-11-16 15:26:39,868 - opensuse.py[DEBUG]: Translated ubuntu style network settings # Converted from network_config for distro <class 'cloudinit.distros.opensuse.Distro'
<rharper> and then prints out the eni that got converted from v1 config
<rharper> in distro base-class, it emits the Warning, thand then does a ns = network_state_parse_config(which is fed the v1 which was converted from the network-data json, you can see the v1 config earlyer in the log)
<rharper> then in base distro, it does content = eni.network_state_to_eni() which makes an eni file; and passes that to the distro specify layer (self.apply_network() -> self._write_network())
<robjo> I am confused about where the openstack helper comes into play
<rharper> converting the config drive network_data.json
<rharper> when the Datasource is parsing it's input
<rharper> robjo: cloud-init local runs to find the datasource, detects config-drive and runs the ConfigDrive datasource; in it, if there is a file network_data.json, it calls the helper to convert the json into v1 config and storages it in DataSource class property (network-config);  then as cloud-init local continues, if the datasource has a 'network-config' property; it it's read and "rendered"
<rharper> 2017-11-16 15:26:39,825 - stages.py[DEBUG]: applying net config names for {'version': 1, 'config': [{'subnets': [{u'routes': [{u'netmask': u'0.0.0.0', u'network': u'0.0.0.0', u'gateway': u'192.168.168.1'}], u'netmask': u'255.255.255.0', u'type': 'static', 'ipv4': True, 'address': u'192.168.168.30'}], 'mac_address': u'fa:16:3e:ee:2b:97', u'type': 'physical', 'name': 'eth0', u'mtu': 1500}]}
<rharper> there you can see stages applying the v1 config that it got from the configdrive data source (after the network_data.json conversion)
<robjo> yes, that very early part I ignored and I started with that data and where I think it would end up in the code
<nacc> smoser: sorry for being dense if i'm missing something obvious
<robjo> and as I noted in comment #32 in the bug gateway information is missing from the renedered intermediate format
<rharper> right, that;s due to hoe the eni is rendered
<rharper> the gateway is inside the post-up route add default gw <gatewayip>
<rharper> the net_util.translate does not know how to (yet) extract the gateway value from the eni contents
<rharper> so, I believe in that net_util.translate, you can check if line has 'post-up' and 'default gw' strings
<rharper> and extract out the ip addr in there and use that as the GATEWAY= value in sysconfig
<rharper> you can see the v1 dictionary has 'gateway': 192.168.168.1 which matches the post-up line; post-up route add default gw 192.168.168.1 || true
<robjo> OK I think I think I get how that part of the puzzle fits together
<rharper> the network_data.json file may have other routes that are not default gateway ones, so there may be additional {pre,post}-{up,down} routes added which will also need converting to the sysconfig syntax; depending on how far you want to take the translator
<robjo> well everything that is not "default gw" shoud really go into ifroute-IFACE
<rharper> yes, I think so
<rharper> we generally only had special handling for default gw, vs. route add -net
#cloud-init 2017-11-17
<smoser> blackboxsw: ping
<smoser> hey
<blackboxsw> pong
<smoser> i was just about to accept your merge
<smoser> i was going to just put a comment in
<smoser>                 except UrlError as urle:
<blackboxsw> smoser: sorry was tweaking the description to be more appropriate
<smoser>                     message = str(urle)
<smoser>                     # older versions of requests may not get the url
<smoser>                     # into the message.
<smoser> and fix up the commit message.
<smoser> yeah.
<smoser> and that'd be it.
<blackboxsw> ahh sounds good. want me to push
<smoser> then i'im good.
<smoser> i'll comment and you can take it.
<blackboxsw> +1 will do
<smoser> just approved with changes on the mp
<blackboxsw> smoser: I was gonna land https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/333575
<smoser> please grab
<smoser> thanks
<smoser> i'm fine with that too
<blackboxsw> I can wait til morning on that 2nd branch though
<blackboxsw> ok will do tonight
<smoser> fix c-i first though
<blackboxsw> agreed
<smoser> ie, the urlerror message first
<smoser> just to have less broken tips
<smoser> thanks
<smoser> and i'm out.
<smoser> later
<blackboxsw> thanks have a  good one
<smoser> oh for petes sake jenkins
<smoser> blackbox fixed cloud-init but then jenkins cries
<smoser>  https://jenkins.ubuntu.com/server/job/cloud-init-ci-nightly/161/console
<blackboxsw> smoser: can we rerun nighlty?
<blackboxsw> to get a good value
<blackboxsw> powersj: smoser: I'll fixup qa-scripts/scripts/launch-ec2 for bionic  while smoser is working a unit test for fallback_nic on upgrade
<blackboxsw> launch-ec2 on my end was working from xenial :/ I'll fixit up on bionic now
<smoser> $ echo raw support for rharper | haste -r
<smoser> https://hastebin.com/raw/zurabikoko
<rharper> \o/
<rharper> can you hastebin your haste tool ?
<smoser> blackboxsw: https://hastebin.com/akazezaroh
<smoser> thats what i have so far
<blackboxsw> thx smoser will pull that in
<blackboxsw> and handle other issues
<smoser> blackboxsw: it seems wierd that boto doesnt expose 'InvalidGroup.NotFound'
<smoser> or any of those.
<smoser> or even 'code' on the Error or seomthing
<blackboxsw> Yeah that seems broken. string parsing in the error message is not appropriate
<blackboxsw> not an appropriate design decision
<blackboxsw> I wonder if there's a structure I can import.
<blackboxsw> I'll look at boto3 modules
<blackboxsw> ahh recent python3-boto3 in bionic has some exception goodnees
<blackboxsw> smoser: powersj just pushed qa-scripts/scripts/launch-ec2 for bionic
<blackboxsw> de823f2..58c9a97
<blackboxsw> will test the failed upgrade
<smoser> blackboxsw: https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+ref/bug/1732917-fix-fallback-interface
<smoser> see what you think about that
<smoser> i dont think we actually *need* the setter
<smoser> (though i could add one)
<blackboxsw> smoser: spelling inteface
<blackboxsw> other than the typo on the class variable '_fallback_inteface' it should work.
<smoser> blackboxsw: https://hastebin.com/kivunetopa
<smoser> (hean, and i fixed those and pushed --force)
<smoser> tox now passes
<smoser>  blackboxsw launch_ec2 is really nice.
<blackboxsw> per your latest 'haste' I think boto3 on bionic may have an official exception for that. I'll try (changing my keypair
<blackboxsw> bummer, still botocore.exceptions.ClientError: An error occurred (InvalidKeyPair.NotFound) when calling the DescribeKeyPairs operation: The key pair 'cloud-init-integration-chad' does not exist
<blackboxsw> ok taking your try/except changes
<blackboxsw> and thanks
<blackboxsw> I want to add the ipv6 setup support and hit the blog with it in hand.
<blackboxsw> let's get past this upgrade SRU bump
<smoser> blackboxsw: i laucnhed an instance, ssh'd to it
<smoser> and then it disappeared
<smoser> oh. you terminated it for me ?
<smoser> seems like keep_alive should be default :)
<blackboxsw> smoser: yeah --keep-alive
<blackboxsw> sorry
<blackboxsw> could surface kill-it :)
<blackboxsw> I'll change that param
<blackboxsw> --destroy :)
<smoser> ugh
<smoser> launched instance
<smoser> typed 'apt-get update'
<smoser> 0% [Connecting to security.ubuntu.com (2001:67c:1560:8001::14)]
<smoser> hung
<blackboxsw> same
<blackboxsw> other apt repos  worked
<blackboxsw> all local to amazon though
<blackboxsw> works on 0.7.7
<smoser> so thats bad news
<smoser> the others resolve to ipv4 though maybe ?
<blackboxsw> yet why would 0.7.7 work
<smoser> well, we get ipv5 address
<smoser> so something notices that and returns the ipv6 address for the security.ubuntu.com
<smoser> and then we do not have outbound connectivity i guess
<blackboxsw> upgrading from 0.7.7 xenial (with working apt connectivity) -> 17.1
<smoser> that should be fine, no ?
<blackboxsw> checking to be sure
<smoser> blackboxsw: did you recreate this ?
<smoser> the failure in that bug
<blackboxsw> not yet smoser
<blackboxsw> trying to though
<smoser> i launched instance
<smoser> upgraded
<smoser> rebooted
<smoser> no WARN
<smoser> ugh
<blackboxsw> no error on my side. on upgrade path.
<blackboxsw> trying specifically from 0.7.9~233
<blackboxsw> for my next pass
<smoser> hm.
<blackboxsw> hmm is right, from 0.7.9 -> 17.1 (upgrade without clean) I reboot without error
<blackboxsw> as cloud-init doesn't re-run
<blackboxsw> hrm. checking that bug traceback again
<smoser> blackboxsw: perhaps this is not on ec2
<smoser> he never says he is.
<smoser> definitely datasource got used. but nots ure.
<blackboxsw> smoser: wierd comment from him
<blackboxsw> https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1732917/comments/1 :( hmm
<ubot5> Launchpad bug 1732917 in cloud-init (Ubuntu) "17.1 update breaks EC2 nodes" [Undecided,New]
<blackboxsw> he says the failure happens when restarting the cloud-init? but goes away when restarting the node?
<blackboxsw> I'm misreading that
<blackboxsw> I'm just not really sure what that's saying
<smoser> yeah.
<blackboxsw> ohh maybe running cloud-init init  or something?
<smoser> we could see what happens on openstack if we set it to use the Ec2 datasoruce
<smoser> blackboxsw: good news is that this isnt as serious as it seemed at first
<blackboxsw> yeah, I'm just trying to see if maybe a complex networking setup would cause this?
<blackboxsw> not sure
<smoser> we need to identify the issue witih ipv6 too
<smoser> the hang on security.ubutnu
<blackboxsw> stepping away for 20
<blackboxsw> gotta help w/ lunch
<smoser> blackboxsw: https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/333905
<blackboxsw> ugh
<smoser> its not too big a deal i dont think
<rharper> smoser: blackboxsw: do you have logs from the upgrade cloud-init reboot ?
<smoser> http://paste.ubuntu.com/25982990/
<rharper> thx
<smoser> there is a /var/log/cloud-init
<blackboxsw> http://paste.ubuntu.com/25982994/
<blackboxsw> 2late
<smoser> line 109 is intersting there.
<smoser> oh. thats tmp file deletion
<smoser> lucky it didnt fail
<rharper> smoser: this is both original and upgraded  in the same file ?
<rharper> smoser: your paste looks like launch, reboot, upgrade, reboot ? is that right ?
<blackboxsw> 2017-11-17 18:32:43,838 - stages.py[DEBUG]: cache invalid in datasource: DataSourceEc2
<blackboxsw> first time i;ve seen this
<rharper> smoser: actually, I'm really confused;
<rharper> 2017-11-17 18:17:36,206 - handlers.py[DEBUG]: finish: modules-final: SUCCESS: running modules for final
<rharper> 2017-11-17 18:19:06,783 - util.py[DEBUG]: Cloud-init v. 17.1 running 'single' at Fri, 17 Nov 2017 18:19:06 +0000. Up 102.29 seconds.
<smoser> rharper: that was launch with old, upgrade, reboot, possibly more reboots.
<rharper> 0.7.9 finished at 18:17
<rharper> then there's a single mode ?
<rharper> after upgrade ?
<rharper> I would have expected the reboot
<smoser> i dont thin ki ran that.
<rharper> well
<rharper> your log shows you did
 * rharper looks at blackboxsw 
<smoser> oh
<smoser> chad might have run that for me.
<blackboxsw> --proposed reboots
<rharper> same in changes
<rharper> 2017-11-17 18:31:21,520 - util.py[DEBUG]: Cloud-init v. 17.1 running 'single' at Fri, 17 Nov 2017 18:31:21 +0000. Up 277.24 seconds.
<rharper> 2017-11-17 18:31:21,521 - stages.py[DEBUG]: Using distro class <class 'cloudinit.distros.ubuntu.Distro'>
<rharper> what's 17.1 single mode doing ?
<smoser> hm.
<blackboxsw> yeah not sure there
<smoser> oh
<rharper> don't we just boot; sudo apt update && sudo apt install cloud-init && reboot  ?
<smoser> its the upgrade
<smoser> let me see.
<rharper> what is upgrade doing ?
<rharper> 2017-11-17 18:31:21,523 - cc_apt_pipelining.py[DEBUG]: Wrote /etc/apt/apt.conf.d/90cloud-init-pipelining with apt pipeline depth setting 0
<rharper> 2017-11-17 18:31:21,523 - util.py[DEBUG]: Reading from /proc/uptime (quiet=False)
<rharper> 2017-11-17 18:31:21,523 - util.py[DEBUG]: Read 14 bytes from /proc/uptime
<rharper> 2017-11-17 18:31:21,524 - util.py[DEBUG]: cloud-init mode 'single' took 0.063 seconds (0.06)
<rharper> I guess it's fixing up the apt conf ?
<rharper> but, that reloads the on-disk object prior to reboot
<smoser> debian/cloud-init.postinst
<blackboxsw> which might be what breaks apt to security,ubuntu?
<smoser> it only changes pipelining
<smoser> i'm guesing that code should be version-fixed in some way
<smoser> no
<smoser> but that is just noise
<rharper> ok
<rharper> just walking through the log
<rharper> I didn't expect that
<smoser> yeah, we should probably fix that
<rharper> ah
<rharper> on ec2, instance is always invalid
<rharper> I have a branch, but didn't finish it, to read instance_id from sys/dmi
<rharper> so we never read the cache at local time
<blackboxsw> ahh o
<smoser> ?
<rharper> we need to capture the system_uuid to compare
<smoser> right. we always re-discover. because there is no check
<smoser> yeah
<rharper> we don't do that
<rharper> so, the local cache check says, it;s invalid
<rharper> that's expected at this point (we always do this on ec2)
<blackboxsw> hmmm is it possible that get_fallback_nic returns None on some platforms
<rharper> but it would have blown up in local mode, no ?
<rharper> if I'm reading the bug log right, ti was stage init (versus init-local)
<blackboxsw> yeah it should have fallen apart in init-local
<blackboxsw> right if we were Ec2 proper, we wouldn't actually get to the DatasourceEc2
<rharper> well, Local exits on non-ec2
<blackboxsw> we would've already detected DatasourceEc2Local and not run init-network
<smoser> yeah
<smoser> thats why i asked if he was on Amazon
<smoser> i dont think it is
<smoser> and that we can try on serverstack
<rharper> you can force it to rn ec2 even on Openstack ? instead of the OpenstackDS ?
<blackboxsw> if we run dpkg-reconfigure cloud-init  we can force it right
<blackboxsw> just uncheck OpenStack
<blackboxsw> I *think*
 * blackboxsw fires up my vpn 
<blackboxsw> ok creating a xenial instance and will attempt the upgrade
<blackboxsw> ok clean reboot on 0.7.9 openstack instance w/ OpenstackDatasource gets me a warning banner
<blackboxsw> and upgrading/rebooting doesn't hit that traceback about fallback_nic on  the obj.pkl because Ec2 claimed invalid obj.pk and recreated it.
<blackboxsw> so Openstack images limited to Ec2Datasource can't reproduce this on upgrade path
<blackboxsw> Openstack-ec2datasource: â
<blackboxsw> here are the logs as that's a bit complex
<blackboxsw> here are the logs http://paste.ubuntu.com/25983308/
<blackboxsw> and for the record dpkg-reconfigure cloud-init did allow me to unset OpenstackDatasource on an openstack instance
<smoser> sure. and it should.
<blackboxsw> just felt I needed to affirm my "I *think*" comment
<blackboxsw> smoser: I'm testing your sandbox dhcpclient branch
<blackboxsw> will approve shortly
<rharper> blackboxsw: hrm; so we don't yet have a plausible path where we reload an EC2 datasource
<blackboxsw> yeah not that I can figure currently
<rharper> what about AliYun ?
<rharper> it can run at local and net (DEP_FILESYSTEM, DEP_NETWORK)
<rharper> and it will get the .fallback and the network config properties, but EC2Local won't run
<rharper> which I think get's us the path we're on;  that the variable defaults to None, and no path to set it to a fallback value that's not None
<rharper> smoser: do you have a aliyun account ?
<smoser> rharper: no. idont think so.
<blackboxsw> maybe we need to spitball, but smoser your patch seems like it would fix this path, however we got there
<smoser> blackboxsw: yeah. i think so too. :)
<smoser> and we need the better save too
<blackboxsw> so smoser yeah with public ipv6 configuration, I can't get to security.ubuntu
<blackboxsw> as in, if I dhcp6, apt timesout
<smoser> blackboxsw: is it possible that our security group is just set up incorrectly ?
<smoser> not allowing outbound ipv6
<blackboxsw> ahh very
<smoser> blackboxsw: that does somewhat still identify a regression
<rharper> blackboxsw: I'm happy with the smoser patch; and I suppose that we can't yet find a path to the failure should mean that the impact is narrow; but it's rather frustrating that it;s not obvious how we hit that path
<smoser> but its not really one we could do something about
<smoser> we can't easily enable ipv6 when it was enabled in the metadata and then not have the system use it.
<smoser> blackboxsw: rharper chat ?
<rharper> y
<smoser> https://hangouts.google.com/hangouts/_/canonical.com/cloud-init?authuser=0
<blackboxsw> Yeah lost network there in amin
<blackboxsw> approved https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/333905
<smoser> blackboxsw: old deb
<smoser> wget https://launchpad.net/ubuntu/+archive/primary/+files/cloud-init_0.7.9-233-ge586fe35-0ubuntu1~16.04.2_all.deb
<blackboxsw> thx smoser
<smoser> ddpkg install that deb
<smoser> rm -Rf /var/lib/cloud /var/log/cloud-init
<smoser> reboot
<smoser> apt-get install cloud-init
<smoser> cloud-init init
<smoser> then i tried to fix with  my deb (dpkg -i)
<smoser> and run cloud-init init
<smoser> again
<smoser> 2017-11-17 22:05:41,781 - DataSourceEc2.py[WARNING]: unexpected metadata 'network' key not valid: None
<blackboxsw> ok success
<blackboxsw> functional branch is at
<blackboxsw> https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+ref/fix-ec2-fallback-nic
<blackboxsw> needs tests
#cloud-init 2017-11-18
<smoser> blackboxsw: just maek sure that we're not goign to hit this re-crawl if we're not on ec2.
<smoser> where that network key is never going to get populated
<smoser> (i havent looked that dep yet)
<smoser> and also... i wonder why you:
<smoser>  if 'network' not in self.metadata.keys()
<smoser> why the .keys() ?
<blackboxsw> wasn't sure on py2.6
<blackboxsw> will check
<blackboxsw> didn't need the keys()
<blackboxsw> pulling it. ok will make sure we don't re-crawl on non-ec21
<blackboxsw> ok unit test in place pushing
<blackboxsw> https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/333927
<blackboxsw> one more upgrade test on this
<blackboxsw> ok test worked
