#cloud-init 2014-07-28
<smoser> mgagne, well, with bzr we can even have it magically build dailies on commit to a ppa.
<smoser> you probably can manage that with it polling a github, i dont know.
<harmw> smoser: https://code.launchpad.net/~harmw/cirros/udhcpc-respect-mtu that should do it
<harmw> thought I should dig up what we spoke about in regards of udhcpc and asking for options some months (!) ago
<harmw> can't remember if we settled with this or chose some other aproach
#cloud-init 2014-07-29
<harmw> omgwtfbbqlol
<harmw> libvirt_type = qemu...
<harmw> no wonder my cloud instances are dead slow...
<harmw> seriously, why didn't I figure that out like a year ago
<harmw> (ok, the hardware is slow but still)
<harmw> anyway, centos took a mere 40ish seconds, instead 5+ minutes to boot :p
<smoser> harmw, my plan was to ditch the static/silly CONFIG_IFUPDOWN and make that configurable.
<smoser> at "runtime".
<smoser> ie, just have CONFIG_IFUPDOWN_UDHCPC_CMD_OPTIONS as ""
<smoser> and then wrap udhcp or ifupdown or somethign with a shell that reads options from somewhere in the filesystem.
<harmw> smoser: so let udhcpc run in way the user wants it to run, according to some config file
<harmw> sounds like a plan
<smoser> harmw, i think i might even consider patching busybox to call a different udchpc
<smoser> ie, udhcpc.ifupdown
<smoser> or something. 
<smoser> then we can just provide that.
<smoser> might have to atch busybox to do that
<harmw> hmk
<harmw> isn't a wrapperscript a little more convenient instead of patching busybox?
<harmw> eg. working with a script+config file?
<smoser> well, id consider patching busybox to invoke /sbin/udchpc.ifupdown
<smoser> so that we'd know when *that*program was called it was from ifupdown
<smoser> and not get in the user's way if they ran it : udchpc -i eth0
<smoser> or something like that.
<harmw> ah like that
<harmw> I didnt venture into things that deep :p
<harmw> though I'm starting to like the concept
<harmw> I'm guessin' one hour of work
<harmw> patching busybox... thats just a configuration file thingy I'd guess, right?
<harmw> some of our own config in /etc/sysconfig/udhcpc.options
<smoser> harmw, there is some infrastructure for quilt applying patches in cirros now
<smoser> and i'd just do that to change the string that busybox invokes
<smoser> not bother to do it properly (ie, not do a config option)
<smoser> i think its networking/ifupdown.c
#cloud-init 2014-07-30
<harmw> ill have a look, thanks
<harmw> hm, annoying
<harmw> thre is a wrapper to figure what/how to start udhcpc, and then a default.script gets run to actually -do- stuff
<harmw> hm, it would be sexy if udhcpc could use the wrapper script itself as a default.script, to keep everything in 1 place (in case we want to add support for another option sometime in the future)
<harmw> uhm, that quilt thing isn't able to patch busybox - atleast it doesn't look like it
<harmw> smoser: https://code.launchpad.net/~harmw/cirros/udhcpc-wrapper 
<fifthecho> Has anyone used fpm to build cloud-init?
<fifthecho> Because I'm trying to build current cloud-init packages for older distros (Debian 6, Centos 5) to use on a CloudStack cloud and it's proving quite frustrating.
<smoser> fifthecho, no. i dont know fpm.  if you're interested in pursuing that, patches to make it work would be accepted.
<smoser> also, there is ./packages/brpm
<smoser> but maybe you knew that
<fifthecho> Or, is there some repo out there for cloud-init 0.7.5 for older distros?
<smoser> what is not functional for older distros? just deps ?
<harmw> fifthecho: is there a sane reason to start supporting centos 5 these days?
<harmw> given 7 is out
<smoser> harmw, so i would have just patched the string 'udhcpc' to invoke udhcpc.cirros instead of udhcpc.
<fifthecho> smoser: Older distros have older packages that don't have the CloudStack provider.
<smoser> as you have it i think it is a bit recursive
<fifthecho> smoser: Unfortunately we have customers who are stuck on old releases.
<smoser> fifthecho, right. but why does 0.7.5 not work on older distros. that was the question.
<fifthecho> smoser: It should, but there's no packages I can find and I'm having difficulties building RPMs and DEBs as there's no real instructions on how cloud-init builds their packages.
<smoser> harmw, ie, i was just oging to do this: http://paste.ubuntu.com/7904991/
<smoser> fifthecho, ah. so i'd start either with trunk, using ./packages/brpm
<smoser> or updating something in epel or something.
<harmw> well, we could've just move /sbin/udhcpc to somewhere else and replace it with the wrapper :p
<smoser> correct.
<smoser> but i didnt want to do that.
<smoser> because then /sbin/udchpc would be the wrapper
<harmw> btw your patch keeps feeding udhcpc.cirros with options, wich is not needed anymore either
<harmw> yea
<smoser> so then when a user invoked udhcpc "manually", they'd get our wrapper. and the wrapper wouldn't know if it was invoked via ifupdown or "upser"
<smoser> s/upser/user/
<harmw> uhu
<smoser> so yeah, if you want you can copy the stanza there
<smoser> we can set the options to ""
<harmw> well, if were going to patch busybox anyway I'd opt for adding my patch and keep it nice&tidy
<harmw> instead of ultimatly defacing udhcpc :p
<harmw> (in ifupdown.c that is)
<smoser> thats fine.
<smoser> but call udhcpc, not 'ifupdown'
<smoser> because calling 'ifupdown' is going to call udchpc
<smoser> which is going to call ifupdwon
<harmw> I dont think I'm following that one, my patch adds a call to /sbin/ifupdown for busybox to execute - that script in turn runs udhcpc, which is calling the same script yet again though with the renew|bound|whatever args
<smoser> harmw, you added a entry to ext_dhcp_clients which is part of FEATURE_IFUPDOWN_EXTERNAL_DHCPC
<smoser> i had assumed that what that means is that when the user types:
<smoser>  ifup
<smoser> and the network is ocnfigured for dhcp
<smoser> that 'ifup' would then call udhcp (in the current path)
<smoser> and instead, your patch is making it call
<smoser>  ifupdown up
<smoser> which i assumed was basically the same as calling ifup
<smoser> which would then run that code again ...
<harmw> ah like that, deadlocking
<smoser> right. thats how i read it. i could be wrong.
<smoser> how does it select an entry to run ?
<harmw> either way, there is no ifupdown binary currently - so that atleast shouldn't hurt
<harmw> it tries each entry
<smoser> ah. based on executable_exists
<harmw> indeed
<harmw> which is why I just prepended my code to the struct
<smoser> ah. ok. so now it makes more sense. i'd leave more of it intact though.
<smoser> but you were right in your assesement, that we could just replace /sbin/udhcpc (other than we'd be changing the behavior of that binary by wrapping it)
<smoser> harmw, thank you for humoring me.
<harmw> humoring? realy
<smoser> now i understand much more of your patch
<smoser> and it makes a fair amount of sense.
<smoser> other than i dont like the name "ifupdown"
<harmw> but...
<smoser> cirros-dhcp ?
<harmw> +c
<harmw> or: cirros-dhclient
<harmw> anyway
<harmw> Ill think of something
<smoser> yeah.
<harmw> you read the commitmsg btw?
<smoser> :)
<smoser> no.
<smoser> sorry.
<harmw> lol
<smoser> maybe that would have saved all this, eh?
<harmw> only one way to find out
<smoser> what about the '-h hostname' -i vendor -I client -l leastime stuff ?
<smoser> is that useful to us ?
<harmw> havent given that a thought just yet
<smoser> i'd think hostname probably is
<harmw> my focus was on putting something out there to aid in future enahncements
<harmw> to be able to add things like hostname, indeed
<smoser> and have no idea what: [[ -h %hostname%]]
<smoser> means
<harmw> (which is quite sane, for areas without clouds)
<harmw> *environments
<harmw> anyway, my commitmsg contains a note about beeing able to apply the patch to busybox :p
<harmw> which is why I asked if you read it, realy
<harmw> :)
<smoser> ah. harmw there are ways in buildroot to apply patches
<smoser> http://buildroot.uclibc.org/downloads/manual/manual.html#_providing_patches
<harmw> haha, not looking into the obvious :p
<harmw> oh no wait, my trouble was with busybox
<harmw> hm, lets see
<aneto> Hello, I am currently having trouble trying to get mounts to work in EC2 with cloud-init 0.7.4 installed from repl my config and logs are https://gist.github.com/anthony-neto/1c522c14ef286646cc9a I having being trying to use a bashton AMI that has a working cloud-init as an example but its a old version of cloud-init, I am going crazy
<aneto> If anyone in here is using cloud-init in ec2 with centos, and could gist my a mininmal config to test the mounts I would appreciate it =)
<smoser> aneto, so what is that ?
<smoser> you want to test mounts. how?
<aneto> that gist is what I have been trying, including the log from cloud init, I just want to see cloud-init mount the ephemeral mounts ec2 assigns.
<aneto> I have gotten userdata to work, but for the life of me cannot get the swap or ephemeral mounts to work
<aneto> I see this in the log, Attempting to determine the real name of ephemeral0 | Mapped metadata name ephemeral0 to /dev/sda2 | did not find entry for sda2 in /sys/block | Ignoring nonexistant default named mount ephemeral0
<aneto> smoser, I just want to get the default mounts working for ephemeral0 and swap
<smoser> well, built in "should" work.
<smoser> it should behave properly with no additional config.
<aneto> smoser, if cloud-init doesn't find an entry in /sys/block it ignores the mount?
<smoser> i think it does.
<smoser> maybe you can say "put it there nayway"
<smoser> well, no. it says:
<smoser>  http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config.txt
<smoser> line 185
<smoser> but i think for 'ephemeral0' it has to exist
<smoser> and swap too probably
<aneto> hmm has this always been the case? I installed an older version of cloud-init and got the mounts to work, however the /sys/block looks exactly the same as the problematic ami
<smoser> i dont know . i dont remember to have changed it.
#cloud-init 2014-07-31
<nvucinic> hi guys, anyone here? :)  have some questions about cloud-init 
<harmw> so, I've specified the EXTERNAL_DHCPC=y thingy in buildroot.config, but it still doesn't execute /sbin/ifupdown
<harmw> and calling /sbin/ifupdown directly complains about unknown mappings
<harmw> so prhaps I should take the easy route afterall, and just change udhcpc to /sbin/bla :p
<nvucinic> helo, i have instaled cloud-init on my centos machine, and i am stuck now. 
<nvucinic> it's some strange configuration with centos on hyperv 
<harmw> in which way are you stuck :)
<nvucinic> is there any chance i could add cloud-init config file over mounted disk  or url ?
<nvucinic> well i dont really know what to do now :)
<nvucinic> i want to specify configuration to run at machine boot 
<nvucinic> one time 
<nvucinic> i know what i want in cloud-config and how to define it 
<harmw> you have an amazone-like api machine setup?
<harmw> like openstack, cloudstack 
<nvucinic> i have default installation from yum packet manager on centos
<nvucinic> so, that would be - no
<harmw> ah
<harmw> cloud-init is used to configure cloud images
<harmw> and basically, that requirs a metadata service in your network
<harmw> - the web url you mentioned
<harmw> to download the config from
<harmw> usualy lives at http://169.254.169.254/
<nvucinic> ah 
<nvucinic> ok, is there any way to conigure metadata service only ? 
<nvucinic> i deploy instances in "cloud" manner 
<nvucinic> hmm... maybe i could get some layer in front of hyperv ?
<harmw> just make sure there is a metadata api accessible at that ip, and make sure your vm is able to reach it :)
<nvucinic> ok, but how to install // configure metadata api ?
<nvucinic> it's part of openstack right ?
<harmw> btw, cloud-init can also use configdrive - eg. put the json config on a usb stick (well, image) and add that to the vm
<harmw> yea
<harmw> or any other cloud platform
<harmw> depending on your environment, you could also script something yourself
<harmw> afaik, using configdrive would be sufficient for environments that lack a metadata service - like yours
<nvucinic> great
<harmw> hm, just don't know since whn cloud-init supports it...
<nvucinic> this was very helpful 
<paresh> hey my cloud-init process is not able to get user-data file on my aws setup
<harmw> why not :)
<paresh> 2014-02-12 16:16:13,268 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/i-xxxxxxx/user-data.txt - wb: [384] 0 bytes
<paresh> it says 0 bytes downloaded
<harmw> indeed it does
<nvucinic> do i need to setup something extra for configdrive?
<harmw> readthdocs.org :)
<harmw> about it format
<harmw> docs.openstack.org/user-guide/content/config-drive.html
<nvucinic> yeah, find it 
<harmw> great :)
<nvucinic> http://developer.openstack.org/api-ref.htmlext_config_drive.html not working  :|
<harmw> paresh: you could try curl on the user-data url from within the instance
<harmw> http://docs.openstack.org/user-guide/content/enable_config_drive.html
<harmw> nvucinic: I think you should be able to use that
<smoser> nvucinic, you can use the 'nocloud' datasource, and 'seed_from' to probably accomplish what you want.
<smoser> or you could configure on the disk the url to the ec2 mtadata service and then stand something like that up.
<smoser> nvucinic, http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config-datasources.txt
<smoser> that shows how to configure 'NoCloud' to give it a 'seedfrom'
<smoser> then just stand something up there with user-data nd meta-data as described in 
<smoser> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/files/head:/doc/examples/seed/
<smoser> you can also pass the seedfrom informatoin on the kernel cmdline if you had easier access to that.
<nvucinic> smoser: great, thank you
<smoser> harmw, what complains about unknown mappings ?
<smoser> your patch surely seemed sane to me.
<harmw> nah, forget it :)
<harmw> its almost done, Ill push the last bits tonight
<harmw> smoser: https://code.launchpad.net/~harmw/cirros/udhcpc-wrapper
<harmw> ive had to build this patchd into my updated buildroot env, since thats the one that builds on CentOS
<harmw> though it should compile without issues on ubuntu
<harmw> so: https://code.launchpad.net/~harmw/cirros/udhcpc-wrapper/+merge/229114
<smoser> harmw, seems odd you'd have to set that.
<smoser> CONFIG_FEATURE_IFUPDOWN_EXTERNAL_DHCP
<smoser> since i thought previously we were running ghrough that code
<smoser> in order to get udchpc invoked with the options
<harmw> there is a big if EXTERNAL_BLABLA around the part that decides which dhcp client to run
<harmw> it's N by default
<smoser> so how does udchpc run in 0.3.2 ?
<smoser> that seems "external" :)
<harmw> please check out static int FAST_FUNC dhcp_up(struct interface_defn_t *ifd, execfn *exec) in ifupdown.c
<harmw> it ends with:
<harmw> return execute("udhcpc " UDHCPC_CMD_OPTIONS " -p /var/run/udhcpc.%iface%.pid "
<harmw> and before that function is the stuff that concerns CONFIGURE_FEATURE_IFUPblablabla
<harmw> this all revolvs around L575 :)
<smoser> ah. ok.
<smoser> i believe you
<smoser> :)
<smoser> harmw, that looks really nice.
<smoser> actually..
<smoser> it looks to me like its busted for non-eth0
<smoser> as when busybox calls us, it will set that in the environment i presume
<smoser> but then you override it by reaading /etc/default/udchppc
<smoser> and also i don tlike the global 'DISABLED'
<harmw> you dont?
<harmw> and that interface... hm, I suppose I shouldve paid a little more attention to that
<harmw> ill just remove that from the config file :)
<harmw> and disabled as well, if you dont like it
<harmw> ah, so
<harmw> commnting between watching netflix isn't very wise either :p
<harmw> smoser: udhcpc uses eth0 by default
<harmw> so using a config file to make it changable for whatever reason shouldn't be a problem
<harmw> so, we can configure on which interface we want it to acquire an address
<harmw> and udhdpc will in turn set $interface with the interface it got an adress from/on
<harmw> http://git.busybox.net/busybox/tree/networking/udhcp/README.udhcpc?h=1_1_stable&id=179f41778880d0a233435f5c9ed04e17316f479a#n73 
<harmw> whats wrong with DISABLED, or is there some other easier way of saying 'no, please move on and leave me without an address!'?
<smoser> harmw, hm..
<smoser> i was thinking when someone types:
<smoser>  ifup eth0
<smoser> and /etc/networking/interfaces says to bring eth0 up as dhcp
<smoser> then 'ifup' invokes cirros-dhcpc
<smoser> as:
<smoser>   cirros-dhcpc up
<smoser> i assumed it somehow conveyed which interface should be brought up
<smoser> harmw, yeah, so our command there (line 33 of https://code.launchpad.net/~harmw/cirros/udhcpc-wrapper/+merge/229114)
<smoser> needs to :
<smoser>  cirros-dhcpc up %iface%
<smoser> at least. and can't see why not put in some othe other things avialalbe (like leasetime).
<smoser> then, when cirros-dhcpc calls udhcpc it will pass on the provided %iface.
<smoser> i suspect then that udhcpc then stuffs that interface into the environment
<smoser> and we'll be able to expect it in environment on renew|bound
#cloud-init 2014-08-01
<msitarz> Hi
<msitarz> could somebody tell me how does cloud-init decide which network interface to configure?
<harmw> smoser: ifup $interface
<harmw> damn
<harmw> ok, Ill make some adjustments
<harmw> udhcpc will put the interface where it's gotten an address on in the environment, yes, as $interface
<harmw> and disabling would be done in /etc/networking/interfaces, so no need for a DISABLED (which Ill remove)
<msitarz> harmw: is that related to my question ?
<harmw> haha no, sorry :)
<harmw> I should've bzr copied the default.script btw, since now it looks like cirros-dhcpc is completely new - which it obviously isn't
<harmw> anyway, smoser , the merge request is updated
<smoser> msitarz, it doesn't. 
<smoser> system configuration tells it.
<smoser> on ubuntu, it blocks on anything 'auto' in /etc/network/interfaces.
<msitarz> smoser: so cloud-init does not setup the network
<msitarz> but the default OS configuration is just setup to use DHCP , that's why I get IP on eth0 ?
<smoser> right.
<smoser> thats how i would think things should work
<smoser> eth0 should be expected to come up on dhcp and give route to network
<smoser> now...
<smoser> that is being fixedui do want to fix that htough
<smoser> so that ocnfig drive can correctly insert networking
<harlowja_at_home> smoser, btw, i'm in NY for a couple week, so will be on/off (forgot if i told u :-P)
<smoser> harlowja_at_home, sure. just make sure you vote for my talk. :)
<harlowja_at_home> ??
<harlowja_at_home> where's that
<smoser> http://www.openstack.org/vote-paris/Presentation/ecure-containers-in-openstack-using-lxc-and-user-namespaces
<smoser> and while you're there, you should vote for this one too
<smoser> https://www.openstack.org/vote-paris/Presentation/astrologer
<harlowja_at_home> cool
<harlowja_at_home> will do, will get some others to vote for that also :-P
<smoser> others by canonical at http://insights.ubuntu.com/2014/07/31/voting-begins-for-openstack-summit-sessions-in-paris/
<smoser> look at that second link :)
<smoser> if you have links to yours, i'll + them to
<harlowja_at_home> "highly effective astrological love problem solutions are good enough to solve all your problems - See more at: https://www.openstack.org/vote-paris/Presentation/astrologer#sthash.Gu4vFXan.dpuf"?
<harlowja_at_home> what the
<harlowja_at_home> i didn't put any up this time :-P
<harlowja_at_home> smoser, astrogger even put it under the networking category +1, ha
<harlowja_at_home> different type of networking, ha
<harmw> smoser: ready to accept my merge request? :P
<smoser> let me look. i think we were closer.
<smoser> so the patch to busybox
<smoser> you patch in a patch. is that right ?
<smoser> is thats what happening there?
<smoser> where does leasetime come from, do you know ?
<smoser> ie, is it of any use for us to get that passed in from ifup ?
<smoser> i dont know where it would get it from
<smoser> harmw, do you mind if i bikeshed on some things?
<smoser> i dont want to put you off, and i thikn its probably good enough.
<smoser> but a few things. 
<smoser> here. i'll just make some changes here
<smoser> and then propose for merge into your branch
<smoser> then you sanity check
<harmw> leasetime comes from the dhcpservice, we don't do anything with it
<smoser> hm..
<smoser> i'd have thoguht so
<smoser> but why does ifup have '%leastime%' ?
<smoser> see line 37 of your MP.
<harmw> hm, it does?
<smoser> see how it can call dhcpcd with '-l %leasetime'
<smoser> it seems odd
<smoser> but i didn't understand. so asking you
<smoser> :)
<harmw> hmk, well, not all clients are using the same options though
<harmw> in ifupdown.c 
<harmw> and yes, its a patch that creates a new (patch) file that will add the new dhcp client directive to that struct in ifupdown.c
<harmw> whats so wrong with my stuff for you to b wanting to bikeshed? :p
<smoser> give me 10 minutes. i'll push up a branch.
<harmw> meh
<smoser> just making stuff more localized and such.
<smoser> i'll show
<smoser> nothing big
<smoser> you really did good.
<smoser> thank you!
<harmw> yea right, well obviously not good enough 
<harmw> :p
<smoser> harmw, what is -x
<harmw> as in the udhcpc parameter ?
<smoser> yeah
<harmw> asking for options
<harmw> no
<harmw> 'include option OPT in set packets'
<smoser> ok. so if hostname is empty
<smoser> then we dont want the -x either
<smoser> right ?
<harmw> damn, well... isn't hostname defaulting to cirros?
<smoser> probably. but just in case.
<harmw> god I hate you for that :p
<smoser> i told you bike shed
<harmw> you're probably right
<smoser> i've got it thoguh
<harmw> probably something like: [ -z `hostname` ] && OPT=bla
<harmw> you should merge the updated buildroot as well, someday
<smoser> yes.
<smoser> i will merge that for sure.
<harmw> but lets finish this udhcpc branch first, then you're all set on the buildroot, and then its time to drink beers and enjoy the weekend :>
<smoser> harmw, so every time something ifup dhcps
<smoser> it will kill resolv.conf
<smoser> ie, last one wins
<harmw> but thats not any different from the current situation, right?
<smoser> i dont know.
<smoser> problby not
<smoser> just found it interesting
<smoser> and odd
<smoser> i'm gonna make it only do that if
<smoser> domain or dns is set.
<harmw> that part was already in default.script, which is called at ifup
<smoser> alright.
<smoser> i'll lave it as it is
<harmw> probably wouln't hurt, no
<smoser> we could potentially make that work
<smoser> make it configurable
<smoser> we could
<harmw> i'd opt for first bringing in the major changes, and then perhaps clean this little issue :)
<harmw> *fix
<harmw> grrr, why doesn't centos7 ship with cloud-init
<harmw> I thought RH had added that to RHEL
<harmw> and while the releasenotes of centos7 mention there are cloud images underway, they have yet to arrive
<smoser> harmw, lp:~smoser/cirros/udhcpc-wrapper
<smoser> i'll be back in a bit
<harmw> ok, ill have a look at merging that with my branch
<harmw> done, pushed
<harmw> ok, I'm building a new image now
<harmw> hm, I dont see any reason for it to fail on the current buildroot - but I'm using my 2014 buildroot to testdrive this thing... (since that builds on centos)
<harmw> ill check back in an hour :)
<harmw> and another hour :p
<harmw> so, busybox 1.20.1 in buildroot 2014 :| damn
<smoser> yeah. thats kindof annoying.
<smoser> lets get to a new buildroot
<harmw> haha cool
<harmw> make it happen
<harmw> :)
<smoser> i dont have an easy answer for that patch having to have the busybox verison in it
<smoser> (that is annoying)
<harmw> I had the exact same objection to it
<harmw> but went for it nonetheless
<harmw> anyway, ill be back in 60 minutes
<smoser> k
<smoser> so lets shoot to merge to your 2014.02
<smoser> and then merge to 2014.5
<harmw> (and lets not forget udhcpc)
<smoser> right.
<smoser> holy moley. theres a bunch there.
<smoser> harmw, ok. i hope id ont break anything. but heres my plan.
<smoser>  a.) take the udhcp sutff without the patch disabled in the series file
<smoser> so that it will still build fine, but will be inert
<smoser> b.) get us merged to a newer version and address that later.
<smoser> harmw, ok. pulled iwthout patch.
<harmw> uhm, then you shouldn't have deleted default.script right?
<harmw> since now we're not applying the patch to bb (to make it default to using cirros-dhcpc) it'll start udhcpc just like it always did
<harmw> and that will be done with -s /usr/share/udhcpc/default.script
<harmw> which doesn't exists
<harmw> so please revert that file untill we will use the patch
<harmw> thanks for merging though :)
<harmw> be sure to update the chngelog btw
#cloud-init 2014-08-02
<smoser> yeah, you're right. :)
<smoser> thanks harmw . fixed and pushed in 3.2.
<smoser> revno 312.
<harmw> smoser: lol, nice
<koaps> hello
#cloud-init 2014-08-03
<koaps> does anyone know why my VM's hang on boot for url_helper.py looking for metadata?
<harmw> koaps: sounds like they cannot find any metadata api/url, in which case not routing to 169.254.169.254 is your problem
<koaps> so look like there was some issues with the compute bridge IP, now I'm having issues with dnsmasq
<koaps> it's set to listen on a IP that doesn't exist or not assigned to the bridge
<harmw> its bound to a different namespace
<harmw> ip netns
<harmw> will show you
<koaps> think I got it, killed off the dnsmasq and restarted my nova stuff, it started on the right interface, hopefully it will stay there
#cloud-init 2015-07-27
<cbolt> Odd_Bloke: thanks!
<cbolt> is it possible to have cloud-init default to mounting all ephemeral drives?
<cbolt> ec2 c3.large has 2 ephemeral drives, cloud-init sees both from the metadata but only mounts 1
<cbolt> when using nocloud is it possible to specify what device is ephemeral?
<smoser> cbolt, can't easily telll nocloud which is ephemeral...
<smoser> although if it ahs a filesystem with 'ephemeral' as the filesystme label it might work
#cloud-init 2015-07-28
<minfrin> Quick question - is it possible to run an additional cloud-init script at a later point during or after boot? The reason I ask is that Azure supports cloud-init, but has severe restrictions on template sizes that make cloud init impractical to use. Azure does support a thing called a "CustomScriptForLinux" that allows us to run a script downloaded from somewhere else, and what I want that...
<minfrin> ...script to be is an additional cloud-init script. Is it possible to run additional cloud-init scripts after the initial one is run via some kind of command line option?
<smoser> minfrin, you're limited by size i gather ?
<smoser> the CustomData on azure can be gzip compressed inside there. that will buy you some.
<smoser> also, you can use '#include' style to consume data from elsewhere.
<smoser> see https://help.ubuntu.com/community/CloudInit
<smoser> then your user-data can effectively be anything you can fit inside the content of a url
<minfrin> Gzip is no good for us unfortunately, the thing that is blowing our size limit are the private keys for the machines which aren't compressible in any meaningful way that makes a difference to us.
<minfrin> How do you secure data coming from an URL?
<smoser> minfrin, as in put secret data there? or as in secure against MIM
<smoser> it can read over https so that secures transmission
<smoser> for secret data you can use '#include-once' with a url to a long one time hash
<smoser> but if you want to run stuff after boot, you can
<smoser> cloud-init single --frequency=always --name=<that-module-name>
<minfrin> I tried out cloud-init single, but it forces us to choose a --name option, and I've interpreted that as meaning we are only able to run one single module in our additional cloud-init script. What I need to do is run all modules, is this possible?
<minfrin> We're trying to find a solution that doesn't involve too many hacks given the number of hacks we've had to apply to work around Azure's size limits. :(
<cbolt> smoser: is there a way to make cloud-init reformat a drive on every boot?
<cbolt> currently getting an error stating the filesystem is already mounted
<smoser> minfrin, you can run the stage again 'cloud-init init' or 'cloud-init config'
<cbolt> first boot, it takes the unformatted drive and formats it properly. when i reboot the box it attempts to reformat it but mkfs.ext4 fails stating the drive is already mounted
<smoser> cbolt, probably want/need to take it out of /etc/fstab . at least that could work.
<smoser> you can probably configure 'mounts' to not automatically mount it.
<smoser> minfrin, do you have sensitive data in the user-data ?
<smoser> is that why you're adverse to urls ?
<smoser> you could put the sensitive data (assuming it fits) into the body of the CustomData and #include large code hunks
<cbolt> ok thanks, ill come up with a workaround for this
<smoser> cbolt, http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config.txt#L174
<smoser> i think if you feed it user-data that says 'mounts: [ephemeral0, null]'
<smoser> then it wont getm ounted. and that shoudl fix your reboot issue
<cbolt> yea but then ill need to come up with a solution for mounting it
<smoser> oh. yeah. ok.
<cbolt> really i shouldnt be having to force cloud-init to reformat the drive on every reboot
<minfrin> Yes on the sensitive data - it's all private keys for the machines.
<minfrin> If I was to add an additional cloud-config script, what command would I run? In other words: [additional-cloud-init-command] cloud-init-with-keys
<cbolt> i need to fix the underlying issue
<smoser> minfrin, what is the size limit ?
<smoser> just really curious. i didnt' remember there was one. (initially there wasnt, thats how we implemented custom-data there without azure's knowledge :)
<minfrin> To explain the background, Azure supports customData of up to 64kb in size. Their templates are however constrained to approx 320k in size before failing on deployment. We have 21 machines to deploy as a unit (and climbing), and a customData script of anything bigger than 7k causes the deployment to crash.
<smoser> hm.
<smoser> have you raised that to them ?
<smoser> for "custom cloud-config script" above, what did you mean ?
<smoser> you want to add a config module and run it ?
<minfrin> Yes. Took them a while to figure out the cause was a size limit, we were getting "internal server error" for a while which told us nothing of what was wrong.
<smoser> the 64kb sucks as a limit.
<smoser> on ec2 its 64k
<smoser> but its binary
<smoser> is theirs 64kb of mime-encoded (so it can be shoved safely into xml body) ?
<minfrin> We want to run cloud-init twice - once on boot using their normal process, and a second time at some point after boot using their "CustomScriptForLinux" feature, which while being URL based can be secured. "CustomScriptForLinux" allows us to specify a file to download (which is the cloud-init script in theory) and a command to run (which is in theory "cloud-init [options]").
<smoser> ok. i think maybe i udnerstand.
<smoser> if you can run stuff, then
<smoser> let azure boot,
<smoser> run command taht does: rm -Rf /var/lib/cloud/ /etc/cloud/cloud.cfg.d/*azure*
<smoser> then populates /var/lib/cloud/instance/nocloud/seed/
<smoser> or... possibly simpler.
<smoser> let azure boot
<smoser> hm..
<smoser> well, ^ above shoudl work.
<minfrin> We're getting nowhere near the 64kb size limit, the limit we're hitting is the total size of their deployment template. In theory we're told it is 1MB - itself very small, but we're then told that a UTF8 to UTF16 halves the effective size of the template to 512k, and there are further unspecified limits we hit, bringing the size where we hit the wall to about 320k-ish.
<smoser> ah.ok.
<smoser> what is the user-data (what format) that you'd normally send to cloud-init ?
<smoser> is it cloud-config ?
<minfrin> Normally #cloud-init. It was all working great up to the point where we added the private key.
<minfrin> Sorry, #cloud-config.
<smoser> ok. if its all cloud-config i think you should be able to:
<smoser>  * let azure boot as it would
<smoser>  * run some program that does:
<smoser>    * rm -Rf /var/lib/cloud/instance/
<smoser>   * put your #cloud-config data into /etc/cloud/cloud.cfg.d/your-data.cfg
<smoser>  * reboot or run the stages of cloud-init manually
<minfrin> Let me give this a play and see what I am limited to - I still need to see exactly how the CustomScriptForLinux works and what restrictions are imposed on us. Thank you for confirming the cloud-init steps, I appreciate it.
<smoser> yeah, i dont know how that works at all
<smoser> Odd_Bloke, around ?
<Odd_Bloke> smoser: In and out.
<smoser> http://paste.ubuntu.com/11954733/
<smoser> does that make sense on top of https://review.openstack.org/203533
<openstackgerrit> Merged stackforge/cloud-init: Implement a DictRegistry.  https://review.openstack.org/203094
<openstackgerrit> Merged stackforge/cloud-init: Use a registry to configure reporting handlers.  https://review.openstack.org/203093
<openstackgerrit> Merged stackforge/cloud-init: Make reporting handlers configurable.  https://review.openstack.org/203533
<Odd_Bloke> smoser: Without context, it looks like a pointless change...
<smoser> ah. acutally never mind. you're right
<smoser> i didnt'realize you were using event. in the getLogger call
<smoser> so its not pointless its broken
<Odd_Bloke> Oh, also true. :p
<Odd_Bloke> Great review from me there. >.<
<smoser> i was trying to avoid getLogger and join on every publishEvent
<Odd_Bloke> Ah, I see.
<smoser> Odd_Bloke, http://paste.ubuntu.com/11955467/
<smoser> any reason you use unittest and cloudinit.tests.TestCase ?
<Odd_Bloke> smoser: Force of habit?  (No. :p)
<smoser> thats fine. was just looking at cloud-init 0.7 for this
 * harlowja prefers 'cloudinit.tests.TestCase'
<harlowja> makes it easier to add new base functionality if needed
<harlowja> and/or compat stuff
<openstackgerrit> Scott Moser proposed stackforge/cloud-init: tests: use cloudinit.tests.TestCase everywhere  https://review.openstack.org/206688
<smoser> harlowja, ^
<harlowja> cool
<smoser> hey, you want to take over my "main" ?
<harlowja> hmmmmmmmmm
<smoser> https://review.openstack.org/#/c/202743/
<harlowja> one does not take over main, lol
<harlowja> one does not just takeover main, lol
<openstackgerrit> Scott Moser proposed stackforge/cloud-init: add cloud-init main  https://review.openstack.org/202743
<smoser> why would flake8 blow up
<smoser> not like my ordering ?
<harlowja> smoser http://docs.openstack.org/developer/hacking/#imports
<harlowja> 'Do not import objects, only modules'
<harlowja> might complain, might not
<harlowja> maybe we turned that check off, not sure
<smoser> oh.
<smoser> well, i couldnt get H306 to fail
<smoser> harlowja, http://paste.ubuntu.com/11955830/
<smoser> thoughts ?
<smoser> Odd_Bloke, ^
<smoser> i dont want to pas a handle around everywhere
<Odd_Bloke> smoser: We could do a stack, but that's pretty brittle.
<harlowja> hmmm
<Odd_Bloke> smoser: I don't think there's going to be a good way to do this that won't (a) break unexpectedly, or (b) fail to expand to the next use case we think of.
<harlowja> probably better to use a stack, put it in a thread-local variaible, then have most of this reporting be decorators and not explict calls imho, at least hide the calls to start/end all over
<harlowja> but this is the cost of that level of tracing :-P
<harlowja> *event emission*
<Odd_Bloke> smoser: harlowja: Immediate example of a stack failing: parallel data-source discovery.
<harlowja> yup
<harlowja> thread-local stacks
<Odd_Bloke> harlowja: If that's how we implement parallel discovery, maybe.
<harlowja> depends how u want to do it, without all the async stuff, paralleism probably just easier via spawning threads imho
<Odd_Bloke> But I don't know that I want event reporting to dictate how we parallelise the code which does the actual work. :p
<harlowja> agreed
<Odd_Bloke> Also, given that we expect the output of this to be computer-readable, we need it to be super-hard to break.
<Odd_Bloke> Humans can cope with the occasional weird log message; computers, not so much.
<harlowja> ya, got me, if u want a stack, u probably need a stack structure somewhere :-P
<harlowja> *stack like output
<harlowja> or u have to use the call-stack as the stack, not many other options, ha
<Odd_Bloke> This also feels like we might actually only use it in one or two places, and it might be easier for those places to manage it.
<Odd_Bloke> (e.g. all of those instances will _know_ that they are doing ds-search, so can just use that as their event name)
<Odd_Bloke> s/instances/examples of calls/
<harlowja> sure
<Odd_Bloke> So I also wonder if this is a YAGNI situation.
<harlowja> possibly, lol
<harlowja> https://github.com/jek/blinker doesn't seem to support stacks either
<harlowja> so my guess maybe we overthinking it
<harlowja> either does https://github.com/openstack/taskflow/blob/master/taskflow/types/notifier.py (my event  like thing)
#cloud-init 2015-07-29
<smoser> parllel data source discovery is not example of failurel there
<smoser> as each would just append but never pop to stack
<smoser> harlowja, Odd_Bloke thanks for thoughts.
<harlowja> sureeee
<openstackgerrit> Merged stackforge/cloud-init: tests: use cloudinit.tests.TestCase everywhere  https://review.openstack.org/206688
<harlowja_still_a> smoser,  i'm getting 'On 2015-07-31, 7 days from now, your membership
<harlowja_still_a> in the cloud init development team (cloud-init-dev) Launchpad team' ohhhh my goodddddddd
<harlowja_still_a> oops
<harlowja_still_a> 'On 2015-07-31, 7 days from now, your membership
<harlowja_still_a> in the cloud init development team (cloud-init-dev) Launchpad team
<harlowja_still_a> is due to expire.'
<harlowja_still_a> that one, ha
<harlowja_still_a> i don't wanna die
<harlowja_still_a> keeps on sending that daily, lol
<Odd_Bloke> smoser: Each DS would push on to the stack; you'd end up with .../ec2: started, .../ec2/openstack: started, etc.
<smoser> Odd_Bloke, right.
<Odd_Bloke> So that would be a problem, no?
<smoser> claudiupopa, Odd_Bloke harlowja
<smoser> lets do cloud-inti meeting here.
<smoser> roadmap is https://trello.com/b/HoPNdiTI/cloud-init-development-roadmap
<claudiupopa> hey.
<claudiupopa> Isn't the reporting merged?
<claudiupopa> I saw a couple of patches getting it last night.
<smoser> reviews can be seen at http://bit.ly/cloudinit-reviews-public
<smoser> reporting is in-progress.
<smoser> the reporting infrastructure is there with a logging backend
<smoser> but we'll still need a webhook
<smoser> i started looking at getting that back to cloud-init 0.7 yesterday as I actually need it there RSN
<Odd_Bloke> I've moved the framework to done, and added a new card for the webhook.
<smoser> Odd_Bloke, thanks
<Odd_Bloke> smoser: I feel like we also talked about another thing that needed doing there last week, but I can't remember what it was.
<smoser> my main remains in stub form https://review.openstack.org/#/c/202743/
<smoser> today i have to focus the 0.7 reporting stuff
<smoser> claudiupopa, if you want to take a look and thoughts on main, i'm happy to have input.
<claudiupopa> looking right now.
<smoser> there s still a lot of infrasturcture thre that we need
<smoser>  * tie into reporting
<smoser>  * actually implement the search of datasources
<smoser> claudiupopa, that was the one thing i went looking for you on.
<smoser> didnt' really know how those should be loaded.
<smoser> didnt take a huge look, but you werent in
<claudiupopa> right, I could write a patch for this.
<claudiupopa> basically getting all the possible data sources and calling .load on them.
<claudiupopa> If it returns, that data source is good to go.
<smoser> right. that'd be good.
<smoser> we can do serially loaded now
<claudiupopa> Probably with some filters, since you will probably want only the net datasources.
<claudiupopa> The problem is how to discover them.
<claudiupopa> I mean, where to look for the possible data sources cloud-init has.
<smoser> ok. so lets do 2 paths
<smoser>  a.) directory relative to __file__
<smoser>  b.) osys "plugins" path
<smoser>  under osys.plugin_path() we'd have a 'sources' dir.
<smoser> we would look i guess in 'b' first and then in 'a' ?
<claudiupopa> osys.plugin_path() for returning the path of the data sources or the plugins (config modules)?
<claudiupopa> Since the name seems to suggest the latter.
<Odd_Bloke> What about using namespace packages?
<Odd_Bloke> Let me rephrase that.
<Odd_Bloke> (As everyone Googles namespace packages)
<Odd_Bloke> :p
<openstackgerrit> Scott Moser proposed stackforge/cloud-init: add cloud-init main  https://review.openstack.org/202743
<Odd_Bloke> If we are using a plugin mechanism, I don't think we should have data sources be importable via normal means at all.
<smoser> claudiupopa, i was saying the sources woudl be loaded from osys.plugin_path() + os.path.sep + "sources"
<smoser> in one way or another.
<Odd_Bloke> Or we need to ensure that they are _all_ importable via normal means; even those that we don't distribute with cloud-init itself.
<claudiupopa> Odd_Bloke: makes sense.
<Odd_Bloke> Namespace packages would be one way of doing the latter.
<Odd_Bloke> Though they might be PITA on Python 2.6.
<smoser> Odd_Bloke, for the very clueless (smoser) explain a bit more ?
<Odd_Bloke> So namespace packages basically let different Python "packages" distribute modules within a particular namespace.
<Odd_Bloke> So 'pip install cloud-init' could install cloudinit.sources.foo, and 'pip install cloud-init-obscure-cloud' could install cloudinit.sources.obscure_cloud.
<Odd_Bloke> Presumably downstream packagers have tooling to mirror this.
<Odd_Bloke> (Certainly we do, I think oslo is distributed this way)
<claudiupopa> Odd_Bloke: how does the discovery works in that case? Is there an api for retrieving the packages from a given root point?
<claudiupopa> Not very familiar with them either ;-)
<Odd_Bloke> We would still need a discovery mechanism, I think.
<smoser> :)
<smoser> thanks for asking for  me, claudiupopa . i'm aware you're just trying to make me feel like less of an idiot
<Odd_Bloke> Possibly our best bet here is to describe what we want from a plugin-type system, and let claudiupopa go away and work something out.
<Odd_Bloke> Then we can tell him he's wrong next week. ;)
<claudiupopa> ;-)
<Odd_Bloke> But I feel like if we're diving in to the paths that plugins will be loaded from, we may be dismissing other solutions.
<smoser> ok. claudiupopa you want to think about that a bit ? you can add a card there if you want.
<smoser> Odd_Bloke, how so ?
<smoser> Odd_Bloke, i very much want to be able to document "if you put sources in this static directory path, then cloud-init will find them"
<smoser> and the same for config modules
<smoser> and part handlers
<smoser> and such
<Odd_Bloke> Yeah, absolutely.
<claudiupopa> yeah, I'll look into this, since it's something that I should do for cloudbase-init as well.
<smoser> i dont care necessarily how we get there, but that is something i want.
<Odd_Bloke> But, for example, do we mean static, or do we mean static-relative-to-the-cloudinit-python-package?
<Odd_Bloke> And if the former, where in the FHS should that live.
<Odd_Bloke> etc.
<Odd_Bloke> I'm on another call now, so I won't reply for a while.
<Odd_Bloke> But will catch up when I'm off.
<smoser> Odd_Bloke, we mean both.
<smoser> :)
<smoser> as in ubuntu packages should be able to install sources alongside of cloudinit.<stuff>
<smoser> and vendors or developers shoudl be able to put them in /var/lib/cloudinit/sources
<claudiupopa> smoser: left a couple of comments on the main executable thing.
<smoser> claudiupopa, thank you!
<claudiupopa> yeah, the last comment does not make sense.
<claudiupopa> On the review I mean.
<smoser> it confused me
<smoser> You mean val.get('opts', {})? Otherwise the unpacking will fail in the case opts does not exists.
<smoser> isn't that what i have ?
<smoser> (i was typing that there)
<claudiupopa> Yeah, I didn't notice that opts is already a tuple and for a, b in {} works, since it's empty.
<smoser> ok. i see. does *not* make sense.
<smoser> i completely ignored the word 'not' above
<Odd_Bloke> smoser: I haven't used it, but do you think it's worth having the meeting bot for these weekly meetings?
<smoser> i'm not opposed to it.
<smoser> i went asking for ubuntu logger bot the other day
<smoser> *really* want that
<smoser> i dont really care about the meeting bot so much
<smoser> but want logging and "bug links"
<Odd_Bloke> Better record an action with the meeting bot.
<Odd_Bloke> ;)
<harlowja> sup
<harlowja> u guys want to have meetings?
<harlowja> like via http://eavesdrop.openstack.org/ ?
<harlowja> 'Meeting schedule' there?
<smatzek> I would be in favor of scheduled meetings to discuss design, direction, issues, etc.  I actually have to drop now so I can't chat more, just putting my 2 cents in.
<smoser> harlowja, smatzek well, we kind of have a 10:00 US/Eastern meeting on Wednesdays
<smoser> and you're both invited.
<harlowja> ah
<harlowja> damn thats like super-early
<harlowja> lol
 * smoser thinks that smaztek is less lazy than harlow and might be up then
<harlowja> ha
<smoser> yeah, 10:00 sure is early
<harlowja> 7am
<harlowja> (my time)
<smoser> relocate to where the sun shines in proper hours
<harlowja> lol
<smoser> hey, you can probably help me
<harlowja> well i was gonna move into your basement
<smoser> looking at my main thingy
<harlowja> butttt u already occuping your basement
<smoser> https://review.openstack.org/#/c/202743/5
<harlowja> u have killed it
<harlowja> lol
<harlowja> *killed jenkins, lol
<smoser> i addressed / was addressing cladiu's request to not pass function names in but actually callables
<harlowja> k
<smoser> but then my test started to fail because mock mocks "too late" I think
<smoser> what is there + http://paste.ubuntu.com/11961057/
<smoser> shows the issue. mock doesnt realize that main_version got called
<smoser> but it very clearly does
<smoser>  http://paste.ubuntu.com/11961063/
<harlowja> hmm
<harlowja> ya, its probably all those functions getting captured into SUBCOMMANDS
<smoser> right.
<smoser> SUBCOMMANDS gets a reference
<smoser> and mock doesnt patch that ref.
<harlowja> right
<harlowja> if its get_subcommands() that returns a dictionary, i wonder if it would work then
<harlowja> late binding them
<harlowja> *could even return the same dictionary, but python binding i think will then happen when the dictionary is created, and therefore mock can affect things
<smoser> :)
<smoser> i think i'm gonna just drop the test as it doesn't actualy do anything that wasn't done other place.
<harlowja> or that
<smoser> as i was testing hat main_version has expected output :)
<harlowja> ya, those kind of tests i always wonder about
<harlowja> aka, sorta feel they are useless lol
<openstackgerrit> Scott Moser proposed stackforge/cloud-init: add cloud-init main  https://review.openstack.org/202743
#cloud-init 2015-07-30
<harlowja> smoser hey, i'm gonna ask the infra folks to remerge over the 0.7.x branch from bzr -> stackforge
<harlowja> https://github.com/harlowja/cloud-init/tree/0.7.x is the updated version (with the latest changes from launchpad)
<harlowja> okie dokie with u?
<harlowja> basically just replaces https://github.com/stackforge/cloud-init/tree/0.7.x
<smoser> harlowja, thats fine. how can we keep that indefinitely working ?
<harlowja> smoser hehe, good question, monty was/is like 'i'll do that once more, but no more syncs'
<harlowja> :-/
<harlowja> we could just accept things only new 0.7.x via stackforge :-/
<smoser> hm.. well hm..
<smoser> i'd not be terribly opposed to that i guess.
<smoser> would using git on laucnhpad help any?
<smoser> then it'd be straight mirror.
<smoser> i got to run..
<smoser> the sun has gone down where i live.
<harlowja> smoser oh, maybe, that'd be cool
<harlowja> is it possible to have git on launchpad?
 * harlowja really just wants git to make some other y! folks lives easier, ha
<harlowja> if its not 0.7.x on stackforge, thats cool
<smoser> http://blog.launchpad.net/general/git-code-hosting-beta
<smoser> later dude
<harlowja> oh nice
<harlowja> yes, that'd be best, then can just drop 0.7.x on stackforge
<harlowja> and still have git on the launchpad 'legacy' stuff
<harlowja> *and leave stackforge just for all the new hotness
<Odd_Bloke> smoser: Can you remind me how I can check if cloud-init has finished running?
<Odd_Bloke> smoser: (By looking at the filesystem)
<smoser>  /run/cloud-init/*.json
<smoser> Odd_Bloke, ^
<Odd_Bloke> smoser: Cheers.
<Odd_Bloke> smoser: I think https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1440263 was mistakenly marked as fixed in the latest cloud-init release.
<Odd_Bloke> (Because I mistakenly told you it was, so you marked it with --fixes on a merge)
<smoser> hm.
<smoser> well then. we need to fix that.
<harlowja> smoser hey, so enabling 	http://blog.launchpad.net/general/git-code-hosting-beta for cloud-init (legacy) would be suppperrrr
<harlowja> if that could be done, it'd save some pain for me (and/or infra)
<harlowja> cause the yahoo build systems really only understand git (and not bzr)
<smoser> Odd_Bloke, what do you think ?
 * smoser is mostly just joking here as he knows what Odd_Bloke thinks
<harlowja> i don't think
<harlowja> lol
<harlowja> so i never have that problem :-P
<Odd_Bloke> YES YES A THOUSAND TIMES YES
 * Odd_Bloke --> pub
<harlowja> smoser do u know if anything special is needed to turn on http://blog.launchpad.net/general/git-code-hosting-beta ?
<smoser> i dont think anything special.
<smoser> moving 0.7.x trunk off of bzr is not offensive to me.
<smoser> but dont knwo what to do with the bzr packaging branches
<harlowja> cause i trieeddd
<harlowja> Warning: Permanently added 'git.launchpad.net,162.213.33.96' (RSA) to the list of known hosts.
<harlowja> fatal: remote error: Repository 'cloud-init' not found.
<harlowja> :-/
<harlowja>  git clone git+ssh://git.launchpad.net/cloud-init
<harlowja> *but maybe not doing it right or something
<smoser> oh. you probably hve to push some button.
<smoser> andyou'd probably have to push to it before you could clone it :)
<harlowja> smoser can u push the button
<harlowja> i am blind-poor-soul that does not know where the button is
<smoser> k
<smoser> i'll look
<harlowja> thx for helping poor-blind-sould
<harlowja> *soul
 * harlowja is so blind he can't even type
<smoser> harlowja, so in theory, you have the 0.7.x branch imported into stackforgae
<smoser> right ?
<harlowja> right, but its not the same as whats on launchpad unless https://github.com/harlowja/cloud-init/tree/0.7.x resyncs (i just recreated that last night)
<harlowja> ^ https://github.com/harlowja/cloud-init/tree/0.7.x should have all the recent launchpad stuff up to yesterday
<smoser> right. ok. and that is 116 commits ahea dof tackforge:0.7.x
<harlowja> ya, something like that
<smoser> how did you do the sync ?
<harlowja> smoser basically http://paste.openstack.org/show/406483/
<harlowja> +- that
<smoser> harlowja, http://paste.ubuntu.com/11968557/
<smoser> http://paste.ubuntu.com/11968562/
<smoser> your git branch has much less tags.
<harlowja> ya
<harlowja> *git repo, not branch
<harlowja> i don't think the infra people will sync all those tags anyway
<harlowja> *openstack-infra people
<harlowja> although maybe they will
<harlowja> https://github.com/stackforge/cloud-init/tags has only a subset of those
<harlowja> not sure if it should even have the ubuntu-* ones?
<smoser> i dont care about the ubuntu- ones
<smoser> but i'd like to get the X.Y.Z tags
<harlowja> kk, so let me avoid trashing those and repost
<harlowja> smoser http://paste.openstack.org/show/406497/ a full script that can do this resync stuff
<smoser> nice
 * smoser executes harlow's rootkit
<harlowja> ha
<harlowja> smoser or https://github.com/harlowja/cloud-init/tree/0.7.x
<harlowja> has that updated, with all the tags pushed
<harlowja> (minus ^ubuntu ones and trunk-karmic or whatever
<smoser> we need to update some of the tools to not use bzr
<smoser> and the one thing that i like about bzr ...
<smoser> revision numbers
<smoser> that is used in changelog.in (build-deb)
<harlowja> hmmm
<harlowja> oooorrr http://blog.launchpad.net/general/git-code-hosting-beta
<harlowja> i'm fine with that to :)
<harlowja> just some git, so that yahoo build-systems can be happy :-P
<smoser> harlowja, i have to be able to ./packages/build-deb
<smoser> which uses bzr export tarball (through make-dist-tarball)
<harlowja> hmmm, that can probably be switched pretty easy i think
<harlowja> pretty sure git can make a tarball or something
<harlowja> smoser ok, found another issue that probably should get merged; https://code.launchpad.net/~harlowja/cloud-init/encode-resolve-fix if u get time
<harlowja> pretty straightforward, make sure its a string before writing it...
<harlowja> smoser perhaps https://www.kernel.org/pub/software/scm/git/docs/git-archive.html works for bzr export thing
<harlowja> ya $ git archive --format=tar.gz  HEAD > thing.tar.gz seems to work
#cloud-init 2015-07-31
<smoser> harlowja, yeah. git archive is probably sufficient. i know we can find stuff.
<smoser> fiddle.
<smoser> harlowja, you can push that if you want.
<smoser> Odd_Bloke, https://code.launchpad.net/~smoser/cloud-init/trunk.reporting/
<smoser> its not in a great state right now (not acutally functiona) but i'd appreciate your thoughts on the chagnes i've made / additions to the reporting stuff.
<smoser> i'll flesh it out more today
<Odd_Bloke> smoser: Oh, hm, I was going to refactor the reporting stuff a bit, which might make it hard to take changes back.
<smoser> well more consider this as input to the work you're doing.
<smoser> things really there are
<smoser>  a.) i've got a context handler that seems useful (ReportingStack)
<smoser>  b.) i made event have success/fail/warn
<Odd_Bloke> smoser: Would it make sense to submit those changes to 2.0, so that we can maintain a wholesale backport?
<smoser> yeah.
<smoser> i'm just in need of getting a reporting functional on 1. so i'm pushing through implementation to learn things and see how it goes.
<smoser> and those things that i thin i've learned, i'm interested in your thoughts on
<smoser> that make sense?
<smoser> s/1/0.7x./
<smoser> s/typo/other typo/
<Odd_Bloke> smoser: Yeah, both the context manager and the WARN addition look broadly sensible.
<Odd_Bloke> smoser: I do have some comments, but it'd be easier to make those in some sort of code review tool *COUGH*gerrit*COUGH*. ;)
<smoser> well, i can propose to merge into 0.7
<smoser> then you can use that other code review tool if you'd like.
<Odd_Bloke> Doing it that way around will make backporting any other reporting changes in the future more painful.
<Odd_Bloke> I'd like to get reporting to a done-ish state before we backport it.
<Odd_Bloke> If that's a problem, ask Andreas who on his team he's going to commit to cloud-init work. Â¬.Â¬
<Odd_Bloke> Sorry, shouldn't be so grumpy.
<Odd_Bloke> But I think it will cause pain in the future.
<smoser> bah. proposing for review does not mean accepting for review
<smoser> you mentioned you wanted to comment on it.
<smoser> i proposed a way you could comment
<smoser> that i'd then address and get into suitable path
<smoser> which woudl then be submitted for review to 2.0
<Odd_Bloke> Sure.
<Odd_Bloke> Let's do that. :)
<smoser> Odd_Bloke, https://code.launchpad.net/~smoser/cloud-init/trunk.reporting/+merge/266578
<smoser> Odd_Bloke, if you coudl take a look i'd appreciate it.
<smoser> i'm planning on spending the rest of my day on this, so ..
<Odd_Bloke> Yeah, looking now.
<Odd_Bloke> smoser: There you go.
<smoser> mercy
<smoser> oh this is not good
<smoser> i can't see your comments
<smoser> :)
<smoser> do you see them. ?
<smoser> Odd_Bloke,
<Odd_Bloke> smoser: No, I think because you pushed new commits.
<Odd_Bloke> s/new/different/
<smoser> joy
<smoser> new
<smoser> not different
<Odd_Bloke> mumble mumble gerrit mumble :p
<Odd_Bloke> They _might_ be in the email LP should send at some point.
<harlowja> what u guys doing, lol
<Odd_Bloke> So lets wait for that before I go through and try and remember everything.
<smoser> Odd_Bloke, yeah, i'm looking at email
<smoser> Odd_Bloke, the printHandler is just garbage for debugging . was easier than opening up the log :)
<smoser> stderr makes sense for sure.
<smoser> but so does possibly just dropping it.
<Odd_Bloke> smoser: Right; either drop it or remove it from default config. :)
<smoser> comments back!
<smoser> hm.. or they were.
<smoser> odd
<openstackgerrit> Daniel Watkins proposed stackforge/cloud-init: Refactor handlers out of main reporting module.  https://review.openstack.org/207979
<smoser> Odd_Bloke, sometimes its good to explain *why* you're making a change.
<openstackgerrit> Daniel Watkins proposed stackforge/cloud-init: Refactor handlers out of main reporting module.  https://review.openstack.org/207979
<Odd_Bloke> smoser: Yep, not totally used to the commit _being_ the review proposal yet.
<Odd_Bloke> Updated with more reasoning.
<openstackgerrit> Merged stackforge/cloud-init: Refactor handlers out of main reporting module.  https://review.openstack.org/207979
<claudiupopa> Odd_Bloke, smoser: late to the party, I left a comment for #207979.
#cloud-init 2015-08-01
<Wulf> Good Morning!
#cloud-init 2016-08-01
<Odd_Bloke> smoser: So is git now the official cloud-init VCS?
<smoser> Odd_Bloke, yes
<Odd_Bloke> \o/ \o/ \o/
<rharper> smoser: https://lists.ubuntu.com/archives/ubuntu-devel/2016-August/039470.html
<harlowja> and also http://lists.openstack.org/pipermail/openstack-dev/2016-July/100476.html
<harlowja> smoser  https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301727
<harlowja> that should get us to a place where tags and such can be used
<harlowja> oddly though, where does these branches proposed for merges show up
<harlowja> https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301728
<harlowja> https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301729
<harlowja> https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301730
<harlowja> https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/301731
#cloud-init 2016-08-02
<smoser> harlowja, thank you.
<smoser> harlowja, you can see things for proposed merge into cloud-init's git trunk this way...
<smoser>  all the cloud-init repos that launchpad knows about (~user/cloud-init) are viewable at  https://code.launchpad.net/cloud-init
<smoser>  lp:cloud-init (aka ~cloud-init-dev/cloud-init) git repo is at https://code.launchpad.net/~cloud-init-dev/cloud-init/+git/cloud-init
<smoser> a list of all cloud-init-dev's cloud-init branches can be seen here
<smoser>  https://code.launchpad.net/~cloud-init-dev/cloud-init/
<smoser> the 'master' branch https://code.launchpad.net/~cloud-init-dev/cloud-init/+git/cloud-init/+ref/master
<smoser> and that has 'Branch merges' "7 branches proposed for merging into this one"
<smoser> which points https://code.launchpad.net/~cloud-init-dev/cloud-init/+git/cloud-init/+ref/master/+activereviews
<smoser> all of that is navigatable but sometimes non-obvious
<smoser> s/sometimes/
<smoser> larsks, around ?
<larsks> Sort of :) Phone meeting. I saw your changes; hope to look at that this afternoon.
<smoser> larsks, alternatively if you can tell me how to test that i did not break your work with bdrpm i'd just merge
<smoser> larsks, ok. so i think i'm going to just pull my changes.
<smoser> brpm works for me as via:
<smoser>  http://paste.ubuntu.com/21909186/
<larsks> smoser: okay.  totally going to look at things this afternoon; yesterday was just busy :)
<smoser> larsks, i understand not geting aroudn to things.
<larsks> But if it works for you that is probably the best test.
<smoser> you have been a big help. thank you.
<smoser> see that paste there, i built successfully with rpmbuild
<smoser> your repo needs to get tags ro 'git describe' doesnt work
<smoser> and i thikn your repo needs new repo basically (since i re-imported)
<smoser> end result though, trunk should bddeb and brpm shortly
<larsks> I think the existing code is post-your-re-import.  But yeah, may be missing tags.
<smoser> yeah, just missing tags then.
<smoser> that makes more sense.
<smoser> rharper, powersj harlowja ^ above paste points to building cloud-init in an centos7 lxc.
<smoser> there are some less-than-ideals there (i had to copy the list of rpms to install rather than using something in trunk to generate it) and i the script assumes it has to branch a remote repo, but generally did work.
<rharper> smoser: what's with the ssh stuff in the build env?
<rharper> it's a builder script?
<harlowja> whats up
<harlowja> i like building things :-P
<harlowja> now just need to see if lxc is on our build slaves, lol
<harlowja> larsks is it prefered to use lxc, or use the mock thing i  was reading about
<harlowja> https://fedoraproject.org/wiki/Mock
<larsks> mock is generally what people use for building packages in pristine environments.
<harlowja> is that like glacial sprint pristine?
<harlowja> *glacial spring
<harlowja> or like my local pond pristine
<harlowja> lol
<larsks> Oh, maybe, you had me confused for a minute.
<harlowja> the above is just messing around
<harlowja> lol
<harlowja> (well the mock question isn't)
<harlowja> but the spring part is
<larsks> I had figured that out, but thanks :)
 * rharper enjoys the artic air during his sprints
<harlowja> lol
<larsks> harlowja: incidentally, you'll note that brpm now has a --srpm option which will *just* build a source rpm, which you can then hand off to mock.
<harlowja> cool
<harlowja> gotta get my jenkins build scripts up to date to try some of this
 * harlowja has been recently messing with http://docs.openstack.org/infra/jenkins-job-builder/ 
<harlowja> and getting the CI guy onboard with using that
<rangerpb> smoser, hey around? i was talking with larsks about the problem with entry_points, bloating /usr/bin/, and fully qualified path names to executables .. he had a nice idea where instead of using entry_point, the scripts call python -m instead
<rangerpb> it is a good compromise, the only  downside is the python version is not figured out like it is with entry_point
<smoser> rangerpb, right. and that is fine.
<smoser> except...
<smoser> which python do you call ?
<smoser> python2 or python3
<rangerpb> well exactly .. thats the point I was bringing up
<smoser> maybe its easier to come up with that.
<rangerpb> i was thinking so ... but this is new territory for me as well
<rangerpb> i did see python version is determined in the make file
<rangerpb> hey harlowja are you aware of a way that setup.py could be used to replace a variable in a file prior to install ?
<harlowja> nope :-/
<rangerpb> how about running a command which in turn could do that?
<rangerpb> larsks, smoser http://bazaar.launchpad.net/~bbaude/cloud-init/azure_dhcp/revision/1253  <-- how much does that make you throwup in your mouth?
<larsks> rangerpb: yuuuuuuck.  So, for the record: I think that templating the hook scripts seems unnecessary.  I think we should trust the Python installer to put scripts where they will be in $PATH, and then just call them with an unqualified name.  If someone *really* wants to override things by mucking about with $PATH, more power to them!
<smoser> that dhclient hooks is *such* a crap interface
<larsks> I mean, we trust it to put cloud-init in the right place...
<rangerpb> smoser, i dont follow..
<smoser> larsks, thats not it though..
<smoser> the template is needed to get the right python binary
<smoser> either python2 or python3
<smoser> as it is now.
<larsks> It's not, in fact; this was all rangerpb trying to work around a prior requirement :)
<larsks> I think we should just replace the call to '$PYTHON ...' in the hook scripts with 'cloudinit-dhclient-hook', no full path, and it all becomes much easier.
<smoser> so, yes. anoterh alternative is to put a easyinstall program into /usr/bin/
<rangerpb> there we are couple of pieces to it. the right version, the right binary, the right install path
<larsks> Which is exactly where console_scripts entrypoints get placed.
<smoser> except then you have a crappy program that is not a user executable in /usr/bin/
<smoser> thats what i didn't like.
<larsks> I guess.  It seems like a lot of effort to work around it.
<smoser> i'm polluting /usr/bin/ and destroying tab completion for cloud-init with some sillyness.
<larsks> You have the cloudinit-dhclient-hook script bail out with an error message if called without the appropriate environment.
<larsks> s/have/could have/
<smoser> the right place for 'cloud-inti-dhclient-hooks' is /usr/lib/cloud-init/
<smoser> which isn't going to be in a PATH (and for good reason)
<larsks> The alternative -- which is my preference -- is to leave installation of the hook scripts up to the package.  I know we've talked about this before, but I don't generally expect my setup.py to be setting up init scripts etc. for me.
<larsks> The right place for the hook helper is probably distribution dependent.
<smoser> sure.
<smoser> but not in $PATH
<larsks> (which is why I think it's a packaging issue)
<smoser> and that is reasonable.
<smoser> but then later on i break you, because your dealing with that now has to deal with a change we made upstream.
<smoser> we're trying to make setup.py do the right thing.
<smoser> i think the link there is pretty good.
<larsks> I understand your goal, but I think that trying to turn setup.py into a multi-distribution system configuration tool is the wrong place to put our effort.
<rangerpb> wait ... smoser did you think what I did was somewhat ok ?
<smoser> not really a system configuration tool. just an install tool.
<smoser> i know its a pain.
<larsks> Well, setting up init/systemd/etc...I think of that more as system config.
<smoser> i dont know what other things do here.
<smoser> i generally agree, but cloud-init kind of has more involved init/systemd than other things.
<larsks> A lot of things just install the executables via setup.py, and leave service configuration up to the packager (e.g., I think all the openstack services do that, for example).
<smoser> its trying to be part of the os
<smoser> i'm willing to accept that maybe this is over-doing things.
<larsks> Eh, ultimately you're the driver :)
<smoser> i think the link there is actually pretty good though
<rangerpb> i think it is the closest to accomplishing what you wanting without total insanity
<rangerpb> just like ... partial insanity
<smoser> some nit picks i have , but largely good.
<rangerpb> comment away
<smoser> that dhclient hooks interface is complete crap
<larsks> So, what if we unlitaterally install the script into /usr/lib/..., and just call that explicitly from the hook scripts, which then don't require templating...
<larsks> ...and packagers can patch setup.py if they want the script in a different place?
<smoser> you could actually unintentionally and unknowingly break some other program's hook because you are setting PYTHON_BINARY
<smoser> what crap that is.
<larsks> Hence this ^^^ proposal :)
<rangerpb> larsks, but installing into a static location doesnt fix which python version should be used
<smoser> but you just added a level of indirction
<larsks> Argh, right, I keep forgetting that silly detail.
<smoser> you have to template the /usr/lib/ thing
<smoser> yeah.
<larsks> How about:
<larsks> setup.py installs it in /usr/bin, and the package moves it? :)
<larsks> Nah.
<smoser> this is also sovled by piggy-backing on the cloud-init in /usr/bin/
<larsks> Never mind.
<larsks> Oh, that's an even better idea.
<larsks> Just making it a subcommand, you mean?
<rangerpb> i dont follow that
<larsks> Like 'cloud-init dhclient-hook'?
<smoser> yeah.
<rangerpb> wait !!!!!!!!!1 i suggested that before
<larsks> Well hell, that fixes everything.
<larsks> rangerpb: pics or it didn't happen :)
<smoser> the only thing i didnt' lke there is that the main has a bunch of imports and stuff that the dhclient doesnt
<smoser> making it much slower.
<rangerpb> oh it happened :)
<smoser> larsks, rangerpb i really appreciate boht of your inputs
<rangerpb> smoser, so are you saying youll take the perf hit to make it a subcommand ?
<larsks> I think that seems cleanest, honestly.
<rangerpb> yeah but I gotta hear the man say it
<smoser> :)
<larsks> Oh sure, I'm just opining.
<smoser> well, it would be slower
<larsks> Opinionating.
<smoser> thats what i'm saying.
<rangerpb> so i can take ap icure ...
<smoser> and slowness is a pita.
<larsks> I guess.  We'll already have run cloud-init once at that point, so hopefully most of the disk accesses are already cached.
<rangerpb> cloud-init runs *after* dhclient does it not?
<larsks> Well, point being, we run it twice in either case.
<larsks> You could run some short tests to quantify the performance difference.
<rangerpb> smoser, we could conditionally import stuff based on what cloud-init is doing
<larsks> AHHHH  STOP ADDING LOGIC! :)
<larsks> Seriously, that would be premature optimization unless you first demonstrate the magnitude of the performance hit.
<rangerpb> im a people pleaser
<smoser> actually, yeah.
<smoser> the crap that i have for the stages and paths makes it bad.
<smoser> larsks, here is time info
<smoser> http://paste.ubuntu.com/21937409/
<rangerpb> and your conclusion is ?
<smoser> rangerpb, ok. i'm fine with a sub-command of cloud-init
<larsks> \o/
<rangerpb> and a subcommand would be something like 'init' already is right>
<rangerpb> ?
<smoser> http://paste.ubuntu.com/21937756/
<smoser> right.
<rangerpb> ok, ill get cracking on that ... any advise on how you would see that being done?
<smoser> it is very real.
<rangerpb> would you detect if the subcommand was given and jump from there asap ?
<smoser> yeah, just register the subcommand with the args as the other ones are.
<larsks> rangerpb: see the 'main' method in cloudinit/cmd/main.py
<smoser> the performance difference is ~ .03 seconds between the 2
<rangerpb> larsks, yeah i was poking around in there to see as well
<harlowja> so when can we merge https://code.launchpad.net/~larsks/cloud-init/+git/cloud-init/+merge/301310 :-P
<harlowja> larsks is that ready?
<harlowja> killing my package building CI job without that ;)
<larsks> harlowja: *I* think it's ready, if you and smoser are happy with it...
<mgagne> so I tested latest cloud-init 0.7.7~bzr1256-0ubuntu1~16.04.1 and it fails with a known bug we identified a couple of weeks ago where the bond link name attribute is expected by cloud-init but configdrive in OpenStack doesn't provide it since it's not part of the spec.
<mgagne> this fix didn't make it afaik: http://paste.ubuntu.com/20492837/
<mgagne> in fact, I think this bug hasn't been fixed yet: https://bugs.launchpad.net/cloud-init/+bug/1605749
<harlowja> mgagne cool, something to wrok on
<harlowja> once i get this git crap building, ha
<mgagne> :P
<harlowja> or others can, now that we have git!
<harlowja> ha
<mgagne> (wink wink)
 * harlowja winks back at u
<harlowja> lol
<harlowja> ha
<harlowja> did u say u wanted to do that if we moved to git?
<harlowja> or was that someone else, ha
<mgagne> well, would make it less painful
<mgagne> I know smoser worked on fixing those issues AFAIK
<mgagne> in fact, this paste fixes the bond link name issue but also reproduces the issue where bond_links aren't physical nic but reference to link ids
<mgagne> anyway
<mgagne> will schedule some time to work on it tomorrow
#cloud-init 2016-08-03
<harlowja> if u want
<mgagne> harlowja: how can I fork or work on cloud-init in LP git?
<mgagne> ok so I create a new repo in my own account and push a clone of cloud-init in their? there is no fork button?
<harlowja> mgagne it seems so
<harlowja> larks documented this in https://code.launchpad.net/~larsks/cloud-init/+git/cloud-init/+merge/301310
<harlowja> its a little different than github
<larsks> harlowja: yes.
<larsks> harlowja: but do you have a specific change you'd like to see?
<mgagne> will read
<harlowja> not me, mgagne was wondering where is the fork button :-P
<larsks> Oh sorry.
<larsks> Ha!
 * larsks wishes we would all just use gerrit.
<mgagne> looks to not be that bad (LP)
<mgagne> time will tell =)
<harlowja> its better than bzr
<harlowja> imho :-P
<harlowja> cause i only ever understood the basics of bzr, lol
<smoser> harlowja, going to get over to get completely here shortly.
<larsks> no argument here :)
<smoser> with the changge s that larsks had and mine there.
<harlowja> cool
<smoser> and then its not ata ll impossible for us to have a c-i that mirrors launchpad to github
<harlowja> then my jenkins pipeline will work!
<harlowja> lol
<smoser> dont know how i feel about that thoguh
<smoser> ie, i coudl put github/cloud-init-dev/cloud-init to jsut be pushed to from lp:cloud-init by a bot
<harlowja> meh, i'm ok with this launchpad bzr
<harlowja> not perfect imho, but good enough
<harlowja> github not perfect imho either
<smoser> the big thing is that if i do it there, then it implies we'd take reviews from there.
<harlowja> ya, meh, don't do it then :-P
<harlowja> or u can do what openstack-infra does, and they shutdown all PR(s) that come through that
<harlowja> auto-shutdown
<smoser> oh.
<smoser> hm.
<harlowja> https://github.com/openstack/taskflow/pull/3
<harlowja> for example
<harlowja> i'm pretty sure that person though never followed through with the PR
<harlowja> (for worse imho)
<harlowja> so just a warning :-P
<harlowja> and i don't think people in projects know about how many/if PR(s) have been auto-closed
<harlowja> for example
<harlowja> https://github.com/openstack/nova/pulls?q=is%3Apr+is%3Aclosed
<harlowja> there are 65 lost reviews there?
 * harlowja i think i complained about that once
<harlowja> (ie at least send an email to the PTL when a PR gets auto-closed)
<harlowja> so PTL can at least follow up with person (if they want)
<JayF> harlowja: I think monty emailed the list about improving their generic message pushed up to github, to make it more clear how toa ctually contribute
<harlowja> JayF possibly
<harlowja> i'd like a 'cc' to the PTL that it got closed on also
<harlowja> i'm pretty sure that not many people (PTL or other) know to go look and see whats been closed
<JayF> I mean
<JayF> PTLs haev no time as is
<harlowja> meh
<harlowja> life is hard
<JayF> that would get filed into a folder and never read ever
<JayF> at least for the major projects
<harlowja> i don't accept the life is hard part
<harlowja> life is hard, therefore /dev/null
<harlowja> sounds like a cop-out to me ;)
<harlowja> back to sucking my thumb in the corner
<harlowja> ha
<JayF> it's all a matter of priorities right?
<harlowja> sure
<harlowja> but if right now it goes to /dev/null u can't even assign someone from the PTL(s) team to even look at that stuff
<JayF> like folks would rather 1) work a reasonable number of hours and 2) spend those hours working on project priorities, instead of holding the hand of someone who can't read a $#@%$!@#ing doc about how to contribue :P
<harlowja> thats what interns are for?
<harlowja> 3) assign task to followup with the folks that got PR closed on them (that may or may not have submitted code to the gerrit system) and be nice and kind to them, and see if they need help
<harlowja> treat others like u would want to be treated, blah blah
<mgagne> are unit tests really trying to read files in /proc ?
<mgagne> harlowja: how do you link your personal repo to the upstream one? I'm following steps found in README, reading LP help page and not much details here, there is no Propose for merge in my repo
<harlowja> hmmm
<harlowja> on for example
<harlowja> https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+ref/fix-distro-tags
<mgagne> yep, link is showing here
<harlowja> there was a 'propose for merging' that i clikced thed
<harlowja> u don't got one?
<mgagne> there is none for me: https://code.launchpad.net/~mgagne/+git/cloud-init/+ref/bond_name
<harlowja> *clicked there
<harlowja> hmmm
<mgagne> so I think the repo isn't linked with "upstream"
<mgagne> and I'm not sure how to fix that
<harlowja> maybe it also needs some commits?
<mgagne> there is a commit
<mgagne> which I'm trying to propose for merge
<harlowja> ah, kk, it shows i think the author as scott
<harlowja> unsure here
<mgagne> yes, I didn't code that feature
<harlowja> i call in smoser who might know, ha
<mgagne> so I'm not putting my name in there
<harlowja> k
<mgagne> harlowja: found it
<mgagne> Change repository details -> Target: Project: cloud-init
<mgagne> this detail isn't part of any documentation if found (cloud-init or LP)
<mgagne> harlowja: ok merge proposal done =)
<smoser> mgagne, whats up?
<mgagne> smoser: a review request? =)
<mgagne> smoser: was struggling to find how to link my personal repo to the upstream one
<mgagne> now it's done with above steps
<mgagne> smoser: I think the test you are referring to was to test bond_slaves, not bond name. Maybe i'm wrong: http://paste.ubuntu.com/20492837/
<mgagne> let me know if I misunderstood the purpose of the test
#cloud-init 2016-08-04
<rangerpb> smoser, larsks mind peeking at https://code.launchpad.net/~bbaude/cloud-init/azure_dhcp ? it contains the rework we discussed yesterday.
<smoser> rangerpb, reading
<rangerpb> im updating the commit msg now too
<rangerpb> smoser, also wondering if you wouldnt mind peeking at https://code.launchpad.net/~bbaude/cloud-init/rh_sub_rm_first which is a trivial change too.  <-- larsks fyi
<rangerpb> headed offline for a bit...back in about 90 minutes or so
<smoser> k. i'll have it for you when you return
<smoser> harlowja, please ping when you're in.
<harlowja> smoser ping a ling
<harlowja> so mr.version
<harlowja> ha
<smoser> hey
<smoser> yeah.
<smoser> so. we definitely can't pick up a runtime dependency for something so silly
<harlowja> k
<harlowja> that should be fine
<harlowja> we should be able to just do
<smoser> and even build-time is a pain. i'm perfectly fine to take the convention of pbr
<harlowja> >>> import pkg_resources
<harlowja> >>> pkg_resources.get_distribution("pbr").version
<harlowja> for example
<smoser> but only if you have that available in build
<smoser> right?
<harlowja> (to get whatever version is installed)
<harlowja> everyone has pkg_resources
<harlowja> :-P
<harlowja> replace 'pbr' there with 'cloud-init'
<harlowja> ha
<smoser> i'm confused.
<harlowja> so the runtime dependency would be for version.py?
<harlowja> maybe i'm confused eto
<harlowja> ha
<smoser> so runtime, yeah.
<smoser> so can't pick up runtime
<smoser> but even the build time is a pita
<smoser> and i'd like to avoid it
<harlowja> meh
<smoser> i think its more than just pkg_resources
<harlowja> for build time, meh, what do u do
<smoser> i have to have pbr at some sufficient version
<harlowja> u either create your own pbr thing
<harlowja> or use pbr lol
<smoser> how hard is this though?
<harlowja> if u want to actually provide a useful deb and rpm version at build time
<smoser> i'm fine to follow their versioning scheme
<harlowja> its sorta a pita that i don't want to recreate, lol
<smoser> how is it hard?
<harlowja> its complicated, not hard
<smoser> it can't be that complicated.
<smoser> i'm happy with git-describe
<smoser> it gives you 3 tokens of information
<smoser>  last_tag_on_branch, commits_since_tag, hash
<smoser> you can only put those in so many different orders
<harlowja> ya, hash is useless in redhat
<harlowja> in rpms
<smoser> its useful
<smoser> you just hide it behind something like commits_since_tag
<harlowja> https://github.com/openstack-dev/pbr/blob/master/pbr/version.py#L353 has some details around why its such a pita
<harlowja> i just don't have much desire to recreate any of that crap
<harlowja> https://specs.openstack.org/openstack/oslo-specs/specs/juno/pbr-semver.html#proposed-change has more details
<harlowja> if u really want to recreate all that, go ahead i guess, i just don't care much to, lol
<harlowja> "Provide a new command (deb-version) to output Debian package version compatible version strings. This primarily involves translating PEP-440 precedence rules into Debian ~ and . component separators.
<harlowja> Provide a new command (rpm-version) to output RPM version metadata for incorporation in RPM versions using the ENVRA format. As RPM lacks a âbeforeâ operator (~) the primary method for translation is to treat pre-release and dev builds as release builds of next lowest version to drive the sort order above all actual releases of the version below. We assume that no version will ever have more than 9998 patch/minor releases. E.g. 1.2.0.dev5 is
<harlowja>  rendered as 1.1.9999.dev5. 1.0.0.dev5 would be rendered as 0.0.9999.dev5 and finally 0.0.0.dev5 would also be rendered as 0.0.0.dev5 to avoid negative version numbers.
<harlowja> "
<smoser> harlowja, you are correct that this is a pita
<harlowja> ya, i mean, ya, we can do something similar
<harlowja> sure, (although i don't really want to code it/verify its correctness, ha)
<smoser> can pbr parse a string for me ?
<smoser> and tell me what it thinks debiand and rpm versions shoudl be ?
<harlowja> hmmmm
<harlowja> don't think so
<harlowja> some other package might be able to
<smoser> this is crap harlow
<smoser> version.SemanticVersion(0, 7, 6, dev_count=100).debian_string()
<smoser> 0.7.6~dev100
<smoser> i'd have thought i was saying 0.7.6 + 100 commits
<smoser> and it gave me ~ which is < 0.7.6
<harlowja> lol
<harlowja> u might want to try https://github.com/openstack-dev/pbr/blob/master/pbr/version.py#L121 ?
<harlowja> or seeing what https://github.com/openstack-dev/pbr/blob/master/pbr/packaging.py#L592 does
<harlowja> or https://github.com/openstack-dev/pbr/blob/master/pbr/packaging.py#L691
<smoser> harlowja, it sure looks to me like my versioning scheme works for both debian and rh
<smoser> if it is a tag, then you use Major.Minor.Micro
<smoser> if not, then you use
<smoser>  Major.Minor.Micro+Commits.gHASH
<harlowja> so "+" is addition there?
<smoser> it just means greater
<smoser> i am pretty sure.
<harlowja> ya, i don't think that works in rpms from what i remember
<harlowja> but maybe larsks can comment
<larsks> harlowja: what am I commenting on?
 * larsks reads...
<larsks> I think we should be following https://fedoraproject.org/wiki/Packaging:Naming?rd=Packaging:NamingGuidelines w/r/t rpm versioning.
<larsks> I suspect you can use '+' in a release if you want (that is, I don't think it's prohibitied), but it's not magic in any way.
<smoser> larsks, i'm not sure i follow
<larsks> I think you are asking if "Major.Minor.Micro+Commits.gHASH" is a good versioning scheme for packages, yes?  Or did I misread? Apologies if so.
<smoser> i'm saying that for snapshots of upstream, calling that the "upstream version" is sufficient.
<smoser> if you want to suggest something else, i'm ok with that.
<larsks> I'm still not following, sorry.  You are proposing that e.g., brpm should produce packages with this versioning scheme?  Or something else?  If it's just a documentation reference to the upstream version that seems lovely.
<smoser> so, yeah. if you run 'brpm' on trunk, you'd get a rpm that had a version like that.
<smoser> if you ran it with 0.7.7, you'd get '0.7.7'
<larsks> I guess if those rpms never really go anywhere other than a local development host it doesn't really matter.
<smoser> what versoinoing would you use for your packaging ?
<smoser> you'd probably just use a serial incrementer on the releases youd done
<smoser> right?
<larsks> The doc I pointed at (the fedora packaging guidlines), which is pretty much what I implemented in my changes to brpm.
<larsks> So, <name>-<version>-<release>.<date>git<commit_id>
<harlowja> * if those rpms never really go anywhere other than a local development host it doesn't really matter.
<harlowja> hmmm, but they will be going somewhere else, ha
<larsks> harlowja: are they?
<larsks> harlowja: none of the downstream rpm distributions use brpm for building packages.
<harlowja> sucks to be them
<harlowja> ?
<larsks> I dunno about that.
<larsks> But that is the current situation.
<smoser> :)
<harlowja> reality is i don't plan on waiting around for downstream rpm distributions :-P
<larsks> harlowja: i am not sure I follow your comments.
<larsks> In what way would you need to wait on them?
<harlowja> wait for a change to filter through the pipeline ---> downstream rpm distributions
<harlowja> to images that i can them use
<harlowja> *then use
<harlowja> (in the godaddy cloud)
<larsks> Ah, I see.
<harlowja> also need to add in a godaddy specific module to cloud-init, so i'll be repacking anyway, and cloud.cfg will require tweaks for that
<larsks> It seems like that would happen in any case, regardless of what they were using for the spec file, especially for enterprise distributions, right?
<larsks> There is almost always going to be delay between upstream release and downstream packaging.
<harlowja> idk, i'm not a enterprise distributor :-P
<harlowja> in the world of the future, no i don't like that delay and i think its messed up
<harlowja> in the world of today, sure it probably is a delay
<larsks> Okay.  Anyway, back to smoser's question, I would personally adopt the versioning scheme recommended in the fedora docs, but for our purposes here I don't think it matters one way or the other.  As long as release <N+1> is lexically greather than release <N>, it should all be okay.
<harlowja> is that in part cause of the lack of usage of brpm?
<harlowja> (why it doesn't matter)
<smoser> http://paste.ubuntu.com/22207121/
<smoser> that compares version info like
<smoser> http://paste.ubuntu.com/22207177/
<harlowja> woah, shell
<harlowja> lol
<larsks> smoser: I was curious about whether something like 0.7.7-10-g1234567 will be > 0.7.7-1-g1234567.  If not, one could zero-pad the commit count.
<smoser> you can't put a - in an upstream version in debian
<smoser> stupid
<smoser> wait
<smoser> not saying *you're* stupid
<smoser> stupid that debian issue
<smoser> larsks, the + does seem to work, and honestly make sense.
<smoser> right now we're 0.7.6+some-commits
<larsks> Awesome. Have at it :)
<smoser> the read-version stuff would have to be updated as it gets 0.7.7 now.
<smoser> but tath is ok.
<harlowja> and for a change of topic
<harlowja> smoser any thoughts on https://issues.jenkins-ci.org/browse/JENKINS-30183
<harlowja> https://github.com/jenkinsci/acceptance-test-harness/blob/master/src/test/resources/openstack_plugin/cloud-init  is the cloud-init file that uses
<harlowja> i think in part its because runcmd probably shouldn't be used
<harlowja> or the module ordering needs to be adjusted
<harlowja> https://issues.jenkins-ci.org/browse/JENKINS-30183?focusedCommentId=266029&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-266029
<harlowja> or someones package doesn't set the right ordering for sshd to start
<harlowja> it almost seems like whats wanted is a init-run-cmds there
<harlowja> or moving runcmd into the cloud_init_modules for this case
<harlowja> so that sshd doesn't start until after thats done
<smoser> so they want to lock sshd starting until its done?
<harlowja> ya
<smoser> ah. i see.
<smoser> jenkins ssh's in
<harlowja> any other way besides adjusting the module ordering that u can think of?
<harlowja> ya, they do https://github.com/jenkinsci/acceptance-test-harness/blob/master/src/test/resources/openstack_plugin/cloud-init#L16 and line 22 there
<harlowja> to work around this
<smoser> does jenkins retry ?
<smoser> if it can't get in.
<smoser> i suspect yes
<harlowja> ya, but the desire i think is to not allow sshd at all until user data scripts have all ran
<harlowja> so that they know that user data scripts have finished before jenkins (the master) starts trying to connect in
<harlowja> which it does via ssh
<harlowja> (to establish the master <-> slave connection)
<smoser> well, a boothook could disable sshd (and the stop it to make sure)
<smoser> and then start it in runcmd
<harlowja> right, that'd work to
<smoser> also, if they did not put the ssh-authorized-keys in under 'users'
<harlowja> the weird part (or harder part) is that that plugin allows for user specified cloud userdata
<smoser> but wrote them when they're done
<harlowja> so people can override it and not do that
<harlowja> and screw themselves, ha
<harlowja> bb
<smoser> i've wanted to be able to block ssh from starting until users sare configured before.
<smoser> so that if you ever got denied you'd not need to try again.  either connection closed or youre in
<smoser> but never wanted to not start sshd till later.
<smoser> ubuntu can disable sshd easily enough
<smoser> harlowja, 2 other options
<smoser> a.) rather than having jenkins just bang on the ssh port until gets in, you could have phone_home tell it it is ready
<smoser> b.) you coudl have jenkins block until a /run/all-done is there
<smoser> (or cloud-init's /run/cloud-init/result.json)
<harlowja> ah, ya, the '/run/all-done' might be a good idea
<hans__> hey folks, had a question on whether anything has changed in recent ubuntu images as far as initial mount and format of the ephemeral drive. we have started seeing this lately: Stderr: 'mke2fs 1.42.13 (17-May-2015)\n/dev/sdb1 is mounted; will not make a filesystem here!\n'
<hans__> we think this might be a race condition between mnt.mount and cloud-config.service, but it seems to be a new issue
<smoser> hans__, this is azure ?
<hans__> @smoser, yes
<hans__> @smoser: it looks like mnt.mount is doing a mount before cloud-init gets there sometimes, and we see 'Device /dev/sdb1 has Temporary Storage ntfs'
<hans__> @smoser, then mke2fs fails - is it expected that the drive is unmounted before cloud-init gets there?
<smoser> hans__, yeah, thats probably true.
<smoser> nothing is stiopping the mounts from occuring
#cloud-init 2016-08-05
<tz__> Good morning, Iâm having trouble debugging my cloud-config script.  Iâm on ubuntu 16.04.  I edit /var/lib/cloud/instance/user-data.txt and run cloud-init --debug init, but i get the same error message of âUnhandled non-multipart (text/x-not-multipart) userdata: âbâ{ââ¦ââ every time i run it, regardless of changes to user-data.txt
<smoser> tz__, cloud-init does not re-read that file.
<smoser> it reads form the datasource.
<tz__> can i edit the data source on digital ocean after the VPS has been launched?
<smoser> not terribly easily.
<smoser> for debugging quick launches i suggest using lxc
<tz__> how can i iterate quickly on the cloud-config file then without having to destroy, create a droplet?
<smoser> since you already have one up, try this
<smoser> sudo lxd init
<smoser> ancer its questions
<smoser> then as non-root
<smoser> s/ancer/answer/
<smoser> then as non-root
<smoser> lxc image copy ubuntu-daily:xenial local: --alias xenial --auto-update
<smoser> actually, i woudl install zfs first (apt-get install zfs)
<smoser> then you can use zfs with lxc which is much faster.
<smoser> then you launch new instances with
<smoser>  lxc image copy ubuntu-daily:xenial local: --alias xenial --auto-update
<smoser> fudge
<smoser> sorr.
<smoser> launch a new instance with user data in file 'user-data' like:
<smoser> lxc launch ubuntu-daily:xenial my-name "--config=user.user-data=$(cat ./user-data)"
<tz__> okay cool, trying it out now
<tz__> thanks smoser!
<tz__> is there anyway to see cloud init output inline when i launch?
<tz__> or whats the best way to see the cloud init output?
<smoser> no. actually.
<smoser> that sucks.
<smoser> at least not easily
<smoser> you probably can
<smoser> but you have to fiddle with the config options of the container
<smoser> lxc used to put console output right in front of you
<smoser> but lxc2.0 does not.
<tz__> hmmm so how do people build the cloud config the first time? im probably gonna have to edit it 100 times before i get it right
<smoser> tz__ but you can easily enough do things like:
<smoser>  lxc exec NAME tail -f /var/log/cloud-init.log
<smoser> tz__, if you're passing cloud-config, the first thing is that it needs to be valid yaml
<smoser> so checking that is as simple as:
<smoser>  python -c 'import yaml; yaml.load(open("your-user-data"))'
<tz__> okay, that passes at least
<tz__> got âFailed to shellifyâ then a dumb of all the runcmds
<tz__> okay, i think itâs because i have blank lines (trying to split up the runcmds into sensible chunks)
<tz__> smoser: thanks for your help, iâm now seeing cloud-init debug information
<smoser> tz__, couple things...
<smoser> you can do a print() of the loaded yaml to make sure it looks like you epxect
<smoser> and also yaml "anchors" are useful for making things readable
<smoser> look at http://paste.ubuntu.com/22322345/
<tz__> okay, iâll check that out
<tz__> thanks!
<tz__> is there such a thing as cloud config snippets? a repo of best installations of, for example, redis?
<tz__> node, databases, node app
<tz__> sshd_config
<smoser> tz__, no.
<tz__> sad.
<tz__> seems like it would be super valuable since cloud configs are cross platform(ish)
<tz__> check out what you want on the box, and it generates the cloud config for you
<smoser> harlowja, around ?
<harlowja> sup
<smoser> https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+ref/version-info
<smoser> actually..
<smoser> let me propose for merge to make it easier for you
<harlowja> thx :-P
<harlowja> do we need to start using git rebase more also?
<harlowja> to avoid crap histories?
<harlowja> rebase then push --force
<smoser> https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/302176
<smoser> harlowja, are you trolling ?
<harlowja> not really :-P
<smoser> thats been my biggest pain ever.
<smoser> 2 things that bzr did better than git
<smoser> a.) bzr commit -p
<harlowja> u just gotta know how to use git right ;)
<smoser>   (this just shows diff in the EDITOR window, so you can write your message and look at the diff)
<harlowja> http://blog.adamspiers.org/2015/03/24/why-and-how-to-correctly-amend-github-pull-requests/#sec-3
<smoser> b.) uses '--first-parent' by default behaviori for git log
<harlowja> its the same crap for github PRs that most people don't know how to do :-P
<harlowja> http://blog.adamspiers.org/2015/03/24/why-and-how-to-correctly-amend-github-pull-requests/
<smoser> using first-parent by default is genious
<smoser> genius
<harlowja> ya, bzr prob did that nicer
<harlowja> gerrit forces it
<harlowja> but github doesn't
<smoser> it means that people dont have to squash and throw away data
<smoser> in order to avoid crap history
<harlowja> ya, fair
<smoser> i almost consider just getting crap history
<smoser> and saying "please use --first-parent"
<smoser> so there... at that mp, i tested building of rpm.
<harlowja> cool
<harlowja> how'd that go
<harlowja> ha
<smoser>  http://paste.ubuntu.com/22339840/
<smoser> the way to build is this:
<smoser>  http://paste.ubuntu.com/22339872/
<harlowja> cools
<smoser> that does it all in an lxc container, but basic path is there.
<harlowja> will use that :-P
<smoser> right. no one. although maybe sometime soon honestly as we want to get c-i on centos
<smoser> larsks, you probably have thoughts on my brpm stuff
<smoser>  spec looks like thsi: http://paste.ubuntu.com/22339840/
<harlowja> so only other thing i'm not sure about is
<harlowja> say i have a addon that doesn't really belong in cloud-init, but i need to add it into cloud-init
<harlowja> i can show u this addon if u really care
<harlowja> how do i make sure that the version number won't mess with the upstream one
<smoser> by default, we dont bump the 'Release' as iunderstand thats like "packaging release" in debian and in order to get a new version of that, you  kind of had to bump the source release so 'Version' would go up.
<harlowja> k
<smoser> example ?
<harlowja> a godaddy module addon
<harlowja> config module
<smoser> ah.
<smoser> and you have patches/ that you use --patches ?
<harlowja> this one its not yet a patch, but ya, its equivalent to
<larsks> This is where I wish cloud-init using something like stevedore for discovering out-of-tree modules...
<harlowja> right
<harlowja> https://gist.github.com/harlowja/e6d79d1de62e0edd821bedf76bfd5a20
<harlowja> smoser thats part of that module
<harlowja> bunch of crap that getting out of puppet (doing it in puppet and half of it in cloud-init was causing stupid issues)
<harlowja> so fixing that by just moving some of it to a cloud-nit module
<smoser> right.
<smoser> so how should we do that?
<harlowja> so stevedore is one option
<smoser> you want yours to be bigger than trunk or something.
<smoser> the version ?
<smoser> is that what we're talking about ?
<harlowja> well i either need the release to be bigger or smaller :-P
<smoser> yeah, larsks i do want to support out-of-tree moduels.
<smoser> and datasources.
<harlowja> oorrr go the path of larsks with stevedore
<harlowja> i can cook up osmething with stevedore
<smoser> we already have enough dynamic loading
<harlowja> (yet another runtime dep...)
<smoser> we dont need to add stevedoor for it.
<harlowja> stevedore is just a tiny wrapper over entrypoints
<harlowja> which is a python thing
<smoser> and /me cries everytime he runs a stevedore-using executable
<harlowja> stevedore shouldn't make u cry
<smoser> stat stat stat stat stat disk io disk io
<harlowja> its pretty dinky in all reality
 * larsks cries every time he sees another project re-implement the same basic features.
<harlowja> stevedore is a light layer over the python entrypoints stuff
<harlowja> so either use stevedore, or i just use entrypoints like stevedore is doing
<harlowja> cloud-init right now has its custom thing
<smoser> i dont know.
<harlowja> https://github.com/openstack/stevedore/blob/master/stevedore/extension.py#L147
<harlowja> thats really the 'thing' that does this, lol
<smoser> i'm very much feeling pain of startup performance right now.
<harlowja> 'eps = list(pkg_resources.iter_entry_points(namespace))'
<harlowja> mainly that, ha
<smoser> and also feel pain of external dependencies.
<harlowja> the rest of stevedore is just lue
<harlowja> *glue
<harlowja> so i'm in the middle between larsks and smoser
<harlowja> in that if i could use something like entrypoints
<larsks> An entrypoints-based internal implementation would I guess be fine.
<harlowja> then i can get rid of this complexity :-P
<harlowja> larsks ya, i can easily make something dinky using entrypoints
<harlowja> its not really that hard, ha
<harlowja> https://github.com/openstack/stevedore/blob/master/stevedore/extension.py#L153 is the loop, and the other code is the finding
<harlowja> not magical crazy stuff, ha
<harlowja> which is why stevedore is a dinky layer
<larsks> Yeah.  also, pkg_resources.iter_entry_points may have most of what we need...
<harlowja> yup
<smoser> so 2 thigns i'd like to support out of tree
<smoser> a.) config modules
<smoser> b.) datasources
<smoser> and thats easy enough to "find" them.
<smoser> but getting them into the right order is somethign we dont have. no way to declare a dependency or something.
<smoser> so at the momemnt a list of datasources and a list of modules at each stage.
<smoser> harlowja, so back to your version / thing...
<smoser> what were you saying ?
<larsks> smoser: well, config modules get an ordering from cloud.cfg.
<smoser> does using that version with a git cause issues for you?
<harlowja> smoser i forget
<smoser> larsks, ah. yeah, but not if they're just laid down on disk.
<smoser> ie, ideally you just put new modules in some directory and they get found
<smoser> and datasources too
<harlowja> someway of making sure that a godaddy built cloud-init.rpm doesn't use the same stuff (Version...) as a public cloud-init.rpm
<smoser> harlowja, we could expose branch
<larsks> harlowja: add a suffix to the package name (cloud-init-godaddy) and make it obsolete the other one?
<smoser> in the spec
<harlowja> larsks ah, that might work to
<harlowja> until we have this stevedore and ... stuff
<harlowja> then i can get out of this :-P
<larsks> smoser: looked briefly at your brpm changes, which seem fine, but there is a weird try/except block at the top of brpm that seems to be a noop (it catches any exception...and raises it)
<smoser> larsks, yeah its stupid
<smoser> for flake8
<smoser> it complains about module level imports not at top of filel
<smoser> harlowja, you know of a nicer way to work around that ?
<harlowja> not sure
<larsks> smoser, I don't see that if I remove the try/except block.  no complaints from flake8 (https://asciinema.org/a/bbqseaf84npuoon71hciwvc99)
<larsks> (where flake8 == 2.4.1)
<smoser> how'd you do that larsks ?
<larsks> do which?
<smoser> the asciicinima thing
<larsks> https://asciinema.org/
<larsks> There is a simple 'asciinema' cli tool
<larsks> Looks like you need a ppa for ubuntu.  Seems to be packaged for fedora and debian...
<smoser> well, its really pep8 . pep8 1.7.0 here does not like it.
<larsks> Interesting.  I seem to have 1.5.7.  I guess I would add a comment there, otherwise everyone who looks at the code is going to ask the same question :)
<smoser> yeah
<smoser> deal
<harlowja> hmmm asciinema neat
<smoser> if "avoid-pep8-E402-import-not-top-of-file":
<smoser>   import ...
<smoser> committing that
<smoser> harlowja,
<smoser> https://bugs.launchpad.net/cloud-init/+bug/1603533
<smoser> is fix committed ?
<harlowja> i think so
<harlowja> assuming its still carried forward with the git stuff (and not lost)
<smoser> hey all. i have some changes here locally.
<smoser> i'm going to mark a 0.7.7 release soon. tonight or tomorrow.
<smoser> and call it released.
<smoser> and then we can work towards a quick 0.7.8 iteration on that.
<smoser> the git stuff now in i think.
<JayF> smoser: http://lists.openstack.org/pipermail/openstack-dev/2016-August/101007.html is something you might find interesting
<harlowja> cool
<tz__> anyone have a snippet that shows how to pull a repo from github?  I have http://pastie.org/private/kpnunfkgktxxse2m4xwsga , but problems ensue
#cloud-init 2017-07-31
<smoser> o/
<smoser> o/
<smoser> hello.
<smoser> we'll have a quick meeting here and then just hang out for office hours.
<smoser> https://public.etherpad-mozilla.org/p/cloud-init-meeting
<rharper> o/
<smoser> so the goal of this meeting is just kind of an update / status on whats been happing
<smoser> happening
<smoser> #     Recent Changes / Highlights
<smoser>     COPR builds: https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init/
<smoser> we have these COPR builds up, that repo gets nightly from trunk which is great for anyone wanting to test.
<smoser> if you're using centos/rhel and seeing bugs, we'll probably ask  you to reproduce with trunk also to see if it has been fixed.
<smoser> - systemd unit files are now templates, allowing for upstream having different units per distro.
<smoser> that allows us to have systemd files in trunk that are rendered differently for each distro (rendered in package build time).
<smoser> and we do currently have some differences for centos from what is in ubuntu
<smoser> -     many sysconfig network rendering improvements
<smoser> rharper did most of those, but sysconfig rendering of network configuration is *much* improved recently.
<smoser> see commit messages for bug numbers and such
<smoser> -     python3.6 fixes
<smoser> Ubuntu 17.10 recently moved to python3.6 by default, so our tests in that environment were running with python3.6
<smoser> there were a couple small fixes... mostly they were test cases that were making assumptions.
<smoser> there was 1 fix though, in the use of os.path.join() where we passed a None. that worked in 3.5 but throws valueerror in 3.6
<smoser> - Scaleway datasource added [Julien Castets]
<smoser> that is all thanks to niluje and his great patience.
<smoser> thats all the highlights i had.
<smoser> #     In Progress Development / Highlights
<smoser>     Merge Proposals: https://code.launchpad.net/%7Ecloud-init-dev/cloud-init/+git/cloud-init/+ref/master/+activereviews
<smoser>     Trello Board: [https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin]
<smoser>     Bugs: https://bugs.launchpad.net/cloud-init
<smoser>     AWS ipv6 support: http://pad.lv/1639030
<ubot5> Launchpad bug 1639030 in cloud-init (Ubuntu) "Configure networking based on EC2 metadata source" [Medium,Confirmed]
<smoser>     Opensuse builds: https://build.opensuse.org/package/show/Cloud:Tools:Next/cloud-init
<smoser>     upcoming Stable Release Update (SRU) for Ubuntu 16.04, 17.04.
<smoser> blackboxsw has been working on AWS support for ipv6 and has done a good job there. that branch is looking like it will land this week.
<smoser> the end result will be that on places on AWS that have ipv6 support, you'll get IPV6 configuration by default.
<smoser> then lastly, i've just started at doing another stable release update for Ubuntu. theres a long list of changes since the last.
<smoser> thats it for things...
<smoser> #    Open Discussion
<smoser> # Office Hours
<smoser> i'll be here, and so is rharper so if you have any questions... feel free to just ask.
<smoser> if you have a MP or something that looks like it was ignored.. or a bug you think needs fixing, please feel free to ask (and do so any time).
#cloud-init 2017-08-01
<larsks> smoser: fyi, this just merged in openstack nova: https://review.openstack.org/#/c/467699/
<larsks> That has implications for how cloud-init handles nameservers from openstack data sources.
<smoser> larsks, thanks for the heads up.. .reading.
<smoser> interesting.
<smoser> fwiw, no linux networking configuration system that i'im aware of actually does dns servers specific to interfaces very well.
<larsks> I dunno, my fedora environment seems to handle it pretty well (I have vpn-specific nameservers that are only used when the vpn is up).
<larsks> Although I'm using a local dnsmasq proxy, which means that nothing ever touches /etc/resolv.conf.
<smoser> and you send queries for *.<internal> and network/prefix correctly ?
<larsks> Definitely the former (all *.redhat.com stuff is handled by the vpn nameservers).  Haven't checked the latter, honestly.
<smoser> i suspect you're getting all stuff sent there :)
<smoser> and yeah, network manager does a *reasonable* job.
<larsks> No, only *.redhat.com goes to the vpn nameservers (as verified w/ tcpdump).
<larsks> Requests for other domains go to my local router.
<smoser> nice.
<smoser> note... in the bug there.
<larsks> This works via the dnsmasq --server option ("If one or more optional domains are given, that server is used only for those domains...")
<smoser> it does not show any domain wildcards
<smoser> right?
<smoser> is there information that would say "send *.redhat.com queries here"
<larsks> smoser: in my case, it's part of the vpn configuration.  The bigger use case w/r/t cloud-init is simply the fact that nameservers that are only reachable via a particular interface can be removed from the resolver configuration when the interface goes down, preventing hangs and other related behavior.
<smoser> sure.
<smoser> but if both are up?
<larsks> Then it doesn't matter.
<smoser> yes it does
<smoser> if one has an answer and the other doesnt
<smoser> or conflicting answers
<smoser> i think cloud-init will handle this fine... have you tried ?
<smoser> can you give me an example of the network_config.json ?
<larsks> cloud-init modifies only the global /etc/resolv.conf and does not populate dns information in the interface config files.
<larsks> Because until this change merged, that information simply wasn't available.,
<larsks> There was no way to configure it.
<larsks> We only received global information from openstack.
<smoser> right... ok. it does do that for ENI (puts dns name servers on the interface that htey camne from)
<smoser> do you have an example ?
<smoser> and i guess best thing to do is just make a cloud-init task for that bug
<smoser> i can open that, unless you disagree.
<larsks> Not handy, but the change is minimal: in addition to the global "services" section, there would be a per-network "services" section as well.
<larsks> Yeah, I was thinking of opening a cloud-init bug for it.  So go ahead.
<larsks> I can spin up an environment running the latest nova and produce an example for a bug sometime this week.
<smoser> larsks, just to clarify, it almost certainly does matter which queries go to which server.
<smoser> if you have all the same information in all the dns servers, then it seems like additional services with no value.
<smoser> i realize there are some cases... but it seems that scoping those queries is pretty quickly goign to be important, so you can do things like *.redhat.com -> $REDHAT_DNS.
<larsks> smoser:  assuming that all dns servers are equally able to answer questions, the value is that if you have dns servers that are only reachable via one interface, you can remove them from the resolver configuration if they are unreachable because the interface is down.
<smoser> yeah.
<larsks> I don't think we recieve sufficient information to make domain related decisions.
<smoser> we dont' in the provided network information for sure
<smoser> thats what i'm saying :)
<larsks> That's not even possible to configure in openstack.
<larsks> So this is primarily an issue of reachability, and that is the value.
<smoser> larsks, http://paste.ubuntu.com/25220242/
<smoser> that adds a unit test that i think represents what you're change did.
<larsks> Right, like that.
<smoser> so if you're intereted in telling me what that *hsould* represent in sysconfig ... .patches are quite welcome :)
<larsks> Yup, I figured :)
<smoser> larsks, when you do make that change i suspect you'll break some of the other unit tests cases even test_small_config
<smoser> as that has another network config format that does have dns entries per subnet
<larsks> I think that's okay. I mean, both need to work.
<smoser> right
<smoser> you'll just have to fix the expected output of those tests too
<smoser> larsks, hm..
<smoser> wondering how sysconfig handles this
<rharper> there is per ifcfg DNS_ values
<smoser> https://serverfault.com/questions/423882/configure-a-dns-server-per-nic-interface-eth0-eth1
<rharper> someone filed a bug about getting those in
<rharper> DNS{1,2,3}
<smoser> well, we write IPV4DNS0 and IPV4DNS1
<smoser> but that is per nic
<smoser> not per address
<rharper> it would need the index
<rharper> just like there's IPADDR1, 2, 3
<smoser> the index of what ?
<larsks> smoser: right now, we don't write per-nic dns servers (because they weren't available).
<rharper> address index
<smoser> i dont think it works like that :)
<larsks> That's what this change enabled.
<rharper> it does
<smoser> larsks, right. openstack now provides per *address* dns information
<smoser> (i think)
<smoser> thats what i put in that patch at least.
<smoser> ie, the 'services' are now on a 'network' entry
<smoser> and multiple networks per nic
<larsks> So pick 3, and have it.
<smoser> but the best i can tell is that sysconfig supports only per NIC dns information.
<larsks> Right.
<larsks> DNS config is per NIC.  And the most common config I've seen with openstack is one network/one nic, but sure, you can have multiple ones attached.
<larsks> The assumption there would be that they are all equally valid.
<larsks> I don't think we receive information that would permit us to do per-prefix or per-domain routing of dns requests.
<smoser> well, i guess the thing that makes this ok is that sysconfig only really does things on a per NIC basis anyway
<larsks> Right.
<smoser> rather than a per-address-per-nic
<smoser> so openstack is providing information that technically could represent a superset of what sysconfig can support
<smoser> larsks, http://paste.ubuntu.com/25220399/ . i think that includes the change for openstack.py that would be needed. but need changes in the sysconfig rendere to write the DNS* fields.
<smoser> rather than rendering resolv.conf
<powersj> rharper: All the errors from last night point to epel mirror issues. Was there another failure that I missed since you mentioned copr? I want to make sure I didn't miss a failure.
<powersj> "Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again"
<powersj> from all 3 failures
<smoser> powersj, i saw that too
<smoser> i dont know.
<smoser> its hard to believe that epel would be unreliable
<smoser> even if copr was.
<powersj> Right, re-running just now also resulted in failure
<smoser> rharper suggested we can turn off the yum fastestmirror thing possibly to improve
<powersj> so makes me wonder if something changed
<powersj> I agree with that suggestion
<powersj> as you saw we get very, very odd mirrors (e.g. japan?!)
 * blackboxsw pulls his head out of the AWS-init-local sand and gets onto SRU bug template writeups. The 2nd part (2 of 3) IPv6 AWS branches is up for review https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/328241. 
<rharper> powersj: there was one related to the copr repo timing out
<powersj> hmm I didn't see that last night
<rharper> Timeout on https://copr-be.cloud.fedoraproject.org/results/@cloud-init/el-stable/epel-7-x86_64/repodata/repomd.xml: (28, 'Connection timed out after 30001 milliseconds')
<rharper> disabling fastmirror won't help there
<rharper> I think copr might have been down at the time
<powersj> oh.. I was looking at jenkins runs, we don't have one for that repo
<rharper> right
<powersj> sorry I heard "runs" and immediately jump to jenkins
<rharper> no worries
<rharper> I don't think we have much choice but to add some retries in there
<blackboxsw> smoser: so how do we work this SRU for jsonschema optional dependency,  the upstream comit still contains the dependency defined https://git.launchpad.net/cloud-init/diff/requirements.txt?id=0a448dd034   But, I don't see a comparable filter defined in bddeb to prevent that python-jsonschema dependency from being added to the deb-requires.
<blackboxsw> don't we need to prevent jsonschema from 'leaking' into package deps on xenial/zesty
<rharper> blackboxsw: IIRC, the downstream packaging branch (ubuntu/xenial) will not update the debian/control file with the deps
<rharper> or applies a patch that disabled it (removes it from a lest); but it will essentially always care some "Delta" from master
<rharper> s/a lest/the list
<smoser> blackboxsw, you're right.
<smoser> that is applied to the ubuntu packaging branches at
<smoser>  debian/patches/stable-release-no-jsonschema-dep.patch
<smoser> ah. and look at that... i actually even referenced it in the debian changelog ;)
<smoser> (i figured i'd missed it)
<smoser>   * debian/patches/stable-release-no-jsonschema-dep.patch:
<smoser>     add patch to remove optional dependency on jsonschema.
<smoser> blackboxsw, for bug 1701097, bug 1705147 and bug 1706752 i suggest we verify using tools/net-convert.py
<ubot5> bug 1701097 in cloud-init (Ubuntu Zesty) "eni rendering of ipv6 gateways fails" [Medium,Confirmed] https://launchpad.net/bugs/1701097
<ubot5> bug 1705147 in cloud-init (Ubuntu Zesty) "cloud-init interface renaming should apply .lower() to mac_address values to match sysfs entries" [Medium,Confirmed] https://launchpad.net/bugs/1705147
<ubot5> bug 1706752 in cloud-init (Ubuntu Zesty) "eni rendering broken for bridge params that require repeated key for values" [Medium,Confirmed] https://launchpad.net/bugs/1706752
<smoser> similar template to
<smoser>  https://bugs.launchpad.net/cloud-init/+bug/1684349
<ubot5> Ubuntu bug 1684349 in cloud-init " mask2cidr error with integer value - argument of type 'int' is not iterable" [Medium,Fix committed]
<smoser> we just have to come up with appropriate network config that would show these changes.
<rharper> smoser: blackboxsw: most of the network configs that show it are taken from curtin's vmtests
<rharper> I think many of the bugs include an example config that's broken as well
<blackboxsw> smoser: thanks for confirmation on the patch preventing jsonschema dep.
<blackboxsw> and agreed on using  net_convert
#cloud-init 2017-08-02
<Neepu> Anyone familiar with the Windows port of cloud-init?
<rharper> https://bugs.launchpad.net/cloud-init/+bug/1708255
<ubot5> Ubuntu bug 1708255 in cloud-init "empty or invalid network config dictionaries are not handled well" [Undecided,New]
#cloud-init 2017-08-03
<mbwe> Morning everybody
<smoser> good morning mbwe (or afternoon for you now)
<smoser> powersj, how'd you do the ascii video things ?
<smoser> what'd you use?
<smoser> ascii cinema ?
<powersj> smoser: apt install asciinema
<powersj> make an account
<powersj> and run 'asciinema rec -t "title name"'
<powersj> do whatever, then type exit or Ctrl-D
<smoser> k
<rharper> smoser: I think bug 1672833  is fixed-release now;   after some trial/error with getting user-data into the instance correctly (This is the file I attached in the ec2 console http://paste.ubuntu.com/25234630/)  The i3.large launched and formats/mounts the nvme device just fine.
<ubot5> bug 1672833 in cloud-init "AWS NVME SSD (i3 family) ephemeral storage fails to mount via fs_setup" [Medium,Confirmed] https://launchpad.net/bugs/1672833
<rharper> I suspect the rework on the fs_setup/mount earlier this year resolved the issue
<smoser> rharper, hm..
<smoser> can you let me into your instance ?
<rharper> yes
<smoser> it is quite possible, even probabable that the funny device name/partition delimiter 'p' caused that
<smoser> and now is fixed.
<rharper> yes
<rharper> I think that might have been it
<rharper> following the code path, the nvme devices don't have a "special name" so the translator returns None, it's not a metadata disk, so it should just work as a device; my testing in the unittests showed that; so I was wondering if there was some sort of hotplug or something else going on
<rharper> that user-data in my paste works fine in a ConfigDrive under KVM with an nvme device, so I tested the i3 and it just works as well
<smoser> rharper, right. so that works but you had to know that the ephemeral disk was nvme
<smoser> you can refer to devices as 'ephemeral0'
<rharper> well, it's not clear in the bug what's not working then
<rharper> the config in the bug works (if you append the #cloud-config when adding it via the console) on the latest images
<rharper> smoser: are you suggesting if they replaced '/dev/nvme0' with 'ephemeral0' that should work ?
<smoser> http://paste.ubuntu.com/25234714/
<smoser> right
<smoser> and iv'e wanted to add support for a user to name aliases like that.
<smoser> so the user coudl then just declare that "ephemeral0" is "/dev/nvme0" or whatever
<smoser> and override any thing provided by the md
<rharper> well, the metadata says ephemeral0 is sdb (which I think this was the bug in the metadata you were alluding to)
<smoser> thus making your example user-data more easily modified to work with sdb or nvme or whatever
<smoser> right
<rharper> I see, ideally, you can write a more common user-data
<rharper> and if you launch without an nvme it would still work
<rharper> hrm, I'd like to get clarity; it's not clear from the bug that the submitter intended that
<smoser> yeah, definitely he submitted it with nvme
<smoser> as the path
<smoser> so uyeah
<smoser> blackboxsw, were you reviewing https://code.launchpad.net/~msaikia/cloud-init/+git/cloud-init/+merge/322991 ?
<blackboxsw> smoser: I need to close out on that. just got through the fisrt one. will wrap that and rharper's other branch up today
<smoser> blackboxsw, http://paste.ubuntu.com/25235459/
<smoser> awesome.
<smoser> i failed to execute your script file, but i did *my* job, so exit value 0
<blackboxsw> rharper: addressed most of your review comments on https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/328241 just pushed
<rharper> k
<smoser> i have some in flight there
<rharper> blackboxsw: what about the find first nic with an address
<rharper> smoser: interested in your thoughts on that one too
<rharper> blackboxsw: smoser: also I had a question about changing the metadata date string;  what's the impact there ?  is it SRU'able?
<rharper> I suppose it should be similar to how OpenStack has versioned metadata strings; but we keep those around IIRC
<blackboxsw> rharper: yeah was pondering that comment. so we are naturally sorting that list of interface names before looking for a valid address.
<rharper> blackboxsw: right, but the interface may or may not have an address
<rharper> ideally we filter for all interface with address, then sort
<rharper> maybe I'm being overly pedantic here; but I sortof would want similar heuristics used to select the "right" interface that we pick for fallback networking
<blackboxsw> rharper: I don't follow how that's different?
<blackboxsw> if only eth2 and eth4 have a mac address, how would that sort logic change if we filter out by interfaces with mac's first  or during our loop
<blackboxsw> you are thinking a kernel might bring up a mac address mid-loop?
<rharper> well, that's possible, but that's not waht I was thinking;  I was thinking ip address, not mac;   the mac should be stable
<rharper> but hotplug is an edge case
<rharper> we already have that gap in fallback networking as well (if the "right" one didn't show up till after we've tried to generate a network config)
<blackboxsw> right. yeah I call bad sysfs filename "address" file == mac-address
<rharper> does the fallback code which picks an interface to configure; does that achieve what you want (find me an itnerface to dhcp on)and if so? can we reuse that
<rharper> I think it does (prefers certain names and other heuristics
<rharper> I feel it woudl be good to pick in the same way
<rharper> ah, it's not callback (buried in generate_fallback_config)
<smoser> just hit 'submit' on my comments.
<smoser> rharper, it is a *mac* address.
<smoser> not an ip address.
<rharper> yeah, I realize that now
<rharper> but I'd like to reuse the interface name choice code
<rharper> it was partially lifted from generate_fallback_config; if we made that a stand-along function, it could get reused
<smoser> yeah, there is a lot of duplicate-ish code there
<smoser> a function for 'mac_address(nic)' would have made it clearer too :)
<rharper> right, 75% of generate_fallback_config is "pick the best interface"
<smoser> https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/328542
<smoser> we do have to do something about the api version though. we can't just straight up change that.
<smoser> the thing that i had talked with blackboxsw about was to read the list of available api versions
<smoser> and sect a newer one if we have one.
<smoser> i think we probably have to do something like:
<smoser>  read the old version instance-id
<smoser>  try:
<smoser>    see if "%s/" returns a list of YYYY-MM-DD
<smoser>     if so, check for newer version, and set api_ver = that
<smoser>   except E:
<smoser>    that didnt work, use the old api_version
<smoser> that way we use the old version unless the md tells us they have the newer version available (and amazon's does)
<smoser> if theres a lookalike that doesnt, then they should fix their md service.
<rharper> smoser: do we do that datestring check in openstack ?
<smoser> we do something like it....
<smoser> _find_workign_version
<rharper> yeah, looking at that now
<blackboxsw> +1 on the old/new API version support. I had forgotten about that in the mix. right so we need a fallback to 2009
<rharper> right
<rharper> blackboxsw: right; that was my concern
<rharper> it's shame they can't point to the new url in the old metadata
<rharper> that way we don't take a hit on a  possible not-present URL
<rharper> sort of like a union of metadata with a link to the next set
<smoser> well, they kind of do
<smoser> they list the versions that are present
<blackboxsw> shall we only support  a limited # of release metadata versions ?  SUPPORTED_METADATA_VERSIONS = ['2016-09-02', '2009-04-04',]
<blackboxsw> and attempt to hit each in priority order if present?
<rharper> smoser: oh, nice
<smoser> you can enter into http://169.254.169.254/
<smoser> and i think you get a list.
<smoser> thats what openstack does.
<rharper> ok
<smoser> the issue is that i was saying a lookalike might not do that
<smoser> (aws does)
<smoser> and we've never depended on that list before.
<blackboxsw> roger. but even if there is a list of X releases reported, should cloud-init attempt only the known MD versions that we have tested against (and skip any interim versions we may not understand)
<rharper> smoser: oh, bummer
<rharper> that sorta stinks;  so everyone pays for the hit to the new metadata version if it's not there
<blackboxsw> or everyone pays to hit a list endpoint too I guess
<blackboxsw> we could just make it Ec2Local hitting that MD version
<blackboxsw> but, still most lookalikes I'd imagine would also allow for init-local dhclient init. hmm
<smoser> blackboxsw, right. only hit the two that we've known to work.
<smoser> hit/consider
<smoser> rharper, but on amazon the metdata service is fast. the "hit" is not a big deal :)
<smoser> and on amazon also you can always assume any version that your code knows about is there.
<smoser> interesting that the only value in the list is to provide clients on a lookalike a way to check.
<rharper> 'any version your code knows about is there'  how does that work ?
<blackboxsw> approved https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/328542
<blackboxsw> smoser: aws does provide a list from http://169.254.169.254/
<blackboxsw> not sure if that's considered "faster" than attempting  a version dir in the metadata sevice that doesn't exist
<blackboxsw> in dumb human testing w/ time  it looks like the list http://169.254.169.254/ plus string match is about comparable  to a 404 on version not present
<blackboxsw> maybe a little faster to just try the known version and cope w/ the 404
#cloud-init 2017-08-04
<smoser> blackboxsw, if you're still around, we missed a template on bug 1701097
<ubot5> bug 1701097 in cloud-init (Ubuntu Zesty) "eni rendering of ipv6 gateways fails" [Medium,Confirmed] https://launchpad.net/bugs/1701097
<smoser> i'll get it tomorrow if not
<smoser> :-(
<smoser> later
<mbwe> thanks smoser well overhere its now morning
<mbwe> Europe overhere
<smoser> mbwe, you all keep funny time.
<rharper> powersj: how were you getting that negative value in the cloud-init analyze lxc run?  I'm trying to reproduce here
<powersj> rharper: let me look
<rharper> I just forgot that cloud-init.log has subsecond timestamps now, so it's easier to just cat cloud-init.log | cloudinit-analyze show -i -     versus those journalctl ones
<powersj> rharper: Ah! It was when I was doing my demo script, which uses the following user data: https://github.com/powersj/debconf17/blob/master/demo/user_data.yaml which is setting the timezone to something that isn't correct.
<rharper> oh!
<rharper> yes
<powersj> it will be correct in couple days ;)
<rharper> lol
<smoser> powersj, rharper well, analize is doing bda math then
<smoser> or ignoring time zone
<smoser> er... *not* ignoring time zeon.
<smoser> zeon!
<smoser> it should be doing any time operations on utc.
<smoser> or some constant
<rharper> smoser: we get timestamps from python logging in cloud-init.log
<rharper> when you modify the timezone; logging doesn't know that
<smoser> hm...
<smoser> we'd then need to chage theoutput to log with timezone information. i can't imagine that python logging is simply broken across time zone changes
<smoser> surely it can detect that
<rharper> looks like we;'d need to force logging to use gmt
<blackboxsw> rharper: smoser https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/328241 all comments addressed. performing one more test run on aws now
<blackboxsw> validatred on aws
<smoser> blackboxsw, your comment
<smoser> "Perform dhcp discovery on nic if it is not already up."
<smoser> i dont think that is true
<smoser> "if the nic is down, run dhclient"
<smoser> where is "if" ?
<smoser> i think it just runs dhclient
<blackboxsw> ahh fixing that
<blackboxsw> right that was an earlier iteration on this branch
<smoser> blackboxsw, the other comment is the bit in DataSourceEc2.py
<smoser> dhcp_leases = dhcp.maybe_perform_dhcp_discovery()
<smoser> move that out to anoterh method maybe ?
<smoser> ipv4_info = dhcp.maybe_dhcp_and_get_ipv4_info()
<smoser> if not ipv4_info:
<smoser>   return False
<smoser> with net.EphemeralIPv4Network(**ipv4_info):
<smoser>   ...
<blackboxsw> :w
<blackboxsw> whoops mid-fixing it
<smoser> blackboxsw, have a minute ?
<smoser> http://c.brickies.net/hangout
<blackboxsw> sure, pushed latest changes so we are talking about the same thing
<blackboxsw> smoser: fixes pushed testing on aws now
<smoser> there are merge conflicts tehre.
<smoser> you should rebase
<smoser> other than that i think its probably sane
<smoser> https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+ref/feature/analyze
<smoser> and i gots to go. ^ that is the analyze work that i did, thats all though.
<blackboxsw> ok will shepherd it in next week
<blackboxsw> have a good one
<rharper> thanks
<powersj> thanks smoser
<smoser> its no where near done.. just initil thing.
#cloud-init 2018-07-30
<smoser> powersj: https://jenkins.ubuntu.com/server/view/cloud-init,%20curtin,%20streams/job/cloud-init-integration-lxd-c/
<smoser> how often / how does that run ?
<smoser> it looks like it just stopped running on the 20th
<powersj> lxd is daily
<powersj> hmmm
<powersj> hmm xenial is running daily, but not set to kick the bionic test
<powersj> doh: set to kick project: cloud-init-integration-lxd-a
<smoser> hm..
<powersj> fixed
<[42]> can you point me somewhere for getting started creating a debian template image to be used with qemu (proxmox)?
<blackboxsw> smoser, time to reset cloud-init status meeting date. it doesn't look like we had one since 07/02
<blackboxsw> and it looks like someone kicked off a meeting start a couple days ago by accident
<blackboxsw> https://ubottu.com/meetingology/logs/cloud-init/2018/cloud-init.2018-07-23-16.05.log.txt
<blackboxsw> #endmeeting
<meetingology> Meeting ended Mon Jul 30 16:34:15 2018 UTC.
<meetingology> Minutes:        http://ubottu.com/meetingology/logs/cloud-init/2018/cloud-init.2018-07-23-16.05.moin.txt
<blackboxsw> smoser: think we should host the status meeting next monday?
<smoser> blackboxsw: that sfine with me...
<smoser> (long delayed)
<blackboxsw> lotta vacation
* blackboxsw changed the topic of #cloud-init to: Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting: Monday 8/6 16:00 UTC | cloud-init 18.3 released (06/20/2018)
<TJ-> I'm trying to diagnose why a vagrant ubuntu/bionic64 box doesn't appear to be getting its network configured. As I understand it cloud-init should be doing that on first boot. Are there docs that'll help me figure it out?
<blackboxsw> TJ http://cloudinit.readthedocs.io/en/latest/topics/network-config.html should be an overview of what cloud-init does
<blackboxsw> generally speaking if you are on cloud-init version 17.2 or greater you can run "cloud-init status --long" on the commandline if you can access the serial console of the box to see if there are errors
<blackboxsw> otherwise look in  /var/log/cloud-init.log  for Tracebacks
<blackboxsw> on bionic you should have 18.2 or later cloud-init so should be ok on the version
<TJ-> Thanks; it seems there's no "network:" defined. I presume that is something to do with the vagrant box
<TJ-> no log whatsoever; systemctl status shows all cloud-* services dead
<blackboxsw> hrm, I would've expected "systemctl list-dependencies" to list all cloud-init jobs/units in green status unless dependency chains were broken by some other units/services that are running in the environment
<blackboxsw> root@myb1:~# systemctl list-dependencies | grep cloud
<blackboxsw> â   ââcloud-init.target
<blackboxsw> â   â ââcloud-config.service
<blackboxsw> â   â ââcloud-final.service
<blackboxsw> â   â ââcloud-init-local.service
<blackboxsw> â   â ââcloud-init.service
<blackboxsw> if dependency chains were broken due to ordering cycles, you'd see "Breaking ordering cycle" from "journalctl -b"   which may tell you if cloud-init was not run or disabled by the booting system
<blackboxsw> sudo cloud-init collect-logs   will collect a tarfile of cloud-init logs that you could post to me if you'd like me to peek at what may be the symptom. but generally "cloud-init status --long" of look in /var/log/cloud-init.log for any Tracebacks
<TJ-> I'm not sure that vagrant relies on cloud-init - lack of authoritative docs though. I've just read an overview says vagrant box /should/ be configured to use DHCP. My situation is even the vagrant management interface isn't UP, let alone running dhclient to get an IP address.
<blackboxsw> so artful/bionic should have switched from using /etc/network/interfaces to /etc/netplan. I'd expect you'd see /etc/netplan/50-cloud-init.yaml if cloud-init was in charge of your networking.
<blackboxsw> if your vagrant image contains network configuration in /etc/network/interfaces(.d) it's likely misconfigured
<blackboxsw> netplan *should* be relying on networkd to do all your dhclient work from artful/bionic or later
<TJ-> right, no netplan config, nothing under /run/systemd/network/ or whatever
<TJ-> This is the ubuntu/bionic64 vagrant box , so very weird
<TJ-> OK, I've solved it! The problem is vagrant-mutate. The Ubuntu boxes are for VirtualBox and have 2 vmdk disk images, 1 of which is  cloudimage-configdrive. vagrant-mutate only copies over the root-fs so there's no config data for cloud-init
<TJ-> looks like vagrant-mutate is no longer maintained by its author either. I'll have to fork and see if I can get a handle on the code
<TJ-> Thanks for the pointers; helped me reduce the problem space considerably
<arthurl> hi guys- I'm trying to pass a cloud-init.yml to packer to build a custom ami in aws ec2- the ami is being built but i see none of the cloud-init actually gets run...
<blackboxsw> thanks for the write back TJ
#cloud-init 2018-07-31
<smoser> arthurl: can you say waht the OS is ? and what (if any) data is in /run/cloud-init/
<smoser> and then run 'cloud-init collect-logs' and open a bug.
<smoser> https://bugs.launchpad.net/cloud-init/+filebug
<arthurl>  @smoser thanks for that- let me give that a try
<arthurl> in the result.kson I see 1 failure in 1 attempted commands- how can I determine what that actual failure was?
<blackboxsw> arthurl: cloud-init status --long? may work. otherwise you can look for the  "Traceback" in /var/log/cloud-init.log to see the whole trace
<blackboxsw> arthurl: additionally, if it's a user script that had caused the error you'll see any stderr emitted in /var/log/cloud-init-output.log at that failure timestamp
<arthurl> @blackboxsw awesome- I found what I was looking for in cloud-init-output.log but cloud-init status --long seems useful too- thank you!
<blackboxsw> good deal arthurl
<blackboxsw> smoser: rharper sru-regression fix https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/351920
<smoser> reviewd, marked. go lander.
<blackboxsw> thx for the review. lander won it :)
<blackboxsw> s/won/owned/
<blackboxsw> smoser: upload MP up for review in cosmic https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/351925
<rharper> blackboxsw: +1
<smoser> blackboxsw: i'll land and upload shortly
<smoser> thanks
<blackboxsw> cherry pick up for bionic, artful and xenial to come  https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/351926
<blackboxsw> I probably should go through the final upload steps for cosmic on our next run past today's we'll sort the SRU first to get things queued
<smoser> blackboxsw: sure.
<blackboxsw> I'm expecting I'll hit some missing dev environment dependency or something
<smoser> i'm
<smoser> blackboxsw: http://paste.ubuntu.com/p/KNbzjPD3Xd/
<smoser> is my 'build-and-push' that i do.
<smoser> the 'build-package' you have
<smoser> sbuild-it i suspect you dont, but that is basically just doing a binary package build tomake sure it builds.
<smoser> http://smoser.brickies.net/git/?p=tildabin.git;a=blob;f=sbuild-it
<blackboxsw> ok looks like I have perms https://bugs.launchpad.net/ubuntu-community/+bug/1784421 or at least the bug went through
<ubot5> Ubuntu bug 1784421 in ubuntu-community "[TB/DMB] PPU additions for chad.smith" [Undecided,Fix released]
<blackboxsw> smoser: looks like you merged my bionic branch true?
<blackboxsw> or was that the lander?
<blackboxsw> artful up: https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/351937
<blackboxsw> xenial up: https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/351938
<blackboxsw> looks like bionic already got queued to unapproved status.... ok just artful and xenial to go
<blackboxsw> I'm going to test this built package over on fginther's oracle instance
<blackboxsw> smoser: I just validated on live Oracle instance clean and dirty reboots w/ bionic and the package built from our upstream ubuntu/bionic  branch.
<blackboxsw> all good there. thanks
<blackboxsw> once artful/xenial are approved/merged I'll ping in ubuntu-release
<smoser> blackboxsw: its on its way up. building here and uploading.
<blackboxsw> smoser: I'm still missing artful it looks like
<blackboxsw> ahh
<blackboxsw> https://meta.askubuntu.com/questions/18032/eol-notice-artful-aardvark-17-10-reached-end-of-life-on-july-19-2018
<blackboxsw> just have to wait long enough on an SRU to avoid updating
#cloud-init 2018-08-01
<nb16> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<nb16> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<nb16> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<nb16> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Tycale6> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Tycale6> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Tycale6> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Tycale6> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Guest90850> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Guest90850> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Guest90850> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Guest90850> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Cprossu0> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Cprossu0> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Cprossu0> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Cprossu0> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<AlwaysHigh19> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<AlwaysHigh19> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<AlwaysHigh19> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<AlwaysHigh19> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<eNigmaFx5> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<eNigmaFx5> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<eNigmaFx5> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<eNigmaFx5> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<theWhisper_17> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<theWhisper_17> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<theWhisper_17> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<theWhisper_17> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<justyns> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<justyns> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<justyns> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<justyns> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<circle> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<circle> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<circle> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<circle> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<NSCLRP-1> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<NSCLRP-1> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<NSCLRP-1> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<NSCLRP-1> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<CrunchyChewie22> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<CrunchyChewie22> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<CrunchyChewie22> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<CrunchyChewie22> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<christophegx> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<christophegx> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<christophegx> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<christophegx> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<insidious24> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<insidious24> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<insidious24> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<insidious24> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<morsik5> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<morsik5> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<morsik5> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<morsik5> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<L0S> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<L0S> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<L0S> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<L0S> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<GodSkinS> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<GodSkinS> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<GodSkinS> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<GodSkinS> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Guest2162> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Guest2162> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Guest2162> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Guest2162> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<lbft26> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<lbft26> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<lbft26> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<lbft26> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Xe12> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Xe12> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Xe12> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Xe12> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Chords> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Chords> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Chords> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Chords> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<acuzio24> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<acuzio24> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<acuzio24> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<acuzio24> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Oats87> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Oats87> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Oats87> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Oats87> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<em> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<em> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<em> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<em> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<mort3> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<mort3> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<eth218> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<eth218> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<eth218> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<eth218> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<lutoma4> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<lutoma4> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<lutoma4> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<lutoma4> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<l4z4i> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<l4z4i> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<l4z4i> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<l4z4i> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<em> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<em> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<em> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<em> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Guest37268> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Guest37268> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Guest37268> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Guest37268> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<pwillard29> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<pwillard29> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<pwillard29> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<pwillard29> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<pixdamix> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<pixdamix> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<pixdamix> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<pixdamix> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<ldunn15> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<ldunn15> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<ldunn15> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<ldunn15> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<jimby8> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<jimby8> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<jimby8> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<jimby8> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<nosbig14> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<nosbig14> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<nosbig14> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<nosbig14> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<AlexZ0> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<AlexZ0> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<AlexZ0> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<AlexZ0> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<ketas26> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<ketas26> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<ketas26> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<ketas26> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Sove> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Sove> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Sove> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Sove> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Natechip> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Natechip> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Natechip> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<nandub> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<nandub> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<nandub> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<nandub> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<ajvpot13> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<ajvpot13> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<ajvpot13> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<ajvpot13> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<blacksyke28> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<blacksyke28> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<blacksyke28> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<blacksyke28> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Dworf> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Dworf> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Dworf> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<ktechmidas> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<ktechmidas> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<ktechmidas> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<ktechmidas> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<YuGiOhJCJ> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<YuGiOhJCJ> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<YuGiOhJCJ> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<ski_> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<ski_> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<ski_> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<ski_> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<bigpresh3> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<bigpresh3> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<bigpresh3> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<bigpresh3> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<ugjka20> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<ugjka20> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<ugjka20> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<ugjka20> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<thevdude0> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<thevdude0> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<thevdude0> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<thevdude0> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<programmerq16> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<programmerq16> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<programmerq16> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<programmerq16> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<raynold> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<raynold> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<raynold> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Adbray29> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Adbray29> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<berndj23> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<berndj23> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<berndj23> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<berndj23> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<Typhon4> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<Typhon4> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<Typhon4> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<Typhon4> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<sjums> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<sjums> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<sjums> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<sjums> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<smoser> wonder why this channel gets spammed
<vectr0n> maybe because it starts w/ c it would be near the top of a list dump?
<powersj> I did ping a freenode admin about it
<vectr0n> might be worth while putting the chan +r till it calms down?
<smoser> vectr0n: yeah, maybe.
<smoser> but #curtin didn't get it
<vectr0n> who knows lol
<cwre> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<cwre> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<cwre> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<cwre> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<rharper> smoser: should we set registered user mode only ?
<rharper> the freenode operators suggested we +r the channel
<smoser> i dont know. i'm not opposed to it. but it is a pain for someone who just wants to join.
<richvdh25> With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/
<richvdh25> I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/
<richvdh25> Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate
<richvdh25> A fascinating blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/
<rharper> smoser: any of the ops can do it; so I'm fine with someone turning it on;  I can also ignore the spam too
<smoser> might as well turn it on for now i guess.
#cloud-init 2018-08-02
<blackboxsw> smoser: quick review on https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/352010
<smoser> thank you blackboxsw
<smoser> blackboxsw: that is a good comment
#cloud-init 2018-08-03
<smoser> rharper or blackboxsw https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/352027 would be nice. should seem non-contentios
<rharper> smoser: did you already confirm that the tab-completion still works for devel now that we have more than one element there ?
<rharper> smoser: wanted to make sure the bash-compeletion didn't need updating (it shouldn't it should dump the devel schema, which I think you updated already in that branch)
<smoser> rharper: i did not update bash completion. or look at it at all.
<rharper> smoser: I don't think we need to, but just  wanted to check it out.
<smoser> powersj: https://jenkins.ubuntu.com/server/job/cloud-init-ci/189/console
<smoser> that ran, but did not post to https://code.launchpad.net/~sw37th/cloud-init/+git/cloud-init/+merge/350428
<powersj> line 4: MERGE_REVISION: unbound variable
<powersj> when you launched it you did not provide a candidate_revision
<powersj> https://jenkins.ubuntu.com/server/job/cloud-init-ci/189/parameters/
<smoser> where do you see that ?
<smoser> and fwi, you didn't use to have to provide candidate_revision. it assumed HEAD
<powersj> smoser, I don't recall the job ever not requiring that
<powersj> it is how it keeps track of what revision is has tested
<powersj> I found the unbound variable in the job it kicked off next, the vote job
<smoser> it just defaults to HEAD
<smoser> i'm sure of it.
<smoser> nd why *not* default to head.
<powersj> tests take time, and if a user quickly pushes another fix it could vote on the wrong version
<smoser> well no. it would test *a* version and it would log that and report that it tested that veresion.
<smoser> i'm almost certain that this is why we have this message in there
<smoser> Checking out Revision 3f8f3249643056c751efe11ce27edfe56d9cec3d (refs/remotes/origin/opennebula_fix_null_gateway6)
<blackboxsw> approved https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/352010
<blackboxsw> lander should get that
#cloud-init 2019-07-29
<chillysurfer> the merging doc (https://cloudinit.readthedocs.io/en/latest/topics/merging.html) seems to explicitly talk about merging userdata. is vendordata subject to these same principles? (default merging, ability to specify merge-types in the vendordata cloud-config, etc.)?
<chillysurfer> my testing is hinting at a "no". if userdata has a runcmd(s) and vendordata has runcmd(s), then the vendordata runcmd(s) is/are not run
<chillysurfer> wondering if i could specify `merge_how` in vendordata cloud-config  to change this behavior though
<chillysurfer> actually i think the vendordata section answers my question: "if both vendor and user have provided âruncmdâ then the default merge handler will cause the userâs runcmd to override the one provided by the vendor. To append to âruncmdâ, the user could better provide multipart input with a cloud-config-jsonp part"
<chillysurfer> so the end-user (supplying userdata) would have to be _very_ explicit to allow vendordata to not get overwritten
<Odd_Bloke> chillysurfer: Yes, the intent is that if a user provides a key then it completely overrides the vendor's key.
<Odd_Bloke> (Unless they opt-out of that behaviour, as you noted.)
<chillysurfer> Odd_Bloke: ok cool thanks for the confirmation. and besides the user specifying a cloud-config-jsonp part, there is nothing vendordata could do otherwise, right? can't specify a different merge-type in vendordata?
<Odd_Bloke> chillysurfer: As far as I know, the power lies entirely with the user.
<Odd_Bloke> (By design. :)
<chillysurfer> yep that's what the docs seem to indicate as well :)
<rharper> chillysurfer: I've not explored it yet; but I wanted to see if we could have a place to set the default merge-how values; rather than a per user-data file;   it's awkward to have to have a merge-how in each input file you might use
<rharper> that said, I do think the need to merge vendor-data with user-data usually indicates that what's happening in vendor-data probably should be baked into the image in some other way; for example a stand-alone vendor package integrated into the image;  or event a dynamic vendor config module;
<chillysurfer> rharper: yep makes total sense
<chillysurfer> i have a question and some findings that are way too long for irc. what's the cloud-init mailing list that i could use?
<Odd_Bloke> chillysurfer: It's the mailing list attached to this Launchpad team: https://launchpad.net/~cloud-init
<chillysurfer> Odd_Bloke: perfect thanks!
<AnhVoMSFT> rharper : Sorry it took a while but I pushed an update to https://code.launchpad.net/~vtqanh/cloud-init/+git/cloud-init/+merge/369785
<rharper> AnhVoMSFT: no worries; thanks
#cloud-init 2019-07-30
<Goneri> hi, could you take a look at https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/368507 and https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/365641
<Goneri> I built images that include the two patches: http://bsd-cloud-image.org/
<schlitzer> hello, is there a guide on how to write a custom plugin for cloud-init?
<Odd_Bloke> rharper: Is there a way for a user (or metadata source on an ISO) to change network configuration to happen on every boot?  Or is that only configurable on the DataSource in the code?
<tribaal> Odd_Bloke: vaguely related: I'm trying to do the opposite (modify the list in cloud_config_modules in the conf from our datasource). I'm not sure I understand how it works - what reads that config key? I can change the behavior manually just fine (by playing with conf files), but I don't understand where/how to do the same programatically
<tribaal> I'm not sure another datasource does anything similar (nothing I could find seems to tweak that particular set of entries). But if I knew where it's read it would already help me quite a bit :)
<tribaal> Odd_Bloke: nevermind, enough digging yield the answer - for the record, the solution is that the datasource's self.extra_config is "util.mergemanydicts" 'd with the conf. So adding my config tweak to the datasource's extra_config does what I want. \o/
<Odd_Bloke> tribaal: Great!
<rharper> Odd_Bloke: (updating network-config on each boot is an event type that we've sent in a few datasources) however, in the hotplug branch; I've changes that allow user-data to opt-in to changes for each boot;  that is, for datasources which allow per-boot events; users can opt-in or out of what boot events to respond to;
<Odd_Bloke> Thanks!
<Odd_Bloke> This was for triage, and I wasn't even entirely sure that's what they were asking about, so I left a more general message in the end.
<rharper> for ConfigDrive, or NoCloud; we'd need to update the datasources to allow EventType.BOOT (we default to EventType.BOOT_NEW_INSTANCE); and then user data could add the update_events: config to opt-in to that
<rharper> Odd_Bloke: I think I know which one you're talking about; we have had general questions about allowing an updated iso or configdrive/nocloud to be applied on each boot
<rharper> so once a rebased hotplug branch lands, we can update other datasources to accept EventType.BOOT as well
<Odd_Bloke> Yeah, I think that would be a nice enhancement.
<rharper> effectively, any datasource which has a network_config property could allow users to opt-in
<AnhVoMSFT> hey folks if I have a simple cloud-config with packages/write_files/runcmd, is there anyway to quickly test it on an already-provisioned VM ?
<Odd_Bloke> rharper: https://bugs.launchpad.net/cloud-init/+bug/1834875 is feeling more and more like it's a fundamental enough bug in {kernel,systemd,?} that we can't just workaround it from our perspective.
<ubot5> Ubuntu bug 1834875 in cloud-init "cloud-init growpart race with udev" [Medium,In progress]
 * rharper reads the update
<rharper> Odd_Bloke: can we vary to a previous image which older kernel ?
<rharper> or possible boot generic kernel instead?
<rharper> that might help point to kernel regression
<rharper> I agree it appears to be out of our hands
<Odd_Bloke> rharper: Yeah, though the report is cosmic and later.
<Odd_Bloke> Oh, right, but on Azure that may mean less with the custom kernel.
<Odd_Bloke> rharper: Hmph, cosmic and bionic look to me like they had the same linux-azure version (because HWE), which suggests it isn't the kernel.
<rharper> Odd_Bloke: surely we've had updates to the linux-azure kernel
<rharper> like security fixes or other refreshes?
<rharper> and we can still test a generic kernel to see if it reproduces
<rharper> something underneath the instances may have changed which exposes the issue
<Odd_Bloke> rharper: The issue is reported as present in cosmic and not in bionic.
<Odd_Bloke> And cosmic and bionic were on the same linux-azure until cosmic EOL'd.
<Odd_Bloke> I've looked at the udev diff from bionic to cosmic and it was fairly minimal, though.
<Odd_Bloke> So I'll ask for a linux-generic run and see if that turns up anything interesting.
<rharper> well, what about Disco or E
<rharper> as cosmic os EOL
<Odd_Bloke> My thinking was that as we started seeing this in cosmic, changes that weren't present in cosmic are unlikely to be the cause.
<Odd_Bloke> (I mean, I also haven't ever really looked at the udev source before, so I could very easily be missing an important change that I don't understand.)
<rharper> right, systemd-udev is userspace; but the event order is coming from the kernel
<Odd_Bloke> The kernel event order is the same on failure and passing, I think.
<Odd_Bloke> Let me double check that.
<Odd_Bloke> Yes, the kernel emits (sda, sda1, sda14, sda15) in that order in both cases.
<Odd_Bloke> Uh, sorry, the kernel emits change events for (sda, sda1, sda14, sda15) and then the add/remove for sda14 (from the partx call).
<Odd_Bloke> But the same order on both hosts.
<Odd_Bloke> And udev on the passing instance processes (sda, sda15, sda14, sda14, sda14, sda1), whereas on the failing instance it's (sda, sda15, sda1, sda14, sda14, sda14).
<Odd_Bloke> (In both cases, the three sda14 events are in the order (change, remove, add).)
<Odd_Bloke> So my intuition is that the kernel emits the sda1 event while the partition changes are still in flux somehow, and we fail if udev processes the sda1 event early enough to hit that "in flux" period.
#cloud-init 2019-07-31
<tribaal> rharper: thanks for the review :)
<rharper> tribaal: thanks for the Datasource
<rharper> tribaal: I want Odd_Bloke to look at it as well; final pass but I'd like to land it today or tomorrow
<tribaal> rharper: no worries
<rharper> tribaal: once landed, I think we'll need to fix up the release branches to include the ExoScale in the datasource list in the dpg
<rharper> dpkg
<tribaal> rharper: yep. Nitpick though: Exoscale (no camelcase :P )
<rharper> yes =)
<Odd_Bloke> eXoscaLe
<tribaal> don't make me say "ubantu" out loud please :P
<rharper> lol
<ijohnson> hi folks, I'm looking for any code in cloud-init that might be calling `snap wait system seed.loaded`. I have a server image which without a snap seed my user is created almost instantly in the boot process with the NoCloud datasource. However, if I add the seed, then it takes almost 15 minutes for the user to be created
<Odd_Bloke> ijohnson: Ubuntu systems are configured so that systemd won't run certain stages of cloud-init until all software is installed on the system; there's a known issue with snapd where seeding is very slow, but 15 minutes seems extreme even for that.
<Odd_Bloke> ijohnson: https://bugs.launchpad.net/cloud-init/+bug/1834190 <-- that's the bug, I believe
<ubot5> Ubuntu bug 1834190 in snapd "snapd.seeded.service blocks for a long time preventing cloud-init from completing" [Undecided,New]
<Odd_Bloke> Though, actually, there's definitely one that's older than June this year somewhere.
<Odd_Bloke> ijohnson: Right, it's in systemd/cloud-config.service.tmpl in the cloud-init repo.
<Odd_Bloke> (I'm running to a meeting now, else you'd have got an actual link. ;)
<ijohnson> thanks Odd_Bloke I'll look at the bug
<ijohnson> ah yes that systemd unit is exactly what I was looking for
<ijohnson> I know why it takes 15 minutes to seed all the snaps on my system, what I'm really wondering is how I can workaround that to prevent cloud-init from blocking on snapd seeding since seeding takes so long on this system
<Odd_Bloke> It basically boils down to: cloud-init would regress from pre-snap-seeding behaviour if we didn't wait for all seeded software to be installed before we apply (most) user-provided configuration, so we have to wait.
<Odd_Bloke> I don't know if there's a good way to stop that in a generic way, but you could easily override the unit in your case.
<ijohnson> yes I'm just looking to overwrite this in my particular image
<chillysurfer> i just pushed a package to my launchpad ppa. this was maybe 2 hours ago. dput said success, but i see nothing in my launchpad ppa yet. is there usually a long lag time on this if anybody has any experience with this?
<Odd_Bloke> chillysurfer: If you go to "View package details" (or append /+packages to the URL), do you see anything?
<chillysurfer> Odd_Bloke: "This PPA does not contain any packages yet" :(
<Odd_Bloke> chillysurfer: What was your dput command?
<chillysurfer> dput ppa:trstringer/ppa cloud-init_19.2-6-g96d31049-1~bddeb_amd64.changes
<chillysurfer> Odd_Bloke: what's interesting, though, is my ppa name is "trstringer_ppa"
<chillysurfer> i'm wondering if it should be s/ppa/trstringer_ppa
<Odd_Bloke> Yes, I expect so.
<chillysurfer> Odd_Bloke: but the funny thing is, if i go to my ppa https://launchpad.net/~trstringer/+archive/ubuntu/ppa
<chillysurfer> it says...
<chillysurfer> You can upload packages to this PPA using:
<chillysurfer> dput ppa:trstringer/ppa <source.changes>
<Odd_Bloke> Oh, right, I see what you mean.
<Odd_Bloke> Yeah, the PPA is called "ppa", it has a title of "trstringer_ppa".
<chillysurfer> maybe the "name" isn't part of the dput arg?
<chillysurfer> ahhh
<chillysurfer> got it
<chillysurfer> ugh
<chillysurfer> so my dput was correct then
<Odd_Bloke> Yeah, looks that way to me.
<Odd_Bloke> Let me check if there are any known LP outages.
<chillysurfer> Odd_Bloke: maybe it's worth mentioning, i pushed my pgp key to the key server maybe 10 or 15 min before i dput'd
<chillysurfer> maybe it's a weird timing issue?
<Odd_Bloke> chillysurfer: Nothing is being talked about in the usual places; I'd retry the dput and see what happens.
<Odd_Bloke> Worst case, it just rejects it.
<chillysurfer> Odd_Bloke: because no changes, it doesn't like it
<chillysurfer> Package has already been uploaded to ppa on ppa.launchpad.net
<chillysurfer> Nothing more to do for cloud-init_19.2-6-g96d31049-1~bddeb_amd64.changes
<chillysurfer> Odd_Bloke: weird right?
<Odd_Bloke> Yeah.
<chillysurfer> Odd_Bloke: maybe i should just be patient
<chillysurfer> forcing dput gives me the same result (nothing more to do)
<Odd_Bloke> If I had to take an educated guess about how PPA uploads work, it's that it accepts all of them that are valid and not duplicate, and then another process comes along and actually pops them in the PPA.
<Odd_Bloke> So perhaps that second part is either wedged or is taking a long time to reach you.
<chillysurfer> Odd_Bloke: i would assume the same. i think i should just be patient for the next few hours
<Odd_Bloke> 2 hours would be a long time for that, though.
<chillysurfer> Odd_Bloke: my thoughts exactly
<Odd_Bloke> (I think.)
<Odd_Bloke> Let me ask.
<chillysurfer> Odd_Bloke: thanks so much... my fear is i'm just being impatient and i'm wasting all of our time
 * chillysurfer starts sweating
<Odd_Bloke> Nah, 2 hours is way too long to be patient for that.
<Odd_Bloke> Odds are it either fell in a crack or we're reporting an outage.
<Odd_Bloke> FWIW, I just asked in #launchpad on Freenode, so you could join and follow along if you wanted.
<chillysurfer> ohhhhh nice
<chillysurfer> Odd_Bloke: that really helps
<chillysurfer> i'll just ask there if it doesn't "resolve itself" soon
<Odd_Bloke> I've already asked, so I would counsel patience on asking again. :)
<chillysurfer> Odd_Bloke: haha yeah i'm going to give it until tomorrow
<Odd_Bloke> OK, cool, wasn't sure what timescale "soon" was on. ^_^
<chillysurfer> if nothing shows up tomorrow then maybe i'll ask in #launchpad for some troubleshooting steps or how to do a force push to the ppa
<chillysurfer> Odd_Bloke: i really appreciate the help :)
<chillysurfer> Odd_Bloke: on an unrelated point, a couple of days ago i asked a question on the mailing list (cloud-init@lists.launchpad.net). if my assumption is correct, you're on the cloud-init team so you might be getting mailing list messages? could you just confirm that i did infact send it correctly? the "from" is thstring@microsoft.com :)
<chillysurfer> and a follow up to that is, it seems like non-team members can't subscribe to the mailing list? is that accurate?
<rharper> chillysurfer: lemme check
<chillysurfer> rharper: awesome thank you so much
<chillysurfer> and that's for sure not a nudge for an answer, just want to make sure the question didn't get lost in the internet ether
<rharper> yeah , looks like you need to join the team  https://launchpad.net/~cloud-init
<rharper> I don't see a message posted, https://lists.launchpad.net/cloud-init/maillist.html
<chillysurfer> rharper: ok requesting to join the team now
<chillysurfer> do you need to be a team member to post to the mailing list?
<rharper> yes, but it's an open team
<rharper> so you should just automatically join
<chillysurfer> ah cool
<chillysurfer> i'm glad i asked then! i'll join the team and re-ask the question :)
<chillysurfer> rharper: thanks so much!
<chillysurfer> ok i joined the team and resent the email to the mailing list (cloud-init@lists.launchpad.net). hopefully it hits this time :)
<rharper> k
<Odd_Bloke> chillysurfer: See #launchpad. :)
<chillysurfer> Odd_Bloke: thanks! i'm going to join in the conversation :)
<chillysurfer> another general canonical irc question... i see the ubuntu irc channel list (https://wiki.ubuntu.com/IRC/ChannelList), but it doesn't seem to include many irc channels, like #cloud-init and #launchpad. is there a _full_ list of canonical irc channels?
<chillysurfer> the reason i ask is i'm specifically curious about an irc channel that is focused on python-apt
<Odd_Bloke> chillysurfer: I don't know of any such list, I'm afraid.  That said, we keep apt and its friends in sync with Debian, so most of the Ubuntu apt discussion will happen wherever the Debian apt discussion does.
<chillysurfer> Odd_Bloke: cool that makes sense!!
<chillysurfer> thanks!
<Odd_Bloke> No worries, happy to help!
#cloud-init 2019-08-01
<chillysurfer> out of curiosity, does cloud-init (when installed on a target machine) utilize a virtual environment? or does it just deal with shared python dependencies on the machine?
<powersj> chillysurfer, the ubuntu packaging uses python packages from the distro
<powersj> other distros do the same
<chillysurfer> powersj: correct me if i'm wrong, but using python packages from the distro is the equivalent of shared deps for python
<Odd_Bloke> chillysurfer: That's correct.
<Odd_Bloke> When `import X` runs, the Python interpreter runs through PYTHONPATH to find what's being imported, and we ensure that all dependencies for cloud-init are in the system PYTHONPATH entries (by expressing dependencies in our Debian packaging).
<chillysurfer> Odd_Bloke: yep that makes sense!
<chillysurfer> funny that there haven't been nightmares of dep conflicts though..?
<Odd_Bloke> That's what a lot of distro work is: getting all the packages you want to ship on to versions that are mutually compatible
<Odd_Bloke> We're also very conservative about adding new dependencies to cloud-init, for precisely this reason.
<Odd_Bloke> It might save us a few hours or days of development time, but we'll be on the hook for support and maintenance for that version of that dependency for the lifetime of the Ubuntu releases it's in.
<Odd_Bloke> Which in the case of bionic, at least, is until 2028.
<Odd_Bloke> chillysurfer: You'll probably also want `--release eoan` or similar, else you still won't be able to upload it.
<chillysurfer> Odd_Bloke: ah ok. i'm building this for bionic though so i'll specify that instead
<Odd_Bloke> Yep!
<AnhVoMSFT> rharper regarding this https://code.launchpad.net/~vtqanh/cloud-init/+git/cloud-init/+merge/369785 - I have some concern about mocking the systemctl call as you suggested because I actually want to test against the real systemd output. In a non-systemd system we're not interested in the boot telemetry at all (for now) and the fact that the code isn
<AnhVoMSFT> 't exercised in that case is a desirable outcome. My main concern is that if there's a behavior change say in systemctl and the output is now formatted in a way that is not compatible with what the function expected; we want to catch that in unit test
<Odd_Bloke> AnhVoMSFT: Unit tests are not the appropriate place to be testing integration with system utilities.  As Ryan notes, the unit tests shouldn't be dependent on whether or not systemd is installed (or, for that matter, what version of systemd you have installed).  This is the sort of issue that we would expect to catch in integration testing (i.e. cloud_tests).
<Odd_Bloke> If we found an issue there, we would then modify the unit tests' test data to reflect the discrepancy, and fix the issue so that those tests were passing again.
<Odd_Bloke> To put it another way, the unit tests are there to ensure that _we_ haven't broken our interactions with systemctl when we make changes in the future.
<Odd_Bloke> (Just having the unit tests run against the system's systemctl doesn't really help in the case where there is a change in behaviour, regardless.  We would need to test against _both_ behaviours, and the system systemctl will obviously only exhibit one of the behaviours.)
<rharper> AnhVoMSFT: we only run unittests against a particular host os; so a unittest won't ever verify that systemctl isn't broken on any other os;  as Odd_Bloke said, one could write an integration test which runs on multiple OSes and versions of systemd to ensure systemdctl isn't broken but I don' t _really_ think we need that level of testing for  the systemctl command output ;
<AnhVoMSFT> rharper Odd_Bloke I don't disagree with the intention of unittests here but the the only thing the function does is making those two calls foo and bar, and if mock both foo and bar, what exactly are we testing?
<Odd_Bloke> AnhVoMSFT: So I haven't reviewed the MP in question, but if you're worried about the systemctl output then we are presumably testing that we parse the current systemctl output correctly?
<rharper> AnhVoMSFT: we want to exercise the various code paths;  systemctl provides no output, systemctl throws an exception, systemctl provides stdout (but not in the expected format Foo=abc) , systemct provides output in the correct format
<rharper> we can mock the return value (and side-effect) of the subp call to show that cloud-init does the expected thing in each of those possible scenarios
<AnhVoMSFT> ok, that seems a bit clearer on the intention - thanks guys
<AnhVoMSFT> my original intention was to basically test two things: if the VM has systemd, I want to see xyz as output, if no systemd, I want to see no output
#cloud-init 2019-08-02
<Odd_Bloke> tribaal: o/ I'm running through a review of the data source ATM.  I've found a few minor bits and pieces I'd like cleaned up, but I won't want to block merge of the initial work on those.  Are you comfortable committing to cleaning up those bits in a follow-up MP so we can land this one sooner?
<Odd_Bloke> (Note that I haven't finished my review yet, and I do have a couple of more major things that I've noted I need to think through, so there may be a couple of things I want addressed before we land.  I'll call those out specifically if they come to pass.)
<blackboxsw> back from long vacation. hope everybody is doing well
#cloud-init 2019-08-04
<muhaha> growroot needs restart ?
<muhaha> Or is possible to resize root partition with cloud-init without reboot?
<icarusfactor> At least Openstack only talks about on first boot.
<icarusfactor> muhaha, ^
<icarusfactor> muhaha, and limited to OS and versions.
#cloud-init 2020-07-27
<smoser> bad design (and implementation) in openstack
<smoser> each time you do a request to the metadata service it ends up in lots of messages on the bus to get the information. at points you could almost DOS a openstack installation by just hitting the metadata server.
<smoser> meena: ^
<smoser> https://github.com/cirros-dev/cirros/issues/8
<smoser> that is a very typical discussion on the topic.
<smoser> i'd like to highlight this comment:
<smoser> In these test runs, how long did it take cirros to boot? Lets say it took 2 minutes. That means you booted virtual hardware (bios, kernel,disk, ....) in 2 minutes, but a HTTP couldn't be satisfied in 10 seconds?
<mock> Question: I am adding the package_update: true to my cloud-config userdata. My package manager is yum. Will this be the equivalent of a yum update or yum update --security? And if not the second, is there a cloud-init way to do the security only update or do I just need to runcmd the command.
<mock> ?
<Odd_Bloke> mock: I don't believe it will be the second, and I don't believe that there is a cloud-init way to express a security-only update.
<Odd_Bloke> In the apt world, or Ubuntu at least, security updates aren't special at the package manager level; if you only want security updates, then you change the package sources to only include security repos and then `apt update; apt upgrade` as normal.)
<Odd_Bloke> So I think this is a case where the "lowest common denominator" package management module can't express such a requirement (though I haven't given it too much thought and would happily be contradicted).
<mock> Ok, thanks
<mock> I will use the runcmd avenue
#cloud-init 2020-07-28
<chillysurfer> good day, all! could i get a maintainer to review this pr? https://github.com/canonical/cloud-init/pull/509 thanks so much!
<blackboxsw_> apoligies I've got something funky going on with my irc proxy. So, I'll  have to kick off this meeting as dunder bbsw
<blackboxsw_> #startmeeting cloud-init status meeting
<meetingology> Meeting started Tue Jul 28 16:55:17 2020 UTC.  The chair is blackboxsw_. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
<meetingology> Available commands: action commands idea info link nick
<blackboxsw_> community notice: time for another bi-weekly (or semi-monthly if you prefer) cloud-init community status meeting
<blackboxsw_> #chair Odd_Bloke smoser rharper
<meetingology> Current chairs: Odd_Bloke blackboxsw_ rharper smoser
<blackboxsw_> Hello folks, cloud-init community status roundup. We gather here in this IRC channel every 2 weeks to discuss current development tasks and progress on cloud-init.
<blackboxsw_> All questions. side-conversations and interruptions are welcome
<blackboxsw_> Last meeting minutes live here
<blackboxsw_> #link https://cloud-init.github.io/status-2020-07-14.html#status-2020-07-14
<blackboxsw_> he topics we'll cover today: Previous Actions, Recent Changes, In-progress Development, Community Charter, Office Hours (~30 mins).
<blackboxsw_> #topic Previous Actions
<blackboxsw_> None found in meeting minutes from last session.
 * blackboxsw_ sets the topic for next meeting.
<blackboxsw_> +2 weeks from now
* blackboxsw_ changed the topic of #cloud-init to: pull-requests https://git.io/JeVed | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting Aug 11 16:15 UTC | 20.2 (Apr 26) |  https://bugs.launchpad.net/cloud-init/+filebug
<blackboxsw_> August 11th 16:15 UTC
<blackboxsw_> #topic Recent Changes
<blackboxsw_> The following commits have been landed on master of upstream branch since last meeting: found via git log --since 2020-07-14
<blackboxsw_> #link https://paste.ubuntu.com/p/RjZcwtk6Hd/
<blackboxsw_> features of note:
<blackboxsw_>  - azure: avoid bouncing hostname if set hostname fails
<blackboxsw_>  - vmware: new defaults for post customization script overrides on vCloud
<blackboxsw_> - azure ValueError raised if JsonDecodeErrors is not available when parsing metadata
<blackboxsw_> Thanks Goneri otubo anhVo and dermotbradley for community contributions this round
<blackboxsw_> #topic In-progress Development
<blackboxsw_> Current projects for cloud-init are leading us to additional features:
<blackboxsw_>   - network device hot plug support for cloud-init post-boot
<blackboxsw_>  - better integration testing on other clouds, Oracle support
<blackboxsw_>  - extended json schema validation and publishing full static schema versions for external tools
<blackboxsw_> #topic Community Charter
<blackboxsw_> The following topics are still topics for ongoing community development:
<blackboxsw_>  - JSON schema extensions to validate user-data before instance launch: https://bugs.launchpad.net/cloud-init/?field.tag=bitesize
<blackboxsw_> - Datasource documentation and updates
<blackboxsw_> - cloudinit.net refactor into distro-specific networking subclasses cloudinit.distros.networking: https://bugs.launchpad.net/cloud-init/+bugs?field.tag=net-refactor
<blackboxsw_> As always: thank you all for bug contributions, PR submissions, triage and discussion participation.
<blackboxsw_> If anyone would like to be involved more than they currently are, please feel free to contact us here in IRC #cloud-init on Freenode or on the mailing list cloud-init@lists.launchpad.net and we can see how best we can get you "set up"
<blackboxsw_> #topic Office Hours (next ~30 mins)
<blackboxsw_> This time of the meeting is really just an open door for any discussions, concerns, bugs, questions or general prodding of upstream devs to make sure existing development work is unblocked where possible.
<blackboxsw_> In the absence of discussions, review of existing PRs is addressed.
<blackboxsw_> Thanks for tuning in folks. have a good one!
<blackboxsw_> #endmeeting
<meetingology> Meeting ended Tue Jul 28 17:56:15 2020 UTC.
<meetingology> Minutes:        http://ubottu.com/meetingology/logs/cloud-init/2020/cloud-init.2020-07-28-16.55.moin.txt
<Odd_Bloke> rharper: If you have a minute, could you take a look at https://bugs.launchpad.net/cloud-init/+bug/1888822 for me?  The submitter has done a good job of triaging the issue themselves, and the commit "to blame" mentions you and a bug, but doesn't include enough info for me to fully understand the root of the change (including a lack of link to the mentioned bug).
<ubot5> Ubuntu bug 1888822 in cloud-init "cloud-init caches files and never checks again" [Undecided,New]
<rharper> Odd_Bloke: yeah, I looked at it the other day ... and I got as far as not understanding how it worked in the first place; let me look at if there are any updates
<Odd_Bloke> rharper: There was a comment an hour ago with the triage I mentioned.
<rharper> yeah, I see it
<rharper> I had suspected the change but haven't worked out what's happening yet
<Odd_Bloke> So we've expanded the set of Content-Types that we consider "in need of determination of Content-Type" from TYPE_NEEDED to also include all Content-Types that are used for including too.
<rharper> right
<rharper> the issue is that their second item in their payload is not *present* during boot
<rharper> so when we go to read the content type of the payload; it returns empty/none;  so that modifies the mime-type
<Odd_Bloke> But this means that if an include sets its Content-Type but _does not_ include the corresponding first line, it will be detected as x-shellscript.
<rharper> then they *rerun* cloud-init.service *inside* their boot hook
<rharper> it is not
<rharper> it is set as an unmapped type
<Odd_Bloke> I think this is the line that indicates the problem:  2020-07-24 09:26:04,614 - __init__.py[DEBUG]: {'Content-Type': 'text/x-shellscript', 'Content-Disposition': 'attachment; filename="part-001"'}
<rharper> the logs are not clear; there are two days, 2020-07-23 using 19.4 ... and it emits an interesting like the 'Empty payload' which is when it tries to read /etc/secret-userdata.txt (and the file exists but is empty);
<Odd_Bloke> I've just tested locally, and using user-data generated by `./tools/make-mime.py -a boothook:cloud-boothook` where `boothook` has the header and doesn't have the header behaves differently: namely, it is interpreted as x-shellscript without the header.
<rharper> this is not present at all in the 07-24 run ... so I'm not sure we're getting clean logs rom them
<rharper> that's the one of the points; if you provide a payload, we'll look at the content
<rharper> an set the mime type by the content
<rharper> their issue is that the content is not yet present when we look now
<Odd_Bloke> Well, our docs say "Begins with: #cloud-boothook or Content-Type: text/cloud-boothook when using a MIME archive."
<Odd_Bloke> Not "and".
<rharper> they have two payloads
<rharper> one is  a boot hook script, which runs, to fetch the second contents for the second payload
<Odd_Bloke> Right, and the boothook doesn't get run at boothook time any longer, it gets run as a user-data shellscript.
<rharper> it does
<rharper> the issue is the resulting cloud-config.txt does not include the secret-userdata
<rharper> so their restart of cloud-init.service does not have any new user-data (From the secret they downloaded) to consume
<rharper> the first payload very much does run
<Odd_Bloke> I never said it didn't run.
<rharper> why do you think it runs as a user-data script instead of the boot hook ?
<Odd_Bloke> rharper: The output comes after this line: Cloud-init v. 20.2-45-g5f7825e2-0ubuntu1~18.04.1 running 'modules:config' at Fri, 24 Jul 2020 09:26:07 +0000. Up 112.48 seconds.
<Odd_Bloke> Whereas I believe a boothook would have its output come substantially earlier than that?
<Odd_Bloke> (In cloud-init-output.log.)
<rharper> yeah, I think I can see that;   I'm not sure that having the script run as boot hook would fix the issue; but maybe it would mean that the consume-data semaphor won't have been touched yet ?
<rharper> I'm going to ask for a collect-logs from a successful run;
<rharper> the logs provided are somewhat confusing as mentioned
<rharper> Odd_Bloke: do we trust the payload or the mime-types ?    previously, if you said it was a boot-hook and payload was #!/bin/sh it ran at boot-hook time;  now we look at the payload and if we don't see the #! then it's a shell script;  the payload would need to have #cloud-boothook ;
<Odd_Bloke> rharper: I believe the linked commit moved us from "trust the mime-type, infer from the content if either we don't have a mime-type or the mime-type is in cloudinit.user_data.TYPE_NEEDED" to the same but add cloudinit.handlers.INCLUSION_TYPES_MAP.values() to the set of mime-types we won't trust.
<Odd_Bloke> And I think that means that we will now not trust the mime-types for ~all mime-types people might pass to us.
<Odd_Bloke> (So effectively we've moved to "always infer from the content, even if we've already been told what's in there".)
<rharper> Yeah; I agree.  And if we revert that; then the older scenario which always sets a mime-type to x-shell-script but provides #cloud-config instead stays broken;
<Odd_Bloke> And boothooks have to start with a shebang (or perhaps be a binary executable? I haven't thought that through too much), which we detect as x-shellscript.
<rharper> alternatively, the newer bits which want boot-hook can also provide payload with the right header for payload detection
<Odd_Bloke> (I agree that I don't fully understand why their config was working before, to be clear. :p)
<rharper> I see some clear paths with (if the mime-type and content-type match; continue on) if there's a mismatch;  what do we want to do?  I guess reverting makes the most sense as this regresses existing payloads which were tagged correctly (and mapped their payload to a reasonable handler)
<Odd_Bloke> Yeah, I think we should revert; in the older scenario, is the first line of the content a reliable indicator of what's in there?  (i.e. Is it true that they want us to treat text/x-shellscript as text/plain?  Or do they want us to treat it as text/cloud-config?)
<Odd_Bloke> So I just tested and if an x-shellscript part does not start with a shebang then we fail to execute it.
<Odd_Bloke> So I think we can safely check the content on x-shellscript parts, because we will either (a) redetect it as x-shellscript because it starts with #!, or (b) were going to fail to run it later anyway because it didn't.
<Odd_Bloke> (By "fail to execute it", I mean: 2020-07-28 18:48:36,307 - subp.py[DEBUG]: Exec format error. Missing #! in script?
<rharper> that failed to execute happens when the first pass of boot attempts to run the empty file (secret-userdata.txt) file is present but empty (no content);
<Odd_Bloke> Nope, I'm talking about a local run in a lxd container.
<Odd_Bloke> Not their logs.
<rharper> oh, I see, yes
<rharper> that at least plugs the old hole, without harming this new breakage ;   no need to check content of boot-hook; the only "known" case of mislabeled content is x-shell-script
<Odd_Bloke> Right, I think the previous fix just spreads its net too wide.
<rharper> ack
<rharper> Odd_Bloke: https://paste.ubuntu.com/p/ZPydQdt8kY/    I think that should cover what we wanted;
<rharper> I tried to recreate the bug's scenario in a unittest but I've not had any luck getting it to work;  AFAICT, if the /etc/secret-userdata.txt file isn't present during boot; cloud-init will fail saying missing file import or something like that;  if the file is present but empty (touch /etc/secret-userdata.txt) then we take a different path where the payload has it's mime-type changed to x-not-multipart due to not being able to
<rharper> read the content; and subsequent consume_userdata() calls don't then view the payload as a #include url anymore;
<Odd_Bloke> rharper: I've just posted a comment to the bug, could you quickly review it before I ping other folks to let them know about the regression?
<rharper> yeah
<Odd_Bloke> Thx
<rharper> Odd_Bloke: yeah, that looks good.
<Odd_Bloke> rick_h: blackboxsw_: falcojr: lucasmoura: FYI, we've had a regression from the last SRU reported as https://bugs.launchpad.net/cloud-init/+bug/1888822; I've summarised the discussion that Ryan and I have been having into my most recent comment.
<ubot5> Ubuntu bug 1888822 in cloud-init "cloud-init caches files and never checks again" [Critical,Triaged]
<rharper> I guess we will also need a boothook/shellscript multi-part mime test
<Odd_Bloke> The title of that bug is misleading, let me update that.
<rharper> yeah; though ... I'm super interested in seeing what they describe actually work (and understanding why) ... it really boggles my mind on including a non-existing file and restarting cloud-init ...
<rharper> what they want is something like we discussed about having cloud-init fetch secrets from aws/ebs encrypted store a while back
<rharper> no restarting of cloud-init.service needed
<Odd_Bloke> We vaguely discussed previously having some sort of x-config-fetcher part which would be a Python script that would emit config on stdout.
<Odd_Bloke> Yeah, I'm thinking of the same convo.
<rharper> yeah
<Odd_Bloke> Well, I guess it need not be Python if the interface is stdout.
<Odd_Bloke> rharper: I would expand that comment to mention that we can safely round-trip true x-shellscript parts through find_ctype, that's why it's safe to carve that exception out; other than that, I agree that's the change we want/need.
<Odd_Bloke> Are you working on this/tests for it ATM, or shall I pick it up?
<Odd_Bloke> smoser: BTW, I believe https://github.com/canonical/cloud-init/pull/493 is now ready for review; as you implemented the Oracle DS originally, I wanted to give you a chance to review it.
<rharper> Odd_Bloke: there's an existing test in-tree that tests the case where content is not shellscript; and we consume it correctly ( content is cloud-config) ;   what test remains that we ned ?
<rharper> Odd_Bloke: I'm not actively working on a new unittest;  I was attempting to test that bookhook + x-include-url file://path scenario ... but I can't make it work;
<Odd_Bloke> rharper: I wasn't sure, just knew you were poking at some testing. :)
<rharper> fruitlessly; so thanks for spotting the initial issue
#cloud-init 2020-07-29
<catphish> is there a short answer to how to start building a datasource and make it execute?
<catphish> ah, it's documented at https://cloudinit.readthedocs.io/en/latest/topics/datasources.html - duh
<catphish> though i'm still not very clear about how to get started
<catphish> anyway, i'd really appreciate it if anyone could point me in the direction of some very simple instructions on getting started, essentially i just want to place a datasource module somewhere and have it be executed to i can start developing, either by cloning the repo and running it, or by droping my module into a default ubuntu install, whichever is simlpler
<catphish> my initial attempt to place a file into cloudinit/sources and running "cloud-init init" do not result in my code running, so i'm clearly missing several steps to load the module and have it be identified as the active datasource
<otubo> rharper, yes, RHEL8 is NetworkManager only (sorry for delay, was on PTO)
<catphish> good morning sirs, same question as last night, would anyone be able to point me in the direction of getting started developing for cloud-init, specifically i'm looking to create a datasource, but i need some basic pointers about how to run cloud-init from the repo, and how to add a datasource class so that it executes
<catphish> hopefully i can use this as a template and try again https://github.com/canonical/cloud-init/commit/e80517ae6aea49c9ab3bd622a33fee44014f485f
<Odd_Bloke> catphish: What platform are you creating a datasource for, out of interest?  To test cloud-init as if it's a first boot, you'll want to look at `cloud-init clean`; that will remove all of cloud-init's state.  Also note that the set of datasources cloud-init will consider is generally configured in /etc/cloud/cloud.cfg (or /e/c/cloud.cfg.d/*) so your source will need to be in that list to even be
<Odd_Bloke> considered.
<Odd_Bloke> (I would suggest adding it as the _only_ DS while you're iterating on it.)
<catphish> Odd_Bloke: i'm writing a datasource for a new cloud provider - https://katapult.io/ - but my intention is for it to be somewhat different from the existing datasources, specifically because it will use http over ipv6 link local (possibly with multicast discovery), rather than DHCP or APIPA, if at all possible i will make it generic rather than proprietary
<catphish> Odd_Bloke: thanks for the pointer, i will try adding my datasource module and adding it to /etc/cloud/cloud.cfg - i have one other question though, should i be hacking on an ubuntu install, or directly on the source repo, i couldn't work out how to execute the latter, i'm actually not that familiar with python packages
<Odd_Bloke> catphish: It's probably easiest to iterate in an install; cloud-init is system software, so running it "from source" is tricky.  We generally use `./packages/bddeb` to build debs from the tree, which you can then install in an Ubuntu system to test.
<catphish> thanks! i'll give that all a try
<Odd_Bloke> rharper: You mentioned "there's an existing test in-tree that tests the case where content is not shellscript" but I just added `assert False` to the top of UserDataProcessor._process_msg and it didn't cause any tests to fail.  Am I missing something?
<Odd_Bloke> (Perhaps it's a cloud test, I realise immediately after sending that message.)
<Odd_Bloke> Ooor perhaps I hadn't saved the file?  I'm seeing failures now.
<Odd_Bloke> rharper: Disregard the above. :)
<otubo> Odd_Bloke, smoser quick question about cloud-utils[-growpart]: What's the difference between these two? IIUC -growpart is the package shipped by distros that includes just the growpart script from cloud-utils. Is that correct?
<otubo> Odd_Bloke, also, I was on PTO. Gonna fix that PR this week :-)
<smoser> otubo: i can't really speak to that.
<smoser> ubuntu ships 'cloud-image-utils' and 'cloud-guest-utils'.  growpart is included in cloud-guest-utils.
<smoser> i really don't know about other distros.
<smoser> i can agree that with 10  years of hindsight, the connection of 'growpart' to 'ec2metadata' seems silly.  it is "obvious" that I just should have made a separate upstream package for growpart.
<smoser> what is more obvious is that I should have just written such a thing in C ;)
<meena> *cough* rust *cough*
<smoser> meena: yes. but please give the smoser from 10 years ago a break on that.
<smoser> he justu didn't think that rust was going to pan out.
<meena> fair
<meena> i think ten years ago i still held onto the believe that C is a real programming language.
<meena> instead of 38962 different dialects
<smoser> but for what its worth, if you're interrested in a rust rewrite of growpart
<smoser>  https://github.com/bottlerocket-os/bottlerocket/tree/develop/sources/growpart
<smoser> might be a place to start.
<meena> cool
<otubo> oh nice
<otubo> smoser, thanks. Yeah, and the mystery of why distros ship growpart as a different package continues.
<otubo> if I get any interesting info on that I'll leave some bits here :-)
<Odd_Bloke> https://github.com/canonical/cloud-init/pull/512 <-- short-term fix for the pylint CI failures we've started seeing since pytest 6.x released
<Odd_Bloke> rharper: ^ should fix your PR's CI failure.
<soasada> Hi guys, I need some help. I have an Ubuntu 18.04 LTS VPS on OVH and has installed cloud-init. The thing is that I have just removed the "set_hostname" from "cloud_init_modules" in the "/etc/cloud/cloud.cfg" file and I reboot the server, the problem is that now I cannot connect to the server, is completely down. It is possible to recover it?
<Odd_Bloke> soasada: I don't have experience with OVH; do you have access to a serial console for the machine?
<soasada> No, I haven't
<Odd_Bloke> Hmm, then I don't know how much help we'll be able to offer here, you may need to follow up in OVH support channels.
<Odd_Bloke> (FWIW, I wouldn't expect that change to take a server down completely, which is why I suspect this may be an OVH-specific issue.)
<soasada> ok, so I have to contact then
<soasada> thank you so much Odd_Bloke
<Odd_Bloke> soasada: Good luck! :)
<Odd_Bloke> blackboxsw_: That pytest change hasn't been released yet; any objection to landing that fix so CI will run, and backing it out once we do have a release?
 * blackboxsw_ was wondering about us actually pinning https://github.com/pytest-dev/pytest/commit/d46fe88ec33f3ee2aa8addc56ee090b0119474d3 for the moment.
<blackboxsw_> in our deps. but, let's just go with the simpler <6
<blackboxsw_> can we add a comment about the pytest issue Odd_Bloke for reference
<blackboxsw_> or make that the commit msg
<blackboxsw_> either way, a bread crumb would be nice to the issue so future us can track closure and back this out
<Odd_Bloke> blackboxsw_: Done.
<blackboxsw_> and Approved thx
<rharper> Odd_Bloke: thanks, i'll update
#cloud-init 2020-07-30
<otubo> Odd_Bloke, I figured I could remove fstab writing safely from those tests. Not sure why I kept them... Anyways, they're still wrong please let me know. I really need to pick up the pace on these pytest stuff
<rharper> Odd_Bloke: https://github.com/canonical/cloud-init/pull/511/checks?check_run_id=924499934 ;  the status on the PR page isn't syncing with travis;, when I click through to the 'queued' jobs; they're all green.
<rcj> This question is a follow-up to discussions about a grub regression in #ubuntu-release...
<rcj> Odd_Bloke: Can you think of other distros that need to know about this because they use cloud-init?  deb-based distros can be affected but is there something similar for the rpm-based folks?
<rcj> The issue is summarized @ https://wiki.ubuntu.com/SecurityTeam/KnowledgeBase/GRUB2SecureBootBypass#Existing_Cloud_.2F_MAAS_instances_may_fail_reboot
<Odd_Bloke> rcj: There isn't a specific module that does anything along these lines for anything but dpkg-based distros, and "grub" does not appear in cloudinit/distros/* (so it isn't implemented in there either).
<Odd_Bloke> So I think this is just Debian derivatives.
<rcj> Okay, thx.  Otherwise I'd ping some other folks.
#cloud-init 2020-07-31
<twouters> any idea what causes /var/lib/cloud/instance to become a directory instead of a symlink? i've seen this happening when /var filesystem is full (cloud-init fails with OSError: [Errno 28] No space left on device) and fails on subsequent reboots with IsADirectoryError: [Errno 21] Is a directory: '/var/lib/cloud/instance'
<twouters> I understand that a full filesystem is likely to result in unexpected behaviour, but I'm trying to figure out when the path is created as a directory instead of symlink
<twouters> I'm guessing a mkdir -p alike situation where /var/lib/cloud/instance doesn't exist as a symlink, but can't seem to spot it in the code
<chillysurfer> thanks for the review smoser on https://github.com/canonical/cloud-init/pull/509! was there anything else you wanted to see changed or answered?
<rharper> twouters: https://bugs.launchpad.net/cloud-init/+bug/1883903
<ubot5> Ubuntu bug 1883903 in cloud-init "cc_final_message will create /var/lib/cloud/instance as a directory if the symlink does not exist" [Undecided,Fix committed]
<twouters> oh, how could i miss that
<rharper> it's recent fix
<twouters> thanks, i know what to do now :)
<rharper> twouters: cool!
<chillysurfer> smoser: thanks for the comment on the pr! you ask about documentation, this seems like it's more helpful for the developer than the user. you think documentation in the form of comments in the get_public_ssh_keys function should suffice as long as it is verbose enough?
<smoser> chillysurfer: some datasource
<smoser> IBMCloud has a good docstring at the top of DAtaSourceIBMCloud
 * smoser out
<chillysurfer> smoser: "some datasource" -> not sure i understand? you mean document this behavior in the docstring at the top of DataSourceAzure?
<Odd_Bloke> https://github.com/canonical/cloud-init/pull/515 <-- pytest have released 6.0.1 which fixes the pylint failures we saw with 6.0.0
<smoser> chillysurfer: yes. that is a good place to document the general behavior of the IMDS/datasource
