#cloud-init 2013-09-30
<smoser> samppah, you may need a newre cloud-init than in your 6.4 centos.
<smoser> i'm not sure.
<samppah> smoser: ok, thanks.. i'll try to check that out :)
<smoser> cloud-init config-drive came in 0.7.0. and one fix came in 0.7.2 that affected older 2.6 kernels (added by RH engineer)
<samppah> ah
<smoser> so you may even need 0.7.2
<samppah> i think it's something like 0.6 in epel
<smoser> yeah, not good enough there.
<smoser> 'make rpm' from trunk should / might give you something. harlowja uses that, but i've notn used rpms in probably 5 years.
<gholms> samppah: You could build always try building fedora's source rpm on rhel.
<gholms> You would have to tweak its config a bit, but it's probably doable.
<smoser> gholms, have you ever used 'make rpm' ?
<smoser> that "shoudl work" ?
<smoser> if less than ideal in a trivially fixable way, we could fix it.
<gholms> smoser: Can't say I have.
<gholms> Given that it's likely to include config for the wrong init system I jusy use the same packaging code as fedora all the time.
<harlowja> ya, make rpm should work, if not then its a bug i think 
 * harlowja just got in, catching up
#cloud-init 2013-10-02
<pedroalvarez> Hi! me again :) 
<pedroalvarez> For use user-data from openstack, I have to add a datasource
<pedroalvarez> Config Drive with a configuration as this is enough? http://paste.ubuntu.com/6184639/
<pedroalvarez> In the cloud.cfg file.
<smoser> pedroalvarez, you dont have to do that at all.
<smoser> you just have to enable the datasource
<pedroalvarez> smoser: but, how to enable them?
<smoser> they're all enabled by default. there is a builtin list.
<smoser> but if you want to select
<smoser> datasource_list: [ NoCloud, ConfigDrive, OpenNebula, Azure, AltCloud, OVF, MAAS, Ec2, CloudStack, None ]
<pedroalvarez> smoser: also in cloud.cfg?
<smoser> yeah. or cluod.cfg.d/foo.cfg
<pedroalvarez> smoser: 
<pedroalvarez> thanks
#cloud-init 2014-09-29
<smoser> harmw, you there?
<harmw> nah, not realy
<harmw> daytime = worktime
<harmw> smoser: just give it a try and hope I'll reply :p
<smoser> so on freebsd you said that blkid depends on e2fsproggs ?
<harmw> yea, I had to install that first (from either pkg or ports)
<harmw> it then drops a binary in /usr/local/sbin
<harmw> hiren_: you can probably confirm this :) I'm not aware of something in BASE that does the same though, so perhaps there is a reasonable alternative we could/should use instead
<smoser> well, blkid is pretty much a dependency at the moment. 
<smoser> if it depnds on e2fsprogrs (possibly for identifying / getting info  on extX partitions) then it seems like a bug if freebsd doesn't identify that dependency
<harmw> in order to get that blkid binary, I must install that e2fsprogs pkg
<harmw> but its no problem
<harmw> just another dependency
<harmw> did my comments make sense btw?
<smoser> mostly, yeah. i have one fix there. and thank you for your work on that.
<smoser> so freebsd doens't resolve dependencies in any way?
<smoser> the user has to explicitly know that ?
<smoser> (that blkid depends on e2fsprogs?)
<harmw> ah no
<harmw> perhaps i wasn't clear on it :) blkid isn't a package on its own
<harmw> the binary comes with the e2fsprogs package
<smoser> so i think we need to add /usr/local/sbin to PATH in the sysvinit/freebsd ?
<smoser> harmw, ok. that clears that up. thanks.
<harmw> that sounds quite reasonable, yes
<harmw> (that's where all non-BASE binaries end up)
<smoser> alright. well, take another look when you ahve a chance.
<smoser> the PATH stuff i just simply declared. and you're welcome to improve the static definition. another route is to do: PATH="$PATH:/usr/local/sbin:/usr/local/bin"
<harmw> Ill have a look tonight
<harmw> thanks!
<harmw> smoser: I no longer have to hardcode any datasource in the config now, so thats another change you may apply
<hiren_> harmw: hey!
<harmw> noes!
<hiren_> didn't get your que about blkid
<harmw> oh it's nothing :)
<harmw> thats just some binary that c-i needs
<hiren_> ah. to understand blkid?
<harmw> something like that
<harlowja> hiren_ so we all safe from the bash stuff now :-P
<harlowja> for a few days??
<harlowja> lol
<harlowja> until the next crazy issue thats been existing for 20+ years, lol
<harmw> lol
<JayF> blkid is great stuff
<harmw> proofs once again opentsouce hasn't got anything to do with security :P
<hiren_> hah
<hiren_> yeah, bash: the gift that keeps on giving :-)
<harmw> :>
<smoser> harmw, explain that ?
<harmw> in the freebsd config file there is a line hardcoding the DS to use, currently thats openstack
<harmw> I just ran it without that hackery, and it found the attached configdrive without issues
<harmw> so whatever caused it not to find any DS earlier on is probably fixed
<smoser> ok. so i'll just drop that.
<harmw> yea, ill run a test with DHCP enabled on this instance in a sec
<smoser> hm..
<smoser> well, lets just add configdrive
<smoser> to the list.
<smoser> for now.
<harmw> could go that route as well
<smoser> rather than letting it go willy nilly
<harmw> willy nilly...
<harmw> wtf
<smoser> i'm sure there are other sources that have some issue. that are probably ignorable
<smoser> but will pointlessly put 'WARN' in your log
<harmw> ah yeas
<smoser> yeah, for now lets do that.
<harmw> how can I test/read which DS has been used (in freebsd.py)?
<smoser> whihc has bee nused ?
<harmw> yea, I want to set static networking when it's used configdrive
<harmw> seemed legit
<smoser> well, config drive doesnt necessarily imply static networking
<smoser> cloud-init has a bunchy of networkign cleanups to do. really. an dfreebsd will have to come along.
<harmw> if info.get('bootproto') == 'static':
<harmw> I'm not realy getting that :P where is bootproto set/specified?
<smoser> i'm not sure. there is a lot of growing that needs to occur here. 
<harmw> aw crap, wasn't it some annoying feature in openstack that it would specify network setup in a configdrive?
<JayF> <.< >.>
<harmw> uhm, whre it would specifically NOT do that
<JayF> harmw: right now, Openstack transmits network information to cloud-init by way of a debian-style /etc/network/interfaces file
<harmw> and thats in a configdrive?
<JayF> harmw: there's a spec up for K (that Rackspace implements /today/ in vendor_data.json) that represents the network information from neutron for parsing as json
<JayF> harmw: yeah
<harmw> ok
<harmw> so that should be something in /var/lib/cloud representing the ntwork settings
<JayF> in my setup
<JayF> it's a pointer to something in openstack/content/
<harmw> obj.pkl has the netsetup
<harmw> or atleast the address
<JayF> There's basically a template that nova uses to write network configuration out to the configdrive 
<JayF> I guess it's possible for that to not be enabled
<harmw> oh, but thats local-ipv4 I'm seeing - no gateway
<harmw> ok JayF 
<harmw> http://docs.openstack.org/user-guide/content/enable_config_drive.html
<harmw> that shows how to inject the interfaces file with a networksetup inside, but thats just hackerish
<JayF> that's not the most awesome document I've ever seen
<JayF> nova /does/ 99% of that for you iirc
<harmw> well I was hoping that it would do just that :P do you have a link to some more helpfull/updated docs?
<JayF> I don't know much about generically operating Openstack
<JayF> I *do* know about how we operate openstack
<JayF> so I can show you what we do 
<harmw> sure thing :)
<JayF> harmw: injected_network_template= in nova.conf [DEFAULT] 
<harmw> aha
<harmw> so JayF , that configitem is specific to configdrive?
<JayF> I think so?
<JayF> ask me questions about Ironic if you want more certain responses :P
<harmw> haha
<harmw> :P
<harmw> /etc/nova/nova.conf:injected_network_template = /usr/share/nova/interfaces.template
<harmw> so, I'm already using that in my config
<JayF> does that file exist and seem reasonably formatted?
<JayF> we had to make a custom one :)
<harmw> oh that file exists, and looks reasonable
<harmw> harlowja: how would c-i drop that file or the content in /var/lib/cloud?
<harlowja> hmmm, maybe from http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/sources/DataSourceConfigDrive.py#L197 ?
<harmw> I can see that's where it wants to apply it, yes
<harmw> but I'm to lazy to find out how openstack hands the network config (through configdrive) to c-i :p
<harlowja> should just be a file present on the config-drive
<harlowja> http://paste.ubuntu.com/8460729/
<harmw> ah so, well, I'm missing that content folder
<harlowja> harmw that would do it :-P
<harmw> could you give it back, please
<harlowja> nope, ha
<harlowja> u might want to ensure the following
<harlowja> https://github.com/openstack/nova/blob/stable/havana/etc/nova/nova.conf.sample#L1862
<harlowja> i think it needs to be 'always'
<harmw> force_config_drive=always
<harmw> but I don't want configdrive, only on specific instances... which is why I've got this 1 instance booted with --configdrive (or something)
<harlowja> hmmm, i didn't think u could target it to specific instances
<harmw> nova boot --configdrive 
<harlowja> hmmm, guess that must be new
<harmw> --config-drive=true
<harmw> since havana, I guess
<harmw> python-novaclient-2.13.0-1.fc19.noarch
<harmw> (damn old, but hey)
<JayF> You absolutely can use --config-drive=true
<JayF> in newer novaclient
<JayF> you should probably upgrade fwiw
 * JayF wishes everyone would use pip versions of openstack clients
<harmw> uhm, I was just saying I *am* using --config-drive=true :)
<harlowja> ya, for some reason u aren't getting into the code @ https://github.com/openstack/nova/blob/master/nova/virt/netutils.py#L69
<harlowja> or @ https://github.com/openstack/nova/blob/master/nova/api/metadata/base.py#L170
<harlowja> https://github.com/openstack/nova/blob/master/nova/virt/netutils.py#L107 is a common cause afaik
<harlowja> probably line 107 there thats stopping this
<harmw> could be, yea
<harlowja> i belieeve if u can comment that crap out, a file will appear, ha
<harmw> lol
<harlowja> or somehow set your network to 'injected' 
<harmw> and that would be on the controller's end, right?
<harlowja> ya
<harlowja> is this your own openstack, or someone elses :-P
<harmw> so how is injected set in the first place  :p
<harmw> my own
<harlowja> somewhere when a network is setup i think
<harlowja> something like that, ha
<harmw> don't you just love openstack at these moments
<harmw> I know I do :>
<harlowja> that just scratches the surface ;)
<harlowja> i've seen things man
<harmw> :P
<harlowja> lol
<harlowja> seen horrible things
<harlowja> 3 trillion config options...
<harmw> and a multitude of possible fuckups
<harlowja> openstack is like a crossword puzzle with about 10k pieces
<harlowja> all sorta the same color to
<harmw> yup
<harmw> luckily, I'm colorblind
<harlowja> ha
<harlowja> bb
<harmw> hiren_: ping
<harmw> ok, so it's definately failing because there is no 'injected' meta
<harlowja> ya
<harlowja> thought so
<harmw> compute is logging 'meta': {'injected': False,
<harmw> god this is tiring
<harmw> damn you, openstack!
<harlowja> :-/
<harlowja> how did u setup your networks that nova is using?
<harmw> with neutron?
<harlowja> hmmm, ya, got me then, ha
<harmw> harlowja: Ive hacked out L108 and L107 and now I finally have content :> but it contains a litteral $bla on about every line
<harmw> L108 and L117, oops
<nvucinic> je
<harlowja> hmmm, literal blah, must not be replacing stuff it needs from the thing u commented out, haha
<harlowja> my guess is https://github.com/openstack/nova/blob/master/nova/virt/netutils.py#L146 is empty
<harlowja> or missing
#cloud-init 2014-09-30
<hiren_> harmw: supp
<harmw> hiren_: nvm, my configdrive still sucks
<smoser> harlowja_away, did you get sorted above ?
<smoser> i've never been able to make a network-interfaces appear in config drive either
<smoser> harmw, did you re-try my config drive ?
<harmw> smoser: apart from networking, it all works
<harmw> didn't I say that the othr day? :)
<smoser> i'm sorry. i didn't see that. 
<smoser> ok. i'll pull that in.
<harmw> needo
<harmw> hm, no rhel6 juno packages from rdo :|
<harlowja> hmmm
<harlowja> harmw did u figure out the networking stuff?
<harlowja> if u want u can ping 'markmcclain'  in #openstack-neutron, he might be able to help, say josh sent u :-P
<harmw> nope, I got to tired
<harmw> haha
<harlowja> and don't forget to do the secret handshake
<harlowja> otherwise, bad  things...
<harmw> ah yea
<harmw> I once forgot
<harmw> ..
<harmw> since thn, alwys shake
<harlowja> ya, the saga of harmw and his lost fingers, lol
<harmw> fingrs? which it ended there...
<harlowja> never again!
<harlowja> lol
<harmw> kinda sux thre apparently are no el6 packages for juno
<harmw> I wasn't planing to upgrade both openstack + centos
<harlowja> u can generate them
<harlowja> using anvil ;)
<harlowja> if it works for juno (i haven't tried)
<harmw> anvil?
<harlowja> my other project, lol
<harmw> hmk
<harlowja> http://anvil.readthedocs.org/en/latest/topics/summary.html :-P
<harmw> so anvil takes care of sysvinit vs systemd?
<harlowja> well it does that by just using sysvinit, lol
<harmw> lol!
<harlowja> for now :_P
<harlowja> :-P
<harmw> aren't you clever :>
<harlowja> it might not work for juno though, i haven't tried since juno not out yet, lol
<harlowja> but it does build things for icehouse
<harmw> well, RDO covers icehouse so thats no problem :p
<harlowja> :)
<harmw> its juno I worry about
<harlowja> http://paste.ubuntu.com/8467423/ is a script that will run anvil if u want to try juno
<harlowja> u should just be able to point anvil @ https://github.com/stackforge/anvil/blob/master/conf/origins/master.yaml
<harlowja> although harmw if u are feeling lucky u can try 'http://209.132.178.33/repos/current/' 
<harlowja> which i believe the RH guys are creating from juno
<harlowja> using https://github.com/openstack-packages/delorean
<harmw> fc20? rdo.fedorapeople also has that :)
<harlowja> kk
<harlowja> well nm, let me know if anvil works ;)
<harlowja> if u want, haha
<harmw> hihi
<harmw> I'm used to using this repo: https://repos.fedorapeople.org/repos/openstack/openstack-juno/
<harlowja> kk, seems similar
<harlowja> except none of thats for el6
 * harlowja i talked to RH about this, they don't seem to care much about el6 
<harlowja> :-/
<harmw> hmk, well that sucks
<harmw> kinda
<harlowja> ya, sucks for people that can't just jump to RHEL7
<harlowja> which is like all the enterprise folks, lol
<harlowja> *aka me*
<harmw> indeed
<smoser> harmw, it freebsed config drive merged
<harmw> I noticed
<harmw> great
<harmw> though I want static networking covered/fixed but can't test that on my cloud :(
<harmw> it *should* work just fine, but I can't test it (regarding configdrive)
<harlowja> hiren_ hopefully can test that
<harlowja> since y! uses static networking  + configdrive still
<harlowja> *for better or worse*
<harmw> harlowja: when did you say you wre planning on visitting RH to do/package Juno for el6?
<harlowja> whenever y! get around to it, probably when juno gets released :-P
<harmw> hehe
<harmw> i'd vouch for Juno rc1 :P
<harlowja> :)
<harlowja> liekly
<harlowja> haha
<harlowja> not many people here trust openstack enough to run trunk ;)
<harmw> hehe
<harmw> wise
<harmw> but rc1 stage is a good stage to start harassing RH people they need to package for el6, instead of just 7 :p
<harlowja> ya, well we already harassed them a little
<harlowja> we'll probably harass them again, lol
<harmw> please do
<harlowja> since if they are going to do what anvil is doing, we need to make sure they have el6 packages
<harmw> the world depends on you!
<harlowja> otherwise anvil has to keep on doing what it does (building packages)
<harlowja> although i don't think y! signed the RDO support stuff, so they might not care
<harlowja> lol
<harmw> lol
<harlowja> u not pay us $$
<harlowja> u no get packages, lol
<harmw> hhe
<harlowja> *which is ok, cause i can automatclaly already build the packages, lol
<harlowja> but they said, u not get packages that we verified, and we like 'meh'
<harlowja> lol
<harlowja> * not how it went down, lol
<harmw> :p
<harlowja> arg, i get scared of the proposals peopel put up for openstack alot :(
<harlowja> today was https://wiki.openstack.org/wiki/OpenStack_cascading_solution#Architecture 
<harlowja> ^ makes me so sad
<harlowja> my response http://lists.openstack.org/pipermail/openstack-dev/2014-September/047507.html 
<harlowja> we should add a few more layers in there
<harlowja> then we will get a 7 layer openstack burrito
<JayF> the problem is
<JayF> Ironic is a burrito bowl
<JayF> so is vmware
<JayF> and openstack only wants to support wraps
<JayF> no respect for those of us intolerant of hypervisors :(
<harlowja> lol
<harlowja> taco bell would be so proud
<harlowja> ha
<harlowja> https://github.com/stackforge/tricircle/tree/master/novaproxy is interesting from a theortical perspective, but imho its not helping make openstack more reliable 
<harlowja> thats just my 2 cents
<harlowja> ^ the above is what that cascading thing would use to proxy from one nova to another
<harlowja> 'Nova-Proxy acts as the same role of Nova-Compute in cascading OpenStack. Nova-Proxy treats cascaded Nova as its hypervisor, convert the internal request message from the message bus to restful API calling to cascaded Nova.'
<harlowja> which seems sorta like ummm wrong
<harlowja> go all the way through the nova API, quota system, database calls... to just at the final end pop out a call to another nova
<harlowja> and the 'nova API, quota system, database calls' initially there are already sorta messed up
<harlowja> so X levels of messed up doesn't seem like an improvement, lol
<harlowja> *same with https://github.com/stackforge/tricircle/tree/master/cinderproxy (which does the same thing for cinder)
<harlowja> once u hit â levels all problems go away ;)
<harlowja> *because nothing ever finishes, haha
#cloud-init 2014-10-01
<harmw> harlowja_away: I'm looking over netutils.py again, and isn't Nova just by design skipping the network_template stuff when it's dealing with a dhcp'ed network?
<smoser> harmw, that may well be how it works.
<smoser> i have never succeeded in getting a template available in config drive :)
<smoser> its definitely something that needs work.
<harmw> yea, well
<smoser> i really hope to fix openstack and cloud-init to do sane things
<harmw> I hope to get a better look at things tonight
<harmw> and that'll could very well result in a bugreport
<harmw> not that it is a true bug, but given how it currently looks it should just do what I want - not what someone decided it 'should probably do in this or that situation'
<glev> Hello, I have a question about fs_setup.  Does it need to be entered under the cloud configmodules section in cloud.cfg to be used in my user-data cloud-config file?
<glev> It appears cc.disk_setup is the appropriate module, but it doesnt seem to work.  Where is this all documented?  The documentation I find shows configuration examples, but they don't work for my CentOS instance.  The log seems to not provide much useful information.
<glev> Am I in the correct place for cloud-init project related information?
<harmw> harlowja: ping
<harlowja> harmw pong
<harmw> wow
<harlowja> ping pong
<harmw> I which everything was so low-latency
<gholms> Heh
<harlowja> well u can't play ping pong at high latency
<harlowja> it doesn't work out so well
<harmw> true, true
<harmw> listen, I'm checking out netinfo.py in nova
<harlowja> kk
<harmw> return template.render({'interfaces': nets, 'use_ipv6': ipv6_is_available})
<harmw> that should print the template with everything substituted
<harlowja> right
<harmw> but it doesn't
<harmw> it just prints the original template
<harlowja> is there anyway u can add in a print before/after the render?
<harlowja> and the variables to
<harlowja> maybe this is a juno bug :-/
<harmw> (im on icehouse)
<harlowja> oh
<harlowja> can u add those prints anyway
<harmw> ofc
<harmw> in build_template i've added print of very function (template.render())
<harmw> which just prints out the original template
<harmw> with the nasty literal $ signs
<harmw> the net_info that is set on L136 contains proper values
<harmw> eg. this works: LOG.debug("NET_INFO: eth%s %s %s %s %s", ifc_num, address, netmask, gateway, dns)
<harmw> and this prins the original template: LOG.debug("OK, got template with contents: %s" % template.render({'interfaces': nets, 'use_ipv6': ipv6_is_available}))
<harmw> this jinja2 stuff is way to complex for me :P
<harmw> (btw, I hacked away 2 continue statements to make it this far)
<harmw> nets[0]: {'broadcast': '192.168.168.255', 'netmask': '255.255.255.0', 'gateway_v6': '', 'name': 'eth0', 'dns': u'212.54.35.25', 'address': u'192.168.168.66', 'gateway': u'192.168.168.1', 'netmask_v6': None, 'address_v6': None} 
<harmw> so thats good
<smoser> JayF, in thinking about netowrking and cloud-init
<smoser> and that netowkring config.
<smoser> i think i'd like some bit of info that says "block boot on this adapter"
<smoser> or something like that
<JayF> that info comes from neutron atm
<JayF> idk if there's a way to link that in, but I'd make a comment on the blueprint if you have desires
<smoser> i'm saying the new format.
<smoser> i realize the information comes from nuetron
<smoser> the reason being, if cloud-init gets this info on a config drive.
<smoser> its going to configure networking. 
<smoser> but then it should not let user code run until sane bits of netwoork are up.
<smoser> hm..
<JayF> who is to decide what is a sane amount of network?
<JayF> I'm always careful about that
<smoser> right.
<JayF> bceause in our cloud, at least, you can spin up boxes in 100% private networks
<smoser> its non-trivial
<JayF> so we can't depend on things for the general case that require internet to exist
<smoser> so heres the thing i was thinking about.
<smoser> system boots. reads some network from config drive.
<smoser> user hotplugs network device
<smoser> cloud-init reads config from MD server for that device and configures it.
<smoser> should cloud-init block the next boot on presense of that device ?
<JayF> I don't know about that specific scenario
<JayF> but I'd say cloud-init should do as much as it can regardless of some failures
<JayF> (and then give me a way as a developer to instead have it throw a hissy fit, aka exception, if something breaks)
<smoser> theres a difference between failure
<smoser> and "hasnt hotplugged yet"
<JayF> Ah
<smoser> and no real way to know the difference
<JayF> Yeah, IDK about those cases
<JayF> because I don't hotplug things :)
<JayF> atm at least
<smoser> right. 
<smoser> so vm, neutron hotplug the thing.
<smoser> on reboot, i dont know if i should expect that nic or not.
<smoser> and dont know if i've missed an event that would have pulled it from my config.
<smoser> ie, missed the hotplug remove during the reboot.
<JayF> this is why in the model I proposed for event-driven changes
<JayF> that you just pass the event and the updated data and say "make it right"
<JayF> you can't manage this as a certain number of steps to perform in order every time imo
<JayF> you have to just take the data you're given and do a thing with it, and if, for instance, that data references a network drive that the hypervisor didn't hook up, or that the OS image didn't wait to be ready before cloud-init runs
<JayF> then that can't be a cloud-init problem, it's an environmental problem
<smoser> hm.
<smoser> thats a good point. 
<smoser> but then makes the data on config-drive possibly stale.
<JayF> yes
<smoser> and at that point in boot, before a network is available to the MD thats all i have.
<JayF> I mean, should we support a mix of sources generally? Do we?
<JayF> ooooooooh.
<smoser> so if data on config drive had 'required' or not.
<smoser> then i could block until all required.
<smoser> and just "never unplug stuff that is required" rule.
<smoser> and then once required are up i can hit MD and get updated info
<JayF> Yeah I'm not sure I have a good informed opinion on this, and I have other stuff I need to finish this week, but I'm going to let that bake for a while and think about it
<smoser> :)
<smoser> this was bout all the time i could spend on this now too
<smoser> i'll comment in gerrit
<harlowja> harmw so it seems like the template is getting populated then right?
<harmw> it's beeing read just fine, but nothing gets replaced inside... so the original template is copied with any specifics in to the configdrive
<harlowja> ya, thats totally weird
<harmw> what else is new :P
<harlowja> is it possible to open a nova bug for this, it seems like someting is wrong ;)
<harlowja> although i'm not sure why y! hasn't seen it since we run that code 
<harlowja> and are running icehouse
<harlowja> so that part doesn't quite make sense, ha
<harmw> haha
<harmw> template.render(nets[0])
<harmw> that still produces the original template :|
<harmw> I don't get it
<harlowja> that should run via https://github.com/openstack/nova/blob/stable/icehouse/nova/virt/netutils.py#L150 right?
 * JayF suspects requirements differences
<harmw> yup
<harmw> JayF: I'm growing a feeling in that direction as well
<harlowja> but jinja2 not processing variables would be like a bad bug, ha
<harlowja> or not processing in the same manner, lol
<harmw> indeed, so I'm doubting my own setup as well :p
<harlowja> can u dump 'nets' and such, then try to load the template in a smaller script that uses your own code, try to replace nets in that code instead?
<harlowja> just to cause the same failure
<harmw> already trying something like that
<harlowja> hmmm, and your template is https://github.com/openstack/nova/blob/stable/icehouse/nova/virt/interfaces.template right?
<harmw> oh my
<harmw> no
<harmw> nowhere near
<harlowja> hmmm
<harlowja> that might be it
<harmw> uhu
<harlowja> it seems https://github.com/openstack/nova/commit/fa0d61084e50 changed cheetah -> jinja2
<harmw> I'm reading the vars in the template should be {{ bla }} as well
<harlowja> did RDO forget to pickup that file :-/
<harmw> which is merely a $bla in my template
<harlowja> ya, thats cheetah syntax :(
<harmw> ah!
<harlowja> so ya, that would be an issue ;)
<harlowja> u used RDO?
<harmw> ofcourse
<harmw> centos6 + rdo icehouse
<harlowja> ah, ya, rdo packagign bug i think
<harlowja> wonder if thats been reported
<harmw> # rpm -qf /usr/share/nova/interfaces.template 
<harmw> openstack-nova-common-2014.1.2-1.el6.noarch
<harmw> though thats the old file
<harlowja> ya, i bet they forgot to update it or something
<harlowja> *they being RH
<harmw> uhu
<harmw> you know where their bugzilla is at?
<harlowja> not sure, ha
<harmw> h
<harmw> bugzilla.rh
<harlowja> something like that
<harlowja> u can poke 'dprince_' in #openstack-dev
<harlowja> he might be able to get that to occur quicker
<harlowja> afaik he knows the people that know the people
<harmw> ah ok
<harlowja> *and also works @ RH
<harmw> hhe
<harmw> openstack-dev, guess I should join that channel as well then
<harlowja> ya, openstack has tons of channels, lol
<harmw> my irssi is getting a bit crowded
<harlowja> :)
<harmw> does he share your TZ?
<harlowja> harmw i think EST us
<harlowja> so not mine (PST)
<harmw> ok, well added a bug
<harmw> hhee, well that apparently did the trick - I changed the template to be jinja2
<harmw> so I now have a configdrive with proper network file, woohoo
<harmw> hm, well, this *almost* works 
<harmw> nova just told my vm something about eth0... and freebsd only knows of vtnet0
<harmw> so thats something to look into in freebsd.py
<harmw> but other than that, configdrive works and even configured the network (eventhough it was the wrong interface)
<harmw> and now it's bedtime :)
<harlowja> harmw coolness
#cloud-init 2014-10-02
<harmw> anyone aware of issues with efnet?
<smoser> haven't been there in probably 10 years ;)
<harmw> hehe, yea well I 'need' it for some bsd channels
<JayF> Yes
<JayF> it's a cesspool :)
<JayF> that's the main issue I'm aware of with efnet
<JayF> hehehe
<harmw> hm, interesting
<harmw> configdrive has no MTU support, so it seem
<harlowja> could be, does nova/neutron have the ability to set that?
<harlowja> *or even pass it in?
<harmw> nope, not that I know off :) dnsmasq does, but thats outside either one
<harlowja> :)
<harlowja> hiren_ have u had any luck getting all this going, i think harmw found a vtnet0 thing that u'll probably hit to
<hiren_> harlowja: fighting other fires :-(
<harmw> puh
<harmw> lame
<harlowja> still bash stuff?
<hiren_> nah. But I punted a lot of things to this week while takig care of bash drama.
<hiren_> so..
<harlowja> gotcha
<hiren_> I want to get back in a day or so if I can. 
<hiren_> just as an fyi, I am doing things with just libvirt/virsh right now. 
<hiren_> no openstak involved.
<harmw> ok
<hiren_> Is is something you guys want me to test
<hiren_> ?
<hiren_> on yahoo openstack cluster?
<harmw> almost :)
<hiren_> okay.
<harlowja> hiren_ well u can get the libvirt + config-drive copy working to start
<harlowja> i think u are doing that, which acts close enough to a openstack cluster to begin with
<hiren_> "working" is a stong word :-)
<harlowja> ya, get that 'working' first, then we can graduate u to openstack cluster ;)
<harmw> hihi
<hiren_> hah
<hiren_> fair enough.
<harlowja> :)
<harlowja> harmw i think is using a mini-cluster
<harlowja> and finding out how much fun it all is
<harlowja> haha
<harmw> lol
<harmw> 2 nodes :>
<hiren_> btw, so when I feed that config drive as 'vdb', and start the VM, I should mount that on the way of bring up, right?
<hiren_>     <disk type='file' device='disk'>
<hiren_>       <driver name='qemu' type='qcow2' cache='none'/>
<hiren_>       <source file='/home/openstack/7.6-YAHOO-20140927.openstack.qcow2'/>
<hiren_>       <target dev='vda' bus='virtio'/>
<hiren_>     </disk>
<hiren_>     <disk type='file' device='disk'>
<hiren_>       <driver name='qemu' type='raw' cache='none'/>
<hiren_>       <source file='/home/openstack/configdrive.dsk'/>
<hiren_>       <target dev='vdb' bus='virtio'/>
<hiren_>     </disk>
<hiren_> for example.
<harmw> why not use it as cd0?
<hiren_> thats better? sure.
<harlowja> hiren_ cloud-init should be finding it/mounting it
<hiren_> ah
<hiren_> thats what I wanted to know 
<hiren_> so
<hiren_> harlowja: I am at a point now, I need to put clound-inint
<hiren_> in the image
<harlowja> i defer to harmw , hopefully he knows that ;)
<harmw> pkg install bzr
<harmw> hiren_: are we doing this right now? Since I thought you whre busy :)
<harmw> else Ill get some popcorn first
<harmw> :>
<hiren_> hah
<hiren_> true. 
<hiren_> stupid timezones.
<harlowja> lol, popcorn
<hiren_> harmw: will you be around after 12:00 PST, that is 1.5 hrs from now?
<harmw> oh, I was refering to "hiren_ and the super aweseme cloud-init of joy" :P
<hiren_> lol
<harmw> it's 19:30 here
<harmw> so I have a few more hrs :)
<harlowja> sleep is for babies
<harmw> lol
<hiren_> and for the weak.
<hiren_> harmw: okay, lets try this now. 
<harmw> in that case: instal bzr
<hiren_> where? 
<hiren_> so
<harmw> in your vm
<harmw> pkg install bzr
<hiren_> after it comes up. okay.
<harmw> you've got a dhcp address I asume?
<harmw> after it comes up... well, yes
<harmw> :p
<hiren_> now are are talking. not really. Let me fix that. 
<hiren_> hum. how do I provide "networking" to the vm from config.xml file?
<hiren_> this is a fedora box I am doing this on
<hiren_> it has virbr0
<harlowja> :)
<hiren_> harlowja: :-) you know aht I am talking about.
<harlowja> u'll probably have to get libvirt to setup a nat, since afaik u won't have a real address
<harlowja> * http://libvirt.org/formatnetwork.html#examplesNAT
<harmw> why bother with xml?
<harmw> fedora has the boxes interface to manage all this
<harmw> or virt-manager
<harlowja> could try that to
<harlowja> hiren_ although since u are using a box that doesn't have a network that has extra 'ips' for vms, u're likely gonna have to do the NAT approach or try virt-manager (which probably does soemthing similar)
<harlowja> the real openstack clusters have whole ranges of ips that we can suck from and release...
<hiren_> okay. I am trying NAT
<harlowja> networking is oh so much fun :(
<harlowja> lol
<hiren_> heh
<hiren_> obviously it didnt work :/
<hiren_> how should the n/w interface look like inside the vm?
<harmw> vtnet0
<harmw> I'm asuming you use virtio
<hiren_> this is what I have in config.xml I am using:
<hiren_>       <network>
<hiren_>         <name>default</name>
<hiren_>         <bridge name="virbr0" />
<hiren_>         <forward mode="nat"/>
<hiren_>         <ip address="192.168.122.1" netmask="255.255.255.0">
<hiren_>           <dhcp>
<hiren_>             <range start="192.168.122.2" end="192.168.122.254" />
<hiren_>           </dhcp>
<hiren_>         </ip>
<hiren_>       </network>
<harmw> thats just a network, like a switch
<harmw> you also need an adapter, like a nice
<harmw> nic
<harmw> for your vm
<hiren_> http://pastebin.com/WQQGtaBd - config.xml
<hiren_> ah
<harlowja> hiren_ this is where virt-manager might help :)
<harmw> :p
<harlowja> i think it manages alot of this
<harmw> indeed
<hiren_> is it a pkg I should install on this fedora box?
<harmw> yum install virt-manager
<hiren_> k, installing.
<hiren_> ugh, its just gui?
<harlowja> :)
<harlowja> might have a cli, not sure
<harlowja> i've not used it so much
<harmw> you needed easy, right :P
<harlowja> turtles all the way down, it just gets more and more complicated ;)
<harlowja> thats why i stay away from networking lol
<hiren_> heh. I needed something that works. :-) 
<hiren_> so no virt-mgr for me. lets see how else can I get netowkring to work for me.
<harlowja> kk
<harlowja> there are a bunch of instructions online, i thought it would mostly just work, but maybe requires some tweaking
<hiren_> rigth.
<harmw> I think I've got this thing sorted now
<harmw> regarding static networking/configdrive on fbsd
<harlowja> harmw what u find?
<harlowja> probably just need to remap ethX to freebsdX
<harmw> find? was I looking for something? :P
<harmw> yup
<harlowja> u looking for the solution
<harlowja> haha
<harmw> and the ifup was way off as well
<harmw> haha
<harmw> lol
 * hiren_ takes off to actual $work. 
<hiren_> I'll poke at this tonight
<hiren_> to get n/w working.
<harlowja> kk
<harmw> smoser: I've got just one little patch remaining 
<harmw> http://dpaste.com/27KPCX2
<harmw> aw crap, I see vim put in a tab :|
<smoser> we've got a lot of networking things to do in the next 6 months.
<harmw> yea whatever, just apply&commit this :>
<harmw> or should I properly branch and such?
<smoser> i'll pull it
<harmw> sweeeeeet
<harmw> and please change that TAB :)
<smoser> harm type a commit message
<harmw> fix: Make sure to use the proper virtio FreeBSD network interface name.
<harmw> that'll do
<harlowja> harmw send me 100$
<harmw> it doesn't work that way harlowja 
 * harlowja waiting...
<harlowja> damn it
<harlowja> ha
<harmw> only smoser has such powers
<harlowja> :(
<smoser> these are not the droids you're looking for ?
<harmw> :p
<harmw> smoser: are there critical bugfixes needed before officially throwing out 0.7.6?
<smoser> well, other than some bozo, not mentining any names harmw, broke 'make test' with his suggested patch.
<harmw> bozo.. hm, that'll probably be that same guy who'd last broke something
<smoser> http://paste.ubuntu.com/8481544/
<harmw> ah
<harmw> hehe
<harmw> so I expect after changing lines 226 and 227 to read vtnet instead of eth that'll be fixed :p
<harmw> (lines in the netconfig test)
<smoser> pushed.
<harmw> why are there so many non-related lines changed?
<harmw> because of make tests or something?
<harmw> (pylint and such)
<harmw> hiren_: you should checkout trunk for your tests, and test it :)
<harmw> hm, I should've just branched.. now I don't get no points!
<hiren_> harmw: I'll have to start n/w firtst right? to get bzr installed
#cloud-init 2014-10-03
<harmw> yes hiren_ 
<harmw> else you can't use pkg :)
<smoser> harlowja, around ?
<harlowja> yo yo
<harlowja> sup dawg
<smoser> bah. i meant harmw but that cna't hurt
<smoser> :)
<smoser> bsd.
 * harlowja runs away
<smoser> well, at some poitn i want to talk to harm.
<harlowja> sooo bsd
<smoser> wanted to know what harm was going to do there.
<smoser> upload to bsd. not sure what that really means. other than being available in ports.
<smoser> if it means something more or there is a path to something more.
<smoser> i dont know much about freebsd.
<harmw> I'm not here smoser 
<harmw> this is my answering robot
<harlowja> lol
<harmw> and you should be afraid of this robot 
<harmw> especially you harlowja 
<harlowja> ha
<harlowja> my robot is bigger than yours
<harlowja> maybe u should be afraid instead 
<harlowja> ha
<harmw> no-o
 * smoser tries to think of some joke wrt yahoo and robots.
<harmw> mine is bigger than the world
<smoser> but fails
<harlowja> mine is like infinity big
<harlowja> ha
<smoser> surely there is some way i can make fun of yahoo here. but i'm missing it. i must be worn out.
<harmw> na'ah, that doesn't count
<harlowja> :)
<harmw> smoser: why make fun of Y!, if you have harlowja around :>
<JayF> smoser: how about a Yawho?
<harlowja> isn't it the same thing ;)
<harlowja> harlowja == yahoo :-P
<harmw> anyway, smoser sup
<harmw> hehe, that would explain the rapid decline of their stock value
<harlowja> lol
 * JayF shouldn't make fun of Y! after that amazing food at the Ironic mid-cycle last year
<harlowja> ha
<harlowja> my hope is that marissa starts spending that $$ from alibaby on something good
<harlowja> alibaba
<harlowja> or whatever
<harlowja> and a personal jet for me wouldbe cool
<harmw> personal? since no pilot would want to fly you around? :P
<harlowja> robot pilot ftw
<harmw> smoser: what you wanted to know
<smoser> what did you intend to do wrt freebsd ?
<smoser> you'd upload it somewhere ? i know nothing about this process.
<harmw> I already have a makefile and such so Ill upload that at bugs.freebsd.org
<harmw> once thats accepted, it lives in ports
<harmw> and ppl can install it using pkg install py27-cloud-init
<harmw> just requires a version bump on your end
#cloud-init 2014-10-04
<harmw> harlowja_at_home: seems CERN is interested in RDO for el6, just like Y!
<harlowja_at_home> hmmmm, RDO
<harlowja_at_home> i'm not sure RH wants to further RDO for el6 though ;)
<harlowja_at_home> el7 is the future, ha
<harmw> lol yes
<harmw> but I want just Juno, not an upgrade to centos7 as well :p
<harmw> not yet
<harlowja_at_home> ya, well wait till juno is released in like 2 weeks, lol
<harmw> there is a topic on the RDO ML about this
<harlowja_at_home> once y! starts to get juno running (without RDO) i'll have the packages that u can use
<harlowja_at_home> *without RDO
<harmw> and how whould they differ from RDO?
<harlowja_at_home> do u have a link to the RDO ML?
<harmw> sure
<harmw> sec
<harmw> https://www.redhat.com/archives/rdo-list/2014-September/msg00122.html 
<harlowja_at_home> harmw, packages are built from upstream, so in reality they don't differ much,  https://github.com/stackforge/anvil builds them automatically, code is all there and stuff if u want to use it
<harlowja_at_home> specs @ https://github.com/stackforge/anvil/tree/master/conf/templates/packaging/specs
<harmw> and those specs do the same stuff RDO does?
<harlowja_at_home> pretty much
<harmw> 'pretty much' :p
<harmw> you already got that stuff packaged somewhere?
<harlowja_at_home> get a VM, and it will pop out the packages for u ;)
<harlowja_at_home> it generates the packages, lol
<harmw> I'm not particularly interested in building a crapload of Juno packages by myself :P
<harlowja_at_home> ya, i can probably get them published somewhere when i/others get anvil working for juno
<harlowja_at_home> those spec files probably need some tweaking for juno
<harmw> ye, they look old :p
<harlowja_at_home> but the idea for anvil is that it sucks down git repos, analyzes there python requirements, fills out those spec files, finds and downloads dependencies, builds those all into rpms
<harlowja_at_home> *builds those into source rpms
<harlowja_at_home> and then has a further stage to convert all that into rpms
<harmw> just like rpmbuild handles my perl dependencies (sometimes wrong or competely unwanted) 
<harlowja_at_home> :)
<harlowja_at_home> hey, u can change anvil at least ;)
<harmw> hehe
<harlowja_at_home> i even made terminal recordsing @ http://anvil.readthedocs.org/en/latest/topics/examples.html
<harlowja_at_home> a while ago, haha
<harlowja_at_home> u can watch it
<harlowja_at_home> haha
<harmw> lol
<harlowja_at_home> *super exciting*
<harmw> omfg
<harlowja_at_home> lol
<harmw> superdupercool :P
<harlowja_at_home> ha
<harlowja_at_home> ya, showterm.io was neat when i found it
<harlowja_at_home> lol
<harmw> indeed
<harlowja_at_home> although the viewer is still somewhat buggy
<harlowja_at_home> still doesn't always scroll right
<harmw> so what if I wanted to build me some el65 packages for Juno RC1
<harmw> ha no
<harlowja_at_home> first watch http://showterm.io/12c29e87094f128d945fa/ 
<harlowja_at_home> but anyway, u can try anvil
<harlowja_at_home> and give ./smithy a --origin option
<harlowja_at_home> and make an origin file for https://github.com/stackforge/anvil/tree/master/conf/origins for juno rc1
<harlowja_at_home> and see how it goes
<harmw> what's it doing with passwords in that 'movie' ?
<harmw> and this stuff doesn't require root, right?
<harlowja_at_home> building rpms does
<harmw> still?
<harmw> wasn't that squashed ages ago?
<harlowja_at_home> for when say later if anvil installs say mysql, it will need root to, and a password
<harmw> could be wrong on that though
<harmw> ah ok
<harmw> makes sense
<harlowja_at_home> maybe, i haven't tried turning it off in a while
<harmw> you happen to know if epel6 contains all requirements?
<harlowja_at_home> not likely
<harmw> regarding python versions and stuff
<harlowja_at_home> anvil will scan epel for requirements so that it doesn't have to build them
<harlowja_at_home> it usually doesn't find them all there
<harlowja_at_home> seems to always be some missing, lol
<harmw> hehe, but if that happens anvil will build it itself?
<harlowja_at_home> yup
<harlowja_at_home> or try to 
<harmw> sounds like anvil is holy grail
<harlowja_at_home> ha
<harlowja_at_home> depends if it works for u
<harmw> ok, so the movie says packaging blabla
<harmw> but that is not rpmbuild, right?
<harlowja_at_home> thats a source rpmbuild build
<harlowja_at_home> which doesn't require root
<harmw> ah ok
<harlowja_at_home> the non-source-rpm (build stage) requires root
<harlowja_at_home> http://anvil.readthedocs.org/en/latest/topics/examples.html#building
<harlowja_at_home> ah one reason it requires root to build
<harlowja_at_home> 'Installing build requirements'
<harlowja_at_home> that one won't work so well without root ;)
<harmw> makes sense :)
<harlowja_at_home> so ya, anvil builds its own RDO in a way :-P
<harmw> hehe
<harmw> well my bigest concern with packaging by myself is keeping it updated is also something I need to do, by myself
<harmw> and that sucks
<harlowja_at_home> ya, well for just local testing and stuff, it should be fine
<harlowja_at_home> if u don't have resources to keep on running anvil and stuff, then ya, go with RDO
<harmw> yea well
<harmw> :)
<harlowja_at_home> although y! and godaddy (which have been using anvil) aren't running against trunk openstack, so we sometimes hit anvil issues when moving to the next release, aka, binaries move around or some crap
<harmw> hmk
<harmw> well I'm not neccesarily after trunk
<harmw> stable is fine :p
<harlowja_at_home> ya, i don't think stable is juno yet though ;)
<harlowja_at_home> so u are in a transitionary period, lol
<harmw> :P
<harmw> I guess it'll take another 10 yrs for openstack to reach a true stable point :>
<harlowja_at_home> 100
<harlowja_at_home> 1000 years
<harlowja_at_home> lol
<harmw> hm, or I could just start reading about this inplace centos6>7 upgrade path
<harlowja_at_home> or that
<harmw> hm hm, well that actually sounds like a plan
<harlowja_at_home> let me know how it goes
<harmw> it'll nuke my designate setup, probably
<harlowja_at_home> ya, thats the part i'm worried about, epseciall when u have alot of hypervisors on el6 (like yahoo does)
<harlowja_at_home> not so good to nuke hypervisors
<harlowja_at_home> lol
<harmw> designate isn't the hypervisor
<harlowja_at_home> *that sort breaks the inplace upgrade, haha
<harmw> its openstack-designate, which I'm running in this centos6 vm I'm targetting for an upgrade :p
<harlowja_at_home> gotcha
<harmw> and I built the rpm myself, so ditched systemd
<harmw> http://wiki.centos.org/TipsAndTricks/CentOSUpgradeTool
<harmw> ok, lets just see what happens :)
<harlowja_at_home> good luck
<harlowja_at_home> haha
<harlowja_at_home> hmmm https://www.redhat.com/archives/rdo-list/2014-September/msg00128.html
<harlowja_at_home> i know perry
<harlowja_at_home> hmmm
<harlowja_at_home> sucks for people using RDO :(
<harlowja_at_home> * https://www.redhat.com/archives/rdo-list/2014-September/msg00133.html
<harlowja_at_home> they know about anvil to... argggggg
<harlowja_at_home> *at least some of that team knows about anvil
<harlowja_at_home> stuff like 'We'd love to have people outside of the Red Hat engineering team doing some of the packaging work that we lack either the time or the priority to do. ' when anvil does exactly this confuse me :-P
<harlowja_at_home> maybe i should get on that ML and post, anvil...
<harmw> well, yes
<harlowja_at_home> lets see if i can do that
<harlowja_at_home> they'll probably get all pissed at me, why would people pay for RDO then haha
<harmw> but isn't RDO the free countrpart of rhel openstack? I never realy got that :p
<harlowja_at_home> hmmm, not sure, anything associated with RH makes me think of $$
<harlowja_at_home> lol
<harmw> hehe
<harlowja_at_home> i guess i've been in to many of there meetings which end up becoming, use RH product XYZ at yahoo :-P
<harmw> hhe
<harmw> I only know of CentOS :p
<harmw> though that's about to change, since new $job uses RHEL
<harmw> and AIX :>
<harmw> ok, running # preupg -s CentOS6_7
<harlowja_at_home> ya, i'm debating on whether i want to send mail to rdo list about anvil
<harlowja_at_home> lol
<harmw> just go for it :>
<harlowja_at_home> hmmm, maybe
<harlowja_at_home> lol
<harlowja_at_home> bb, i'll think about it, ha
#cloud-init 2015-09-28
<jtheuer> I want to sign the newly generated ssh host keys (obviously between key generation and sshd start) -- how could I do that with cloud-init?
<smoser> jtheuer, wouldn't you need to have access to the signing private key on the instance ?
<smoser> natorious, as much as possible, yeah. that would be the case.
<jtheuer> smoser, yes, I would need the key but fetching it isn't the difficult part.
<openstackgerrit> Scott Moser proposed stackforge/cloud-init: LICENSE: correct wording with respect to Apache 2  https://review.openstack.org/228471
<larsks> Merging: can I specify cloud_final_modules in usedata and have it *merge* with whatever the defaults are in /etc/cloud/cloud.cfg? I think I recall recently talking to smoser about this and the answer was "no", but I forget...
<smoser> well, you can . but "merge" is sucky for lists by default.
<smoser> as for lists, merge is just 'replace'
<smoser> you can manage to prepend or append with cloud-config-jsonp
<larsks> Actually, I think that may not be true.  I found our prior conversation, in which you saidL
<larsks> <smoser> it appears that you cant patch builtin config
<larsks> <smoser> you can re-define the whole list though
<smoser> yeah.
<smoser> :-(
<larsks> Yeah.  No worries. I knew we had talked about it recently.
<smoser> probably thats right. i do recall that conversation.
<smoser> jtheuer, you could probably manage to write a upstart job that would do it. (or systemd job)
<smoser> and have a bootcommand write the upstart job
<smoser> have that start on starting sshd
<smoser> and it could pull your keys (i'm guessing from a https:// do the sign and destroy the keys)
<smoser> or you coudl have a key signing post service that it could post to that you'd give it one time use url
<jtheuer> smoser, I currently do it in runcmd but then also needs an ssh restart. But if there is no clean way to do it between key creation and ssh start I'm fine with that.
<smoser> you should be able to write systemd or upstart jobs that start on starting ssh
<smoser> and do it there.
<jtheuer> I would to do it *before*
<smoser> well, 'starting' is before
<jtheuer> hmm, didn't know that... !
<jtheuer> thanks for the pointer
<smoser> starting(7)
<smoser> starting - event signalling that a job is starting
<smoser> jtheuer, http://manpages.ubuntu.com/manpages/trusty/man7/starting.7.html
<smoser> that is probably good for upstart, and i'm sure you can find a similar thing in systemd, but i dont know it off the top of my head.
<smoser> jtheuer, i'm curious how you're acommplishing it, and i'd be open to something to do it in a sane generic way in cloud-init
<smoser> the other thing you could do is pass the keys into the instance
<jtheuer> Haven't followed the last war on system init, currently we use ubuntu LTS which uses upstart. I'll check if that is still true for the next LTS version
<smoser> it will change to systemd in 16.04. Horay for time lost to re-doing things that are already done!
<smoser> you *can* pass keys into the instance reasonably securely, with #include-once
<jtheuer> smoser, that would be cool. I see two ways depending on the user's needs: First, add private, public and cert host to cloud-init file. Or second let the instance generate new keys and sign it with a supplied private ca key. (I currently fetch the key from AWS S3 via a script but that is probably not the final version)
<jtheuer> Then, a line for the host keys certificate has to be added to the sshd config: "HostCertificate /etc/ssh/ssh_host_rsa_key-cert.pub" >> /etc/ssh/sshd_config
<jtheuer> Adding all three files (private, public, cert) to cloud-config.yaml has security advantages, the ca file stays on your local computer.
<smoser> yeah, but without '#include-once', and a read-once url, you've provided non-root user onthe instance with the ssh private key
<jtheuer> You mean all users can read the original cloud-init.yaml file?
<jtheuer> So this is a general issue when you add an ssh private key to cloud-init. Never thought about it... Luckily we only have admins users so far but still not very nice.
<smoser> jtheuer, well, the user can't read the file on disk
<smoser> but if it came from an http web service, it is still probably there (ie, on ec2 its in ec2 user-data that is accessible to anyone on that host)
<jtheuer> Then, I probably didn't get what you mean with your concern about the ssh private key in cloud-init (like it is possible now)
<jtheuer> or do you refer to the AWS metadata implementation as http endpoint? This is indeed a bit strange that everybody can read the full instance details.
#cloud-init 2015-09-29
<harlowja> smoser how goes the git stuff??
<harlowja> did u get gitty
<harlowja> gitty with it
<smoser> git gitty with it.
<smoser> we can get there, harlow.
<harlowja> :)
<harlowja> do i have to git gitty with it to?
<harlowja> lol
<smoser> na na nanananaaannannan
<harlowja> :-P
<pm90_> hello, my backslashes in a #cloud-config file get replaced by undefined on the cloud server....is this a known issue in cloud init?
<smoser> pm90_, example ?
<pm90_> smoser: https://gist.github.com/pratikmallya/71562b7afc643b20e0be
<smoser> pm90_, hm. it should not do that
<harlowja> smoser have u finished getting gittty with it
<harlowja> lol
<harlowja> daily gitty with it reminder
<harlowja> lol
<smoser> tell you what.
<smoser> you figure out why iscsi root has transient resolvconf issues
<smoser> and i'll do the git thing
<smoser> :)
<harlowja> :-P
<harlowja> k, i fixed it
<harlowja> transient, cause of the moon
<harlowja> so the red-moon thing passed, it should be all fixed
#cloud-init 2015-09-30
<openstackgerrit> Nate House proposed stackforge/cloud-init: [WIP] Updating openstack.base to support configdrive inheritance  https://review.openstack.org/225465
<openstackgerrit> Nate House proposed stackforge/cloud-init: Removed buffer return from http source vendor data  https://review.openstack.org/229246
<natorious> morning!
<smoser> good morning natorious
<natorious> hopefully I did that patch right ^^.  Didn't see anything else mentioned to directly address.  I'd fixed that spelling thing via amend.
<claudiupopa> hey folks.
<claudiupopa> Are we doing a meeting today?
<smoser> yes.
<natorious> got a ways on configdrive and was looking at doing that as a separate request for that and it'd make it read a whole lot better w/ that in etc.
<openstackgerrit> Merged stackforge/cloud-init: LICENSE: correct wording with respect to Apache 2  https://review.openstack.org/228471
<natorious> was planning a separate request for network json config too.  Feel free to correct me if I'm going about things wrong or backwards
<natorious> hi claudiupopa o/
<claudiupopa> Hi natorious \o!
<smoser> hey.
<claudiupopa> so irc or hangouts?
<smoser> irc i think for now.
<smoser> ok. so well walk over list of reviews quickly, and then open discussion and we can talk to natorious some there on his questions.
<smoser> https://review.openstack.org/#/q/project:stackforge/cloud-init+status:open,n,z
<smoser> i just accepted a license file change.
<smoser>  https://review.openstack.org/#/c/228471/
<smoser> the change was to make the LICENSE file not self contradictory.
<smoser>  https://review.openstack.org/#/c/228471/1/LICENSE
<claudiupopa> I'm not a lawyer, but looks good to me. ;-)
<smoser> as previously it basically said "if you want to offer as apache 2.0 only, then put the GPLv3 header above"
<smoser> which doens't make sense :). now its says
<jgrimm> +1
<openstackgerrit> Nate House proposed stackforge/cloud-init: [WIP] Updating openstack.base to support configdrive inheritance  https://review.openstack.org/225465
<smoser>  "if you want to offer as apache 2.0 only, then put apache header above"
<smoser> thansk to jgrimm for that.
<smoser> i think reasonably thats all we have for the reviews. i think we should just help natorious a bit, with his questions and then we should try to get moving.
<smoser> i have a couple topics for open discussion
<smoser> is that ok? (i'm not skipping the reviews, i do want to talk about them, but i dont think they're jsut "quick")
<smoser> Odd_Bloke, you here ?
<claudiupopa> Sure, let's go with open discussion then and see where natorious can use our help.
<Odd_Bloke> o/
<smoser> k
<smoser> heres hwat i had foro open discussion topics
<smoser>  * python2.6 support: shall we bother with this?
<smoser>     + need someone to look at viability of 2.7 on rhel 6.
<smoser> the primary immediate driver of this is desire of using taskflow.
<smoser> but generally.. if we can do without 2.6, that might be nice.
<smoser> the goal would be to have *some* solution for running this on rhel 6
<claudiupopa> It would be nice to not need it, yes, but it depends after all on your supported platforms.
<smoser> and probably some sles, but i'm not so familiar with sles family of products.
<claudiupopa> so the problem is rhel 6?
<smoser> well, and probably some sles.
<claudiupopa> when it's scheduled for EOL?
<Odd_Bloke> The heat death of the universe (which will largely be caused by people having to maintain software for RHEL 6). :p
<Odd_Bloke> Wikipedia says 2020.
<natorious> they have a 10 year cycle :/
<claudiupopa> in the same year as python 2.7
<claudiupopa> not a coincidence.
<Odd_Bloke> More, if we want to cover their extended lifecycle support. :p
<smoser> i'm very  much ok to say "supported with python2.7 on rhel 6" if there is some sane path to getting python 2.7 on rhel 6.
<smoser> but not if that path includes "go to some dude's web site" or "download python2.6 and compile"
<smoser> ie, i'd accept 'enable EPEL'
<Odd_Bloke> Is there going to be a path to installing cloud-init 2.0 on RHEL6 that fits your criteria?
<smoser> thats what i want to know. i'd like someone to dig on that a bit.
<smoser> smatzek, might know. also might have info on sles
<natorious> what about pinning taskflow to before the 2.7 dep change?
<claudiupopa> that doesn't sound that good.
<claudiupopa> And if I'm not mistaken, the problem is with a dependency of taskflow.
<smoser> well, apparently taskflow isn't itself 2.7 only.
<smoser> right.
<smoser> but generally speaking if we can say "no 2.6" then that woudl make life easier.
<smoser> so i'm looking ofr a way, and a sane path for that.
<smoser> i kond of hoped someone woudl take this and run with it.
<smoser> i guess i can do that a bit after meeting
<smoser> ok. so next topic, then.
<claudiupopa> Btw, let's use trello for this as well.
<smatzek> I would have the same criteria as smoser, if there is a sane path to getting 2.7 on rhel 6.  I don't have info on if there is a sane path for that or not.  I don't have the "python version included with" info for SLES 11 or 12 but could dig it up without much trouble.
<smoser> claudiupopa, good idea.
<smoser> claudiupopa, you want to add a task and give it to me ?
<claudiupopa> yep.
<smoser> smatzek, ok. you can help me :). thanks for offering.
<smoser> google doesn't seem to know about assane path to 2.6 that does not include "people.readhat.com/some-dude" (yes, thats better than some.dude.com, but...)
<smoser> anyway..
<Odd_Bloke> That's because you have to be insane to still be running RHEL6. :p
<natorious> we could make a rhel/cent rpm for alt installing python2.7 that would work though
<smoser> natorious, as in smoser maintain python2.7 on rhel ?
<smatzek> I still like Odd_Bloke's suggestion from several weeks past that those who bundle / ship python 2.6 after its EOL be made to sit in a corner and think about what they've done.
<smoser> smoser == some-dude-i-wouldn't-trust
<natorious> iir it can't replace the python2.6 due to yum or some such
<smoser> natorious, yeah, python2.7 would be the name installed. i'd never expect that /usr/bin/python would be 2.7.
<smatzek> maybe we just hande RHEL 6 and those other OS levels on 2.6 out to dry with cloud-init 2.0 and if those distros really want it, they get python 2.7, on their distro.
<smoser> but i really dont want to maintain pytthon package.
<smoser> smoser, that is how i am leaning
<smatzek> hande = =hang
<smoser> ok.
<smoser> lets move on.
<smoser> actually one more thing here.
<Odd_Bloke> Well, they could also maintain distro patches for taskflow/networkx/cloud-init 2.0 which would make it work on Python 2.6; the point is it's their problem. :p
<smoser> please everyone here just join cloud-init mailing list . https://launchpad.net/~cloud-init
<smoser> and i'll use that list to write to.
<smoser> k?
<Odd_Bloke> Done.
<smoser> smatzek, natorious claudiupopa you are all actioned to join that :)
<smoser> ok. natorious now lets move on.
<smoser> you want to ask some questions to the channel ?
<natorious> yeah, sure
<natorious> so configdrive
<natorious> do we want to support v1 and v2?
<claudiupopa> done.
<natorious> are the only differences vfat vs iso
<smoser> i really dont care about v1.
<claudiupopa> What's the difference between v1 and v2? We're also interested in vfat for cloudbaseinit for instance.
<smoser> it was pretty bad i think
<smoser> and i think that v2 has been in place for probably 2 years ?
<smoser> https://review.openstack.org/#/c/11184/
<smoser> s/2/3/
<smoser> no one other than possibly natorious's employer is running a 3 year old openstack cloud :)
<natorious> burn
<natorious> -.-
<smoser> natorious, do you have a need to support config drive v1 ?
<smoser> i'm reallynot terribly opposed to it.
<smoser> but it woudl not land high on my priority list
<natorious> I somewhat feel like most of the detection can be refactored down to detect umounted blockdevs searching f iso9660 and vfat fs w/ the same dir structure
<smatzek> anyone have links to a spec or other doc that describes v1 and v2?
<smoser> well, we dont want to assume 'unmounted'
<natorious> k
<smoser> really we want to look for the filesystem label
<natorious> k, that might make it even easier
<smoser> that seems to me to be the biggest thing.
<smoser> it would seem to me to be a low possibility of non-desired false positive if you found a drive with a label 'config-v2'
<smoser> s/drive/drive or partition/
<smoser> ie, if someone put something like that... they probably did it so you would find it.
<smoser> now... obviously such a person could be malicious
<natorious> are we still doing an on first boot hook?
<smoser> thats a good question
<natorious> didn't see any examples of that in the http ds
<smoser> cloud-init 0.7 basically always searches for a datasource.
<smoser> (on every boot)
<smoser> unles the  user sets 'manual_cache_clean' or some such
<smoser> the reason for that is so that it can detect "new instance".
<smoser> so that the user who does:
<smoser>   * boot instance
<smoser>  * install a package
<smoser>  * shutdown and snapshot
<smoser> doesn't have to also do: 'run cloud-init clean'
<natorious> right
<smoser> now that search is painful
<smoser> in many ways.
<natorious> its also ordered and limited by the datasources defined config right?
<smoser> as for config drive, it means that if you messed up the label, then the instance would get all foobarred.
<smoser> right.
<smoser> we could possibly take a position that we do a quick check to see 'is this a new instance'
<smoser> and only go looking further if that quick check failed
<smoser> and if that quick check was buggy in that it did not recognize a new instance when it should have, we tell the user 'well, run "clean"'
<smoser> i'm open to ideas here, and i dont knwo what a good "quick check" is.
<smoser> i suspect that a lot of people do something like the above
<Odd_Bloke> BTW, on Python 2.7 in RHEL6: https://twitter.com/ncoghlan_dev/status/649228375135903744
<smoser> so that probably needs to generally work
<smoser> Odd_Bloke, scl that references http://people.redhat.com
<Odd_Bloke> I found this: https://www.softwarecollections.org/en/scls/rhscl/python27/
<smoser> natorious, you ahve thoguths on that ?
<smoser> i'd like to not have to search every boot
<natorious> stuck on quick check.  So if the instance is new but from a snap
<natorious> it could have previous instance data in it
<smoser> right.
<smatzek> I don't think there is a "quick check".  I think the only real check you can make is to query datasources and get the instance ID from them to compare with ID on disk.
<natorious> instance uuid is sourced from ds so we couldn't use that to determine caching of ds method
<smoser> well, there might be a heuristic that is sane.
<smoser> similar to what MS does / did when you bought a new hard drive and you had to call their customer support to allow you to run windows again.
<natorious> there's also hw identifiers that can be used no?
<smoser> ie some look at the system (hard drive ids, cpu number ... might be a suffiicent quick check)
<natorious> so like tie an instance id data to hw identifiers somehow
<natorious> yeh
<smatzek> checking hw identifiers doesn't work because you could have done a cold VM migration.
<smatzek> or live followed by a reboot.
<smoser> in which case the 'quick check' said "looks like it changed".
<smoser> so we'd just do a longer check.
<smoser> and if thats annoying to the user, then can still manually set "manual clean" or something.
<smoser> you see what i mean ?
<natorious> live migrate wouldn't hit cloud-init until a new reboot too
<smatzek> I've struggled with these checks to see if you're a new instance for several years with both cloud-init and non-cloud-init based VM activation.  I've found that cloud-init has the best way to check, with instance ID.
<smoser> well, and live migrate would surely have the same (virtual) hard drive id
<natorious> so processing the ds detection and determining the same instance uuid should keep things going still
<smoser> right.
<smatzek> a quick check to see if HW changed would not trigger a deeper check if you too a VM snapshot and created a new VM using the snap on the same physical hardware.
<smoser> smatzek, correct.
<smoser> in which case, a user there would have to tell cluod-init that it should check every time.
<smoser> i dont knwo how i feel about it.
<smatzek> if we had a setting like that users would have to flip it on if they plan to snap and deploy snaps.  Also, wouldn't a hardware quick check require some hardware info in the base image for the public images you create?  I suppose you could put bogus hw info in there so as not to disclose the actual HW you're creating them on.
<smoser> smatzek, well,. to be fair, on a sane cloud.. if you did that, and your new isntance landed on the same "host".
<smoser> (or similarly configured host)
<smoser> you'd still hope that you'd prvide new "hard drive" serial number to the virtual disk.
<smoser> i guess if in fact you'd  handed the guest the same physical disk..
<smoser> smatzek, well on the offical published images, we'd clean whatever data is tehre.
<smoser> but the ubuntu image build process never invokes cloud-init
<claudiupopa> what about windows?
<natorious> there could be instances that use vdi chains for image caching who share a common base
<smoser> claudiupopa, thats why you're here :)
<natorious> but I think the vdi snaps have unique sn's
<claudiupopa> First of all, I don't think I know what's HW. :-)
<smoser> i'd think they would, natorious . but you're right, they dont necessarily. but you'd think it would be desireable.
<smoser> ok..
<smoser> so lets say this:
<smoser>  * cloud-init should have a cleanly documented way to explicitly disable or explicitly enable per-boot checking for instance id
<smoser>  * cloud-init may provide a "auto" mode for that that does some heuristics based checking or other quick checking
<smoser>    the auto mode shoudl generally be *very* quick and not be skewed to not have false negatives (when it missed a change that it should have seen)
<smoser> does that seem sane ?
<natorious> * cloud-init cares not re v1 configdrive
<smoser> unless natorious makes it care, which is ok
<natorious> yea
<smoser> one example i'd like to give of a very sane and quick check would be on lxd.
<smoser> lxd will expose its datasource via a socket /dev/lxd.  it seems a sane "quick check" to do the unix-socket query 'get-instance-id' if /dev/lxd is present.
<smoser> that suggests that the quick check should have access to the datasource that was used last... so that it woudl know it was lxd, and if previous was lxd and there is no /dev/lxd, then probably a change occurred.
<natorious> interesting, I'd have to look into that and see about setting up a test env too
<natorious> being socket I take it means not listed under block devs
<smoser> yeah. its just a unix socket that lxd exposes an api on from outside.
<smoser> i think that vmware has a similar thing.
<smoser> did you have other question, natorious ?
<natorious> think thats it
<natorious> got what I need to get another request in.  its going to depend on the previous but should be alright
<smoser> natorious, cool
<smoser> claudiupopa, you going to stick around for a while ?
<smoser> or are you EOD
<claudiupopa> I'll be here for some time.
<smoser> ok. i'll ping in a bit
<stanguturi> Hi, I am working on cloud-init 0.7.6. I was able to execute './packages/bddeb' command and successfully build the debian package. I installed the final debian package in Ubuntu machine. Can anyone provide some instructions to test the changes. I mean, are there any log files which say that the package has been run, etc.
<natorious> hi stanguturi!
<natorious> cloud-init --version should print out the installed version
<stanguturi> hi natorious
<natorious> if your looking for some specific source change you made prior to building the pkg
<natorious> you can check the installed source too
<natorious> typically in /usr/lib64/pythonX.X/site-packages/cloudinit etc
<natorious> X.X being 2.7 or 3.4 and whatnot etc
<natorious> depending on the targetted init system specified when building the pkg would depend on where what init scripts or service files get installed too
<stanguturi> great. Wher are the log messages that get printed when the cloud-init runs on machine boot
<natorious> but like if its not running on boot, perhaps you targetted the incorrect init sys for your distro version
<natorious>  /var/log/cloud-init.log or /var/log/cloud-init-output.log
<natorious> one of the previous versions had a thing where most log msgs would go to syslog though
<natorious> to change that you can comment the log line in /etc/cloud/cloud.cfg.d/05_logging.cfg for it
<stanguturi> :x
<natorious> like # - [ *log_base, *log_syslog ]
<natorious> leaving  - [ *log_base, *log_file ] uncommented etc
<natorious> might not be an issue for you.  Thought worthy of a mention though
<stanguturi> ok. What is the best approach to test the changes. I am thinking of following 'make changes, build deb, install deb and reboot'.. Is there any other simple way to test the changes/
<stanguturi> :x
<natorious> what init system are you using?
<natorious> after making the change and reinstalling, rm your /var/lib/cloud dir and it'll be like its bran new etc
<natorious> don't think you necessarily need to reboot either
<natorious> you could probably just fire off the init scripts again
#cloud-init 2015-10-01
<Odd_Bloke> smoser: So I think https://access.redhat.com/documentation/en-US/Red_Hat_Software_Collections/1/html/1.0_Release_Notes/chap-RHSCL.html#sect-RHSCL-Features is enough for us to be happy that Python 2.7 is viable on RHEL6.
<Odd_Bloke> "all Red Hat Software Collections components are fully supported under Red Hat Enterprise Linux Subscription Level Agreements, are functionally complete, and are intended for production use."
<Odd_Bloke> smoser: And, in fact, GCE Red Hat 6 instances come with the Python 2.7 SCL already installed.
<gin> Hello, would you be kind to help me out with Cloud-Init configuration? I want to bootstrap Puppet agent for CentOS 7 (AWS AMI). The given example http://cloudinit.readthedocs.org/en/latest/topics/examples.html#setup-and-run-puppet does not work. From the logs I can see puppet package is missing.
<gin> Is this the expected behaviour and I need to add puppet repository myself?
<smoser> Odd_Bloke, that is odd. why is that not even remotely what you get when you google python 2.7 rhel
<smoser> as recent as july, you have posts like this:
<smoser>  http://tecadmin.net/install-python-2-7-on-centos-rhel/#
<Odd_Bloke> smoser: If you're a RHEL user, I don't think you have to Google to find solutions.
<Odd_Bloke> You have Red Hat to tell you about them. :p
<smoser> that just doesnt make sense though.
<smoser> and if people are writing things like that blog post (who apparently *are* redhat customers) then surely they're not just doing it because they like compiling their own python
<Odd_Bloke> Well, that blog post is about CentOS, not RHEL.
<Odd_Bloke> smoser: CentOS appears to have SCLs as well.
<smoser> https://wiki.centos.org/AdditionalResources/Repositories/SCL#head-9c6aea9c13b921d5258446c4c5e5886571bdb741
<smoser> yeah.
<smoser> (x86_64 only)
<smoser> ok. i'm sold on 2.7 for rhel
<Odd_Bloke> smoser: GCE image for CentOS 6 also has Python 2.7 installed via SCL, so I think it's viable for both.
<gin> How to enforce sequential execution in cloud-init script? The following script adds repository after the attempt to install the package that requires it. http://pastebin.com/YK2SRBnZ
<gin> I can't seem to find any documentation regarding this...
<larsks> gin: modules are executed in the order in which they are listed in your cloud-init configuration (usually, /etc/cloud/cloud.cfg + whatever #cloud-config data you pass in)
<larsks> Typically, runcmd happens near the end of things.
<larsks> However, there is an "add repository" module that may be enabled to run before "
<larsks> ...before "packages"
<larsks> E.g., this example: http://cloudinit.readthedocs.org/en/latest/topics/examples.html#adding-a-yum-repository
<larsks> In centos and RHEL, the yum-add-repo module is enabled by default and configured to run before package-update-upgrade-install.
<gin> thanks larsks for calrifying. Will try it out now.
<gin> has anyone tried cloud-init puppet conf? (http://cloudinit.readthedocs.org/en/latest/topics/examples.html#setup-and-run-puppet)
<gin> for CentOS7
<gin> I get warning and hence empty puppet.conf file:  util.py[WARNING]: Running puppet (<module 'cloudinit.config.cc_puppet' from '/usr/lib/python2.7/site-packages/cloudinit/config/cc_puppet.pyc'>) failed
<smoser> gin,  i suspect it needs fixing.
<gin> smoser, seems like so.
<smoser> Odd_Bloke, https://code.launchpad.net/~daniel-thewatkins/cloud-init/shim_fixes/+merge/269199
<smoser> comments tehre.
<Odd_Bloke> smoser: Oh, yeah, I'd lost track of that.
<Odd_Bloke> Thanks!
#cloud-init 2015-10-03
<finisherr> Could anyone tell me how cloud init sets the routes?
<finisherr> I added a route to an image because it was having trouble accessing metadata though 169.254.169.254, which worked. But when I boot it, it totally removes the route I added
<finisherr> this is for ubuntu 14.04 cloud image in openstack
#cloud-init 2015-10-04
<hoonetorg> hi
<hoonetorg> isn't it a security issue, if vfat, iso9660 or http(s) is used as datasource?
<hoonetorg> if i put f.e. private ssh hostkeys into user-data, and the datasource is available via http
<hoonetorg> anybody can see the private ssh host keys with wget or curl
<hoonetorg> with vfat or iso9660 it's not that easy, but
<hoonetorg> imagine you use a cloud instance as terminal server (x2go+lxde or similar f.e.), then could any user mount the datasource via filemanager
<hoonetorg> it's mounted as user and the user has full read access to vfat or iso9660
<hoonetorg> afaik xfs, ext4 or btrfs ... are not supported as datasource filesystem
<hoonetorg> but they could solve that problem and make the data on the datasource virtual drive as secure as on the rootfs.
<hoonetorg> anybody thought about that yet?
<hoonetorg> I know, it's not the fault of cloud-init, because it only implements what cloud providers did.
<hoonetorg> probably i'm here at the wrong time to ask and should ask again, when people write each other here.
<hoonetorg> my conclusion is: do not put sensitive data into meta-data, user-data, vendor-data.
<kwadronaut> hoonetorg: it's irc. just stick around. for a long time. if you can't try the mailinglists.
<kwadronaut> sometimes it's quiet here for days, sometimes a lot of activity.
<hoonetorg> kwadronaut: hi, that's exactly what i thought. normally i connect to a channel, am quiet as a little mouse and hv a look what's going on in that channel
<hoonetorg> kwadronaut: but today i did opposite. i was so surprised, that datasources are so insecure, that i must ask immediately if somebody else thinks the same way.
<hoonetorg> will put this channel to my list and stay and reconnect and wait for "traffic" :)
<kwadronaut> if you take the example of private host keys, i wonder why you would preseed them and not generate them on your 'secure' cloud instance? entropy is already a big issue there, in general.
<kwadronaut> if your 'cloud provider' is simply you/your employer, ie, a private cloud, the issue should remain on the internal network anyway.
<kwadronaut> but i'm sure others can give you better answers.
<hoonetorg> should only be an example
<hoonetorg> i thought on deploying my salt minion keys via cloud-init
<hoonetorg> instances are deployed and redeployed multiple times, so salt-minion-keys must find their way to the instance.
<hoonetorg> once salt is running i can deploy ssh host keys and certificates and other sensitive stuff more or less secure via salt.
<hoonetorg> ...... but to get salt running .... i scratch my head
<hoonetorg> probably it's a better approach to do this more dynamic
<hoonetorg> create new keys on every new deploy of the same instance.
<hoonetorg> .... but that is like with ssh host keys then: your ssh hostkey changed.... man in the middle attack... do you want to remove the old key from ...line...
<hoonetorg> ------> yes
<hoonetorg> (did i reinstall that machine???)
<hoonetorg> kwadronaut: thx so far, your answer implies somehow my initial conclusion: no sensitive data in datasources.
<kwadronaut> yeah, i've also ran into that problem, and distributing changed ssh keys upon reinstalling yadayada, i'm not happy with how we do things now, and improvements would be welcome, as usual.
<kwadronaut> and there will be sensitive data in there, but we try to minimalise it.
<hoonetorg> every new  bare metal i use, gets an usbstick formatted with ext4 label "sensitive".  i put salt-keys on it (manually generated), chmod 400, chattr +i
<hoonetorg> that way data on the stick should be as secure as on rootfs (/)
<hoonetorg> but with vfat or iso9660 it's difficult
<hoonetorg> yeah but entropy is still an issue, for sad i hv no QKD device lying around for producing real random :)
<hoonetorg> (sorry for OT, i'm quite now)
#cloud-init 2016-10-03
<JayF> does the new support for network json support bonded interfaces or not?
<jroll> JayF: heh, thank you for asking for me
<rharper> JayF: jroll: yes, it supports creating bonds
<JayF> thanks, that's what I thought
<rharper> sure
<jroll> thanks!
<harlowja> smoser nrezinorn ok, a big one for u guys https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/307485
<harlowja> templatizes cloud.cfg
<harlowja> makes it so one file can serve multiple distros
<harlowja> ^ includes associating refactoring to make that easier to understand
#cloud-init 2016-10-04
<smoser> harlowja, hm.. i was suggesting you could render at setup.py (install) time.
<rharper> smoser: that hang we saw ... I think it may be related to system-resolvd;  I don't think that's getting started prior to cloud-init-local/net which means if we bring up networking but we don't have resolvd running, I believe it times out resolving (in my case, the GCE datasource)
<smoser> ah.
<rharper> I added After=systemd-resolved.service; Requires=systemd-resolved.service to cloud-init.service (which is when netmode runs)
<smoser> that could do it.
<rharper> we should chat with pitti maybe and see where that ought to come up
<smoser> http://pad.lv/1629868
<smoser> yeah.
<smoser> i treid to reproduce with the webhook reporter
<smoser> but it didn't work
<smoser> because i was using by ip address
<rharper> ah
<smoser> you dont have to add the 'Requires'
<smoser> and probably shouldnt really...
<rharper> hrm
<rharper> well
<smoser> After is sufficent
<rharper> it's sufficient, the question is if dns doesn't come up
<rharper> is it worth running ?
<smoser> yeah, i dont know whow i feel
<rharper> if we requires=networking , why not dns too ?
<smoser> networking doesnt require dns :)
<rharper> fair
<smoser> the internet worked without dns for quite some time
<smoser> heres my /etc/hosts file
<smoser> :)
<rharper> I need to play with the unit deps; systemd said I had a loop
<smoser> hm..
<smoser> lets just ask pitti
<smoser> i think he's in
<rharper> ok
<rharper> smoser: the digitalocean merge is causing unittest failures due to older mock (I'm on xenial, no mock 2.0 here), http://paste.ubuntu.com/23275308/
<smoser> rharper, fiddle.
<smoser> thank you for catching.
<smoser> proposed that merge ?
<rharper> y
<rharper> smoser: https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/307583
<harlowja> smoser well its either render at install, or render at runtime, i'm ok with either :-P
<smoser> it kinda seems overkill to render at runtime to me.
<harlowja> k
<smoser> rharper, where idid you get a mock that did not have that ?
<smoser> the assert_called ?
<smoser> bah.
<smoser> never mind.
<smoser> i was running your branch and trying to see it fail
<smoser> it doesnt fail there ! :)
<harlowja> alright, smoser https://git.launchpad.net/~harlowja/cloud-init/commit/?id=8a445f0811e8deca935f5781a84e8d08067b1b1e much smaller now
<harlowja> a new makefile step will render that
<harlowja> don't see how i can easily do it in setup.py time
<harlowja> so makefile step it is :-P
<harlowja> smoser rharper mock had some busted version a while ago that let u do self.assert_anything_u_want and it'd say true
<rharper> harlowja: yeah; I had seen that discussion
<harlowja> because it was thinking u were mocking a new method 'assert_anything_u_want'
<harlowja> vs checking prior asserts, lol
<rharper> heh
#cloud-init 2016-10-05
<smoser> harlowja, yeah. i fixed, and added a tox with an 1.3 mock version to test.
<smoser> well, rharper fixed.
<Odd_Bloke> rharper: cloudinit-analyze sounds <3 <3 <3
<smoser> magicalChicken, around ?
<jgrimm> smoser, fwiw, Wed are class days, so expect delayed response
<smoser> k. mail sent.
<magicalChicken> smoser, jgrimm: I just got the email about syslog support, I'll start adding that in to the cloud-init logging documentation later today or early tomorrow
<magicalChicken> It might actually be useful to put logging documentation into a separate section in the docs since its used everywhere
<jgrimm> thanks magicalChicken!
#cloud-init 2016-10-06
<powersj> smoser, time to chat?
<smoser> powersj, sure.
<powersj> smoser, sent your email a ho link
<rharper> smoser: when you're free, I wanted to chat about some snappy cloud-config
<smoser> ok. can i have 30 minutes to write some thoughts for powersj ? 3:00 eastern ?
<rharper> sure
<smoser> https://hangouts.google.com/hangouts/_/canonical.com/hangout-smoser <-- rharper
<rharper> y
<rharper> smoser: hrm, so I want to use ExtendedTempFile (which gives me the filehandle of the tempfile) and pass that into write_file; but it doesn't appear to take a fh ... ;  that'd be nice unless there's a better way (like maybe added use_tempfile=True to write_file) ?
<smoser> rharper, hm.
<smoser> so many things have been done so many times.
<smoser> would atomic_helper.write_file() be useful ?
 * rharper looks
<rharper> I think I'm OK to use it with my own context with util.ExtendedTempFile ;
<rharper> users of write_file know the name of the file they want to write
<rharper> I think adding temp to it would make it confusing
#cloud-init 2016-10-07
<prometheanfire> may be able to work on that gentoo stuff
<prometheanfire> finally
<prometheanfire> however, before I can switch my images from glean to cloud-init I need to figure out why I need to reboot the server to get networking working
<prometheanfire> because that's a thing
<prometheanfire> on first boot it doesn't find the networking config, second boot it doe
<prometheanfire> on first boot it doesn't find the networking config, second boot it does
<prometheanfire> uses the same data source both times
<prometheanfire> https://gist.github.com/prometheanfire/e7fe7d6b70ebcb8710ef6dfc1a681112
<prometheanfire> smoser: looks like anything using the old network backend won't bring up the interfaces automatically
<prometheanfire> in distros/__init__.py apply_network_config is run with bring_up=false
<prometheanfire> this calls the old backend with that false value
<prometheanfire> the old backend defaulted to true
<prometheanfire> smoser: I'm confused why you changed the default to false here 9c0a2abc8d2c0e390745ddb163f5eae07b20d61d
<prometheanfire> smoser: the default is true everywhere else
<prometheanfire> apply_network_config was run with bring_up set to false
<phatypus> hello, i have installed cloud-init 0.7.5 on centos 7.2.  i have a basic user-data file ready after reading the cloud-init examples (https://cloudinit.readthedocs.io/en/latest/topics/examples.html), but i don't know where to put the user-data file and how to force cloud-init to interpret the user-data file.  can someone advise?
<smoser> prometheanfire, it runs with bringup = false because it should run before the OS goes to bring up networking.
<smoser> so the OS then brings up the network as it would on normal boot.
<prometheanfire> smoser: on gentoo at least, for the first boot (when you add a service to startup automatically like is done) it needs to be brought up manually
<prometheanfire>  /etc/init.d/net.eth0 start
<prometheanfire> etc
<smoser> prometheanfire, hmm.
<smoser> so during boot, you can't add new network configuration.
<prometheanfire> not within the same runlevel
<prometheanfire> if you configured in the boot runlevel it'd work iirc
<smoser> do you change from 'boot' runlevel to another during normal boot ?
<smoser> (ie, can we just put cloud-init in that boot runlevel ? thats basically what we do other places, and is kind of the intent)
<prometheanfire> yep
<prometheanfire> I can test with that
<smoser> initially to me that sounds like the thing to do.
<smoser> cloud-init and cloud-init-local should run "as early as possible"
<prometheanfire> let me make sure that's what I do with glean, but iirc that's how it was fixed
<smoser> if that doesn't work, then we can change the caller to pass a flag other than bringup , or extend that one... saying that it is expecting the OS to bring this up.
<prometheanfire> yep
<prometheanfire>  subprocess.call(['rc-update', 'add', 'glean', 'boot'])
<prometheanfire> I think it's diskimage-builder that needs updating for it, but I'll work on it
<smoser> well can't the install somehow say that ?
<prometheanfire> this isn't debian where we instantiate services on install :P
<prometheanfire> cloud-init-local and cloud-init
<smoser> its not instantiating the service on install.
<prometheanfire> can/should the others be moved to boot?
<smoser> its saying when the service should run, or what services it should have.
<prometheanfire> config and final
<prometheanfire> well, for gentoo that's the same thing, given how service deps work
<smoser> config later, final "rc.local like"
<prometheanfire> well, they have deps between then, so ordering isn'g a problem
<prometheanfire> /etc/init.d/cloud-init-local
<prometheanfire> that should be before netmount right?
<prometheanfire> I imagine so because it's a datasource for configuring the network
<smoser> yes.
<prometheanfire> ok, I'll modify the package, think that is what is causing problems as well
<prometheanfire> oh, those are installed by cloud-init itself, https://git.launchpad.net/cloud-init/tree/sysvinit/gentoo
<prometheanfire> well, for now as a test I'll just sed that out
<prometheanfire> smoser: well, not sure if this is the full fix, but it did start networking properly https://gist.github.com/prometheanfire/9d248da8e6f42fd29a51bc1aa3ece7e9
<smoser> \o/
<prometheanfire> smoser: why is it trying to run that before net is set up?
<smoser> well, 'init' should not run until after net is up
<smoser> oh.
<smoser> i confused you
<smoser> i'm sorry
<smoser> i was wrong
<prometheanfire> ah
<prometheanfire> that's fixable then
<smoser> cloud-init local should run before net is up
<smoser> and block net coming up
<prometheanfire> k, this is fixable :P
<smoser> and cloud-init(-network) would then run *after* it is up
<prometheanfire> let me submit a pr
<prometheanfire> net generally comes up in the default run level
<smoser> gists are pretty wonderful
<prometheanfire> does cloud-init lay down the config or or cloud-init-local
<smoser> cldou-init-local will always render a networking configuration
<prometheanfire> cool
<smoser> well, once per instance.
<prometheanfire> just being clear :D
<smoser> i really need to write stage descriptions out better.
<prometheanfire> this is my diff
<prometheanfire> https://gist.github.com/cbf4a784c30156909809228f2cc1091f
<prometheanfire> and just cloud-init-local will be in the boot runlevel
<prometheanfire> cloud-init should actually be need net I think
<prometheanfire> actually, after is still better in this case
<smoser> prometheanfire, is 'netmount' "configure the network" ?
<smoser> or "mount network filesystems"
<prometheanfire> smoser: second one
<prometheanfire> but netmount needs net iirc
<smoser> why not be more direct ?
<smoser> before net
<prometheanfire> I am
<prometheanfire> you'll see
<smoser> ok.
<smoser> i was just reading that gist
<prometheanfire> :D
<prometheanfire> I changed since then
<prometheanfire> that was long long ago
<prometheanfire> now lets see if I can remember how to submit this
<prometheanfire> https://code.launchpad.net/~prometheanfire/cloud-init/+git/cloud-init/+merge/307969
<prometheanfire> think that's right
<smoser> Odd_Bloke, rharper cpaelzer i'm going to do something generally wrong..
 * rharper takes note
<smoser> in order to get Odd_Bloke's fix into
<smoser> yakkety
<smoser>  (systemd: Run cloud-init.service Before dbus.socket)
<smoser> i'm going to re-order the last 3 commits
<smoser> and re-write his changelog entry
<smoser> so that the ubuntu upload has only his change
<rharper> yeah
<smoser> and has a sane commit message
<rharper> just the single fix
<Odd_Bloke> FTR, I think my commit message is fine; people in the future will be less interested in "fixed something broken" than "here is why this line is here".  But feel free to do whatever you want. :)
<Odd_Bloke> +1 on the general approach.
<prometheanfire> smoser: let me know if any other info is needed on that pr
<smoser> Odd_Bloke, yes, but the previous commit message described that well
<smoser> so no need to duplicate that
<Odd_Bloke> Right, but wouldn't show up in blame.
<smoser>  http://paste.ubuntu.com/23289574/
<smoser> bah
<smoser> dbus.service
<Odd_Bloke> smoser: dbus.service?
<smoser> have you tested your chagen ?
<Odd_Bloke> By making that manual change a couple of days ago, yes.  Let me test again now to confirm.
<Odd_Bloke> smoser: Confirmed.
<rharper> smoser: I tested the dbus.socket in yakkety-daily instance on serverstack
<rharper> smoser: in particular, if you check, systemctl list-dependencies dbus.service; you can see it depends on dbus.socket
<rharper> and we'd like to ensure cloud-init runs before the dbus.socket is active
<rharper> to ensure nss fails fast
<Odd_Bloke> ^ this
<smoser> right.
<smoser> deed is done.
<smoser> pushed to master with re-worked history.
<smoser> 109340b3c16857f526a75d4237d753b8cb82035e was pre-rework
<smoser> https://git.launchpad.net/cloud-init/log/?h=ubuntu/devel now updated
<smoser> building and uploading
<smoser> uploaded http://paste.ubuntu.com/23289681/
<cpaelzer> smoser: rewriting history is what winners do, so it can't be too bad
<sather> hey, cloud-final service module is failing, where can I obtain logs for that?
<sather> cloud-init-output.log is not showing anything
<sather> and journalctl -u cloud-final.service is sparse as well
<prometheanfire> sather: /var/log/cloud-init.log ?
<prometheanfire> that's where I have it write to
<prometheanfire> doesn't even use a logging service and still writes there
#cloud-init 2016-10-08
<openstackgerrit> melissaml proposed openstack/cloud-init: Change assertTrue(isinstance()) by optimal assert  https://review.openstack.org/384009
<openstackgerrit> melissaml proposed openstack/cloud-init: Modify use of assertTrue(A in B)  https://review.openstack.org/384011
#cloud-init 2016-10-09
<hemebond> I'm trying to use cloud-config and the hostname setting with Debian Jessie on AWS EC2 but it's not setting the hostname. I've had to resort to my_set_hostname and my_etc_hosts. Is there a way to find out why it's not working?
<hemebond> I haven't found any errors in the cloud-init* logs. Also, when I reboot the instance it gets the hostname "localhost"
#cloud-init 2017-10-02
<robjo> smoser: blackboxsw So where do we stand with https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/331149
<robjo> From my point of view things become ever more confusing
<robjo> if we agree on the format then I will carry the patch that we have in the SUSE package, I need to move forward on that front
<smoser> robjo, sorry... we stand where i was in a sprint all last week and delinquent
<robjo> if the patch cannot move forward for whatever reasons then so be it, given the current state I do not see how fiddling anymore with this patch set before committing makes any sense as we have comments upon comments upon changes and more comments, I've lost track of it all
<smoser> i'll look and/or talk to chad today. oh. looks like chad just commented 11 minutes ago.
<smoser>  i'll read some
<robjo> thanks,
<robjo> smoser: and I still cannot reproduce the bot failure locally :( worse yet I do not understand what the log is telling me
<smoser> robjo, ok. i have one other thing i need to be looking at at the moment, but someone will get a look at it today
<robjo> smoser: thanks
<blackboxsw> robjo: I'm backing down on the patch suggestion, but I'll add some inline comments about some of the code in add zypper to reorder a bit of the required/defaulted  checks to avoid what feels like a bit of extra work.
<blackboxsw> meh robjo you took the patch. sorry 'bout that. nits etc.
<robjo> blackboxsw: took some of the changes yes, will look at the next set of comments
<blackboxsw> I'm looking over the latest and it feels good. just CI complaints on testing. I'm giving it a thorough review now (not a mid-trevel review)
<blackboxsw> mid-travel rather
<ajorg> question: I thought if my boothook had output I'd see it on the console and in the cloud-init-output.log, but I don't. Is this expected behavior? (on Ubuntu 16.04 LTS)
<smoser> ajorg, you should yes.
<ajorg> hmm, will have to troubleshoot after the meeting then
<ajorg> thanks
<smoser> yikes.
<smoser> meeting
<smoser> blackboxsw, rharper
<blackboxsw> oops I set wrong UTC time
<ajorg> too early? :-)
<blackboxsw> right should be now
<smoser> htis is right.
<smoser> yeah
* blackboxsw changed the topic of #cloud-init to: is Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj |  Next status meeting: Monday 10/2 16:00 UTC | cloud-init 17.1 released
<blackboxsw> oops
<smoser> is the but here, blackboxsw  ?
<smoser> should that work ?
<rharper> bah
<rharper> here
<rharper> bot
<blackboxsw> yep it should
<rharper> smoser: I see meetingology
<smoser> #startmeeting cloud-init
<meetingology> Meeting started Mon Oct  2 16:05:47 2017 UTC.  The chair is smoser. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
<meetingology> Available commands: action commands idea info link nick
<smoser> #topic Recent Changes / Highlights
<smoser> hm..
<smoser> 17.1 released https://lists.launchpad.net/cloud-init/msg00106.html
<smoser> thats the biggest thing and only thing i have for this topic
<blackboxsw> #link  https://lists.launchpad.net/cloud-init/msg00106.html
<smoser> horay for a release, thanks to those who contributed.
<rharper> \o/
<smoser> 17.2 is set for 2017-12-14
<smoser>  https://launchpad.net/cloud-init/+milestone/17.2
<ajorg> yay!
<smoser> feel free to target bugs to that release.
<rharper> #link https://launchpad.net/cloud-init/+milestone/17.2
<rharper> #info please target bugs to the next release
<smoser> does that do anyting ?
<rharper> in the meeting summary, it does
<blackboxsw> smoser: the links showed up in meeting minutes last time
<smoser> k. i always expected the bot to tell me that in a pm
<blackboxsw> meetingology didn't echo though
<rharper> well, smoser may have to do those
<smoser> #topic In Progress Development / Highlights
<blackboxsw> same. but something is up that needs attention.
<blackboxsw> I'll properly handle it when publishing
<rharper> cool
<smoser> Merge Proposals
<smoser> #link http://bit.ly/ci-reviews
<smoser> there are some there for sure. i know that robjo has some he's interested in, and i think ajorg's instance-identity deserves a look
<smoser> as well as simpletable for 17.2
<ajorg> I'd be grateful for both, yes.
<smoser> i am looking at the 'networkd' one which is a blocker as currently Ubuntu does not work properly on azure
<blackboxsw> we should have more bandwidth this week
<smoser> and all sysstemd-networkd systems will not work properly on CloudStack.
<smoser> anything else there?
<robjo> The task limit increase has been tested and solves our problem, thus merging would be great
<smoser> robjo, yeah. athat does seem unlikely to cause issue
<smoser> Trello Board
<smoser> # https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin
<rharper> #link  ?
<smoser> #link https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin
<robjo> https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+ref/taskLimit that's the branch for the task limit
<robjo> Sorry, I can never find the link for the open merge proposals :( problem exists between the keyboard and the chair
<blackboxsw> will get eyes on that today robjo as your addZypper is about wrapped
<smoser> Bugs
<smoser> #link https://bugs.launchpad.net/cloud-init
<smoser> just mostly pointers... above.
<smoser> for Ubuntu in 16.04, there is one SRU in progress that blackboxsw and i need to verify today
<smoser> and then we will be soon looking at doing an SRU of 17.1-ish
<robjo> Thanks, still in europe, my day is about done, will pick it up tomorrow
<smoser> robjo, link is in topic
<smoser> http://bit.ly/ci-reviews
<smoser> ok. so open discussion i guess. no w?
<smoser> #topic Open Discussion / Office Hours
<smoser> we'll hang around and pay attention to pings and discussion fro the next 30  minutes or so.
<smoser> and after that, feel free to ping.
<ajorg> I'm writing a unit test for a bugfix I'm preparing to submit. It works, but it's slow because readurl retries and waits a second each time
<ajorg> I'm a bit of a mock / patch newb. Anyone can help me see how to patch that argument?
<smoser> you can feel free to  mock readurl
<smoser> and set its return_value and look at calls
<smoser> then you dont have to httppretty.
<ajorg> The only thing I want to change about it is the sleep though. Is there a straightforward way to just change one arg?
<blackboxsw> tests/unittests/test_handler/test_handler_chef.py is an example and tests/unittests/test_datasource/test_maas.py
<blackboxsw> I think
<ajorg> Hmm, I could do that, yeah. Would be sufficient for this.
<ajorg> I'd still like to know if there's an easy way to just change one argument to something that gets called somewhere else.
<smoser> you could also mock the time.sleep from url_helper
<ajorg> hahaha, yes. that's what I need to do.
<rharper> ajorg: if you want to mock the sleep you can decorate the test_ method with @mock.patch(time.sleep)
<smoser> we're suggesting other ways generally because there isn't :) at least that i know of.
<ajorg> that's cool, patching time.sleep will do nicely
<ajorg> is there an integration test that looks at what lands on the console?
<ajorg> (going back to my question from before the meeting)
<smoser> no. there coudl be on the nocloud-kvm backend
<smoser> but i think there is no console access currently on lxd
<smoser> and i think that we do not collect console access on nocloud-kvm
<smoser> but as you suggest we should for sure
<ajorg> k
<ajorg> I wondered if systemd might be swallowing my output, or maybe python (boothook in this case is a python script) is making a strange choice when it sets up logging.
<rharper> certainly possible;  I know there were issues with cloud-init starting before say rsyslog on non-systemd boots
<rharper> and the python logging has changed w.r.t the default configuration;  cloud-init main sort of expects this transition as it starts up in init-local and it has not yet read the cloud-config for logging configuration yet, so it reads that and then does some replay
<ajorg> interesting
<smoser> ajorg, you're writing to stdout/err with logging from a boothook ?
<ajorg> at least I would have thought cloud-init-output.log would contain my logs though, since it's more a redirect of stderr and stdout, right?
<ajorg> yeah
<rharper> correct
<rharper> this is from a bootcmd ?
<ajorg> no, #cloud-boothook
 * rharper hasn't used boothook
<smoser> hm..
<ajorg> boothooks are super useful
<rharper> smoser: when do boot_hooks run ?  local ?  net ?
<rharper> I'm not sure yet where stdout/err for boot hooks occur, but you can dump each from the units via:  journalctl -u cloud-init-local.service   (or cloud-init.service)
<blackboxsw> "This is the earliest hook available. Note, that there is no mechanism provided for running only once"
<blackboxsw> I'm checking the code now (local/pre-local maybe)?
<rharper> yeah, saw that;  it runs in-image scripts programs
<ajorg> I did that too, and don't see anything.
<ajorg> (journalctl)
<rharper> you should see some output
<rharper> but not seeing your hook in there ?
<ajorg> I can tell by other means that the hook ran
<ajorg> just don't see that it printed anything
<ajorg> my prime suspect if systemd can't do something bad here is that python makes some decision about not actually logging.
<ajorg> and I need to be more explicit that it should log
<rharper> so, the boot_hook will sub out each part, I see no capture on the stderr/stdout ; so I would expect those to go to whatever is currently capturing those;
<rharper> s/sub/subprocess
<ajorg> yup, that's what I expect too, and what I recall seeing on Amazon Linux
<rharper> since it's using cloud-init's util.subp, there *should* be a debug level message saying 'Running command %s with allowed return codes %s' which maches a path to the boothook
<rharper> I would expect to see that in the cloud-init.log
<smoser> http://paste.ubuntu.com/25661692/
<ajorg> yup, I do see that
<rharper> the output of script, I *think* should go to cloud-init-output.log
<smoser> i'm testing ^ on serverstack now.
<smoser> i verified the commands run on lxd but not look at console there.
<ajorg> thanks
<smoser> actually.. yeah, BOOTHOOK does run, but doesnt seem to have stdout tied to same place as bootcmd
<ajorg> oh?
<ajorg> I'm slightly surprised it's not just me
<smoser> http://paste.ubuntu.com/25661714/
<rharper> it really should be in the journal/console if stdout is not redirected to a file, cloud-init-local service
<smoser> that reproduces in lxc
<rharper> smoser: in your  instance on serverstack, do you see your BOOTHOOK in journalctl -b -u cloud-init-local.service ?
<smoser> ajorg, please go ahead and open a bug.
<ajorg> cool (not cool :-P)
<ajorg> will do
<smoser> # journalctl -b -u cloud-init.service | grep BOOT
<smoser> Oct 02 16:37:41 a5 cloud-init[121]: BOOTCMD: Mon, 02 Oct 2017 16:37:41 +0000: a5
<smoser> ^ that is from lxd
<rharper> and what about the welcome message from cloud-init ?
<smoser> i think not.
<rharper> bootcmd does:  util.subp(cmd, env=env, capture=False) ; where boot_hook does: util.subp([filepath], env=env)
<smoser> yeah, so boot_hook definitely swallowed. and should not be.
<rharper> yeah
<ajorg> https://bugs.launchpad.net/cloud-init/+bug/1720841
<ubot5> Ubuntu bug 1720841 in cloud-init "Output from boothook is not logged" [Undecided,New]
<ajorg> Meanwhile I need to setup more explicit logging from that boothook anyway, so I'm okay.
<blackboxsw> #link https://bugs.launchpad.net/cloud-init/+bug/1720841
<ubot5> Ubuntu bug 1720841 in cloud-init "Output from boothook is not logged" [Medium,Confirmed]
<ajorg> Here's another one, which I've confirmed with a unit test and proposed a patch: https://bugs.launchpad.net/cloud-init/+bug/1720844
<ubot5> Ubuntu bug 1720844 in cloud-init "UrlError from #include aborts stage" [Undecided,New]
<ajorg> (what is this #link thing you're doing there?)
<smoser> well, the bot is supposed to care and do somethign with it.
<smoser> but /me thinks it doesnt work right :)
<blackboxsw> ajorg: since meetingology is still active, the theory is it is keeping track of links  during this meeting
<smoser> (during the meeting)
<ajorg> ah, okay
<ajorg> I think simpletable is completely ready to merge, btw. Any objections? https://code.launchpad.net/~ajorgens/cloud-init/+git/cloud-init/+merge/330525
<blackboxsw> The theory is it writes the meeting notes out for us so we can publish to https://cloud-init.github.io
<ajorg> #link https://code.launchpad.net/~ajorgens/cloud-init/+git/cloud-init/+merge/330525 ;-)
<blackboxsw> geg
<blackboxsw> heh
<blackboxsw> ajorg: I think we were good on that changeset, and we only wanted to wait post 17.1 cut to avoid potential regression
<ajorg> great
<blackboxsw> I'm +1 on that will give it a spin today and then I think we can land it
<smoser> blackboxsw, ajorg my thoughts on the simpletable...
<ajorg> here it comes...
<ajorg> :-P
<blackboxsw> :)
<smoser> i would like to have a machine friendly output available
<smoser> human friendly is good, but machine friendly solves the actual goal of writing the stuff.
<ajorg> So I modified it to display *exactly* as prettytable did
<smoser> oh really.
<smoser> wow
<ajorg> which is admittedly not machine friendly
<ajorg> not especially
<smoser> then i guess i can't object at all.
<ajorg> but it's at least as good as prettytable
<smoser> in that its backwards compat
<smoser> right.
<ajorg> ^ this was my goal, to get you to not object
<smoser> i still think we should probaly additionally write some machine friendly json
<ajorg> agreed, that would be better
<smoser> i tihnk having something human friendly is good though
<smoser> as i know *I* look at that output
<ajorg> it meets my goal of not depending on prettytable
<smoser> and parsing json would be less nice
<smoser> yeah
<smoser> so based on your assertion that it outputs the same as pretty table, i have no objections. only future hopes.
<smoser> and i do agree dropping pretty table is nice
<ajorg> winning
<blackboxsw> smoser: yeah I was wondering how we generally expect people/machines to parse cloud-init-output.log. Right now it's kindof hard to do machine parsing of cloud-init-output.log.
<smoser> blackboxsw, yeah. cloud-init-output.log is not intended to be machine friendly.
<blackboxsw> do we know already of non-human consumers of the formatting in cloud-init.output.log?
<smoser> but the console actually has value in having machien friendly things on it.
<smoser> my feeling is if you have access to /var/log/cloud-init.log, then you could very easily have written whatever you wanted to another file that was purely machine friendly.
<blackboxsw> was wondering where we would intend to dump machine-friendly json
<blackboxsw> yeah
<smoser> but the console (/dev/ttyS0) boot log is different
<smoser> in that it can give you a couple things you'd not find easy access to
<smoser> a.) ssh public keys for the system (providing out of band communication of this data)
<smoser> b.) network configuration info: providn useful bits of data on how you might get to this system
<smoser> systemd makes it "fun" to get that data to the console in a safe way
<ajorg> ugh yes. systemd like to stomp all over your consoles.
<blackboxsw> ok so did we want to iterate on simpletable to dump json, or just look for cloud-init to write supplementary json files under /run/cloud-init
<blackboxsw> .. after we land ajorg's branch
<ajorg> I think it's a good idea to dump json fragments to console for some things, but I suspect smoser will still not want to break anyone who might be using the tables, so probably best to land my branch as-is
<smoser> yeah, i agree with ajorg. for now, we can just take a replacement that drops prettytable
<smoser> there are improvments to be done there, but this is a simple win
<smoser> we'll go ahead and end meeting here.
<smoser> i'll still be around. thanks all!
<smoser> #endmeeting
<meetingology> Meeting ended Mon Oct  2 17:29:26 2017 UTC.
<meetingology> Minutes:        http://ubottu.com/meetingology/logs/cloud-init/2017/cloud-init.2017-10-02-16.05.moin.txt
<smoser> blackboxsw, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/331664
<smoser> would appreciate input there.
<blackboxsw> looks like links got handled by meetingology, not infos. ok will fix that for next meeting
<smoser> blackboxsw, http://paste.ubuntu.com/25662210/
<smoser> that was my "mark-released" that i used for cloud-init
<smoser> powersj, ^. mentioned we'd want to put that somewhere and then can improve on it for other things too
<smoser> mainly i'm using pastebin and irc logs as a persistent storage system right now
<powersj> smoser: thx
<blackboxsw> smoser: powersj pushing it to qa-scripts
<blackboxsw> on the review too
<blackboxsw> ajorg: landed simpletable
<ajorg> \o/
<ajorg> thank you
<blackboxsw> robjo, sorry I forgot to hit approve button w/ final comments https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/331149
* blackboxsw changed the topic of #cloud-init to: is Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj |  Next status meeting: Monday 10/16 16:00 UTC | cloud-init 17.1 released
<blackboxsw> smoser: powersj rharper time/date check on next cloud-init meeting in topic
<smoser> k
<blackboxsw> so I don't mess that up again
<rharper> % date --utc -d "2 weeks"
<rharper> Mon Oct 16 19:11:22 UTC 2017
<blackboxsw> :) just checking that we weren't all up to something else (trips/early Halloween etc.)
<powersj> I think we are good
<rharper> lol
<rharper> should be fine
<smoser> blackboxsw,
<smoser> $ python3 -c 'from cloudinit.simpletable import SimpleTable'
<smoser> Traceback (most recent call last):
<smoser>   File "<string>", line 1, in <module>
<smoser> ModuleNotFoundError: No module named 'cloudinit.simpletable'
<smoser> i think somehow you didn't get the simpletable.py file added
<blackboxsw> bah, merge fail. grabbing it
<blackboxsw> pushed
<blackboxsw> going fresh checkout again to be sure
<blackboxsw> tox passes clean master
<blackboxsw> starting off the week right
<smoser> powersj, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/331670
<smoser> if you review that i'll get the other branches
<smoser> by 'get', i mean i'll just commit to the ubuntu packaging branches.
<blackboxsw> smoser: approved w/ nits https://code.launchpad.net/~smoser/curtin/trunk.doc-interface/+merge/331608
<blackboxsw> wrong channel
<powersj> CI isn't happy at the moment... ERROR: Failure: ImportError (No module named simpletable)
<powersj> smoser: looking
<powersj> +1
<blackboxsw> powersj: just pushed a fix
<blackboxsw> b3acdff390329c8091b880b490e1f27441a7a486
<blackboxsw> I had missed it when merging ajorg's branch
<blackboxsw> I forgot to 'git add' the new files
<ajorg> oh?
<smoser> blackboxsw, did you say you did https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/331664
<smoser> i dont see a comment
<blackboxsw> sorry ajorg, I shouldn't have hilighted you. it was PEBKAC on my merge of your branch
<blackboxsw> all good now
<ajorg> blackboxsw: did I prepare that MP in some wrong way that lead to the mistake?
<blackboxsw> nope, all my fault
<blackboxsw> won't happen next merge
<ajorg> k
<ajorg> no judgement
<blackboxsw> <- the judgement is all in this guy's head
<blackboxsw> smoser: I hadn't yet commenting now, wrapped up your curtin branch and robjo's, now the dhcp
<blackboxsw> only comment I have so far smoser is that the sort order should be reversed I think for systemd's/leases directory (isn't higher # more recent? )
<blackboxsw> testing meetingology for a sec
<blackboxsw> #startmeeting Test captured commands
<meetingology> Meeting started Mon Oct  2 21:14:28 2017 UTC.  The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
<meetingology> Available commands: action commands idea info link nick
<blackboxsw> #commands
<meetingology> Available commands: action commands idea info link nick
<blackboxsw> #info test info
<blackboxsw> #link test link
<blackboxsw> #nick test nick
<blackboxsw> #idea test idea
<blackboxsw> #meetingtopic topic change doesn't work
<blackboxsw> #topic do topics work?
<sedition> lol
<blackboxsw> #action sedition joins in on the fun (but actions don't work)
<meetingology> ACTION: sedition joins in on the fun (but actions don't work)
<blackboxsw> ahh thank you meetingology
<blackboxsw> #idea I think only circle of friends commands work here
<blackboxsw> guess not
<blackboxsw> #subtopic sub topic doesn't work
<blackboxsw> #endmeeting
<meetingology> Meeting ended Mon Oct  2 21:17:34 2017 UTC.
<meetingology> Minutes:        http://ubottu.com/meetingology/logs/cloud-init/2017/cloud-init.2017-10-02-21.14.moin.txt
<blackboxsw> this concludes our broken broadcast. I'm hitting up #meetingology to see what gives
<sedition> what an interesting concept
<blackboxsw> looks like action, idea, subtopic work
<blackboxsw> :)
<sedition> i dont think ive ever seen a meeting minutes bot
<blackboxsw> heh, it's supposed to make meeting minutes publishing easier... per commands listed here: https://wiki.ubuntu.com/meetingology
<blackboxsw> ok looks like most of those commands were properly handled
<blackboxsw> just that meetingology doesn't repeat (it's silent)
<blackboxsw> ok I think we are good then.
<smoser> blackboxsw,  https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/331664 updated
<smoser> i can hp back in later tonight and upload that or address further feedback
<blackboxsw> smoser: looks good on the changeset.
<blackboxsw> I'll approve and give it a whirl on on upgrade azure
<blackboxsw> upgraded
<blackboxsw> && lxd
<powersj> blackboxsw: https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/331678 (if you have time)
<blackboxsw> should be able to get that today
<blackboxsw> straight forward fix
#cloud-init 2017-10-03
<smoser> blackboxsw, still there?
<smoser> you probably shouldnt be.
<blackboxsw> smoser: sorry was doing dinner and bedtime: just ran on azure artful (from zesty upgrade) with and without ifupdown. Worked like a charm in both cases.
<blackboxsw> approved your branch
<blackboxsw> for tomorrow
<smoser> hm...
<smoser> rharper, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+ref/ubuntu/sm-proposed-artful
<rharper> raharper/ubuntu-devel-new-artful-release-v3
<blkadder> Question for the hive-mind: I am using cloud-init to spin up AWS instances via the aws CLI + yaml files. I am in the process of writing a wrapper script as among other things I want to be able to set the hostname for a given instance on launch. I am wondering if it is possible to merge the existing YAML file with the command line parameters I am setting?
<smoser> the aws cli only will allow you to pass one "user-data" blob.
<smoser> you can merge all your yaml files to your liking ahead of that one blob
<smoser> or you can combine multiple blobs into a mime-multipart message and putt hat as your blob and cloud-init will merge yaml on its side.
<blkadder> smoser, Thanks, I was trying to sort through that last option but couldn't find any examples of how to do so.
<blkadder> Otherwise I was just going to have the script treat the yaml file as a template and sed the hostname in.
<smoser> blkadder, https://help.ubuntu.com/community/CloudInit
<smoser> that shows how to create a mime multipart
<blkadder> Awesome thanks!
<blkadder> Had been looking for something like that.
<blkadder> Although looking at it, if I need to go through those gyrations, it is probably simpler to just sub in the hostname with sed as that's the only thing I need to do at present.
<blkadder> In any event appreciate the assist.
<powersj> blackboxsw: https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/331736
<powersj> another https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/331743
<blackboxsw> I'm glad someone is making progress
<blackboxsw> I'm trying to clear out from under unified getdata
<powersj> well that 2nd merge was written by smoser and I just went and did the work and tested it ;)
#cloud-init 2017-10-04
<none_> With a default install on Centos 7, if I pass user-data through Cloudformation, will those directives get stacked with could.cfg or will they preempt it?
<none_> s/could.cfg/cloud.cfg/g
<none_> I feel like I must be doing something wrong. Can I get a radio check?
<blackboxsw> bah timeout on CI
<blackboxsw> Error: Cannot find a valid baseurl for repo: base
<blackboxsw> Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=os&infra=stock error was
<powersj> blackboxsw: best to just re-run sadly
<blackboxsw> yeah I kicked it a bit ago. all green now
<rharper> blackboxsw: did we have a trello card for the in-flight sru tests ?
<blackboxsw> rharper: kinda.... empty
<blackboxsw> https://trello.com/c/wROS4mKT/458-sru-171
<blackboxsw> moved it to doing, I'll start adding a checklist for the bugs related
<blackboxsw> I wasn't sure how far you got yesterday
<blackboxsw> rharper: did you have a copy of the generated changelog, I can doctor that up to markdown linked bugs etc.
<rharper> yeah, one sec
<rharper> blackboxsw: http://paste.ubuntu.com/25675009/
<blackboxsw> thanks
<rharper> blackboxsw: when you get a chance, https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/331755
<blackboxsw> rharper: will do
<rharper> blackboxsw: thanks, and your datasource get/crawl branch needs review as well ?
<blackboxsw> can you make sure I didn't do anything nasty in my last few checkins https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330115
<blackboxsw> yeah was just sayin ' )
<blackboxsw> I tabled surfacing a standardized "platform" key for a followup as this branch is growing to a point it's getting hard to review
<blackboxsw> cloud-name would be something akin to "juju clouds" 'cloud' column
<blackboxsw> platform would be something like ec2, lxd, configdrive, openstack etc.
<blackboxsw> again w/ the instance-data.json. I want to iterate quickly if we can to sort an official 'v1' of our metadata generalized/standardized keys
<rharper> do you have a link to that "juju clouds" column ?
<blackboxsw> ahh and validated that py2 json can serialiaze bytestrings, so userdata not necessarily always get b64encoded
<blackboxsw> rharper: http://pastebin.ubuntu.com/25675178/
<rharper> I wonder if it's worth doing a directory of versioned files
<rharper> similar to metadata services
<rharper> so someone could always use the first version, but new fields/changes can happen
<rharper> that does mean a growing set of file writes if it changes frequently
<rharper> blackboxsw: interesting
<rharper> the juju clouds output
<blackboxsw> here's an official doc https://jujucharms.com/docs/devel/clouds
<blackboxsw> there is an interest for us to generally align across our products where it makes sense per https://trello.com/c/CD9xT2nG/45-cloud-id-output-match-juju-cloud-formatting
<blackboxsw> rharper: https://bugs.launchpad.net/cloud-init/+bug/1718287 is that actually fix committed? or still in progress
<ubot5> Ubuntu bug 1718287 in cloud-init "systemd mount targets fail due to device busy or already mounted" [High,In progress]
<rharper> Merged into cloud-init:master at revision da6562e21d0b17a0957adc0c5a2c9da076e0d219
<rharper> should be fix committed
<blackboxsw> ok tweaked the bg
<blackboxsw> ok tweaked the bug
<rharper> k
<blackboxsw> ok rharper I think this captures the bug list https://trello.com/c/wROS4mKT/458-sru-171
<rharper> y
<rangerpb> can someone remind me how the file /run/cloud-init/enabled gets written ?
<rangerpb> smoser, ^^ by chance ?
<blackboxsw> ... looking to be sure
<powersj> Looks like systemd/cloud-init-generator maybe?
<rangerpb> powersj, thanks man
<blackboxsw> takes me too long. +1 powersj ./systemd/cloud-init-generator
<blackboxsw> geez man. I need to tone down the caffeine
<blackboxsw> makes me jumpy
<powersj> lol
<blackboxsw> .... you can imagine how loud my typing is today
<powersj> I can almost hear it ;)
<rharper> lol
<sedition> you know you've had enough when your hands start shaking
 * dpb1 likes coffeee
<rharper> sedition <--- knows coffee
<dpb1> oh?
<sedition> i switched to an aeropress a few months ago. truly a wonder purchase
<sedition> i can't believe i didn't do it sooner
<sedition> wonderful*
 * rharper loves aeropress 
<dpb1> aeropress is good
 * dpb1 used one for a couple years
<dpb1> but my espresso machine replaced it
<dpb1> :)
<sedition> if i had an espresso machine you'd have to chain my to my desk to keep me from flying away
<dpb1> yes!
<sedition> hm. suddenly my init script doesn't want to add this PPA. interesting
<sedition> https://paste.ubuntu.com/25676096/
<sedition> manually running apt-add-repository ppa:x2go/stable seems to work though
<sedition> i moved the PPA command above the package command and it worked. black magic
<sedition> will do more digging
<gholms> I think I've run across a cc_ntp-related bug, but I'm not sure whether it's in the unit tests or the handler itself.
<gholms> When I run the unit tests in a chroot that I'm using to build packages, the cc_ntp code can't fine a package manager, so it writes out a systemd-timesyncd config file instead of ntp.conf.
<gholms> The unit test then blows up because it can't find ntp.conf.
<gholms> s/fine/find/
<gholms> Eh, I'll just file it anyway.
<gholms> Can figure out how to fix it later
#cloud-init 2017-10-05
<sedition> anyone have an idea on why an init script would be completely ignoring the packages and runcmd blocks? it sets up my users just fine and then fails to install packages/run commands silently. nothing in the cloud-init-output.log indicates that it even tried
<rharper> sedition: /var/log/cloud-init.log should have some info on what got run (it won't have your script output);  but maybe a reason why it couldn't run it;
<smoser> sedition, did you find something out on your  ppa thing ?
<smoser> the only thing i can come up with is that you did not have networkign at that poign.
<smoser> and then as rharper stated, /var/log/cloud-init.log and var/log/cloud-init-output.log probaly have more info
<smoser> gholms, it sounds like a unit test bug
<davydotcom> morning, trying to use cloud-init on Debian 8.3 image however there a known bug fixed in cloud-init 0.7.7 but apt-get only pulls down 0.7.6, how can i get 0.7.7 is on this version of debian
<smoser> davydotcom, https://packages.qa.debian.org/c/cloud-init.html
<smoser> stable has 0.7.9
<smoser> you might be able to just use that package, or build your own for older debian. i'm not aware of any othe rbackported option
<davydotcom> smoser: thanks ill try that out
<powersj> smoser: https://bugs.launchpad.net/cloud-init/+bug/1721243 might be worth looking at, he came back with a patch
<ubot5> Ubuntu bug 1721243 in cloud-init "cc_resizefs for zfs/zpool" [Medium,New]
<smoser> powersj, responded
<powersj> smoser: any comments on the patch? ;)
<smoser> powersj, needs tests
<smoser> but  yeah, i do have comments on it.
<smoser> gholms, tox-venv py3 python3 -m nose tests/unittests/test_handler/test_handler_ntp.py
<smoser> bah
<smoser> gholms, https://bugs.launchpad.net/cloud-init/+bug/1721573
<ubot5> Ubuntu bug 1721573 in cloud-init "ntp unit tests broken if no package program available in test environment" [Low,Confirmed]
<smoser> and rharper ^ just fyi.
<rharper> k
<rharper> smoser: huh, schema unit tests
<gholms> smoser: Oh, I already filed a bug for that yesterday
<gholms> https://bugs.launchpad.net/cloud-init/+bug/1721411
<ubot5> Ubuntu bug 1721411 in cloud-init "cc_ntp schema validation tests don't work in the absence of a package manager" [Undecided,New]
<blackboxsw> gholms: thanks for the bug, just marked it duplicate to track under  1721573
<gholms> Thanks
<smoser> thanks gholms sorry for the noise
<gholms> No worries.  Thanks for confirming.
<sedition> smoser: i solved the ppa issue, now moving on to another script for a different server that's silently eating the runcmd and package block. hoping this morning i can attack it with fresh eyes. thank you for asking!
<sedition> great little community
<powersj> smoser: We just got a pull request on github
<powersj> https://github.com/cloud-init/cloud-init/pull/9
<dpb1> heh
<dpb1> nice
<dpb1> what was our policy there?  close and ask to submit to lp?
<dpb1> and sign the cla
<dpb1> :)
<dpb1> larsks might know him actually
<_ix> Good morning friends. Can someone let me in on the secret to versioning from 0.7.9 -> 17.1. How does this match up to EL Repos?
<_ix> Running into this specific issue: https://bugs.launchpad.net/cloud-init/+bug/1699282
<ubot5> Ubuntu bug 1699282 in cloud-init "cloud-init process fails to configure puppet" [Medium,Fix released]
<blackboxsw> hi _ix. EL repos should eventually update to 17.1. It's a recent release versioning decision that we just came too a few weeks ago with cloud-vendors & distro vendors.  More details here FYI https://lists.launchpad.net/cloud-init/msg00106.html
<blackboxsw> checking the bug details now
<_ix> OK. Thanks very much! I'll check back in just a few minutes.
<blackboxsw> _ix have you been able to check our latest cloud-init builds as built in our copr repo? https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/
<blackboxsw> repos there have tip  (which will have version 17.1+)
<blackboxsw> in which I think we have a fix for puppet config
<sather> Question, where do I start debugging when I get 'Failed to start Initial cloud-init job (metadata service crawler)?
<sather> nothing 'pops' out when grepping the source code
<sather> I ran `cloud-init --debug init`, also nothing popping out as an 'ERROR'
<sedition> my script problem was 'sedition is an idiot'
<sedition> apt will just silently zombie forever without internet access.
 * sedition facedesk
<sather> I'm running version: 0.7.5 on centos
<_ix> blackboxsw: I'll take a look at that. Thanks!
<rharper> smoser: https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/331892
<blackboxsw> couple comments rharper
<blackboxsw> https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/331892
<rharper> blackboxsw: well this is the merge, that's already committed upstream; this is into release
<rharper> you're right though
<blackboxsw> ahh right, right
<smoser> blackboxsw, so we need to fix ?
<smoser> upstremam ? did you fidn a b ug there ?
<blackboxsw> smoser: I think I did find a corner case, "if 0" will never get into that non-bool check to convert it to boolean
<rharper> it's best to fix, the network config documents use of strings 'off' and 'on' ; not 0 or 1;  but nothing prevents us from getting such a config
<blackboxsw> bridge_stp = iface.get('bridge_stp')
<blackboxsw> 57	+        if bridge_stp and type(bridge_stp) != bool:
<rharper> blackboxsw: yeah,  I should check if value is not None and type is not bool;
<blackboxsw> yeah one liner
<smoser> ok. then submit that for merge and then re-submit for package ?
<blackboxsw> sure putting it up
<smoser> blackboxsw, ok. i'm waiting on an MP with a fix for that bug you found.
<smoser> then re-MP from raharper.
<blackboxsw> smoser: pushed. writing a proper descriptiojn now
<blackboxsw> https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+ref/network-stp-handle-0
<blackboxsw> https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/331898
<smoser> ship it.
<smoser> $ tox && git push upstream HEAD
<smoser> (that is running here)
<blackboxsw> merged
<blackboxsw> bah
<smoser> you pushed ?
<smoser> thats fine
<blackboxsw> yeah pushed
 * smoser ctrl-c
<blackboxsw> heh thx
<smoser> ok. now raharper for ubuntu points
<smoser> rharper,
<rharper> smoser: ok, I'll re MP
<smoser> good
<rharper> blackboxsw: thanks for spotting that
<blackboxsw> better late than never
<blackboxsw> thanks for going through the upload gauntlet 50 times
<rharper> smosers handy gist is super helpful
<blackboxsw> so for our SRU change log to xenial.... does our card need to actually capture all bugs from 0.7.9~233 until tip?
<smoser> blackboxsw, it will.
<smoser> here.
<blackboxsw> thanks, I was going to post-process the changelog into the trello checklist
<smoser> http://paste.ubuntu.com/25682062/
<blackboxsw> :( why do people do so much work? :)
<smoser> for your reference, what i did:
<smoser>  git checkout ubuntu/xenial
<smoser>  ./debian/new-upstream-snapshot
<smoser> it shows that log in a editor, i copy paste
<smoser> say 'n'
<smoser> then
<smoser> git reset --hard upstream/ubuntu/xenial
<blackboxsw> placing an bash alias function for that in my env
<htaccess> what is the process for newer cloud-inits getting added to a distro?
<htaccess> ie can i expect to see 17.1 in xenial cloud images?
<rharper> htaccess: for Ubuntu, the process is here: https://wiki.ubuntu.com/StableReleaseUpdates;  and yes you can expect 17.1 to come to xenial;
<blackboxsw> htaccess: we are working on uploading an SRU today/tomorrow to xenial at which point we need to go through verification on the related bugs. I'd expect 1-2 weeks on when you will see 17.1 in xenial. folks can correct me if that's too optimistic
<htaccess> thanks!
<blackboxsw> well just wrote a dumb script for trello SRU checklists from debian changelog
<blackboxsw> heh for next time on SRU check http://paste.ubuntu.com/25682341/
<blackboxsw> yields http://pastebin.ubuntu.com/25682350/
<blackboxsw> looks like 13 bugs to verify instead of 7 for xenial I think'
#cloud-init 2017-10-06
<smoser> blackboxsw, \o/ looks good.
<smoser> you can have it drop the bug and the author from the comment.
<smoser> and probably easier (or at least easy) to get log2dch  to do it.
<smoser> blackboxsw, https://gist.github.com/smoser/813c84bc7a79efc75d3f7fc2f383f12f now has '--trello'
<smoser> http://paste.ubuntu.com/25683387/
<cali_boxer> hello
<cali_boxer> we're having an issue with cloud-init on openstack
<cali_boxer> we have a new ga image based on centos 7.3.1611
<cali_boxer> cloud-init is grabbing the first software_config element from the heat template, but it never informs heat that it has completed the operation
<cali_boxer> the stack-build subsequently fails
<cali_boxer> after timing-out
<cali_boxer> I've spent a few days on this, and can't get further on this,
<cali_boxer> we're on openstack-liberty
<cali_boxer> all pre-reqs are in the image
<smoser> cali_boxer, can you give a /var/log/cloud-init.log ?
<smoser> whatever cloud-init would be doing would be in there.
<Junaid> hello
<Junaid> i want to ask something
<smoser> Junaid, whats up?
<cali_box_web> hello
<cali_box_web> I need support for cloud-init
<cali_box_web> is there anyone who can assist?
<smoser> cali_box_web, whats up?
<cali_box_web> I have an openstack build that partially works
<cali_box_web> the system will build the ste stacks
<cali_box_web> but...
<cali_box_web> cloud-init on some of the instances, does not signal the completion of the software_config section
<cali_box_web> the section is correctly deposited onto the system (ssh keys, hosts file)
<cali_box_web> but does not signal completion
<cali_box_web> this does work on other members in the stack that are Centos 7.2
<cali_box_web> the system in target is a customized version of Centos 7.3
<cali_box_web> 1611
<cali_box_web> cloudinit 0.7.9 is used on the new image
<cali_box_web> 0.7/5 is used on the centos 7.3
<cali_box_web> er 7.2
<cali_box_web> the image in question is custom centos 7.2 linux
<cali_box_web> it has all pre-reqs as far as i can see
<smoser> can you pastebin a /var/log/cloud-init.log ?
<cali_box_web> from the client?
<cali_box_web> sure
<cali_box_web> just a few
<smoser> yeah
<cali_box_web> trying to figure out how to pastebin from putty
<cali_box_web> just a sec
<cali_box_web> https://pastebin.com/By0CwW9q
<cali_box_web> i can post a sanitized yaml file from the office
<cali_box_web> this is openstack liberty btw
<smoser> cali_box_web, the error there (search for WARN) i think is that some part of your user-data does not start with '#!'
<cali_box_web> there is no user-data
<smoser> i think there is..
<cali_box_web> https://pastebin.com/QD3cqaBq
<smoser> private paste?
<cali_box_web> web client doesn't allow that
<smoser> oh. i meant it tells me it is a privarte paste
<cali_box_web> yes
<cali_box_web> let me see if I can post as something else
<smoser> is that a public system ? i can hop in if you'd like and poke around, but *something* made cloud-init think that you had a script to be executed (i think heat does that)
<cali_box_web> it's not public
<smoser> but the script did not start with a '#!' or otherwise kernel-recognizable format
<cali_box_web> whats your email i can send you the santized template
<smoser> can you just collect all of /var/log/cloud-init* and /var/lib/cloud/ ?
<smoser> send to scott.moser@canonical.com .
<cali_box_web> yes
<cali_box_web> thanks
<cali_box_web> sent from my work email
<smoser> cali_box_web, i tihnk it borked the content
<Junaid> is there anyway to trace cloud data centre?
<Junaid> i want dataset of locations of data centres
<Junaid> Can anyone tell me about it?
<Junaid> Please
<cali_boxer> @smoser, I'll resend as links
<smoser> Junaid i'm not sure what you're after.
<Junaid> how can i trace x,y coordinates of cloud data centres?
<Junaid> i want to know exact locations of data centres
<gholms> I'm pretty sure that is out of the scope of this channel.
<Junaid> then where can i get it? can you tell me channel name where i can raise this question?
<cali_boxer> smoser, logs have been sent
<cali_boxer> i added the user_data section, which is getting parsed, and deposits /var/lib/cloud/instances/scripts/userdata but the system still does not complete a checkin letting heat know that it has performed the action
<blackboxsw> hrm Junaid I think you'd have to send a request to cloud vendors specifically. and some of them may not provide that specific information due to privacy concerns
<blackboxsw> I know my former company wanted to keep their datacenter locations confidential for one reason or another
<blackboxsw> smoser: shall I start on SRU MP for cloud-init?
<smoser> yeah
<cali_boxer> smoser, were you able to see anything in the logs?
<cali_boxer> on the second set?
<smoser> cali_boxer, i'm not sure you sent what i wanted ?
<smoser> i need /var/lib/cloud/ directory
<cali_boxer> that was in the second link
<cali_boxer> the lins from our sync.domain.com
<smoser> cali_boxer, http://paste.ubuntu.com/25687216/
<smoser> that is in /var/lib/cloud/<instance-id>/user-data.txt
<smoser> whatever sent that user-data, sent an empty file that was "type" of text/x-shellscript;
<smoser> cloud-init did what it is supposed to do (rather blindly) and put that into a file and executed it
<smoser> and that resulted in the error you see
<cali_boxer> ok
<smoser> admittedly cloud-init coudl say "oh that doesnt make any sense" and fail in a nicer way.
<cali_boxer> I modified the yaml file after your suggestion
<smoser> but very clearly that is the "garbage in" part of "garbage in -> garbage out"
<cali_boxer> i now send a valid user_data containing, #!/bin/sh \n echo "hello"
<cali_boxer> which executes
<cali_boxer> but the system still doesn't respond back to the metadata agent saying thatthe step is complete
<smoser> well, what you sent died because of the empty user-data
<smoser> i can't diagnose a different failure without more ifo
<smoser> info
<cali_boxer> fair enough.
<cali_boxer> i'll send a new set
<cali_boxer> stand by
<cali_boxer> items sent
<smoser> blackboxsw, http://paste.ubuntu.com/25687335/
<smoser> (from #ubuntu-devel)
<blackboxsw> thanks joining that channel now too
<smoser> cali_boxer, well, i think i know what is wrong. i think that you didn't actually get networking up before cloud-init went to look for the metadata service (provided by openstack on 169.254.169.254)
<smoser> i'm not sure why that would differ between the two.
<cali_boxer> odd, since the unit file should have a want for networking
<cali_boxer> my co-worker discovered a potential issue as well
<smoser> so there are 2 boots in your log.
<cali_boxer> it looks like salt-master/minion was not on the image
<cali_boxer> there are
<cali_boxer> i may not have cleaned the log after sealing the image
<cali_boxer> i've added it and am uploding the image to OS now
<smoser> i'm not reallky able to look  much deeper.
<smoser> i'm pretty sure that networking doesnt end up coming up.
<smoser> perhaps you have some sysconfig configuration in /etc/sysconfig/network  that is messing it up.
<smoser> in that second log you sent there are 2 boots, one worked, one failed.
<smoser> blackboxsw, so what that means... is that we should (I will) make a metabug
<cali_boxer> ok, thank you for your help,
<cali_boxer> I'll let you know if the salt thing fixes it
<blackboxsw> so smoser as with curtin, cloud-init will link related bugs to the merge proposal, but not list them in final changelog for release. Only list the master bug
<smoser> yeah.
<smoser> bug 1721847
<ubot5> bug 1721847 in cloud-init (Ubuntu Zesty) "sru cloud-init 2017-10-06 (17.1-17-g45d361cb-0ubuntu1)" [Medium,Confirmed] https://launchpad.net/bugs/1721847
<smoser> blackboxsw, ^
<blackboxsw> thx smoser working through https://gist.github.com/smoser/6391b854e6a80475aac473bba4ef0310 right?
<smoser> https://gist.github.com/smoser/6391b854e6a80475aac473bba4ef0310#file-ubuntu-release-process-md
<smoser> yes
<smoser> i'm working on filling out that bug with more info
<smoser> and you will drop the LP: refernces in the generated changelog
<smoser> blackboxsw, fyi, what i did there was just
<smoser>  git cherry-pick 833c9d6570dd2bfe695c596f744bc70c292a25e5
<smoser> on each branch
<smoser> and then manually update changelog
<smoser> and git commit -m "update changelog"
<smoser> the hash above as found on ubuntu/devel branch
<blackboxsw> +1 on that thanks
<smoser> blackboxsw, one thing the changelog generation doesnt do..
<smoser> 1717477 is already fixed, but the changelog will show it
<smoser> so we should drop that line from the commit message
<smoser> (as it was sru'd already)
<blackboxsw> smoser: ok I'm in the hangout to plow through Xenial
<smoser> ok. 5 minutes
<smoser> k ?
<smoser> (ill be there in)
<blackboxsw> yeah no worries
<blackboxsw> will walk you through what I think I've done
<smoser> blackboxsw, i'm here now.
<blackboxsw> smoser: zesty's changelog has mix of zesty-proposed and zesty in recent changelogs
<blackboxsw> shall I make changelog distro zesty-proposed from here on?
<nacc> blackboxsw: it shouldn't really matter
<nacc> blackboxsw: as uploads go to zesty-proposed anyways
<nacc> blackboxsw: (speaking generally)
<blackboxsw> ok nacc was just trying to figure if we needed/wanted  a  convention from here on
<nacc> blackboxsw: yeah, i would suggest using zesty not zesty-proposed, imo
<blackboxsw> as SRUs will go into *-proposed for real
<nacc> (in the changelog)
<nacc> as that is the 'release' you are targetting
<smoser> blackboxsw, yeah, thats what i was saying. it doesnt matter. i prefer -proposed as i just think its cleaner
<nacc> heh
<smoser> but then somebody always messes up
<smoser> (probably it was me)
<smoser> :)
<blackboxsw> don't make me git blame ;)
<smoser> i just like the clean break from 'xenial' to 'xenial-proposed' as it is then easy to see what was GA
<smoser> but really, no one other than me probably ever looks :)
<nacc> smoser: oh that's a good point, xenial-updates would work just as well
<nacc> smoser: but yeah, your point is valid
<blackboxsw> it's good to have a line in the sand or something.  I'm all for doc breadcrumbs whereever we can leave them.
<blackboxsw> :)
<blackboxsw> ok I'll go zesty-proposed for this commit as that's a convention we've kinda used in xenial for cloud-init
<blackboxsw> and attempted in zesty, when we didn't forget
<blackboxsw> it's official, I ð Chris Columbus. Not working on Monday
<smoser> blackboxsw, you were so close to the statue in Central Park.
<blackboxsw> I should go back sometime.... like next week :)
<blackboxsw> smoser: https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/331975
<blackboxsw> I'm testing build now
<blackboxsw> smoser: I just saw Get:1 http://archive.ubuntu.com/ubuntu zesty/universe amd64 python3-httpretty all 0.8.14-1 [19.5 kB] from apt build-dep.... which is ok right because that's pulling build-deps from current published upstream
<blackboxsw> which still depends on httpretty
<blackboxsw> which we are dropping
<smoser> right
<smoser> s/upstream/packaged version/
<smoser> its pulling it from apt source info
<blackboxsw> hrm, why do I see                python3-httpretty, in debian/control on zesty
<blackboxsw> checking the built package details now to make sure it was patched out of there
<blackboxsw> yep safe/correct
<dhill_> hi guys
<smoser> blackboxsw, hey
<smoser> wait. what ?
<blackboxsw> why am I seeing Changelog w/ bugs linkedhmm
<smoser> blackboxsw, i just diffed xenial (your propseod) to artful
<smoser> and found
<smoser>  http://paste.ubuntu.com/25688137/
<blackboxsw> looking at the paste
<smoser>  more useful
<smoser>  http://paste.ubuntu.com/25688142/
<smoser> blackboxsw, ^
<smoser> the thing i noticed really was the apport launcher
<smoser> think we'd want that.
<smoser> :-(
<blackboxsw> right we don't have apport official integration without it right?
<blackboxsw> so xenial and zesty both need it don't they?
<blackboxsw> I was difffing against ubuntu/xenial and didn't see the apport diff
<blackboxsw> *I dont think*
<blackboxsw> sure enough
<smoser> i can account fo rhte other changes there, they make sense.
<smoser> even that last one which looks suspiciouls (-aws) its ok.
<blackboxsw> so I need to resubmit MPs for xenial and zesty and bring in debian/apport-launcher.py
<smoser> but i'm tempted to cherry pick
<smoser>  2baa6bf9c314630ec29a192e1e81f0f4530cb102
<smoser>  fec2d4a87fcff72aa9cafbfb88fff013a17db665
<blackboxsw> should that have been pulled in from debian/new-upstream-snapshot ?
<blackboxsw> no because it is querying tip of master no ubuntu/devel right?
<smoser> no. those are ubuntu/devel changes
<smoser> we dont merge from ubuntu/devel
<smoser> we could .
<smoser>  we dont
<blackboxsw> yeah ok makes sense
<blackboxsw> so cherry pick, new MP including those cherry picks for zesty and xenial?
<blackboxsw> you're EOW , I can hangout if you want to talk quickly through pro/cons
<smoser> yeah, we can chat quick.
<blackboxsw> ok zesty pushed again
<smoser> i did a sbuild && dput && git push
<smoser> so it might all just go
<smoser> and since i 'git pull'ed it should mark your MP as done also
<smoser> thanks chad!
<blackboxsw> I'm testing the ubuntu-bug on my zesty container to be sure
<blackboxsw> thanks again smoser. I'm scrubbing trello bugs listed now
<blackboxsw> for the rest of the afternoon I presume
<smoser> all done i think thanks chad.
<sedition> hey team. fixed all my issues. all user error on my part. appreciate the patience. =)
 * sedition is a derp sometimes
<blackboxsw> I know the feeling sedition
<sedition> im trying to learn terraform and cloud-init at the same time on top of my BAU job
#cloud-init 2019-09-30
<coli_> How can we help to push this forward:
<coli_> https://code.launchpad.net/~adobrawy/cloud-init/+git/cloud-init/+merge/354679
<coli_> ?
<powersj> smoser, ^ any final thoughts on that
 * smoser reads
#cloud-init 2019-10-01
<lcpfnvcy> how
<lcpfnvcy> *hi
<lcpfnvcy> need some help
<lcpfnvcy> I have a cloud-init config file used to create a vm
<lcpfnvcy> I'm trying to write a file into a /home/user directory
<lcpfnvcy> but as the owner: user
<lcpfnvcy> but it keeps being owned by root
<lcpfnvcy> any ideas why ?
<lcpfnvcy> this is with the write_file module
<lcpfnvcy> write_files:
<lcpfnvcy>   - path: /home/user/storage.cfg
<lcpfnvcy>     permissions: '0755'
<lcpfnvcy>     owner: "user:user"
<lcpfnvcy>  
<lcpfnvcy> it's a simple text file
<tribaal> lcpfnvcy: I suspect you're hitting https://bugs.launchpad.net/cloud-init/+bug/1486113
<ubot5> Launchpad bug 1486113 in cloud-init "write_files runs before users/groups, renders "owner" useless" [High,Confirmed]
<lcpfnvcy> ok
<lcpfnvcy> looks like that is it
<lcpfnvcy> I can change the ownership via runcmd then ?
<tribaal> I suspect you could yes - but I'm not sure when runcmd runs relative to the other modules
<tribaal> maybe somebody else can chime in later (a lot of people on this channel are on American timezones)
<lcpfnvcy> ah ok
<lcpfnvcy> runcmd also gives weird user issues
<lcpfnvcy> but ok I'll wait
<lcpfnvcy> https://stackoverflow.com/questions/34095839/cloud-init-what-is-the-execution-order-of-cloud-config-directives
<lcpfnvcy> if this is correct, then the execution order doesn't make sense to me
<lcpfnvcy> why with runcmd, I still cannot 'chown user directory'
<Odd_Bloke> lcpfnvcy: I believe runcmd should work; what issues are you seeing using it?
<lcpfnvcy> Odd_Bloke: I've tried runcmd, with a simple 'chown user directory' but the directy remains root
<lcpfnvcy> unless my runcmd is wrong
<lcpfnvcy> but I dont see errors in the log
<lcpfnvcy> [ chown, lcp, /mnt/blobfusetmp]
<lcpfnvcy> something like that
<Odd_Bloke> Does that do what you expect if you run it by hand after boot?
<lcpfnvcy> Odd_Bloke: yes
<lcpfnvcy> I can chown the directory login into vm
<Odd_Bloke> lcpfnvcy: In that case, please file a bug and attach the output of `cloud-init collect-logs` to it. :)
<lcpfnvcy> ok
<lcpfnvcy> I'm trying to do this with an azure vm
<blackboxsw> Odd_Bloke: forgot to ask about cloud-init SRU, did we get all the verification logs we needed
<blackboxsw> ?
<Odd_Bloke> blackboxsw: I'm not sure, TBH, I've been focused on other stuff.
<blackboxsw> np I'll kick it. I think we are all green all the way
<blackboxsw> yep powersj marked it verfication done yesterday
<blackboxsw> just needed a bump in#ubuntu-release
<Odd_Bloke> OK, nice.
<AnhVoMSFT> blackboxsw rharper has the cloud-init 19.3 SRU been cut yet?
<chillysurfer> now that AnhVoMSFT mentions it, what determines when we increment the minor or the build component for versioning in cloud-init?
<blackboxsw> AnhVoMSFT chillysurfer: I checked in on this internally ~3 hours ago. it was approved but hadn't yet been published. The SRU team did get that upload finalized today, so tonight it should be put into new cloud images
<blackboxsw> chillysurfer: minor is changed with each new upstream cut of cloud-init
<blackboxsw> we number our versions <year>.<upstream_release_count_for_this_year>
<blackboxsw> the subrevision number is generated based on the number of commits that landed since the upstream release cut and we only generate that when we release into an ubuntu series as part of an SRU or package upload into the latest development ubuntu release (Eoan).
<blackboxsw> so 19.2.36 happens to be 36 commits past the 19.2 upstream cut as published into  Xenial, Bionic, Disco and Eoan
 * blackboxsw needs to double check on the subrevision numbering. But, the major and minor is 19 (year), 2 (second upstream release within 2019 [4 per year])
<Odd_Bloke> AnhVoMSFT: We haven't released 19.3 yet; did you mean the 19.2 SRU that Chad is talking about, or something else?
<blackboxsw> chillysurfer: we are *late* on an upstream release of 19.3, but it is just a line in the sand based on trying to get 4 upstream upstream releases per year, it is not specifically gated on a feature set.
<Odd_Bloke> We only have major/minor versions upstream, which follow the pattern Chad mentioned.
<Odd_Bloke> The -36 is Ubuntu-specific.
<blackboxsw> for context 19.2.36 is the version of cloud-init that we are SRUing (publishing stable release update) into Xenial, Bionic and Disco
<Odd_Bloke> It's -36, _not_ .36.
<blackboxsw> :) thx
<Odd_Bloke> We don't follow semver, so we need to be careful about that.
<Odd_Bloke> Because 19.2-37 could have one enormous change compared to 19.2-36, for example.
<blackboxsw> good pt.
<chillysurfer> but the "37" in this case will be pushed to the ppas right?
<chillysurfer> or if not, what determines when a version/update is pushed?
<Odd_Bloke> The daily PPAs get the latest every day.
<Odd_Bloke> (But note that PPAs are Ubuntu-specific, so that doesn't invalidate my point above. :)
<chillysurfer> so SRU != upstream release cut?
<Odd_Bloke> Correct.
<Odd_Bloke> Well, we will SRU new upstream releases, in general.
<Odd_Bloke> But we also SRU snapshots of upstream in Ubuntu.
<chillysurfer> so what is the determining factor to make, for instance, 19.2-37 instead of 19.3-0?
<Odd_Bloke> There isn't a clear rule about it.
<chillysurfer> ah ok
<Odd_Bloke> Because of the nature of cloud-init (which is that it's generally only consumed when integrated into an image by a distributor), having semantic version numbers isn't all that valuable.
<chillysurfer> last question (hopefully :)), is every SRU pushed to the archives for ubuntu? for instance, 19.2-0 all the way to current is pushed? or are some skipped? e.g. 19.2-15 wasn't pushed to archives?
<blackboxsw> chillysurfer: sorry I missed that. for Ubuntu, we actually take tip of tree of master and bring that into each SRU. so all commits make it into the SRU. We also have an obligation to existing ubuntu Xenial, Bionic and Disco users to retain original behavior. For example.  if a new feature that landed in tip changes a default behavior on Xenial, we end up surfacing a configuration option to retain the original
<blackboxsw> defaults in Xenial, but allow cloud-init users to make use of that feature by enabling a configuration option to get the 'new' behavior seen on tip.
<blackboxsw> if we know ahead of time we are changing default cloud-init behavior, we generally surface that #cloud-config configuration  option in tip and just alter the config defaults to turn that new feature 'off' on stable Ubuntu releases
<Odd_Bloke> chillysurfer: I think you're confusing some terminology here.  SRU stands for Stable Release Update, and is the process we follow to get newer versions of cloud-init back into already-released versions of Ubuntu.
<Odd_Bloke> (In general, the software in the Ubuntu Archive does not change after release, to provide stability for its users.)
<Odd_Bloke> chillysurfer: We only create new packages every so often, generally when there's a specific feature or bug fix that would be valuable to older releases.
<Odd_Bloke> The latest such packages were based on the 36th commit since the 19.2 tag, so their versions all have the prefix "19.2-36".
<Odd_Bloke> But you shouldn't expect to see every commit turned into a package in the Ubuntu archive.
<Odd_Bloke> For example, the version prefixes before 19.2-36 were (working from latest to oldest) 19.2-24, 19.2-21, 19.2-13, 19.2-9, 19.2-5, and 19.2.
<Odd_Bloke> chillysurfer: Does that make some sense?
<blackboxsw> thx Odd_Bloke
<chillysurfer> Odd_Bloke blackboxsw that makes total sense! thank you!
#cloud-init 2019-10-02
<aissen> I reported an issue with ubuntu cloud-images arm64 first boot taking a long time a few weeks (months?) ago. I finally took the time to open a ticket and provide a reproducer: https://bugs.launchpad.net/cloud-images/+bug/1846355
<ubot5> Launchpad bug 1846355 in cloud-images "cloud-init very slow to set password on arm64 cloud image" [Undecided,New]
<Odd_Bloke> aissen: Thanks for the bug report!  Unfortunately, I think you are being bitten by snapd.seeded being very slow.
<Odd_Bloke> There are some snapd changes in the works to make that less painful, which _are_ targetted to land in eoan.
<Odd_Bloke> That said, it's pretty close to eoan release day, so they may not make it in quite in time.
<Odd_Bloke> (AIUI, they will be backported to stable releases once they're in, so missing release day isn't quite as bad as it is for most software in the archive.)
<aissen> Odd_Bloke: it might be indeed. Is there a bug report for this specific issue ?
<aissen> is there a specific reason why cloud-init password setting depends on snap.seeded (but not ssh key setup for example) ?
<aissen> the hostname is also setup relatively early.
<Odd_Bloke> aissen: With reference to https://cloudinit.readthedocs.io/en/latest/topics/boot.html, the SSH keys are put in place in the "Network" phase, and passwords are set in the "Config" phase.
<Odd_Bloke> I'm not 100% sure why passwords are set in that later phase.
<aissen> maybe something in the config phase installs snaps ? (seems weird, but maybe people put arbitrary commands that install snaps ?)
<Odd_Bloke> snapd.seeded.service is installing pre-seeded snaps into the system.
<Odd_Bloke> So, for example, lxd gets installed by it.
<Odd_Bloke> runcmd runs in the Config phase, so we need to be sure that all the system software is in place before that happens.
<Odd_Bloke> So it has to block on snapd.seeded.service.
<aissen> that's interesting, thanks.
<Moo464> Good evening, sir,does anyone like to answer a question about importing SSH keys?
<Moo464> I'm having trouble usind Cloud Init
<Odd_Bloke> Moo464: o/ It's best to post what your problem is, then anyone coming along can help you out. :)
<Moo464> Okay.My problem is this. I want to set up a cloud server at Hetzner. It should receive a previously generated key pair. With this key pair the server should be able to make a git clone via ssh. With GitHub I have already deposited the public key. Nevertheless, the server does not have authorization.I have entered the following:>
<Moo464> ssh_keys:  rsa_private: |    -----BEGIN RSA PRIVATE KEY-----    MIIBxwIBAAJhAKD0YSHy73nUgysO13XsJmd4fHiFyQ+00R7VVu2iV9Qcon2LZS/x    1cydPZ4pQpfjEha6WxZ6o8ci/Ea/w0n+0HGPwaxlEG2Z9inNtj3pgFrYcRztfECb    1j6HCibZbAzYtwIBIwJgO8h72WjcmvcpZ8OvHSvTwAguO2TkR6mPgHsgSaKy6GJo    PUJnaZRWuba/HX0KGyhz19nPzLpzG5f0fYahlMJAyc13FV7K6kMBPXTRR6FxgHEg
<Moo464> L0MPC7cdqAwOVNcPY6A7AjEA1bNaIjOzFN2sfZX0j7OMhQuc4zP7r80zaGc5oy6W    p58hRAncFKEvnEq2CeL3vtuZAjEAwNBHpbNsBYTRPCHM7rZuG/iBtwp8Rxhc9I5w    ixvzMgi+HpGLWzUIBS+P/XhekIjPAjA285rVmEP+DR255Ls65QbgYhJmTzIXQ2T9    luLvcmFBC6l35Uc4gTgg4ALsmXLn71MCMGMpSWspEvuGInayTCL+vEjmNBT+FAdO    W7D4zCpI43jRS9U06JVOeSc9CDk2lwiA3wIwCTB/6uc8Cq85D9YqpM10FuHjKpnP
<Moo464> REPPOyrAspdeOAV+6VKRavstea7+2DZmSUgE    -----END RSA PRIVATE KEY-----  rsa_public: ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAGEAoPRhIfLvedSDKw7XdewmZ3h8eIXJD7TRHtVW7aJX1ByifYtlL/HVzJ09nilCl+MSFrpbFnqjxyL8Rr/DSf7QcY/BrGUQbZn2Kc22PemAWthxHO18QJvWPocKJtlsDNi3 smoser@localhost
<Moo464> It seems as if the server does not insert the key pair
<Odd_Bloke> Moo464: Umm, I hope that you aren't too attached to that particular key, because it's now logged in everyone's IRC clients and online.
<Moo464> Yeah those are Dummy keys
<Odd_Bloke> OK, phew.
<Odd_Bloke> Moo464: ssh_keys is used to configure the SSH _host_ keys.  You're probably looking for ssh_authorized_keys instead.
<Odd_Bloke> This is not entirely obvious in the docs, but the example at the bottom of https://cloudinit.readthedocs.io/en/latest/topics/modules.html#ssh is instructive.
<Moo464> Thank you. So there a two lines with  - ssh-rsa AAAAB3Nza[...] Which one is the Public Key and which one the private?
<Moo464> If I am not totally wrong I have to enter something like this:
<Moo464> ssh_authorized_keys:    - ssh-rsa PRIVAT KEY    - ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA3I7VUf2l5gSn5uavROsc5HRDpZ ...
<Moo464> and the other one is the Public key
<Odd_Bloke> Moo464: The private key is private; you keep that locally.
<Odd_Bloke> Moo464: You only need to include the public key in the ssh_authorized_keys list.
<Moo464> I know but in order to use git clone SSH I need to transfer the private key as well, dont I?
<Moo464> Otherwise I would transfer the public key to the server and also to GitHub, but then GitHub could not verify that I am allowed to make changes
<Odd_Bloke> Moo464: Oh, I see what you mean.
<Moo464> Sorry, I am not that good in writing english :D
<Odd_Bloke> Hey, we got there. :)
<Odd_Bloke> cloud-init doesn't provide a way to put private keys in-place for users, because it's a relatively uncommon operation.
<Moo464> Oh okay, now I see
<Moo464> I misunderstood the Docs
<Odd_Bloke> Yeah, they are definitely confusing.
<Odd_Bloke> Most people will approach them assuming they're talking about _user_ keys.
<Odd_Bloke> Let me file/find an issue about that.
<Moo464> So there is no way to deposit a key during the creation?
<Odd_Bloke> https://bugs.launchpad.net/cloud-init/+bug/1827021 <-- there we go, I knew I'd seen one before
<ubot5> Launchpad bug 1827021 in cloud-init "SSH Documentation should mention "Host Key"" [Medium,Triaged]
<Moo464> Good to know I am not the only one
<Odd_Bloke> Moo464: If you know the name of your default user, you could use write_files.
<Moo464> Great idea. I'm going to try this. Thank you very very much!
<Odd_Bloke> Happy I could help!
#cloud-init 2019-10-03
<do3meli> hi there - just a qq: is it a known problem that cloud-init in the debian 10 cloud image is not working as expected after first reboot after deployment. sytemd-networkd seems to be trying to bring up network which takes around 6 min until it times out.
<Odd_Bloke> do3meli: That doesn't sound familiar to me.  Could you file a bug using the link in the topic, please?
<do3meli> @Odd_Bloke: seems they have 18.3-6 version pre-installed in the image. maybe an issue with that one ?
<Odd_Bloke> Could be!
<Odd_Bloke> If you can try with a more recent version, that'd be really helpful.
<Odd_Bloke> (testing and unstable have 19.2.)
<do3meli> is there a .deb package around somewhere for easy upgrade ?
<Odd_Bloke> do3meli: You could grab it off a Debian mirror
<Odd_Bloke> ?
<do3meli> never mind. i found one.
<do3meli> Odd_Bloke: here we go: https://bugs.launchpad.net/cloud-init/+bug/1846513
<ubot5> Launchpad bug 1846513 in cloud-init "debian 10 cloud-init stuck in systemd-networkd after 1. reboot" [Undecided,New]
<Odd_Bloke> do3meli: OK, we're gonna need at least cloud-init.log from there to debug further.  Can you run `cloud-init collect-logs` and attach the tarball to the bug?
<blackboxsw> +1 Odd_Bloke do3meli was just commenting that on the bug
<do3meli> sure 1 sec
<blackboxsw> bah Odd_Bloke beat me
<blackboxsw> ... again
<do3meli> done
<Odd_Bloke> do3meli: Do you have a working OpenStack metadata service?
<rharper> Odd_Bloke: that looks like this one, where the ephemeral dhcp didn't setup the classless staci routes, but after "fallback dhcp" comes up, crawling IMDS works;  https://bugs.launchpad.net/cloud-init/+bug/1821102
<ubot5> Launchpad bug 1821102 in cloud-init "Cloud-init should not setup ephemeral ipv4 if apply_network_config is False for OpenStack" [High,Fix released]
#cloud-init 2019-10-04
<dante-as> hi guys, someone online?
<dante-as> if I have an Ubuntu image which has already the nocloud-net file and I am trying to apply with Terraform something extra on the cloud-init user-data, does someone why is not working?
<dante-as9> hi guys, someone online?
<dante-as> hi guys, someone online?
<tribaal> dante-as: genrally speaking there is always someone online, but maybe not someone who could answer your question. A better approach to the problem here is to simply ask your question and wait for someone to answer :)
<tribaal> (note - it might take a while to get an answer, since as you've noted not everyone gives full IRC attention all the time)
<tribaal> most people active on here are in american timezones, for the record
<dante-as> tribaal: thanks for the details :)
<dante-as> I'm having an Ubuntu image which has already nocloud-net embedded and I'm wondering if it's any way to add also a network configuration via cloud-init and Terraform for some VMs which use this image
<coli_> our change to cloud-init was approved, what else do we need to do for it to be merged and included ?
<coli_> https://code.launchpad.net/~adobrawy/cloud-init/+git/cloud-init/+merge/354679
<Odd_Bloke> coli_: o/ I'm going to kick off a CI run against your branch.
<coli_> Odd_Bloke: thank you.
<Odd_Bloke> Assuming that's all good, we should be able to land it in the next couple of days.
<Odd_Bloke> We're currently dealing with a regression in the last cloud-init snapshot we released to Ubuntu, so we're focused on that ATM.
<Odd_Bloke> Just to set expectations. :)
<Odd_Bloke> coli_: OK, we're seeing CI failures that are fixed in master, so you'll want to rebase onto (or merge from) master to get those fixes in.
<Odd_Bloke> coli_: Ping me once you've done so and I'll kick off another run. :)
<Odd_Bloke> blackboxsw: Once you're around: https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373649
<Odd_Bloke> smoser: If you're around, ^ is pretty urgent for us, it's breaking a bunch of production deploys.
<Odd_Bloke> smoser: blackboxsw: I'm also proposing that we upload this as a hotfix (i.e. https://github.com/cloud-init/qa-scripts/blob/master/doc/ubuntu_release_process.md#cherry-picked-changes) because I want to release the bare minimum on a Friday.
<Odd_Bloke> (And it's been made clear to us that the last SRU will be reverted if we can't address this today, which I would, of course, like to avoid. :)
<blackboxsw> Odd_Bloke: +1 on hotfix. I'm looking at your branch now
<Odd_Bloke> blackboxsw: https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373655
<blackboxsw> validating against what I build now Odd_Bloke
<blackboxsw> Odd_Bloke: my only diff is expected https://paste.ubuntu.com/p/fd8zmgKdZN/
<blackboxsw> awaiting tox and we can manually merge into ubuntu/bionic
<blackboxsw> I may drop into hangout so we can pair/discuss if needed
<Odd_Bloke> blackboxsw: https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373656
<Odd_Bloke> blackboxsw: No need to merge, I'll just upload and push.
<blackboxsw> approved https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373655
<blackboxsw> Odd_Bloke: one sec
<Odd_Bloke> blackboxsw: What's up?
<blackboxsw> ahh you are good on ubuntu/bionic  I botched the package version name with 19.2-36-g059d049c-0ubuntu1~18.04.1ubuntu1 instead of 19.2-36-g059d049c-0ubuntu1~18.04.1
<blackboxsw> I had seen that extra line diff between our branches, but you did it correctly
<Odd_Bloke> OK, good.
<blackboxsw> Odd_Bloke: approved again https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373655
<blackboxsw> Odd_Bloke: approved ubuntu/devel  only metadata diff https://paste.ubuntu.com/p/GxK6rMFZp3/
<Odd_Bloke> Thanks!
<Odd_Bloke> blackboxsw: https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373657 is next up
<blackboxsw> approved disco https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373657
<blackboxsw> Odd_Bloke: I'm ready for xenial when you are
<Odd_Bloke> blackboxsw: Yep, working on it.
<Odd_Bloke> Ran into orig tarball mismatches, so had to regenerate my local packages.
<Odd_Bloke> blackboxsw: https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373660
<blackboxsw> ahh roger, happened to me yesterday
<blackboxsw> approved https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373660
<blackboxsw> I see you've pushed eoan good good
<coli_> Old_Bloke: rebased to master, tested on 18.10 and 19.10 no issues noticed.
<coli_> https://code.launchpad.net/~adobrawy/cloud-init/+git/cloud-init/+merge/354679
<coli_> Old_Bloke: can you kick off another run of CI against our branch (above) ?
<blackboxsw> will do cll
<blackboxsw> coli_:
<blackboxsw> CI rebuild kicked
<smoser> Odd_Bloke: sorry, wasnt around. just saw. i think you're sorted now.
<Odd_Bloke> smoser: Yep, thanks!
<coli_> blackboxsw: thank you.
<DobrawyAdam> I'm trying to make my first changes to the cloud-init project. Is https://jenkins.ubuntu.com publicly available? Today, at different times of the day, I tried and couldn't access (timeout).
<Odd_Bloke> DobrawyAdam: Unfortunately, it isn't.
<Odd_Bloke> It was, but recent Jenkins vulnerabilities have meant that our security team aren't comfortable having it public at all.
<Odd_Bloke> (We're in the process of migrating to a hosted system.)
<coli_> Odd_Bloke: DobrawyAdam is a person who is responsible for the code in:  https://code.launchpad.net/~adobrawy/cloud-init/+git/cloud-init/+merge/354679
<DobrawyAdam> Could somebody provide me a build log for https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/359/ ?
<Odd_Bloke> DobrawyAdam: I can, but the issue is that our integration testing has gotten wedged, so I'm afraid there isn't anything you'll be able to do about it for now.
<Odd_Bloke> DobrawyAdam: https://paste.ubuntu.com/p/MMrCkxcT3y/ <-- there you go
<Odd_Bloke> We'll get that sorted on Monday, we're all into our weekends here (and are still around because of a critical issue, to boot >.>).
<Odd_Bloke> Thank you for your contributions, and sorry that the CI experience is so frustrating at the moment!
<coli_> Odd_Bloke: can we help in some way ?
<DobrawyAdam> After reading the event log in general, I get the impression that the problem is beyond the scope of my patch.
<Odd_Bloke> Not really, though I appreciate the offer!
<Odd_Bloke> The fix itself wasn't too involved, we're just having to do manual testing across all the platforms that we have ready access to.
<Odd_Bloke> We're pretty much there, though.
<coli_> Odd_Bloke: good luck then and hopefully you all will be able to enjoy weekend soon. We will get back to it next week.
<coli_> Odd_Bloke: we are willing to help if these are some jobs like migrating the Jenkins which needs to be taken care of.
<Odd_Bloke> OK, great, we'll definitely bear that in mind!
<Odd_Bloke> Enjoy your weekend too!
<DobrawyAdam> Do you have knowledge of how providers perform continuous cloud-init integration tests with their platforms? I notice that tests/cloud_tests/platforms have integration tests only for EC2, KVM, LXD.
<Odd_Bloke> Those integration tests are run by us on the Jenkins infrastructure I just mentioned.
<Odd_Bloke> (They can run anywhere, they aren't tightly coupled to our infra.)
<DobrawyAdam> I understand that these are integration tests run by Cloud-init. I am thinking about other tests that run providers by self.
<DobrawyAdam> My patch has tests, but I'm afraid - due to the Cloud-init release cycle and all distributions release cycle - that the functionality will be broken and that the broken release will spread.
<Odd_Bloke> I don't know any specifics for other cloud providers, I'm afraid.
<DobrawyAdam> Thank you for the information. In that case, we will have to consider how to carry out such tests periodicly to react quickly, if our data source has been corrupted.
<blackboxsw> DobrawyAdam: we are definitely interested in having the cloud_tests extended to run against other platforms. so if you have interest in adding a new platform to cloud-init's test/cloud_tests
<blackboxsw> it would also help assist us in 1-off validation of needed in the future
<DobrawyAdam> What do you mean by "1-off validation"?
<Odd_Bloke> DobrawyAdam: When we're releasing cloud-init to existing Ubuntu releases, we do verification that it still works.  If we have cloud_tests for a platform, that's a lot easier.
#cloud-init 2019-10-05
<DobrawyAdam> Sounds like a solution to my fears. :)
<blackboxsw> our CI kicks off jenkins jobs against lxd, ec2, kvm  automitcally because they are platform definiitions in tests/cloud_tests. if you had a platform definition there too it should be a small lift to get jenkins jobs running against rbx too
<blackboxsw> DobrawyAdam: here's an example of what Azure folks are putting together https://code.launchpad.net/~ahosmanmsft/cloud-init/+git/cloud-init/+merge/372957
<DobrawyAdam> In the near future I will look into this solution more closely. I am glad that there is openness to run tests on the Rbx platform as well.
<amansi26> Hi, I am trying to deploy a image. But the IP is not getting assigned to the new lpar. I checked the logs. The cloud-final.service is having this piece of log http://paste.openstack.org/show/781184/
<amansi26> Is this the possible reason of IP not getting assigned. The OS is sles 12 and sles 15 .Can someone please suggest some workaround?
<amansi26> Odd_Bloke: I am using cloud-init v19.1
#cloud-init 2019-10-06
<blackboxsw> hi amansi26, it may be worth trying to file a bug and attaching /var/log/cloud-init.lo. What cloud/platform are your trying to deploy on? Openstack or something else?  DataSourceNone is coming up, which generally means cloud-init didn't detect the datasource type properly. It'd also be worth grabbing /run/cloud-init/ds-identify.log  and cloud-init status --long.
<amansi26> Hi, I just trying to capture the image and deploying the lpar on top of it. Openstack is not involved in that.
<amansi26> blackboxsw: Even sometimes I can this following error message also: stages.py[ERROR]: Unable to render networking. Network config is likely broken: No available network renderers found. Searched through list: ['eni', 'sysconfig', 'netplan']
<jgrasser_> exit
