#cloud-init 2013-09-16
<technolo-g> hi all :)
<technolo-g> im having an issue in which cloud-init on rhel is not utilizing the config_drive provided by openstack. im currently running 0.6.3-0.12. Is it just too old of a version to pick up the configdrive (which is properly provided as disk by label config-2)
<smoser> technolo-g, config-drive-v2 support came in 0.7.0
<smoser> ubuntu carries backport to 12.04's 0.6.3
<technolo-g> has anyone built >0.7.x on rhel6 do you know?
<smoser> technolo-g, i dont know. harlowja ^ 
<harlowja> ya, its been built
<harlowja> in active use at y!
<harlowja> rhel version 6.2, 6.3
<technolo-g> great, thanks guys. I'll see if i can track it down.
<technolo-g> once it is installed, it should automatically be looking for that configdrive correct? or do i need to specify a datasource?
<smoser> probably should, yesh.
<technolo-g> excellente, thanks again :)
<technolo-g> ok, sorry for the lazy questions, but im having a hard time finding how to explicitly set a datasource on an installed cloud-init. would it be possible to point me in the right direction? The datasources doc does not (that i can see) demonstrate how to set the datasource
<smoser> technolo-g, well its just a cloud.cfg setting.
<smoser> datasource_list: [ NoCloud, ConfigDrive, OpenNebula, Azure, AltCloud, OVF, MAAS, Ec2, CloudStack, None ]
<smoser> unless your config (or packaged config) overwrote it, it should be there.
<technolo-g> ok cool, i do have that. ill give it a shot
<technolo-g> alrighty, so running 0.7.1 (the only srpm i could find that would build on rhel6), when specifying the ConfigDrive datasource it just doesn't seem to find anything. i am still able to mount the configdrive itself and browse the contents. am i hitting this fix maybe?
<technolo-g> - allow config-drive-data to come from a CD device by more correctly
<technolo-g>    filtering out partitions.  (LP: #1100545)
<technolo-g> that was it :) now to do it cleanly
#cloud-init 2013-09-17
<pedroalvarez_> smoser: do you know which library needs "sources/DataSourceSmartOS.py" when it does `import serial`?
<pedroalvarez_> Because this one doesn't look ok to me: http://pyserial.sourceforge.net/pyserial.html
<smoser> http://pyserial.sourceforge.net/ is it.
<smoser> why "doesn't look ok"
<pedroalvarez_> Just because I don't understand why it needs a library to use the serial port.. 
<smoser> utlemming, ^ . i dont know for sure.
<smoser> i'm open to sipmle file io. on it
<pedroalvarez_> smoser: It also uses "boto", is this one? https://pypi.python.org/pypi/boto
<pedroalvarez_> to access Amazon web services?
<smoser> it uses boto to crawl the metadata service on amazon.
<smoser> basically turn their web service data into a dict.
<pedroalvarez_> smoser: thanks for the info :)
<utlemming> smoser: the answer is yes, because otherwise reading the serial is a real pita
<harlowja> utlemming probably u would have to rewrite a mini-version of pyserial to do it right, which does seem like duplicated work
<harlowja> *thats sort of the whole point of libraries*
<harlowja> pedroalvarez_ if u don't need certain datasources u can remove dependencies
<harlowja> aka if u don't care about enabling DataSourceSmartOS.py remove the pyserial dependency
<smoser> i dont really know about reading form serial devices.
<smoser> i'm perfectly fine to replace it with something else that works.
<smoser> pySerial is ubuntu-main (a requirement in the end). that does'nt mean other things can't be there, and we get them into ubuntu main. but it sure is nice if the thing is already there.
#cloud-init 2013-09-18
<smoser> harlowja, thoughts:
<smoser> https://code.launchpad.net/~vlastimil-holer/cloud-init/net-reconfigure/+merge/186352
<pedroalvarez_> Hi, I have configured cloud-init in a linux image, but when I boot the image, the systemd services crash. When I start them manually it works. What could it be?
<smoser> pedroalvarez_, i'm not sure. i've never used systemd. 
<smoser> what is 'crash' ?
<smoser> logs ?
<pedroalvarez_> smoser: http://paste.ubuntu.com/6124789/
<harlowja> ya, maybe logs, i haven't used systemd either myself (yet)
<smoser> well it sure thinks it failed to create the user
<pedroalvarez_> If I do: `systemctl start cloud-init` it works.
<pedroalvarez_> And if I reboot the image, sometimes it starts. 
<pedroalvarez_> I want cloud-init to execute commands at boot time, through the "customization-script" of OpenStack
<harlowja> ya, thats fine
<harlowja> pretty normal use-case
<pedroalvarez_> And, correct me if I am wrong, the services running are essential
<harlowja> likely :)
<harlowja> *likely u are right
<harlowja> pedroalvarez_ is this systemd via fedora
<harlowja> or systemd via some other distro?
<pedroalvarez_> customized core linux
<pedroalvarez_> baserock
<pedroalvarez_> I have located the systemd files on /etc/systemd/system
<pedroalvarez_> And I made a symlink in /lib/systemd/system/multi.user.bla.wants
<pedroalvarez_> Also in /lib/systemd/system/
<harlowja> hmmm, eck, defintly outside of my area of expertise :(
<pedroalvarez_> smoser: Does the install script try to copy them?
<pedroalvarez_> harlowja: no worries :) tvm
<harlowja> pedroalvarez_ can u just use the init.d versions instead of the sytemd ones?
<harlowja> that might work better, idk
<pedroalvarez_> harlowja: I'll try init.d version then :) thanks
<harlowja> those work on RHEL6 and such
<harlowja> and i know the upstart ones work
<harlowja> since smoser  uses them
<harlowja> the systemd ones i think work with fedora, but i don't know if smoser or me have to much experience with them
<smoser> yeah, i have no idea on systemd.
<smoser> harlowja, did you see above?
<harlowja> smoser that review looks ok, an interesting approach with the 'pretend'
<smoser> why is it 'pretend' ?
<smoser> what was that for. i didn't understand.
<harlowja> def _write_network(self, settings, pretend=False)
<smoser> right. what is that doing ?
<smoser> other than for testing. why would i want that.
<harlowja> ah, k, got your question
<harlowja> digging deeper, incase i missed why its being used, ha
<harlowja> i think its for 
<harlowja> dev_names = self._write_network(settings, pretend=True)
<harlowja> self._bring_down_interfaces(dev_names)
<harlowja> which is like a way to figure out the device names to bring down
<harlowja> although i would think its a little convoluted :-P
<harlowja> just have a 'list_interfaces' function and bring those down, ha
<harlowja> so from my understanding, whats happening
<harlowja> is that its pretending to write out the network interfaces, the result of this gives u the device names, and then the code is ifdown those
<harlowja> and then it writes out in non-pretend mode
<smoser> so the reason for this being is https://bugs.launchpad.net/cloud-init/+bug/1225922
<smoser> in short, if the system is booting, eth0 might have already been 'ifup'ed. and then we write a new /etc/network/interfaces file.
<harlowja> gotcha
<smoser> (ifup'ed successfully or non-successfully)
<smoser> we need to bring it down and then back up.
<smoser> and ideally we dont run 'ifdown -a'
<harlowja> so the pretend thing is really just a way to list the interfaces by using the same code that writes out the interfaces (which returns a list of device names)
<smoser> and 'ifup -a'
<smoser> (although that i a hacky solution)
<smoser> ah. 
<harlowja> pretend make sense now?
<harlowja> thats my interpreation
<smoser> but where does he call with 'pretend'?
<smoser> ah. i see.
<harlowja> 19	+ if bring_up:
<harlowja> 20	+ dev_names = self._write_network(settings, pretend=True)
<harlowja> 21	+ self._bring_down_interfaces(dev_names)
<harlowja> it is a little convoluted i guess, haha
<harlowja> a function like 'list_interfaces(settings)'
<harlowja> that returns the device names
<harlowja> probably would be clearer to understand
<smoser> yeah. 
<smoser> parse the existing
<smoser> parse the new
<harlowja> ya, since all that write_network does is parse settings and write out a file
<harlowja> *or files
<harlowja> want me to comment on that review
<harlowja> or u got it under control boss
<pedroalvarez_> Ok, after see that cloud-init service tries to create an "ubuntu" user, and then another service tries to use "apt", I think it thinks I'm running ubuntu
<harlowja> :)
<harlowja> haha
<pedroalvarez_> hahahahaha
<harlowja> ya, so u need to edit the cloud.cfg for that
<pedroalvarez_> harlowja: good news then?
<harlowja> yup
<harlowja> # System and/or distro specific settings
<harlowja> # (not accessible to handlers/transforms)
<harlowja> system_info:
<harlowja>  # This will affect which distro class gets used
<harlowja>  distro: rhel
<harlowja> something like that
<harlowja> you are probably using the default ('ubuntu')
<harlowja> and the other part is what modules u have left active by default in cloud.cfg
<harlowja> some of them are more ubuntu speicfic
<harlowja> *aka ones starting with 'apt' and such
<harlowja> all smoser  fault, ha
<pedroalvarez_> The other problem of initialization is just because cloud-init needs ssh services running, and cloudinit doesn't wait I think.
<smoser> harlowja, i'd appreciately comement.
<smoser> pedroalvarez_, the user is configurable.
<pedroalvarez_> smoser: that's cool
<pedroalvarez_> I think I found all my problems on this file! :)
<harlowja> :)
<pedroalvarez_> I'm going home happy today
<pedroalvarez_> I owe you a couple of beers
<harlowja> ha
<harlowja> np
<harlowja> cloud.cfg is the magic file
<smoser> cloud.cfg is the magic.
<harlowja> all the magic
#cloud-init 2013-09-19
<pedroalvarez> Hi :)
<pedroalvarez> I'm trying to do a configuration of the magic cloud.cfg. I see a tag to set the users which are going to use some modules. What about if I just have a root user? Is enough to set "disable_root" to false?
<smoser> pedroalvarez, i dont know. root user is untested.
<pedroalvarez> smoser: Morning :)
<smoser> it could work.
<pedroalvarez> I'm missing something, cloud-init executes the cloud.cfg file, but is not receiving the "user-data" script. 
<pedroalvarez> from OpenStack
<pedroalvarez> Hmmm... I see there are another projects: "cloud-utils" and "cloud-initramfs-tools"
<pedroalvarez> Does openstack need them to communicate stuff to the image from the "user-config" box?
<smoser> pedroalvarez, no.
<smoser> can you pastebin /var/log/cloud-init.log ?
<smoser> it should log there.
<pedroalvarez> smoser: it's empty
<smoser> that is odd.
<smoser> i'm really sorry, pedroalvarez . i'm being pulled on oher things right now. i can't help. you probably want to just try runnign 'cloud-init --local init'
<smoser> and then cloud-init init
<smoser> you shoudl be able to run those manually
<smoser> and look at debug output or debug manually
<smoser> and get somewhere.
<smoser> i'm sorry i cna't help more.
<pedroalvarez> ok smoser no worries
<pedroalvarez> you have been a great help this time
<pedroalvarez> :)
#cloud-init 2014-09-15
<m01> Hi people. Does someone know if cloud-init is supposed to be able to setup multiple network interfaces if I launch a VM with multiple network ports in OpenStack, EC2 etc?
<m01> (I'm only seeing the first one being setup for DHCP)
<smoser> m01, there are some bugs in that area. but current cloud-init should be able to set up networking for multiple interfaces if
<m01> ..if?
<smoser> a.) eth0 is configured in image to be dhcp (and can get a dchp address) or otherwise is correctly configured.
<smoser> b.) no other network interfaces are configured in the image
<m01> I think both of those conditions are fulfilled
<smoser> c.) you use config drive
<m01> ok
<smoser> posisbly it would work with openstack metadata service
<smoser> i'm not entirely sure. 
<smoser> the goal is to have these issues fixed so that reliably the networking that is presented would do the right thing
<m01> Ok, excellent - so cloud-init is expected to do this
<smoser> there are definitely bugs
<m01> do you happen to know if the ubuntu 14.04 cloud image should have a recent enough cloud-init for this to work?
<m01> don't worry if you don't know of the top off your head, I'll go investigate
<smoser>  http://paste.ubuntu.com/8350293/
<smoser> 14.04 should be close. and the goal is to at some point have that functional in 14.04.
<m01> excellent
<m01> I'll make sure I enable the config-drive
<m01> and then re-test
<m01> thank you so much!
<m01> so I tried the ubuntu 14.04 image, with config drive (and eth0 working), and unfortunately eth1 isn't setup
<m01> I don't think nova injects an /etc/network/interfaces (at least it's not on the config drive)
<m01> the other interface is just connected to a dhcp network in openstack
<m01> i've got Version: 0.7.5-0ubuntu1
<m01> that's the latest according to launchpad
<smoser> m01, how did you attach ?
<smoser> is /etc/network/interfaces correct?
<smoser> basically what should happen is that cloud-init should find the /etc/network/interfaces file that is provided to it on the config-drive and place that in /etc/network/interfaces. 
<smoser> and 'ifup -a'
<m01> /etc/network/interfaces just has the eth0 config
<m01> ah
<m01> that file isn't in the config drive
<m01> I just mount /dev/sr0 test
<smoser> m01, it should be somewherein there. 
<m01> it wasn't.. I did a find, and an ls -lR
<smoser> whats in files/
<m01> 1s
<m01> well, I need to re-launch the instance actually
<m01> I blew up the networks
<m01> and re-configued my openstack setup
<m01> *launching
<m01> http://pastebin.com/p3PLXjvj
<m01> here's the contents of my drive
<m01> i don't have a files/ directory
<harmw> cirros-0.3.3-x86_64-initrd
<harmw> cirros-0.3.3-arm-blank.img
<harmw> cirros-0.3.3-arm-vmlinuz
<harmw> cirros-0.3.3-arm-initrd
<harmw> Build step 'Execute shell' marked build as failure
<harmw> [ssh-agent] Stopped.
<harmw> Finished: FAILURE
<harmw> hm, now why is Jenkins telling me that
<JayF> exit code
<harmw> yea, that was my first assumption as well :)
<JayF> if your build script doesn't exit 0, it'll report failure
 * JayF not trying to snark
<harmw> snark? 
<JayF> like be sarcastic :)
<harmw> wasn't that some bug in half-life :p
<JayF> I was trying to actually be helpful
<JayF> lol
<harmw> oh np :)
<harmw> it's bin/build-release that probably needs some love now
 * JayF has added ||true to the end of those things to make jenkins pass
<harmw> true, but it looks like it just bails somewhere
<smoser> its probably the stupid tag check
<smoser> it doesn't really support building from trunk
<smoser>  but it shoud lhave been fairly straight forward in saying that
<harmw> hehe
<harmw> can't you just fix that :p
<smoser> well, someone needs to . harmw you can just tag first before you build.
<harmw> wouldn't that limit everything to just my branch?
<smoser> i dont follow
<smoser> basically just do somethin glike:
<smoser> tag=harmw-$(date +%Y%m%d-%S)
<smoser> bzr tag $tag
<smoser> ./bin/build-release $tag
<harmw> uhm, yea ok
<harmw> but there should be a 0.3.3 tag, right?
<smoser> there should, yes. 
<harmw> since I'm currently running bin/build-release 0.3.3
<smoser> http://paste.ubuntu.com/8352141/
<smoser> your branch might not have that tag in it.
<harmw> wll I should be building trunk here
<smoser> well, that shows that there *is* a 0.3.3 tag in trunk
<harmw> indeed
<harmw> JayF: you've used jenkins with the irc plugin?
<JayF> Yes
<harmw> please tell me where I can configure that darn thing :p
<harmw> can't seem to find it
<JayF> um
<JayF> almost all of that kind of plugin setup
<JayF> is shoved somewhere into global setup
<harmw> I thought so, but I'm not seeing it
<JayF> found a gnarly little bug in cloud-init with noblock resizes
<JayF> about to push up a fix
<JayF> smoser: https://bugs.launchpad.net/cloud-init/+bug/1338614 I'm pushing a fix for this
<JayF> smoser: as I just fixed it locally
<JayF> whee
<JayF> the fun thing is
<JayF> the resize still happened
<JayF> just did it blocking in the fg
<JayF> then threw a fun exception
<JayF> smoser: harmw: harlowja_a*: https://code.launchpad.net/~jason-oldos/cloud-init/bug-1338614/+merge/234749 should fix 1338614. I was unable to run tests due to local enviornment problems but will run them as soon as my VM recovers :)
<JayF> I've verified this code fixes the exception shown and makes the resize happen in the background
<JayF> I think I may have broken the bzr
<harlowja_> hmmm, lol, u broken the bzr!
<harlowja_> ha
<JayF> harlowja_: my working copy is an absolute mess
<JayF> harlowja_: I want git reset --hard origin/master
<JayF> except I don't know how to tell bzr to do that :x
<harlowja_> ya, i only know the basics, smoser though probably knows it all
<harlowja_> JayF did u read over all the taskflow stuff?
<harlowja_> :-P
<JayF> I read it, most of it sounded like anti-git propoganda
<JayF> I think I want a direct mapping of commands
<JayF> but there is none
<harlowja_> lol
<JayF> wiki.bazaar.canonical.com/Workflows was not useful to me at all :(
<harlowja_> i wonder how https://github.com/termie/git-bzr-ng works
<JayF> I don't like not knowing things, I'd rather just know how to make this work :)
<harlowja_> :)
<harlowja_> i try to stay using git if i can :-P
<harlowja_> smoser so when cloud-init movign to git ;)
<JayF> if cloud-init moves to github, I'll even setup all the travis ci for it :P
<JayF> and then eat my flip flop in surprise
<harlowja_> nice
<harlowja_> i'd pay to see that
<harlowja_> JayF i think u missed the days when openstack was fully using bzr :-P
<harlowja_> about 2.5 years ago
<harlowja_> there was some fun mailing list threads about that one
<harlowja_> https://lists.launchpad.net/openstack/msg01741.html
<harlowja_> *and many followups*
<JayF> jesus, that sounds awful
<harlowja_> JayF :)
<harlowja_> the good ole days
<harlowja_> lol
<smoser> harlowja_, JayF its not an impossibility
<smoser> moving to git.
<JayF> for every minute I spend working on cloud-init code
<JayF> I spend two figuring out bzr
<JayF> and I'm sure I'll get off that hamster wheel one day, but for now it's annoying
<smoser> i spent a good while today thinking about how networking should work
<JayF> especially seeing that I can't get my local to get up to whatever HEAD is now, I'm a few revisions behind :(
<smoser> in a world where cloud-init boots, and gets networking information from a local source (ie , config drive). and also from a network source / hot plugged.
<JayF> we've been doing some thinking about the ongoing stuff as well
<JayF> i.e. a customer calls an API to add a cloud network, how does that get configured?
<JayF> on a hypervisor you can just drop in an interface
<JayF> without a hypervisor you have to change something (like the metadata in an md service) and have something looking for changes there
<JayF> but we're borderline if it's in cloud-init scope at that point, right?
#cloud-init 2014-09-16
<smoser> hm.. 
<smoser> for virtual nics. the hotplug network item should trigger the cloud-init query to MD
<smoser> and then ifup and away we go.
<smoser> for physical...
<smoser> i dont know
<JayF> physical in our world, we'd add a vlan to a trunk, and somehow send a signal in out of band
<JayF> this is still in the hand-wavy portions of planning
<JayF> hence the hand waving
<smoser> yeah
<smoser> well, in physical world it could also be "ok, configure that nic now"
<smoser> (that was previously un-configured).
<JayF> how does cloud-init know to check?
<JayF> you'd have to have something persistent or cron'd to check
<smoser> but yours is an interesting case for thought also. essentially "hot plug vlan".
<JayF> because no event to the host, like hotplugging
<JayF> yup
<smoser> right. i wouldn't want to do cron. we could... but 
<JayF> this is explicitly "I want you to do something without you having any other indication you should do it other than me telling you"
<JayF> which seems out of cloud-init scope a little
<smoser> we could long-poll
<smoser> i'd not be completely opposed to that. 
<JayF> do you want a model, ever, where cloud-init runs forever?
<smoser> in the past i've very much said "cloud-init is init".
<JayF> yeah that was my impression as well
<smoser> but it seems less of a stretch here because i *want* to be able to handle the event driven portion
<JayF> smoser: btw, you should really merge the branch I pointed you at earlier. Nasty nasty bug.
<smoser> udev hotplug -> cloud-init-query of MD -> ifup
<JayF> smoser: something I was hunting for a while :)
<smoser> which branch ?
<smoser> i might have missed it.
<smoser> so what i was saying is that since i want cloud-init to have code to do the hotplug based on an event , i'm not entirely opposed to somethign in cloud-init's source code being the source of the event.
<smoser> or you coudl have an agent that generated the event to cloud-init.
<JayF> 14:43:32 <JayF> smoser: harmw: harlowja_a*: https://code.launchpad.net/~jason-oldos/cloud-init/bug-1338614/+merge/234749 should fix 1338614. I was unable to
<JayF>                 run tests due to local enviornment problems but will run them as soon as my VM recovers :)
<JayF> If we run an agent, it'll be an agent smart enough to do everything
<smoser> i thought i had fixed that 
<JayF> Like I said, I can't get my bzr to HEAD, so it's possible you did
<JayF> but it's absolutely broken in the current release
<JayF> or at least what I'm building against, which I think is current release
<smoser> how can you "not get your bzr to head"
<JayF> I'm at 'revno 1009' according to my cloud-init, and that's with the two commits I wrote today
<JayF> I want the equivalent of 'git pull' or 'git reset --hard origin/master'
<JayF> I can't get bzr to just give me a clean working copy of HEAD
<smoser> well you could just : bzr branch lp:cloud-init
<smoser> again
<smoser> or bzr pull --overwrite 
<smoser> that git reset --hard
<JayF> the bzr branch command didn't work
<smoser> bzr branch lp:cloud-init
<smoser> did not  work ?
<JayF> it didn't appear to do anything to my current working copy
<JayF> bzr pull --overwrite is exactly what I wanted
<JayF> smoser: that bug exists in HEAD :(
<smoser> irght. 
<smoser> bzr branch lp:cloud-init
<smoser> is equivalent to 
<smoser>  git clone git://foo.git
<smoser> so in a working copy, you probably got a new cloud-init/ dir underneithy
<JayF> aha
<JayF> that's exactly what it did
<JayF> okay I understand a bit better now
<JayF> I just like after I push something being able to get my working copy back to 'master'
<smoser> you dont explicitly need the kwargs portion
<smoser> of that MP.
<smoser> right ?
<JayF> Locally it didn't work until I had that
<JayF> and that was suggested by someone smarter than I (comstud)
<smoser> hm..
<JayF> I know that I had not-working. Apply that patch, it works. Apply that patch sans **kwargs on the function signature, and it fails
<smoser> ok. pushed.
<smoser> thanks 
<smoser> JayF, the idea of "add a vlan" is interesting as itself.
<smoser> as i dont have anything that would generate such an event.
<smoser> ie, for virtio ethernet . i could see getting the nic added, and cloud-init reading the config and the config saying "make a vlan" and cloud-init configuring correctly.
<smoser> but the core event was the nic added.
<JayF> yeah that's what I'm saying
<JayF> in bare metal, no hypervisor, it takes a physical action to generate a hardware event
<JayF> or much fancier bmcs than I have :)
<JayF> [insert hypervisors are for the weak joke here]
<smoser> ok. so i have to think about that some more.
<smoser> have you ever used hackpad ?
<smoser> https://hackpad.com/Cloud-init-Networking-1gtK434RgVg
<smoser> that seemed like a bit more mark-uppy version of etherpad. dont know how i feel
<smoser> but i put some headers in there.
<smoser> i have some content too , but is all very rough
<smoser> i just can't stand typing in anything other than vi
<JayF> ever looked at floobits?
<JayF> their vim plugin is... lacking
<smoser> i'd not seen that. 
<smoser> no content there :)
<JayF> yeah I can tell now, lol
<smoser> i was trying to write how i'd want stuff to work
<smoser> basically, 
<JayF> my answer to that, more or less, is 'well' and 'reliably'
<smoser> cloud-init-local comes up, and blocks hotplug events until its done.
<JayF> hehehe
<smoser> once its done, it "releases" the nic that it deemed (or found in a local datasource) to be the "metadata service link"
<smoser> then once that comes up
<smoser> all the others are un-blocked
<smoser> and they come up via getting information from the MD.
<JayF> I was thinking more that
<smoser> in the case of no network MD, and only provided by config-drive, the "releasing" of those blocked events would then cause read from that cached data
<JayF> you'd add a new stage to cloud-init
<smoser> and they'd come up too
<JayF> that's possibly persistent
<JayF> that handles post-boot events by re-running the relevent part of the other stage
<JayF> i.e. network json changed; call that module to regenerate network from md service
<smoser> i'd liek to not just have "network json changed"
<smoser> but "nic AA:BB:CC:DD:EE:FF added"
<smoser> rather than determining what changed.
<JayF> see I think there are dragons there
<JayF> because by doing it that way you *are* determining what changed
<JayF> or requiring someone outside of cloud-init to give you a specific trigger
<JayF> if you're doing a long poll on a md service
<JayF> you'll be able to quickly know what changed
<smoser> right. 
<smoser> unles sthe long poll is given explicit updates
<smoser> ie, rather than just getting a new version of the json
<JayF> but the idea of trying, on the fly, to 'diff' the old and new config in some meaningful way seems scary
<smoser> it would get the element that changed.
<JayF> yeah but then you've introduced a second api interface
<JayF> one for init and one for ongoing
<smoser> yeah.
<JayF> which I personally dislike
<smoser> well, only added the second api
<smoser> because there was not an event
<smoser> so we had to essentialy receive the event
<smoser> but then i guess what happens if you miss it
<smoser> so it would be good to determine what changed.
<smoser> and *that* is easy enough , at least its seemingly possible.
<smoser> but figuring out what that means in terms of 'ifdown X, ifup Y' is more complex.
<JayF> I'd say roughly determine what changed
<JayF> like in categories
<JayF> like 'ssh keys changed, kickoff add_ssh_keys module'
<JayF> 'network changed, kickoff network module"
<JayF> etc
<smoser> ah. i wasn't goign to go that far.
<JayF> then you make the ability to update part of each module
<smoser> i was only going to take events on network config.
<JayF> they can either do everything over again or just generate a diff
<JayF> why not go the whole way?
<JayF> why not accept an event saying "you have a new cinder volume that wants to be mounted"
<smoser> well, some are not idempotent.
<JayF> assuming cloud-init would know how to react to some things
<JayF> make modules opt in to being event-driven and wrk to make the modules idempotent
<JayF> anything in config management (cloud init kinda is) should strive to be idempotent where possible anyway imo
<smoser> i dont have strong feelings against what you're syaing.
<smoser> but i'd like to have networking first :)
<smoser> as i think that is where we *need* a solution
<JayF> yes, I would like to have networking too
<JayF> I need to get those changes written and up :x
<smoser> the rest of it, i think ihave a fair argument in "let something else do that"
<JayF> well, edited and up, mostly
<JayF> yeah, but the thing is
<smoser> but nothing else really plays in the automatically-configure-hot-plug-netowrking world.
<JayF> *someone* is going to want a conduit for setting up arbitrary things via event
<JayF> and if cloud-init doesn't do it, someone will
<JayF> and I'm trying really hard to not be someone
<smoser> well, there are lots of things that do that arlready.
<JayF> lol
<smoser> juju does it
<smoser> puppet does it
<smoser> rackspace guest agent does it/
<JayF> Nothing does it from the perspective of a provider
<JayF> ugh, lets not bring nova-agent into this :(
<JayF> I try to forget it exists and have success most days
<smoser> so i kind of see the network stuff as cloud-init's job . 
<JayF> but like "mount me a network volume at boot" doesn't fit?
<smoser> but beyond that, i struggle.
<smoser> well at boot, that is there.
<JayF> then just add general support for event-driven do stuff
<JayF> and maybe a certain JayF would end up writing the event provider for cinder
<smoser> how would we get an event on cinder ?
<JayF> I'd presume we'd model drives to mount as an object in user_data+vendor_data
<smoser> oh. duh.
<smoser> sorry
<smoser> well, there is a dataformat for mounts and what to do with mounts in cloud-init as it is.
<JayF> so lets imagine rackspace, we'd just update vendor_data.json to add ['volumes']['my_fancy_volume'] = { 'place': '/data
<JayF> you get the idea
<smoser> it just only fires on those in the block-device-mapping.
<smoser> so it would be very annalogus to networking solution
<smoser> to have it respond to the newly attached disk
<JayF> so if you saw that piece of data change, you'd trigger an event to revaluate that data
<JayF> exactly
<smoser> and hit the MD for "what should i do with that"
<JayF> everything fits this model fairly well
<smoser> so yeah
<smoser> ok. you sold me there on that.
<smoser> but i'm not sold on ssh keys yet :)
<JayF> ssh keys is my personal #1 use case
<JayF> because nova (or Rackspace Cloud, IDK the difference) only lets you inject one in the nova boot command
<JayF> so the ability to trigger more keys gettting added post-boot seems nice
<JayF> and I know of at least one group in Rackspace that's doing something that'd fit this workflow, but in a more gross way
<harlowja_> hmmmm, seems like a job for mr.puppet or mr.chef
<JayF> again though, I'm not the owner of the node
<JayF> I'm just the service provider trying to make my customer click pretty buttons in a panel and make things happen
<smoser> monkeyspeere
<smoser> http://web.monkeysphere.info/getting-started-ssh/
<JayF> smoser: a google of that takes me here first http://www.cracked.com/article_14990_what-monkeysphere.html
<JayF> smoser: which was really perplexing, hahaha
<JayF> smoser: that's amazing
<JayF> This was a good chat, thanks :) I may be away mostly for a couple of days, but I'll be back
<smoser> i was thinking that monkeyspehere had something to manage keys installed
<smoser> but i guess not
<smoser> anyway
<smoser> later JayF thanks for you patch and your patience
<m01> smoser: remember my networking issue yesterday (only my first ethernet connection is setup)? I looked at the OpenStack nova sourcecode. They do have a template for the interfaces file, but it looks like it's only used for bare metal targets (e.g. PXE-booted nodes), it doesn't appear to be used in general..
<smoser> m01, you appear to be correct.
<smoser> it definitely does that in some cases.
<smoser> its not only for bare metal.
<smoser> harmw, https://code.launchpad.net/~smoser/cirros/buildroot-upgrade
<smoser> i think that builds for i386
<smoser> running bin/bundle now
<smoser> hopefully that'd get us to buildroot-2014.08
<m01> hey
<m01> hmm
<m01> Are you saying that nova uses the /etc/network/interfaces template in environments other than baremetal?
<m01> (e.g. libvirt, kvm etc)
<m01> if cloud-init requires nova to stage that template in, then I guess I need to take it up with the openstack community?
<smoser> how did you detemrine that it does not do that ?
<smoser> it was not clear to me.
<smoser> the interface that it uses right now is silly... it declares networking configuration in /etc/network/interfaces format.
<smoser> which is very limited and guest-specific
<smoser> we're wanting to do it in a more generic way. that is something that JayF is working on and pquerna also.
<smoser> see https://review.openstack.org/#/c/85673/
<m01> so I looked in the config drive, and didn't find it there (see http://pastebin.com/p3PLXjvj)
<m01> and then I went and looked at the nova code
<m01> https://github.com/openstack/nova/search?utf8=%E2%9C%93&q=net-dhcp&type=Code
<m01> that's the search for net-dhcp.ubuntu.template
<m01> which is here: https://github.com/openstack/nova/blob/master/nova/virt/baremetal/net-dhcp.ubuntu.template
<m01> I believe that's the file you're referring to, and that you're expecting in the config drive, right?
<m01> I only found that in the baremetal directory under virt, and I didn't spot anything equiavelent for the other "drivers", or whatever they're called
<m01> and yes, I agree that's a silly format..
<JayF> m01: you really shouldn't be using nova-bm at this point
<JayF> m01: it's being deprecated in Juno in favor of Ironic
<m01> I'm not using it
<m01> I just noted that it's the only driver I've identified that actually stages the /etc/networks/interfaces file
<m01> and (therefore) I don't have one in my config drive
<m01> so my non-first network interfaces don't get IP addresses
<m01> (and I'm confused about how this is meant to work)
<JayF> Ah, Ironic driver does too 
<JayF> at least it does in my install of it; but it's always possible we patched it downstream
<m01> I'm using libvirt anyway I think
<smoser> well, we need to fix it.
<smoser> i'm sorry for leading you on a goose hunt. but looking at code i dotn see why it would not get rendered. 
<smoser> dont obviously see why
<m01> smoser: which code are you looking at?
<smoser> nova/api/metadata/base.py
<smoser> bah. someone busted that.
<m01> CONF.injected_network_template
<m01> aha
<m01> interesting
<m01> let me see what that's set to
<m01> nova.conf:#injected_network_template=/usr/share/nova/interfaces.template
<m01> hmmmm
<smoser> m01, its busted. 
<smoser> the metadata service (used by config drive or metadata server)
<smoser> will never call netutils.get_injected_network_template
<smoser> with a template
<smoser> oh wait.
<smoser> no.. yeah, so you were right. 
<smoser> see, not sure why it doenst work
<m01> I'll uncomment that setting 
<m01> and try it
<m01> hmm
<m01> I uncommented that, restarted all nova services, and my config drive still doesn't have the file
<m01> so, I guess it is busted
<harmw> smoser: cool
<harmw> though I did some work in that area already
<smoser> well this builds. just got the build.
<smoser> it doesnt run particularly well.
<smoser> noticing 2 things:
<harmw> ok
<smoser> a.) no lsblk or sfdisk
<smoser> b.) dropbear is broken
<smoser> doesnt start
<harmw> ah
<harmw> i fixed just that 2nd issue
<harmw> and missing lsblk... sounds like some new BB knob
<harmw> dropbear needs to be started with some other argument, iirc
<harmw> please look at my branch, there should hopefully be something good in it :P
<harlowja_> http://online.mirantis.com/openstack-sv-live-feed if u guys interested
<harlowja_> from http://openstacksv.com
<harmw> whats that?
<harlowja_> a sv openstack minidaysummit kind of thing
<harlowja_> if u interested
<harlowja_> with livestream
<harmw> ok, so just what is sv :p
<harmw> the rest was obvious :p
<harlowja_> *silicon valley
<harmw> super vilain?
<harmw> ah
<harlowja_> lol, super vilan
<harmw> :>
<harlowja_> :)
<harmw> smoser: https://code.launchpad.net/~harmw/cirros/cirros-buildroot-2014.02 
<harmw> smoser: new c-i tag
<harmw> when! :p
<smoser> we're closer .
<smoser> maybe i should just call it.
<harmw> +1
<smoser> i was really hoping to get the network config in 
<harmw> nah, who needs networking
#cloud-init 2014-09-17
<ndonegan> How stable is the config stuff in cloud-init? ie, can I add a cc_whatever to the config directory and expect it to work across versions?
<jkraj> HI 
<jkraj> in user-data file, what should I use 'lock-passwd' or 'lock_passwd' ?
<jkraj> cloud-config, users section 
<smoser> ndonegan, the loading of those has changed possibly...
<smoser> let me look.
<smoser> ndonegan, it appears that the interface to loading those has never changed in cloud-init.
<ndonegan> Cool. Will make my life a little easier.
<smoser> jkraj, it looks like it is 'lock_passwd'
<smoser> but it defaults to 'True'
<smoser> so you dont have to do anything.
<jkraj> but 'lock-passwd : false' works for me
<smoser> hm.. you're sure? is that on a non-default user ?
<jkraj>  - name: user1   passwd: pass1   lock-passwd: false
<jkraj> yes
<jkraj> smoser, I am sure  lock-passwd: false &  lock_passwd: false works for me, 
<jkraj> yes for non default user
#cloud-init 2014-09-19
<harmw> smoser: ready for new release?
<smoser> :)
<smoser> dont' you avhe something else to do on friday night then bother me
<harmw> forgive me for asking so often, I'm just eager to submit that fbsd port
<harmw> hehe
<smoser> you are completely justified there, harmw 
 * hiren_ likes harmw's persistence ;-)
<harmw> lol
<harmw> yea
<harlowja> damn i must of missed an important conversation, lol
<harmw> dont you always 
<harlowja> i think so :-P
<harlowja_> of course right when i say that my thunderbolt crap cable comes undone, lol
<harlowja_> thats what i don't like about the cable, it doesn't hold itself in, lol
<harlowja_> stupid apple
<harmw> :p
<harmw> harlowja_: how sucky  is the Y! cloud?
<harlowja_> lol
<harlowja_> define sucky
<harlowja_> lol
<harlowja_> define parameters to be sucky
<harlowja_> haha
<harmw> hiren_'s having a hard time getting cirros up and running
<harlowja_> hmmm, not sure, haven't heard what he's doing
<hiren_> harlowja_: in our internal n/w
<harlowja_> well shouldn't be that hard, idk how well cirros supports config drive
<harlowja_> if it don't support config drive, then it may not work :-P
<harmw> isn't there a normal metadataservice?
<harlowja_> nope
<harlowja_> just config-drive (until we revist that debate)
<harlowja_> our security team didn't want a channel from VM -> controller layer
<harmw> jeez
<hiren_>  http://dpaste.com/184H91R
<harlowja_> and metadata service is one such channel
<harmw> wankers
<harlowja_> ya, i tried fighting that, i gave up
<JayF> Doing a metadata service in a secure and multitenant way is not an easy task :)
<harlowja_> config-drive is nice and read-only, no two way channel...
<JayF> putting an ISO partition on a VM/disk is super easy :)
<harmw> so true
<harlowja_> right, thats partially why its very attractive for security scaredfolks
<JayF> harlowja_: actually, you can put ConfigDrives in vfat according to the spec
<harlowja_> ya, vfat, iso9660
<JayF> harlowja_: so it doesn't have to be r/o... although we implement it as an iso9660
<harlowja_> right
<harlowja_> well r/w it will still be a local thing
<harlowja_> r/o is just more of an optimization
<harlowja_> so hiren_ the following seems odd
<harlowja_> cp: write error: No space left on device
<harlowja_> cp: write error: No space left on device
<harlowja_> cp: write error: No space left on device
<harlowja_> cp: write error: No space left on device
<harlowja_> failed to copy results from configdrive to /run/cirros/datasource
<harlowja_> :-/
<harlowja_> that would seem to be an issue, lol
<hiren_> heh yeah
<harmw> :)
<hiren_> brb.
<harmw> harlowja_: there is dhcp or did they skip that as well?
<harlowja_> not for vms
<harlowja_> for the existing way baremetal is deployed, it uses dhcp
<harlowja_> and pxeboot 
<harmw> so they must get their config from configdrive, right?
<harlowja_> yes
<harmw> hmk
<harmw> interesting
 * harlowja_ we'll move over to ironic which afaik has a similar dhcp, pxeboot, ipmi as the thing we already have (people are shifting from that team to ironic)
<harlowja_> JayF will get a bunch of new iroinc friends (besides rloo)
<harlowja_> soon enough, ha
<harmw> :P
<JayF> You guys going to use ipa?
<harmw> ipa?
<harlowja_> unsure
<harlowja_> JayF probably, although i'm not that in touch with all that stuff, haha
<harmw> whats ipa :)
<JayF> ironic-python-agent
<harmw> ah
<JayF> the newest, sexiest deploy driver for ironic
<JayF> :P
<harmw> lol k
<harmw> i'm still clueless when it comes to ironic :p
 * harlowja_ mostly me too, ha
<harlowja_> can't be everywhere :-P
<harmw> :p
<harlowja_> y! will likely have to figure out a different mechanism than config-drive for ironic stuff though, something like maybe what JayF and folks have done (writing the config-drive on the main disk)
<harlowja_> attaching a cd isn't gonna work, lol
<harlowja_> or maybe we'll revisit that whole networking metadata webservice debate
<harmw> :)
<JayF> Dude, we already have it handled
<harlowja_> even better, ha
<harmw> smoser: did that configdrive change for fbsd got merged yet? from raginbajin?
<smoser> i dont think so.
<smoser> there was some work to still dothere.
<JayF> harlowja_: Yeah, I am not sure if it's fully upstreamed, but it works entirely and we'll help you get it working if you need :)
<harmw> hm ok, 
<JayF> harlowja_: nova is the one who generates teh configdrive... it just gets passed through ironic, to the agent, into the drive
<harmw> well I need it to test with configdrive, since Y! apparently requires/uses that
<harlowja_> JayF ya, that seems nutty :-P
<JayF> harlowja_: we just create a small partition at the end of the disk, and write it out
<harlowja_> to much passing of crap around, lol
<JayF> harlowja_: that's the Openstack Way(tm)
<harlowja_> :(
<smoser> for ocnfig-drive on bare metal...
<JayF> harlowja_: in our original implementation, the agent built the iso :(
<harlowja_> JayF why doesn't nova just pass the data that is the config-drive to ironic
<harlowja_> seems saner
<smoser> my opinion is that long term, nova (or deployer, whatever) has to be *very* stupid.
<harlowja_> then passing a binary disk
<smoser> and just dd stuff to the first disk
<smoser> and then look for a partition on that disk named "config-drive" or re-use the EFI partition or something
<smoser> and write the data in there.
<smoser> in that way, the image is smart
<JayF> harlowja_: because Nova is used to just writing it out itself
<smoser> and tine installer is dumb
<harlowja_> sux
<smoser> and you have a standard.
<JayF> smoser: that's pretty similar to what ipa does
<harlowja_> nova's like an old grandpa or something, lol
<smoser> sort of similar.
<smoser> but ipa (i think) has to know how to "make it boot". ?
<JayF> smoser: puts an image on a disk, looks for a partition labelled 'config-2', if it exists, we dd the configdrive into that partition, if not, we create it at the end of the disk
<JayF> smoser: the bootloader is entirely in the image
<smoser> it dd the disk ?
<JayF> yeah
<smoser> including the partition table ?
<JayF> let me link you the code
<JayF> yeahhh
<smoser> nice. 
<JayF> now we'll have to support the partition-images eventually
<smoser> yeah, that is theo nly way you can actually accomplish secureboot
<JayF> bceause --preserve-ephemeral (hate)
<smoser> meh
<smoser> partition images dont have to be supported.
<JayF> but I would not use it :)
<JayF> smoser: tell that to Devananda, hahaha
<smoser> its just not scalable.
<smoser> it means that the installer has to know information about the things its installing
<JayF> smoser: https://github.com/openstack/ironic-python-agent/blob/master/ironic_python_agent/shell/write_image.sh + https://github.com/openstack/ironic-python-agent/blob/master/ironic_python_agent/shell/copy_configdrive_to_disk.sh
<smoser> --preserve-ephemeral could just be "i write as many blocks as are in the image. no more , no less"
<smoser> you want presevation of data then keep the thing i'm writing small.
<JayF> I'm not sure I agree, but I also hate the same feature you hate which is funny
<harlowja_> haters gonna hate
<smoser> where is /tmp ?
<smoser> is that dis ?
<JayF> smoser: that code runs from inside the ramdisk agent
<smoser> and why do you "erase mbr from device?"
<smoser> so qemu-img convert -O raw $IMAGEFILE /tmp
<JayF> puts it into the disk :)
<JayF> er, into ram
<smoser> means that you have enough ram to contain the entire contents of the disk?
<JayF> yeah, right now
<JayF> we want to make that stream eventually
<smoser> good thing most people have way more ram than disk space
<smoser> oh wait
<smoser> :)
<JayF> heh
<JayF> in this case? every server it's running on
<JayF> has at least as much disk as ram
<JayF> in 2/3 cases it's less disk than ram 
<JayF> (for the OnMetal case specifically)
<smoser> curtin reads tar file extracts to stdout | dd of=/dev/foo
<JayF> yeah qemu-convert doesn't output to stdout
<JayF> if it did I would've done something like that
<smoser> well, you dont have to support a silly format :)
<JayF> someone is working on vhd support, which I think that whole bucket of things will get refactored
<smoser> (and i fully realize i tell everyone they have to support qcow2)
<JayF> It is what it is man :) We're making it better, but it works now, and it works pretty reliably
<JayF> I have trouble finding too much fault with working software :P
<smoser> yeah. 
<smoser> oh.
<smoser> fyi, qemu-img convert -O raw http://foo.bar/my-image /dev/sda
<smoser> that requires seek support in the http server
<smoser> but works otherwise.
<JayF> while this is a shitty answer for purposes of open source
<JayF> when we implemented it, we found it was much faster do convert, then dd
<JayF> because we could set dd options that made imaging on our satadoms much faster
<smoser> yeah. realistically you need to just fix the image format
<smoser> why did you get an image format in qcow2?
<harlowja_> O raw http://foo.bar/my-image /dev/sda would be neat if nova hosted the image
<JayF> this was when this  was still teeth-agent not ironic-python-agent yet
<smoser> that doesn't make any sense.
<harlowja_> or ironic/nova 
<JayF> smoser: I don't know man :) that was a long time ago
<JayF> harlowja_: right now we only support images from swift temp urls
<JayF> harlowja_: so it's already a direct download
<harlowja_> cool
<smoser> so the easiest solution is to fix the producer of the images to create something that streams.
<harlowja_> except for non-swift users, haha
<JayF> Well yeah man, we made it work for our case
<JayF> and now the gaps get filled in as ironic gets more upstream support
<harlowja_> wfm
<JayF> the agent has support for probably a half dozen things that won't land until k
<harlowja_> sounds like openstack
<harlowja_> lol
<harlowja_> reminds me gotta get my plane tickets
<harlowja_> smoser u going to paris?
 * JayF will be in paris
<JayF> we should have a cloud-init design thinger one evening
<harlowja_> sure
<smoser> i will.
<smoser> we should.
<smoser> i was wanting to say we shoudl have a hack fest or somethign.
<JayF> I mean, my brain will be mush, almost certainly
<harlowja_> i'm up for either
<harlowja_> we can have pow wow
<harmw> Paris is first week of november, right?
<smoser> yeah.
<harmw> sad that's not an option though
<smoser> i go from paris (openstack) to re:invent (aws)
<harmw> cool
<smoser> harmw, :-(
<harmw> yea well, it's my first week at $newjob
<harmw> so not wise to jump in plane instead of meating $newppl :P
<harlowja_> meat them
<harlowja_> haha
<harlowja_> those meatbags
<harmw> lol
<harmw> damn
<harlowja_> :)
<harmw> thats probably the most lame typo ever
<harmw> nice, Jenkins finaly archived my 3 shiny cirros images
#cloud-init 2015-09-14
<arkin> smoser: So on line 17 it will print ($HOME) versus the variable contents
<arkin> smoser: that second script looks really good
<smoser> arkin, i was just trying to show that without a shell, enviroment variables are not interpreted.. there is *no* shell involved there to do that expansion/replacement.
<smoser> echo is '/bin/echo' as found in path.
<smoser> probably should have written that, as reading it it is confusing.
<arkin> smoser: Yeah, OK, I think I sort of understand this... bash is still kind of new to me
<arkin> so when you echo using bash, bash first replaces variables and such before sending to echo ?
<arkin> whereas in runcmd if you send an array, it simply pipes to echo but not through bash (if thats possible) ?
<smoser> right.
<smoser> well, not pipe to echo, just doesn't use a shell. uses 'exec'
<arkin> smoser: hmm ok thanks, suprised at how big cloud-init is, had not heard of it until DO but now I'm seeing AWS use it
<Odd_Bloke> arkin: It's supported by AWS, GCE, Azure and OpenStack (to name the big players).
<arkin> Odd_Bloke: Impressive
<Odd_Bloke> arkin: That also means that anywhere that provides an EC2- or OpenStack-compatible interface can normally be enabled pretty easily. :)
<arkin> Odd_Bloke: Yea its quite cool what I've done so far
#cloud-init 2015-09-15
<Odd_Bloke> smoser: We have a CloudStack partner who would like passwords to be checked and possibly reset on each boot.
<Odd_Bloke> smoser: The CloudStack data source does the password fetching etc.; will it be called on each boot?
<smatzek> Odd_Bloke: doesn't datasource search happen on every boot? I think it does and get_data is called on each.  Cloud-init needs to do the datasource search to get the data to get the instance-id before it can check that vs the id on disk an no-op.
<Odd_Bloke> True.
<Odd_Bloke> And we also do filesystem checking at each boot for Azure.
<Odd_Bloke> smatzek: Thanks. :)
<Odd_Bloke> smoser: So I think we'd need to modify the CloudStack DS to enable cc_set_passwords when it detected a new password. Does that sounds right?
<smoser> Odd_Bloke, the cloudstatck datasource is probably read each boot, but probably setting of passwords does not happen.
<smoser> but cc_set_passwords sets for all accounts.
<smoser> so, yeah, you coudl do that. i guess.
<smoser> but it'd be a seemingly likely failure path if they re-set the root password and kept their old known values for other users
<smoser> and then cloud-init re-set those users passwords.
<Odd_Bloke> smoser: Blargh, this sounds painful.
<smoser> Odd_Bloke, we want to fix this in 2.0 with "agent" like thing that can set password for the expected user in an api like way.
<Odd_Bloke> Yeah.
<openstackgerrit> Merged stackforge/cloud-init: Configure basic logging, and make it possible to log to console.  https://review.openstack.org/220536
<openstackgerrit> Merged stackforge/cloud-init: Use a single source for version information.  https://review.openstack.org/220543
#cloud-init 2015-09-16
<smoser> he all.
<smoser> hey.
<smoser> time for cloud-inti meeting.
<smoser> claudiupopa sends regrets as he's on a plane.
<jgrimm> o/
<smoser> we've now got two smatzeks so that is good.
<Odd_Bloke> o/
<smoser> Odd_Bloke, ?
<smoser> there he is
<natorious> o/
<smoser> http://bit.ly/cloudinit-reviews-public
<smoser> we'll run through there, then some open discussion
<smatzek> you can thank Windows7 ability to completely hide / lose track of running applications on sleep/awake for the dual smatzeks.
<smoser> if only there were a high quality freely available desktop operating system that could handle such things.
 * smoser ducks
<Odd_Bloke> <3
<smatzek> :)
<smoser> ah. have to read that spec some. please others take a look there too.
<smoser> for the next one .. that is Odd_Bloke 's  [WIP] TaskFlow for running shell commands
<smoser> can you summarize what you're thinking there, Odd_Bloke
<smoser> ?
<smoser> oh yeah... this was the 2.6 thing that i was supposed to think some on.
 * smoser terrible.
<Odd_Bloke> So taskflow is good at what it does.
<smoser> ok. i will take an action to look at options for non-2.6 python on older OSes.
<Odd_Bloke> I've set up a simple graph of tasks, and it will execute those based on required dependencies etc.
<Odd_Bloke> Importantly, we can save off the state of execution so that we don't have to execute everything all at once.
<smatzek> I read the spec this morning. I agree with it but didn't +1 it.  I'm not sure if we should +1 / merge it or wait for the investigation into running datasource discovery in parallel with modules.  The spec calls out two ways to handle parallel data source discovery and parallel module running each with pros / cons.
<Odd_Bloke> This gives us the 'run everything' on Windows and 'run parts as they become sensible' on non-Windows OSes.
<Odd_Bloke> It could also enable us to do some more complex dependency stuff with config modules etc.
<Odd_Bloke> So say we have a module which puts SSH keys in place which needs users to have been created, we can just declare that dependency and taskflow will ensure they are executed in order.
<smoser> Odd_Bloke, ok. so benefits are clear enough.
<smoser> and 2.6 support in taskflow is not possible ?
<smoser> i'm sorry that my memory is fuzzy
<Odd_Bloke> It's actually one of taskflow's dependencies that has been bumped.
<Odd_Bloke> networkx.
<Odd_Bloke> taskflow may not need to depend on the newer version of networkx, but it started requiring it due to a general OpenStack requirements bump.
<smoser> ok. thanks.
<smoser> lets do 'open discussion' now.
<Odd_Bloke> And, of course, that's only viable as long as there isn't a bug in networkx that we need a fix for, or a new feature that taskflow starts depending on.
<Odd_Bloke> Sure.
<Odd_Bloke> I don't have any open discussion topics.
<smoser> and anyone listenting who has not read claudiu's spec for parallel discover, please do and ahave a think and add comments
<smoser> natorious, you were looking for work last week, i think
<smoser> and quite possibly i left you hanging ?
<smatzek> natorious: you were going to start work on the ConfigDrive datasource, right?
<natorious> smoser: I was.  I've been poking around with ConfigDrive.  Looking to support network json data
<smoser> ah. fun.
<smoser> thats a key thing that we want.
<natorious> I'd imagine we'd need a spec on network config templates?
<smoser> do you want to chat some ? we can do so now or later. i have thoughts on this, and an idea of where i want to go.
<natorious> I can now, sure
<smoser> some others on my team have recently been workign code in curtin ("ubuntu fast path installer"). that takes a network config format and renders it to the system
<smatzek> natorious: if you're starting with the 0.7 configdrive datasource as a base, can you ensure the calls to 'util.mount_cb' and the find_candidate_devs method are architected in such a way that its easy for non-Linux OSes to have their own impls?  Possily have the datasource call the OS distro class to do it?
<smoser> that happens in /etc/network/interfaces (ENI) format now.
<smoser> my plan has been to mvoe that code into a library, and then use it from cloud-init.  cloud-init woudl read the network info from openstack, and then tell the library to render it to the system.
<natorious> so like in 0.7.7 I was using jinja or similar templates to translate net settings to config files to lay down.
<natorious> smatzek: for sure.  I've only been basing off the Openstack http source thats in 2.0
<natorious> so far
<natorious> smoser: so like would we be able to contribute to this code library though still?
<natorious> were doing ha bonding and would need to support that
<natorious> like 802.3 etc
<smoser> bonding for ubuntu is there now.
<smoser> but yes, code contirbution there would be appreciated
<smoser> currently its part of curtin, but he intent to pull it out
<smoser> natorious, the supported config format right now looks like
<smoser>  http://bazaar.launchpad.net/~curtin-dev/curtin/trunk/view/head:/examples/network-all.yaml
<smoser> that is what maas declares.
<smoser> interneally, it reads that, comes up with a model and renders it to the system.
<smoser> so we can also support the openstack networking definition model too
<smoser> and then there is one from amazon that comes in the MD service that we'd also have to support reading
<smoser> that all make sense ?
<natorious> it does.  Not sure how well the yaml would translate ex - https://gist.github.com/naterh/a88646595ad94033a100
<smatzek> I'd expect the library would have different impls of the interface for the various distros, so the network settings can get written to all the appropriate files on the OS such as route files, ifcfg files, hostname files, resolv.conf, etc on RedHat/SLES?
<smatzek> and run commands to set networking vs writing files on non-Linux OSes?
<natorious> that project is also currently deb based only right?  So like we'd need to come up w/ all the other os's config layouts
<smoser> smatzek, right. thats the idea.
<smoser> and then we have changes to the networking also
<smoser> and ideally it can go from A to B with no bouncing or unnecessary down time
<natorious> that gist was openstack's net json btw ^^
<smoser> natorious, you're right. other than s/"deb based"/ENI rendering/
<smoser> right.
<smoser> essentially cloud-init (either through a library or not) needs to have a model of networking
<smoser> and has to read from multiple different formats (maas, openstack, ec2) and "render" to multiple formats (ENI, sysconfig, networkd, 'ip', windows-whatever)
<natorious> so, I might handle that as a separate task and just get a base config drive source in place then?
<smoser> yeah, i think that makes sense.
<smoser> natorious, please feel free to ping me if you're looking for feedback. i do best wtih irc pings.
<smoser> the only other thing i had for this meeting would be to see what people might think of a meeting time change...
<natorious> I'd read thru claudio's parallel spec too.  Had some thoughts on capabilities too
<natorious> smoser: ty
<smoser> but since we have neither our furthest east member (claudio) nor our furthest west member (harlowja)... that seems not a useful conversation to have at the moment.
<smatzek> what timezone is claudio in?
<Odd_Bloke> CET, I think.
<Odd_Bloke> Well, CEST ATM.
<smoser> CET = +0200
<smoser> also atm
<Odd_Bloke> Oh, actually, he's probably on Eastern European Time, which is +0200.
<Odd_Bloke> (CEST is CET+1 for summer)
<Odd_Bloke> Uh, but maybe on Easter European Summer Time, which is UTC+3.
<natorious> is there a preferred template engine for this project?
<natorious> ideally something windows compatible thoughI've no clue on how windows networking works
 * natorious finds jinja2
<Odd_Bloke> Jinja2 would be my preference; don't know what dependencies etc. look like.
<smatzek> the library should probably hide the engine used. I like jinja2 as well. The 'apply network to OS' part will likely need to render files and run commands, possibly even on Linux.
<smoser> harlowja, awake ?
<smoser> or Odd_Bloke
<smoser> as my resident python experts
<smoser> someone is using subprocess.check_call with timeout=
<smoser> which ends up doing Popen.kill
<smoser> which is SIGKILL
<smoser> which is just not nice
<smoser> any ideas on how to get a different signal sent or otherwise easily implement the same functionality as subprocess.check_call(timeout=)
<Odd_Bloke> smoser: Just a heads-up, timeout= is Py3-only.
<smoser> yeah
<Odd_Bloke> smoser: So you may need to solve this problem anyway if this code needs to work on Python 2. :p
<smoser> i think  https://gist.github.com/kirpit/1306188
<smoser> this code didnt need to run py2
<smoser> other than the shlex bit, i think that is sane
<Odd_Bloke> smoser: https://hg.python.org/cpython/file/tip/Lib/subprocess.py#l665 is what is called under the hood.
<Odd_Bloke> smoser: So it doesn't look like there's any way to stop it being kill'd.
<harlowja> smoser suppp
<harlowja> process.kill() nice, ha
<harlowja> u can use https://review.openstack.org/#/c/218597/ :-P
<harlowja> or at least '_try_term_then_kill' from that
<harlowja> ha
<harlowja> my gift to u
<harlowja> lol
<harlowja> i personally like the psutil approach better
<harlowja> but up to u guys :-P
<stanguturi> Hi everyone, I am learning the 'cloud-init' stuff. Need some assistance with the initial dev environment setup. Where can I find the latest 2.0 / 2.x development code for cloud-init, github / https://launchpad.net/cloud-init
<harlowja> https://github.com/stackforge/cloud-init/ stanguturi
<harlowja> 0.7.x is also a branch there (although not exactly in sync with the bzr one)
<harlowja> smoser how goes that bzr + git turning on ;)
<smoser> harlowja, bah
<harlowja> press da button!!! :)
<harlowja> pwease
<harlowja> with sugar
<smoser> hey stanguturi
<smoser> harlowja, i need some sort of workaline for revno
<smoser> revno sucks for all sorts of reasons
<smoser> but its so nice for many others.
<harlowja> workaline?
<smoser> a single increasing number
<smoser> workalike
<harlowja> ah
<harlowja> so that should already exist in 2.x since its using pbr
<smoser> ?
<harlowja> it calculates versions from git commits
<harlowja> http://docs.openstack.org/developer/pbr/#version
<harlowja> it should already be increasing it afaik
<smoser> well that is neat.
<harlowja> and then when u make a tag, pbr should lock to that tag
<harlowja> and thats your version
<harlowja> *signed tag
<harlowja> sooo python setup.py --version should work as u would expect (increasing when on a branch, not increasing on a tag)
<harlowja> ...
<smoser> so. that sonds really neat.
<smoser> but in 0.7 what i need is to replace 'bzr' from tools/*
<harlowja> keep 0.7 on bzr, let it still work there (just have git+bzr launchpad read-only mode enabled or something?)
<harlowja> and then i get openstack-infra people to drop 0.7.x on git/stackforge or something
<harlowja> ?
<harlowja> annnd win?
<harlowja> lol
<harlowja> right now on git stackforge cloud-init head
<harlowja> $ python setup.py --version
<harlowja> 1.9.0.dev172
<smoser> i'm fine to swithc it really
<harlowja> hmmm
<harlowja> for real?
<harlowja> i wonder why pbr picked '1.9.0'
<harlowja> lol
<harlowja> $ git tag
<harlowja> 0.7.0
<harlowja> 0.7.1
<harlowja> 0.7.2
<harlowja> 0.7.3
<harlowja> 0.7.4
<harlowja> 0.7.5
<harlowja> 0.7.6
<harlowja>  (the only tags)
<harlowja> i can replace the tools/* stuff though if u are for real
<harlowja> with josh-awesome-sauce
<smoser> i think i'm ok with that.
<harlowja> k
<harlowja> pinky swear?
<smoser> yeah.
<smoser> so the stuff in tools
<smoser> and then http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/trusty/cloud-init/trusty/view/head:/debian/README.source
<smoser> (New Snapshot) there.
<smoser> i use that wheneer i upload to ubuntu
<smoser> probably you dont have to replace 'bzr' there.
<harlowja> k
<harlowja> i'll get openstack-infra to do the sync they said they would (one last final time)
<smoser> as the packaging can still be bzr and the upstream as git
<harlowja> since confirmed pinky swear happened
<smoser> wait...
<harlowja> oh
<harlowja> k
<smoser> i can't stop committing to bzr until i have those things fixed
<harlowja> pinky swear still in progres
<harlowja> oh
<smoser> then i can
<harlowja> kk
<harlowja> u addicted to bzr
<harlowja> its a drug
<harlowja> get your fix man
<harlowja> lol
<harlowja> detox...
<smoser> 6 years addicted.
<harlowja> lol
<smoser> before i was addicted to the git
<smoser> and thought the bzr was silly
<harlowja> but then u got hooked
<smoser> but the addiction is there now. even if i still think silly.
<harlowja> just say no to drugs man
<harlowja> didn't u learn anything in school
<harlowja> lol
<harlowja> ok, so what needs to get into bzr first, lol
 * harlowja serious-mode-enabled
<smoser> i guess just have it ready in git that we can test.
<harlowja> http://paste.openstack.org/show/465192/ (the handy dandy resync script)
<harlowja> bb, foods
<harlowja> smoser ok https://code.launchpad.net/~harlowja/cloud-init/cloud-init-git-tarball-make/+merge/271394 i think should be part of this
<harlowja> or maybe not, ha, let me know
<harlowja> seems to work with .git or .bzr
#cloud-init 2015-09-17
<Odd_Bloke> stanguturi: Hola! Glad to see you've made it to IRC. :)
<Odd_Bloke> stanguturi: You can find the 2.0 code at https://github.com/stackforge/cloud-init
<Odd_Bloke> stanguturi: We use Stackforge's Gerrit to manage all changes in to the tree: you can see a list of open reviews at http://bit.ly/cloudinit-reviews-public
<Odd_Bloke> stanguturi: http://docs.openstack.org/infra/manual/developers.html#installing-git-review gives you some details about how to work with gerrit and git-review (if you don't already know :).
<Odd_Bloke> stanguturi: Not everything in the OpenStack developers guide is relevant though, as cloud-init isn't an OpenStack project, we just use their CI infrastructure. :)
<abhi_> Hi, I have been trying to use cloud-init. Though seems like only a part of the userdata script is being applied to the VM. For examples, yum packages specified in userdata do not take effect but static file writing works.
<abhi_> this is the error:
<abhi_> Cloud-init v. 0.7.5 running 'init-local' at Wed, 16 Sep 2015 08:39:16 +0000. Up 10.40 seconds. Cloud-init v. 0.7.5 running 'init' at Wed, 16 Sep 2015 08:39:23 +0000. Up 17.72 seconds. ci-info: ++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++ ci-info: +--------+------+-------------+---------------+-------------------+ ci-info: | Device |  Up  |   Address   |      Mask     |     Hw-Address    | ci-info: +--------+-
<abhi_> 2015-09-16 04:39:27,398 - util.py[WARNING]: Failed pickling datasource DataSourceConfigDriveNet [net,ver=2][source=/dev/sr1]
<abhi_> 2015-09-16 04:39:27,545 - cc_write_files.py[WARNING]: Undecodable permissions None, assuming 420
<abhi_> Cloud-init v. 0.7.5 running 'modules:config' at Wed, 16 Sep 2015 08:39:28 +0000. Up 22.82 seconds.
<abhi_> Cloud-init v. 0.7.5 running 'modules:final' at Wed, 16 Sep 2015 08:39:29 +0000. Up 23.51 seconds.
<abhi_> Cloud-init v. 0.7.5 finished at Wed, 16 Sep 2015 08:39:29 +0000. Datasource DataSourceNone.  Up 23.70 seconds
<abhi_> 2015-09-16 04:39:29,526 - cc_final_message.py[WARNING]: Used fallback datasource
<mgagne> I tried to build cloud-init from master with ./packages/brpm and I'm getting error this error: AttributeError: 'module' object has no attribute 'abs_join'
<mgagne> is this script supposed to work or was it not used since refactor?
#cloud-init 2015-09-18
<harlowja> hmmm, supposed to work
<harlowja> but maybe i broke it, ha
<harlowja> abhi_ what is in /etc/cloud/cloud.cfg ?
<harlowja> 'Undecodable permissions None, assuming 420' seems bad
<harlowja> whats the userdata u are sending (no private info in it thx)
<abhi_> User data is:
<abhi_> #cloud-config repo_update: true repo_upgrade: all  packages:  - httpd  - crypto-utils  - mod_perl  - mod_ssl  - mod_wsgi  - mysql-server  - php  - php-gd  - php-pdo  - php-pear  - php-xml  - php-mysql  - git  ssh_authorized_keys:  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+a7nTOBzHq/dcK3j8q6HCcyJDO7g7lv4QSdPGrtIH/M+u3fEqbo4f5JCIouP2yxeCCJolvEXOo0iMvKcfv0z2yiF5c1Xe8ilDwRbbIDAVEcQ1hlIm+LC3ScFTzd0EG/150FCIHAHna5UZfi36ZDm2ofdz1RcL9yMOu+
<abhi_> It is basically bunc onf yum packages, ssh_authorized_keys, write_files. Note that the same user data works when I create the VM and attach the disk. Though it works partiall when I attache the disk to a cloned VM
<abhi_> contents of /etc/cloud/cloud.cfg in cloned VM is same as the created VM on which the userdata works perfectly including yum packages installation.
<abhi_> content of /etc/cloud/cloud.cfg:
<abhi_> users:  - default  disable_root: 1 ssh_pwauth:   0  locale_configfile: /etc/sysconfig/i18n mount_default_fields: [~, ~, 'auto', 'defaults,nofail', '0', '2'] resize_rootfs_tmp: /dev ssh_deletekeys:   0 ssh_genkeytypes:  ~ syslog_fix_perms: ~  cloud_init_modules:  - migrator  - bootcmd  - write-files  - growpart  - resizefs  - set_hostname  - update_hostname  - update_etc_hosts  - rsyslog  - users-groups  - ssh  cloud_config_modules:
<abhi_> - yum-add-repo  - package-update-upgrade-install  - timezone  - puppet  - chef  - salt-minion  - mcollective  - disable-ec2-metadata  - runcmd  cloud_final_modules:  - rightscale_userdata  - scripts-per-once  - scripts-per-boot  - scripts-per-instance  - scripts-user  - ssh-authkey-fingerprints  - keys-to-console  - phone-home  - final-message  system_info:   default_user:     name: centos     lock_passwd: true     gecos: Cloud Use
<abhi_> sudo: ["ALL=(ALL) NOPASSWD:ALL"]     shell: /bin/bash   distro: rhel   paths:     cloud_dir: /var/lib/cloud     templates_dir: /etc/cloud/templates   ssh_svcname: sshd  # vim:syntax=yaml
<harlowja> can u put that in a pastebin somewhere
<harlowja> my guess is some syntax is off somehow
<abhi_> sure, http://pastebin.com/zJDsFaNC
<abhi_> thanks
<abhi_> just want to point out that this cfg works when I create the VM and attach configdrivev2 iso, only during clone case certain things of cuser data like yum, git clone does not work. Userdata: http://pastebin.com/gn0XiVbx
<harlowja> hmmm, guess can't help abhi diagnosis the rest of that issue, lol
#cloud-init 2015-09-20
<natorious> Just pushed this.  Still working on the actual configdrive module but, its a start...  https://review.openstack.org/#/c/225465/
#cloud-init 2016-09-20
<jwilx> http://termbin.com/16tk
<jwilx> why am I getting this error?
<jwilx> is ANYONE around?
<jwilx> ailed to start Initial cloud-init job (pre-networking).
<jwilx> http://termbin.com/8uxe
<jwilx> this is on arch
<jwilx> .
<jwilx> anyone?
<smoser> jwilx, well, you're probalby missing python3 package for pkg_resources
<jwilx> smoser: very helpful... I already corrected that error, only to get this one http://pastebin.com/raw/nS6KXz0w
<jwilx> http://termbin.com/8uxe
<jwilx> it was working fine on arch
<jwilx> I did a clean install, and did nothing but pacman -Syu
<jwilx> and it screwed up so I had to install a bunch of python packages
<jwilx> I think it's cloud-init bugging out
<jwilx> somebody should fix it
<jwilx> come on surely somebody can help...
<smoser> jwilx, are you on the system ?
<smoser> could you try:
<jwilx> i'm on it
<jwilx> yes
<smoser>  python3 -c 'from cloudinit import util; import sys; print(util.load_file(sys.argv[1]))' /sys/class/net/eth0/carrier
<smoser> i think harlowja was seeing that also
<jwilx> it's using python3
<jwilx> 2 *
<jwilx> not 3
<jwilx> [arch@arch ~]$ sudo python2 -c 'from cloudinit import util; import sys; print(util.load_file(sys.argv[1]))' /sys/class/net/eth0/carrier
<jwilx> 1
<smoser> hm.
<jwilx> fyi i'm using systemd networking here
<smoser> that should not cause this error, but cloud-init does not have support at this time for rendering systemd-networkd networking configuration.
<smoser> can you run
<smoser> sudo python2 -c 'from cloudinit.net import generate_fallback_config; print(generate_fallback_config())'
<jwilx> {'version': 1, 'config': [{'subnets': [{'type': 'dhcp'}], 'type': 'physical', 'name': 'eth0', 'mac_address': u'fa:16:3e:37:ae:97'}]}
<smoser> well that is odd.
<smoser> thats where it stacktraced.
<jwilx> hm
<magicalChicken> smoser: I remember this from when I was trying to get fallback_net_config working I think
<magicalChicken> Depending on the state that the net interface is in, reading from the sys info is an IOError with errno 22
<magicalChicken> But later if that net dev is in a clean state it can read
<magicalChicken> I think the issue is in python2 IOError does not inherit from OSError
<smoser> magicalChicken, yeah, i just checked that it doesnt in py3 either.
<magicalChicken> but the catch works in py3
<smoser> i suspect it raises IOError and we're just never catching it
<magicalChicken> yeah
<smoser> but even if we did catch it, i'm not sure what to do
<magicalChicken> the code handles that already
<smoser> i dont think its py3 v py2
<magicalChicken> hang on 1 sec
<magicalChicken> http://paste.ubuntu.com/23207506/
<magicalChicken> also: http://paste.ubuntu.com/23207515/
<smoser> yeah, you're right
<smoser> $ python -c 'print(isinstance(IOError(), OSError))'
<smoser> False
<smoser> $ python3 -c 'print(isinstance(IOError(), OSError))'
<smoser> True
<magicalChicken> I just remember that cause it caused curtin but #1562003
<magicalChicken> which started the clear_holders refactor
<smoser> jwilx, are you able to try this:
<smoser>  http://paste.ubuntu.com/23207571/
<smoser> bug 1562003
<jwilx> does the minus mean to remove?
<magicalChicken> jwilx: you can just use patch with that
<magicalChicken> so clone cloud-init, paste that into a file and do 'patch -p0 < $filename'
<smoser> jwilx, yes. '-' means remove
<smoser> and + add
<smoser> or probably something like:
<smoser>  ( cd /usr/lib/python2.7/site-packages/ && patch --dry-run -p0 ) < that-content
<jwilx> http://termbin.com/zkqf
<jwilx> I get this
<smoser> shoot.s orry, jwilx
<smoser> jwilx, kill the final ')' off that line i think is all you need.
<jwilx> msg = "%s: load file failed with %s" % (fname, e)
<jwilx> this one?
<jwilx> i did and it is giving same error
<jwilx> as above
<jwilx> http://termbin/zoeb this smoser
<jwilx> http://termbin.com/zoeb
<smoser> hm.. jwilx pastebin /var/lib/cloud-init.log ?
<jwilx> nothing there
<jwilx> smoser:  http://termbin.com/93eq
<jwilx> that's output log
<smoser> jwilx, someone was here working on arch support recently...
<smoser> prometheanfire, ^
<smoser> maybe summoning him will make magic occur
<jwilx> hm
<jwilx> prometheanfire:  any idea?
<ThiagoCMC> Hey guys!
<ThiagoCMC> Listen, I launching Ubuntu 16.04 on OpenStack, wired to 3 different subnets: subnet 1 is the default, have dhcp=True, subnet 2 is secondary (no dhcp), subnet 3 is the third (no dhcp). However  if I disable the DHCP for a specific subnet (like I just said), Cloud Init still configured the IP as static!!! But I do not want any setup for the secondarie networks. As a workaroung, I'm doing this on cloud init script: "ifdown ens4 ; ifdown ens5" (while en
<ThiagoCMC> s3 is fine, first one).
<ThiagoCMC> so, how to avoid this workaround and tell cloud-init to configure only the 1 subnet (ens3 vNIC)?
<prometheanfire> smoser: I was working on gentoo
<prometheanfire> I need to get back on working with it, but with the openstack release coming up...
<smoser> prometheanfire, aren't those the same thing
 * smoser ducks
<smoser> sorry for the bad ping.
<prometheanfire> :P
<prometheanfire> all the ricers left gentoo for arch ages ago
<prometheanfire> gentoo is much better off now :P
<jwilx> smoser: so any idea?
<jwilx> :/
<smoser> jwilx, well, the general 'None' bug is probably just not considering no dns information.
<smoser> thats probalby fairly easily fixable.
<smoser> can you file a bug against cloud-init ?
<jwilx> no idea how
<smoser> and state where we've been and where you are now ?
<smoser> https://bugs.launchpad.net/cloud-init/+filebug
<smoser> jsut recap what is here. you can even reference here
<smoser>  https://irclogs.ubuntu.com/2016/09/20/%23cloud-init.html
<jwilx> k
<jwilx> https://bugs.launchpad.net/cloud-init/+bug/1625766
<harlowja> smoser jwilx i've def seen the invalid argument one :-P
<harlowja> weird that it is
<smoser> harlowja, magicalChicken suggests that that happens before nic is really initialized or brought up. and that we're just missing a exception
<harlowja> @smoser talking with nrezinorn and we were thinking about how we could get image building for cent7 into some public image publicing/testing pipeline
<harlowja> cause there be some weirdness with networking starting with DataSourceNoCloud
<harlowja> that may just be some kind of cent7 issue, but is still weird (same with the argparse crap)
<smoser> harlowja, i have to run ... lke right now.
<smoser> but i have been thinking about cloud-init integration test
<smoser> i'll try to put something together on my thoughts and send to list tomorrow.
<smoser> we'lls ee.
<smoser> later.
<harlowja> whattt
<harlowja> lol
<harlowja> a chicken?
<harlowja> i caught the IOError
<harlowja> is this for the /sys/ question
<harlowja> ha
<harlowja> where u running
<harlowja> come back
<harlowja> lol
<harlowja> k
<cwa> any suggestions for setting a http proxy with cloud-init?  I've been trying to do so by exporting the http_proxy env variable via a runcmd:  but it doesn't seem to take (i run a yum update several commands later and it fails because the proxy isn't set)
<smoser> cwa, well, you could probabl do a bootcmd to write /etc/environment . cloud-init does not have any way to affect its environment.
<jwilx> smoser know how long it usually takes?
<jwilx> to get a response
<smoser> here is something i do: http://bazaar.launchpad.net/~smoser/+junk/sstack-proxy/view/head:/sstack-proxy
<smoser> jwilx, to get a response to the bug ?
<jwilx> yeah
<cwa> thanks all, i'll take a look at that
<smoser> well a while.
<smoser> cwa, generally speaking http_proxy globally is a pita
<jwilx> meh guess I can try installing gentoo
<jwilx> while waiting
<smoser> as http_proxy=http://your.proxy/ wget http://localhost:9999/
<smoser> and have that fail
<jwilx> really wanted to use arch but w/e
<smoser> jwilx, what is the provider you ran on ?
<jwilx> lunanode
<smoser> linode ??
<jwilx> luna
<jwilx> https://www.lunanode.com/
<smoser> ah. k
<jwilx> smoser why do you need to know
<smoser> jwilx, did you just pick from the template ?
<jwilx> smoser
<jwilx> yes.
<jwilx> it worked fine. until I did an update
<smoser> ok. i'll maybe take a look tomorrow.
<smoser> just signed up there, gave them $10
<jwilx> ah
<ThiagoCMC> Hey guys, can someone help me?
<harlowja> do u require emergency assistance, if so please dial 911
<x58> Are there any active users of cloud-init on FreeBSD? Anyone know if there is active development for FreeBSD?
<harlowja> harmw might know that x58
<harlowja> i think he did a bunch of the cloud-init stuff for freebsd
<harlowja> though idk if it actually works :-P
<x58> Yeah, I have been talking with him.
<x58> He is no longer doing OpenStack
<x58> So he is no longer able or interested in maintaining cloud-init on FreeBSD.
<x58> harlowja: ^
<harlowja> ok
<harlowja> u can be the replacement :-P
<x58> Yeah, he suggested that too ;-) I was updating the FreeBSD port to 0.7.8 but it's broken due to linuxisms that are not an easy fix.
<harlowja> ??
<x58> https://bugs.launchpad.net/cloud-init/+bug/1625802
<x58> it tries to read from /sys/class/net which doesn't exist on FreeBSD.
<harlowja> ya, that
<harlowja> poop
<x58> I just filed that bug.
<harlowja> k
<x58> There has been a patch for another issue: https://bugs.launchpad.net/cloud-init/+bug/1404745
<harlowja> ya, perhaps we should bundle this into https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+ref/sys-io-errors
<harlowja> which was also simialr
<x58> that has been languishing in review... and tools/freebsd-build* hasn't been updated with appropriate depdendencies either.
<harlowja> ya, never enough people
<harlowja> nrezinorn and i have been working through some cent7 issues
<harlowja> some weirdness there
<x58> I had a patch for CentOS 7 issues, but don't want to go through hassle of signing CLA...
<x58> So I've got a fork running locally.
<x58> (I'm Bert on launchpad)
<harlowja> hassle
<harlowja> hmmm
<harlowja> idk how much of a hassle it is
<harlowja> hard to accept patches afaik without that :-/
<x58> I'd have to get corp legal involved...
<nrezinorn> im pretty hassled harlowja  :P
<x58> Only because of Canonical's policy of not accepting stuff under the GPLv3...
<harlowja> nrezinorn u should be able to contribute to cloud-init without problem
<x58> and wanting what is basically the right to relicense any and all contributions under whatever license.
<harlowja> no hassle for u
<x58> smoser and I had a discussion on the matter already.
<harlowja> k
<harlowja> ya, IANAL
<harlowja> to me the whole license blah blah, is more or/less blah blah
<x58> Hire me, then it won't be a problem ;-)
<harlowja> where are your cent7 patches? ;0
<harlowja> :)
<x58> harlowja: https://code.launchpad.net/~bregeer-ctl/cloud-init/+git/cloud-init/+merge/305058
<harlowja> kk
<harlowja> ya, nrezinorn was seeing something else
<nrezinorn> my "something else" is related to starting a VM locally in libvirt and feeding it a cdrom config drive with some data :P  - pre-cloud QA tests if you will
<harlowja> precloud cloud
<x58> nrezinorn: I've been testing by uploading to glance and starting the image ;-)
<x58> Thankfully my cloud is local, and 1 Gbit/sec uploads into glance, so it doesn't take a long time to do testing.
<harlowja> ya, what nrezinorn say in 0.7.7 is that networking seems to not start initially (when using DataSourceNoCloud) but after reboot it does
<harlowja> which then made me wonder about some thing not recgonizing networking was changed (until after reboot)
 * harlowja not sure if /etc/udev/rules.d/70-persistent-net.rules does that
<harlowja> or could cause that
<harlowja> or something else
<x58> Ah, that was another issue... I had to set cloud-init-local to start Before=NetworkManager.service
<harlowja> all the systemd dependencies (cloud-init-local before networking) seem ok
<x58> https://bugs.launchpad.net/cloud-init/+bug/1620845
<harlowja> well damn
<harlowja> that would probably be it
<harlowja> lol
<x58> cloud-init-local and NetworkManager would start together... and race each other.
<x58> It was glorious (not, I wanted to shoot someone)
<harlowja> ya, sounds about right
<harlowja> nrezinorn may be about to shoot someone
<harlowja> not really sure
<harlowja> lol
<x58> My issue was related to /etc/resolv.conf but it also causes issues with ifcg-* files
<x58> and whatnot
<harlowja> i thought cloud-init-local had before=network-pre
<harlowja> guess that doesn't do it
<harlowja> https://git.launchpad.net/cloud-init/tree/systemd/cloud-init-local.service#n8
<harlowja> assuming thats not enough
<x58> Nope
<x58> not enough.
<harlowja> stupid
<harlowja> lol
<x58> well, NetworkManager doens't have any require=network-pre in it...
<harlowja> gotcha
<x58> so NetworkManager will start even though network-pre is not yet "complete"
<x58> It's a dependency ordering issue.
<harlowja> def
<x58> Changing system shipped service files is not my thing though, so I forced cloud-init-local first by making it Before
<nrezinorn> lets just blame systemd?  thats the cool thing to do right?
<harlowja> ya+1
<x58> lol
<harlowja> torches ready
<harlowja> getting my pitch fork
<x58> Pitchforks for sale here!
<nrezinorn> ill locally make the changes for netmanager tho and see if that helps!
<x58> nrezinorn: no promises ;-)
<nrezinorn> "set cloud-init-local to start Before=NetworkManager.service"   <-- this
<nrezinorn> lol
<harlowja> https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/306283
<harlowja> ya, that should do it ^
<nrezinorn> i was handed down making golden images for the cloud.  i didnt want to push junk into the cloud if it was totally broken, so i import it into libvirt locally and QA a bunch of crap then it ships to cloud (staging)
<x58> Yeah, I am currently trying to build an golden image for FreeBSD 10.3 but cloud-init is broken.
<x58> Now I need to go see how much time I can spend on this, and patch it just enough to work.
<nrezinorn> i have been mucking with this centos7 jazz for ...3 days?  4 days?  i dont even know.
<x58> My local git server now has a fork of cloud-init with a bunch of branches fixing random stuff.
<nrezinorn> i hope that this also fixes the hostname issue i am seeing lol
<x58> nrezinorn: You'll want to look at the patches for /etc/resolv.conf as well (make it a symlink to something else so that NetworkManager leaves it alone)
<x58> Otherwise NetworkManager will happily overwrite your changes to /etc/resolv.conf
<harlowja> x58 can u please sign the CLA
<harlowja> with sugar on top
<harlowja> don't like out of tree fixes that someone has to go discover and recreate
<x58> harlowja: Patches are under GPLv3. Feel free to pull em in. At least then I can be assured my code won't suddenly go closed source (happened with maas-image-builder... for which I had not signed a CLA!)
<harlowja> ya, this is a weird gray area that idk about
<harlowja> like if i just submit your patches under my name
<harlowja> lol
<x58> Sorry, gonna be a stickler about it. I'll reach out to my boss to see if I can sign a CLA for stuff I did on work time.
<x58> harlowja: You'd be infringing my copyright... you can't agree to have canonical relicense something you didn't write.
<x58> Just saying.
<harlowja> but u just said ' Feel free to pull em in.'
<harlowja> lol
<x58> Yeah, GPLv3 allows you do that.
<x58> Just means you can't claim you wrote it :P
<harlowja> gotcha
<x58> and give someone permission to relicense.
<x58> That's what the CLA requires.
<harlowja> but canonical will if i submit them
<harlowja> so if i take them, make new pull request with them
<harlowja> and don't say that i wrote them
<harlowja> someone else will have permission to relicense right
<x58> If someone were to merge those into the tree, and Canonical ata later point in time were to relicense the entire codebase under an alternate license, that would be copyright infringement.
<harlowja> right
<x58> Standard copyright still applies even with GPLv3
<x58> That being said. Let's just say I am Sergeant Schulz. I didn't see nothing. I didn't hear nothing. I don't know nothing.
<harlowja> so pulling them in i guess u are meaning, not into canonical cloud-init code-base
<x58> harlowja: You can pull them in, that would just preclude Canonical from ever relicensing the code base without my explicit approval first.
<harlowja> ya, to much blah blah
<harlowja> lol
<x58> haha
<harlowja> can i sell u some kool-aid
<harlowja> leol
<harlowja> *lol
<x58> harlowja: work with smoser and get cloud-init changed so that it doesn't require the CLA ;-)
<harlowja> ya, i'm not at canonical though :-P
<harlowja> only smoser holds those keys
<x58> Ah, thought maybe you were.
<x58> With the @ and all ;-)
<harlowja> nah, i'm just super-cool
<harlowja> lol
<harlowja> and been doing cloud-init for to long
<harlowja> lol
<nrezinorn> super cool?  that's debatable lol
<harlowja> :)
<harlowja> super-awesome-cool
<harlowja> lol
<nrezinorn> lol
<nrezinorn> x58: your patch didnt fix the issue i was seeing, but maybe you know what i can look through
<nrezinorn> our gold images specifically delete the ifcfg-eth0 file
<nrezinorn> running the 0.7.7 i have to reboot after creating the vm with cloud-init   and then my networking works.
<harlowja> x58 ^ any ideas?
<harlowja> nrezinorn afaik the delete happens way before cloud-init (or even the operating system) starts
<harlowja> in kickstart land
<harlowja> so from the image/operating system its like it never existed
<x58> nrezinorn: does cloud-init mention it creating ifcfg-eth0?
<x58> It should show some debug output for that.
<x58> Can you pastebin your cloud-init output somewhere?
<x58> journalctl -a | grep cloud-init
<x58> Ohhhhh
<x58> cloud-init-local.service, does it have Before=networking.target or something similar?
<harlowja> pretty sure nrezinorn added in the before=NetworkManager and think it has that also, will let nrezinorn respond
<x58> Hmmm, that's cloud-init.service
<x58> there is no "networking.service" in CentOS
<x58> it's "network.service"
<x58> so cloud-init.service depends on the wrong thing,
<harlowja> lol
<x58> Looking at my notes for building this CentOS image.
<harlowja> kk
<harlowja> man this systemd stuff
 * harlowja gets my pitchfork 
<harlowja> lol
<x58> networking.service is a Ubuntuism.
<x58> /etc/init.d/network is CentOS's
<x58> also make sure that network is enabled, especially if NM_MANAGED is set to NO in /etc/ifcfg-eth0
<x58> systemctl enable network
<x58> (It should be by default)
<x58> otherwise a reboot wouldn't fix it.
<x58> You have an ordering issue somewhere... cloud-init-local is not running early enough (before network.service)
<harlowja> so nrezinorn when u get back ^
<x58> nrezinorn: http://pastebin.ubuntu.com/23209143/ is a successful run.
<x58> https://gist.github.com/bertjwregeer/61fede0609fff681fb786bd181a19c8b
<x58> there are my .service files.
<x58> This one is a better journalctl output: http://pastebin.ubuntu.com/23209158/
<x58> includes all the other processes that are started interspersed.
<x58> nrezinorn: here's the systemd critical chain: http://paste.ubuntu.com/23209165/
<x58> (ignore cleanup_defgw.service, it's a work-around for an issue with our private cloud, one I am hoping I can remove in the next month)
<x58> nrezinorn: If NoCloud datasource works like the ConfigDrive datasource, then as long as cloud-init-local runs before NetworkManager and network.service any changes to ifcg-eth0 should be picked up on the first run.
<x58> hostname should be getting changed too using the hostname service, you should see something about dbus activating the org.freedesktop.org.hostname1 unit
<nrezinorn> x58: yes in the intial boot cloud-init makes a new ifcfg-eth0 file
<nrezinorn> but the issue is it requires a reboot for networking to functon - it works fine as-is on 0.7.5  so im trying to track what specifically may be causing the issue
<x58> nrezinorn: I understand that, check the systemd critical-chain and make sure that network.service is running after cloud-init-local.service
<nrezinorn> centos uses systemd too, fwiw, sonds like maybe the names of things need to get fixed up in the systemd service files - sorry i sorta stepped away from this 'fun" lol
<x58> Look at the pastebin I posted above, it should look similar.
<x58> I have only been talking about CentOS...
<x58> nrezinorn: systemd-analyze critical-chain network.service
<x58> does that show cloud-init-local.service as coming before it?
<nrezinorn> yeah, so when the file is NOT there (ifcfg-eth0)  the "network info+++"  stuff says eth0 False  until its rebooted
<nrezinorn> 1s let me get back to teh vm!
<x58> network info +++ is printed by cloud-init
<x58> that's after networking is completed
<x58> that's too late for anything to still be done.
<nrezinorn> yesh im just saying that when it doesnt work, it lists as false for eth0 :)
<nrezinorn> ill pastebin the stuff in a sec
<x58> For what it's worth, I am running 0.7.7 on my CentOS images.
<nrezinorn> systemd-analyze critical-chain network.service  <-- does not show anything cloud-init related in the chain
<x58> That's bad.
<x58> That means network.service is allowed to start before cloud-init-local.service has done it's thing.
<harlowja> man who named these things all differently
<harlowja> lol
<harlowja> wtf people
<nrezinorn> lol
<harlowja> i thought systemd was suppoed to make everything common
<harlowja> but people name the freaking files differently
<harlowja> lol
<nrezinorn> you've seen the distro rosetta stone right harlowja ?  ;P
<x58> nrezinorn: Make sure you have a Before=NetworkManager.service in cloud-init-local.service
<harlowja> is that before or after the crack?
<x58> network.service requires NetworkManager.service
<x58> nrezinorn: IF you modify the file, make sure to call systemctl reload-daemon or whatever that command is.
<nrezinorn> i did put that in there
<x58> Did you reload the daemon files?
<nrezinorn> i did hte netmanager fix up in my golden image and when i boot it -its still misbehaving
<x58> nrezinorn: What's the critical chain for NetworkManager say?
<nrezinorn> i used guestmount for testing stuff :)
<nrezinorn> 1s
<x58> does the critical chain for network.service show NetworkManager.service?
<nrezinorn> for networkmanager crit - it lists zero cloud-init related services lol
 * nrezinorn blames ubuntu!
<harlowja> no no systemd
<x58> nrezinorn: grab my .service file here: https://gist.github.com/bertjwregeer/61fede0609fff681fb786bd181a19c8b
<x58> What version of CentOS?
<nrezinorn> modify both service files?  theres 2
<nrezinorn> centos 7.2
<x58> Yes.
<nrezinorn> its basically the latest centos 7 (from 9/14 lol)  because i dont wnat to wait 30 min for a new image every time
<x58> cloud-init.service should be after network.service and requires=network.service
<x58> instead of networking.service
<nrezinorn> ok 1s let me see how its ordered in my image
<x58> Mine is the same. Was wondering if maybe we had different patch levels :/
<nrezinorn> gotta shut it down to modify
<x58> Modify it, systemctl daemon-reload and blow away /var/lib/cloud
 * nrezinorn activates his libvirt powers lol
<x58> :P
<x58> reboot
<nrezinorn> lol k
<nrezinorn> that seems easier to do
<nrezinorn> haha
<x58> It'll decrease your testing cycle ;-)
<x58> BRB
<nrezinorn> LOL
<x58> Did you figure out your issue?
<nrezinorn> :D
<nrezinorn> the probelm seems to be that the cloud-init.service wants networking.service when its called network.service on centos 7 ;)
<x58> Yes, I even linked to a bug report for that ;-)
<nrezinorn> haha :)
<x58> Wait, sorry
<nrezinorn> i nee dto remember the git comand that says when a file was modified ;)
<x58> I didn't link to it, I mentioned it above^^
<x58> Working now?
<nrezinorn> one "bug" resolved lol
<nrezinorn> my route command errors
<nrezinorn> your output shows it working
<nrezinorn> idk whats up with that
<x58> What route command?
<nrezinorn> ++++++++++++++Route IPv4 info++++++++++++++
<nrezinorn> that block it displayed
<nrezinorn> on mine is breaks
<nrezinorn> and just continues on
<x58> check cloud.cfg and make sure it is correctly set to distro: rhel
<nrezinorn> it is set poeprly
<nrezinorn> to rhel
<x58> hmm, I remember having that issue too :/
<nrezinorn> its next on the bug list LOL
<nrezinorn> along with the hostname not showing proprly until a reboot
<x58> hostname should show properly immediately.
<x58> Mine is :P
<nrezinorn> mine is not haha
<nrezinorn> :D
<nrezinorn> anywhos i gotta go for now
<nrezinorn> perhaps ill be back later, or tomorrow lol
<nrezinorn> thanks for the hot tips tho!
<x58> I did file a bug for this: https://bugs.launchpad.net/cloud-init/+bug/1621671
<x58> the network.service vs networking.service ;_)
<nrezinorn> +1
<harlowja> evil rhel
<nrezinorn> lol
<x58> nrezinorn: I should just link to this: https://bugs.launchpad.net/cloud-init/+bugs?search=Search&field.bug_reporter=bregeer-ctl
<x58> :P
<x58> Alright, time for me to jet. I'm hungry and tired. Enough fighting with cloud-init for today.
<harlowja> by fighting u mean tender loving right
<x58> sure... whatever you want to call it ;-)
<harlowja> lol
<sather> how do I have cloud-init userdata scripts output to console?
<sather> I get an input/output error when I redirect the entire script to /dev/ttyS0
#cloud-init 2016-09-21
<smoser> jwilx, how did you see this failure ?
<smoser> i laucnhed a vm, and it has 0.7.6 , which is not new enough to see the errors you were seeing.
<jwilx> smoser: I had to update it
<jwilx> pacman -Syu
<jwilx> after I did that
<jwilx> it broke
<jwilx> when i rebooted
<smoser> ah. ok.
<jwilx> smoser: did you try it?
<smoser> no. not tired updating. that makes sense though.
<smoser> jwilx, do you know.. is there openstack api access to luna ?
<smoser> or is it just their api, which i think is different.
<jwilx> idk
<jwilx> you can make a ticket and ask
<harlowja> smoser so what was the thoughts on better 'in-image' testing
<harlowja> @nrezinorn ^
<smoser> in-image ?
<smoser> jwilx, i did open a ticket.
<smoser> "We use openstack as our backend, but the api is not exposed to public."
<smoser> which i responded to with something like "kind of pointless to advertise 'openstack' to someone if you're not exposing the API". i can't think of anything else that would make someone care if you were running openstack
<harlowja> smoser like say, building an image with a cloud-init release, then starting it, then, putting it through a few validations, then saying good/bad
<smoser> harlowja, so id' really like to not build an image.
<smoser> but be able to start from an image
<smoser> building is possible, but .. .just a lot less crap if i can pull one.
<harlowja> kk
<harlowja> fair
<harlowja> nrezinorn yt
<smoser> so i have a short term hack plan...
<smoser> to get lxc working
<smoser> generally to allow us to use lxc to pull images, then run arbitrary commands in those images (such as apt-get install) and then to snapshot that and run some tests on it.
<smoser> then, a larger long term goal to generalize such that lxc is just a "backend"
<smoser> and able to plug in other backends
<harlowja> ok, i wonder if nrezinorn and i can pull some strings to get a testing cloud account on godaddy cloud
<harlowja> nrezinorn has been at godaddy longer, might know some folks
<harlowja> then we could just use some real images
<harlowja> *godaddy public cloud
<smoser> harlowja, there are centos images too publicly
<smoser> and we could make a backend that tested them with nocloud
<smoser> but, yeah, for an openstack coud test, we'd need an openstack. :)
<harlowja> ya, nrezinorn is eating or something
<harlowja> so maybe he has strings that i don't have
<smoser> harlowja, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/306391
<smoser> does that look sane to you
<harlowja> looks sane to me
<harlowja> i've been using plumbum recently
<harlowja> its sorta neat
<harlowja> it sorta abstracts subprocess
<harlowja> and makes it work on remote machines also
<harlowja> https://github.com/tomerfiliba/plumbum/#plumbum-shell-combinators
<nrezinorn> harlowja: i dont have magic powers to get hookups on accounts lol
<harlowja> wtf
<harlowja> nrezinorn ma
<harlowja> *man
<harlowja> lol
<harlowja> have u completed the magic power quest
<harlowja> i guess not
<nrezinorn> also the public cloud doesnt expose the openstack apis...  :P
<harlowja> i can work through that
<harlowja> it doesn't, but it exposes 'close-enough'
<nrezinorn> lol
<nrezinorn> what do we need to ask for?  "can we get free vms"  lol
<harlowja> a free account for cloud-init CI
<harlowja> with limited quota
<nrezinorn> ok ill ask this week
<smoser> chain = ls["-a"] | grep["-v", "\\.py"] | wc["-l"]
<smoser> how is that a thing ?
<harlowja> nrezinorn see u do know people
<harlowja> :)
<smoser> is | an operator ?
<harlowja> yup
<harlowja> >>> 1 | 2
<harlowja> 3
<harlowja> bit-wise or
<harlowja> https://github.com/tomerfiliba/plumbum/#remote-commands-over-ssh is the neat part imho
<harlowja> makes the same stuff work on remote machines
<harlowja> which i'm using to build out https://github.com/harlowja/multi-devstack smoser
<harlowja> (yes yes i shoudl just use juju)
<harlowja> i know u about to say that
<harlowja> lol
<smoser> harlowja, or you couild use anvil
<smoser> i think i read about that somewhere
<harlowja> nah
<harlowja> there was only one maintainer of it
<harlowja> and he got tired of being the only one
<harlowja> lol
<harlowja> (swimming against a stream...)
<harlowja> (got boring pretty quick)
<harlowja> well not pretty quick, since i guess i started it 3 years ago
<harlowja> but quick enough
<harlowja> lol
<harlowja> quick like a turtle
<smoser> ah..
<smoser> makes sense.
<smoser> but multi-devstack, that will be different!
<harlowja> lol
<harlowja> ya, its overdue
<harlowja> i need to figure out how to create a overlay network though
<harlowja> because how this works is that it spins up VMs in an existing openstack cloud
<harlowja> then starts targeting each VM as a specific 'kind'
<harlowja> and gets devstack cloned on each VM
<harlowja> then tells devstack to make this node a hypervisor, or make this node a DB
<harlowja> and out pops a mini-openstack
<harlowja> which is running in your other openstack
<harlowja> so u could theortically boot VMs inside this mini-openstack
<harlowja> (but networking is gonna be tricky)
<smoser> harlowja, can you tell me why 'tox -e py3' fails, but
<smoser>  tox-venv py3 nosetests tests/unittests/test_util.py
<smoser> is fine
<smoser> ie, just runnin the one file is good. but the whole thing bad.
<harlowja> its prob u
<harlowja> lol
<harlowja> j/k
<harlowja> seems weird :-P
<harlowja> which `nosetests` is the last one running
<harlowja> is it really running inside the venv?
<smoser>  tox -e py3 tests/unittests/test_util.py
<smoser> that fails too.
<smoser> but tox -e py3
<smoser> er... wait
<smoser>  tox -e py3 tests/unittests/test_util.py
<smoser> works
<smoser>  tox -e py3
<smoser> shows failure
<harlowja> weird
<smoser> http://paste.ubuntu.com/23212980/
<smoser> something is mucking with os.environ
<harlowja> wasn't me
<smoser> cc_growpart module is messing it up
<harlowja> who wrote that crap
<harlowja> lol
<harlowja> hahahaa
<smoser> harlowja, is that mock.patch going to be sufficiently old to run on cent ?
<smoser> or will you not care because unit tests would run in tox ?
<harlowja> ya, unittests in tox
<harlowja> everything is old on cent
<harlowja> its the nature of the game
<harlowja> old == more stable remember
<harlowja> lol
<smoser> harlowja, https://code.launchpad.net/~harlowja/cloud-init/+git/cloud-init/+merge/306283
<smoser> that seems sane..
<smoser> you've tested that it has some affect ?
<harlowja> smoser cancel that one for now
<harlowja> nrezinorn is coming up with those patches
<harlowja> gonna let him submit up
<harlowja> oh, nm
<harlowja> well he may have a new set anyway
<harlowja> lol
<harlowja> but yes that change should be fine, smoser though i don't think its the full thing
<harlowja> might be better to have a cloud-init-local-rhel.service
<harlowja> because there are also stupid name differences
<harlowja> nrezinorn will have a few more changes soonish
<harlowja> smoser did u ever hear back from the amazon folks that dropped by once?
<harlowja> did they go silent again
<nrezinorn> soonish, lol  - im ocd and unless i am happy with it i will have nothing :P
<harlowja> notttthing!
<harlowja> lol
<harlowja> smoser do u know if there is someone who can send a folk here our copy of the signed godaddy + canonical CLA
<harlowja> the folks here don't seem to have one, and when i was checking to make sure nrezinorn was on it, they were like we never got a copy, but nrezinorn should be fine to go ahead and submit stuff
<harlowja> so anyone u know who can send a guy here an email :-P
<harlowja> (with that copy or whatever)
#cloud-init 2016-09-22
<smoser> harlowja, i can get you one, send an email to me asking for it and i'll forward correctly
<apollo13> hi, I am trying the centos 7 cloud images on xenserver and was wondering how/which datasource they try to use
<apollo13> since xenserver obviously does not support cloud-init at all I tried sniffing with tcpdump to see if it tries the EC2 method etcâ¦ but nothing
<smoser> apollo13, it depends on how the image is configured
<smoser> i woudl have thoguht they'd try the ec2, but they might only try openstack or config drive.
<apollo13> smoser: I mounted the raw file, can I easily check? there is no datasource config in /etc/cloudinit, so it should fall back to the default list?
<smoser> probably something like:
<smoser> $ lxc exec x1 -- grep -r datasource /etc/cloud/cloud.cfg.d
<smoser> /etc/cloud/cloud.cfg.d/90_dpkg.cfg:datasource_list: [ NoCloud, ConfigDrive, OpenNebula, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Ec2, CloudStack, None ]
<smoser> if there is no config, it should actually try all of them.
<apollo13> mhm, what are the requirements for EC2? I put the VM onto a network with no DHCP in the hopes that it would assign itself a 169.254 ip addr and then try it
<smoser> apollo13, no. it wont do that. that is to be done... and that is the ultimate goal.
<smoser> but right now, it will fallback to dhcp
<apollo13> ah so I need to serve dhcp with those IPs?
<apollo13> or rather: is there any easy config mechanism I could use in a network which does not really support cloud-init
<smoser> apollo13, http://ubuntu-smoser.blogspot.com/2013/02/using-ubuntu-cloud-images-without-cloud.html
<apollo13> or rather could I use a No Cloud datasource to bootstrap somethng else?
<smoser> that works for ubuntu images.
<smoser> right.
<smoser> you can also actually just add the config inside the image if you're willing to do that
<apollo13> smoser: I would be willing to do that basically I could reconfigure the ec2 source to use an ip on my network, right?
<smoser> you could do that probably yeah.
<smoser> also this:
<smoser>  doc/examples/cloud-config-datasources.txt
<apollo13> assuming I could do this, are there any docs on the ec2 format?
<smoser> so... easiest thing for you to do
<smoser> is to mount the image and add /etc/cloud/cloud.cfg.d/my-stuff.cfg
<smoser> with
<smoser> https://git.launchpad.net/cloud-init/tree/doc/examples/cloud-config-datasources.txt
<smoser> see lines 28 there... you can put user-data and meta-data right inside the image
<smoser> its completely static at that point, but that migth be sufficient for you
<smoser> for ec2 metadata
<smoser> https://gist.github.com/smoser/1278651/
<apollo13> oh I could also set seedfrom for the NoCloud source?
<apollo13> then it wouldn't even be static
<apollo13> https://git.launchpad.net/cloud-init/tree/cloudinit/sources/DataSourceEc2.py#n42 <-- there we go
<apollo13> I need instance-data somewhere :D
<apollo13> and there I can put your server, no?
<smoser> yeah. you can do that.
<smoser> nocloud can also 'seed_from' a url
<apollo13> yeah, but then I do not need to remaster the image and can work with everything
<apollo13> smoser: ok, I can play with that -- thank you so much
<apollo13> I'll report back later, lets see if I can get that to work
<roaksoax> 8
<apollo13> smoser: that thing is weird, it tries to query 20:30:04.279650 IP 172.22.1.225.50786 > gw01.bap.lan.domain: 54179+ A? does-not-exist.example.com. (44) and what not but not instance-data :D gotta setup a password and see what the log says
<harlowja> smoser cool, thx, will send u a offical email
<apollo13> oh it also tries metadata.google.internal -- that one looks like I could hook into
<apollo13> the google metadata server looks nicer anyways (looking at the cloudinit source)
<apollo13> and they actually document it :D
<apollo13> ah amazon too
<smoser> apollo13, yeah, you can read the code for does-not-exist
<smoser> thats trying to work around providers who do dns ...
<smoser> whats that called.
<apollo13> jupp, though I wonder why it checks everything but not ec2, where is the default log?
<smoser> where they give you their web server
<apollo13> dns redirects probably
<smoser> so http://does-not-exist.example.com takes you to your service provider
<apollo13> yeah
<apollo13> smoser: I feel stupid now, the image seems to be buggy :( unexpected error nonetype object has no attribute status_code for the ec2 source :D
<apollo13> lets see where that could happen
<apollo13> does cloud-init also run after starting the machine? it seems that after the machine came up with fallback config it retries to request the config
<rharper> it's part of init; so it runs every boot (but not everything is re-run each boot, there are different frequencies);  I'm not sure; but if it didn't find a datasource then on next boot, it will attempt to find it again
<rharper> if it does find a data source, it will cache that along with the instance id, and as long as that data is present, it won't attempt to acquire datasource again.
<apollo13> rharper: okay, and on the absolute first boot, when I get a login mask -- does cloud-init already have finished at that point?
<rharper> yes
<apollo13> so nothing run in parallel or soâ¦ weird, cause I am getting requests for the ec2 data quite late
<rharper> cloud-init runs through roughly 4 stages;  init --local (looks for a local data source, like config drive, etc);  init-network (bring up networking and look for datasources on the network);  at this point if it doesn't find a data source, it goes to fallback, then modules mode=config (this runs the configuration modules); and then a modules mode=final  which runs any configuration and final boot scripts before exiting a
<rharper> nd letting it finish booting
<rharper> no, cloud-init is quite serial by design
<rharper> if you put ssh keys in your user-data, they need to be imported and available before networking comes up and sshd runs, etc...
<apollo13> okay, I just started a new vm -- lets see, I'll give it five minutes but I think after booting there is another service running requesting metadata
<apollo13> or still cloud-init periodically requesting data till it gets something
<rharper> there's no service running; just cloud-init in 4 distinct phases
<rharper> no
<rharper> no background service
<apollo13> okay, maybe I indeed rebooted the machine, I'll give it a few minutes
<apollo13> 21:27:36.503771 IP 172.22.1.221.60060 > app01.bap.lan.8773: Flags [S], seq 541007663, win 29200, options [mss 1460,sackOK,TS val 4294777761 ecr 0,nop,wscale 5], length 0
<apollo13> ha
<apollo13> that is a minute after the login prompt is there
<apollo13> that is a request to instance-data.:8773
<apollo13> and yet another minute later another try, so something is clearly still running
<apollo13> any ideas where from that would be coming :D
<rharper> I don't know your image
<apollo13> the centos generic cloud image
<rharper> but cloud-init  doenst' background any service
<apollo13> mhm, kinda chicken-egg like :D
<apollo13> I cannot get into the machine till cloud-init is through, but that fails currently :D
<apollo13> maybe I really modify the image to set a password for testing
<rharper> can you get at the image offline?
<rharper> mount it up and extract the /var/log/cloud-init.log; that'd be very useful
<rharper> cloud-init also dumps processing to console-log, so having serial console output is informative too
<apollo13> yeah, let me see if I can kill it without shutting down
<rharper> I'll be back in a bit, so please continue with questions and I'll reply when I can
<apollo13> rharper: ok, can I shutdown the vm or would really killing be better?
<apollo13> oh wait, I just snapshot it, shut down and reset
<apollo13> rharper: http://apolloner.eu/~apollo13/.tmp/cloud-init.log
<apollo13> that 19:27 where it calls instance-data. is when I said "<apollo13> 21:27:36.503771 IP 172.22.1.221.60060 > app01.bap.lan.8773:" -- at this point I already had a login there
<apollo13> though the messages log seems to indicate that multi-user target is reached later, which makes sense, lets try a new vm :D
<apollo13> I tried logging in now, will post auth.log and messages soon
<apollo13> oh wait, I maybe should have mentioned that I am/was trying to login via the console from xenserver which probably shows the single user mode :D
<apollo13> jupp that was it, sorry for beeing so stupid
<apollo13> or not, I learned a lot about cloud-init and audit logs :D
<apollo13> I'll call it a win
<rharper> apollo13: looking at the logs, cloud-init finished here: Cloud-init v. 0.7.5 finished at Thu, 22 Sep 2016 19:30:58 +0000. Datasource DataSourceNone.  Up 312.13 seconds;  after this anything that cloud-init blocked during start-up will continue; including reaching multi-user target later;
<apollo13> rharper: yes and I tried logging in __before__ that on tty1 :(
<apollo13> I just thought: hey vm is up already, wth didn't cloud-init do something
<apollo13> little bit embarrasing ^^
<rharper> no worries; it's rather complicated
<rharper> at least the interplay between all of the systemd units and services
<apollo13> so, the next question is on whether to impelment a google metadata service or ec2, any suggestions?
<rharper> can you attach disks ?
<apollo13> are there any nice docs somewhere on which keys my api should return etc?
<apollo13> jupp I can
<rharper> then I'd use a config drive source
<apollo13> ah no, to lazy
<apollo13> and I need a sideproject :D
<rharper> attaching a blob of yaml formatted as an iso seems a lot easier than implementing a metadata service
<apollo13> but boring
<rharper> surely
<apollo13> google seems to be better documented and from the looks of it it doesn't need more than a few fields
<roaksoax> /w/win 8
<apollo13> ?
<apollo13> https://dpaste.de/5YWj/raw <-- pretty much al that is needed, then throw that behind a nice django iface and I am done
<apollo13> that + xenapi and colleagues in the office don't have to get on my nerves for a new vm, sounds like a win win
<apollo13> mhm, last but not least, can I somehow configure static networking via cloud-init?
<rharper> yes
<rharper> but for centos you'll need 0.7.7
<apollo13> mhm, any docs :D
<rharper> yes, one sec
<apollo13> (I apparently searched with the wrong terms)
<harlowja> ok smoser sent a formal request to yourubuntu email
<smoser> harlowja, k
<rharper> apollo13: doesn't look like it's quite made it into the 0.7.7 docs; but the format is yaml and looks like this: http://curtin.readthedocs.io/en/latest/topics/networking.html
<smoser> its tricky though... that has to exist inside the image (which kind of defeats its purpose)
<smoser> or be read from a static datasource.
<smoser> or, the kernel command line.
<rharper> no, network_data.json
<rharper> from metadata service could work
<rharper> but that looks different
<apollo13> :D
<smoser> no. doesnt work.
<smoser> rharper, because networking is only applied by local datasources.
<rharper> bleh
<apollo13> which kinda makes sense, but reconfig would be nice
<rharper> apollo13: cloud-init has to work with all sorts of cloudes
<rharper> clouds
<rharper> reconfig is on the roadmap , way down at the end
<rharper> but, that's the idea
<rharper> user modifies the instance (hotplug), cloud-init could ask the cloud for new metadata and update config
<rharper> https://specs.openstack.org/openstack/nova-specs/specs/liberty/implemented/metadata-service-network-info.html
<rharper> that's the network_data.json format, cloud-init will read that from a ConfigDrive; but as smoser said, it has to be attached (i.e. a local datasource)
<apollo13> crazy crazy :D
<apollo13> I guess I could just pull in networkd files and restart via thatâ¦
<apollo13> ah damn, not in centos7 yet, well dhcp it is then
<apollo13> oh, it is there, they rolled in 219
<smoser> the way to do this... getting network configuration data from a network datasource is as it is being proposed at
<smoser>  https://code.launchpad.net/~utlemming/cloud-init/+git/cloud-init/+merge/303471
<harlowja> apollo13 perhaps nrezinorn can share a cent7 package with u at some point
<smoser> apollo13 could add a datasource that brings up ipv4 link local networking, finds information about its config, and then reads data and metadata.
<harlowja> he's been rolling through the cent7 changes and adjustments recently
<harlowja> rolling through/into/over
<harlowja> lol
<apollo13> that sounds nice, but for now I can write a small service, that is enough for now
<harlowja> k
<rharper> smoser: that merge from utlemming looks neat
<smoser> rharper, yeah. that is what we want to get to.
<smoser> and can do that on any cloud that clearly identifies itself.
<rharper> yep, see the DMI bits
<smoser> right.
<apollo13> dmi? that will be harder to fake for me I guess :D
<rharper> apollo13: exposing the cloud provider via SMBIOS data (DMI table on x86)
<rharper> linux does a dmidecode (flags to extract a specific field, like BIOS vendor)
<apollo13> yeah, but /me is no cloud, just hijacking requests to metadata servers^^
<rharper> with that, cloud-init can assume it's running on a particular cloud and can use that cloud's datasource code
<rharper> if you launch VMs, you're a cloud =)
<apollo13> I am a rocket first and foremost
<apollo13> or spaceship if you are kind^^
<rharper> xen also supports supplying/injecting SMBios data  (at least HVM)
<rharper> hehe
 * rharper notes nick of apollo13 checks out 
 * rharper relocates, bbiab 
<smemsh> hi, is static network config supposed to work on ubuntu 16.04 ? this has worked fine for me with 14.04 and centos, but 16.04 it seems to just use dhcp and my static config from meta-data is nowhere to be found
<smemsh> the interface name is ens2, which i changed in meta-data, but that still does not fix it
<smemsh> (originally i was using eth0)
#cloud-init 2016-09-23
<smemsh> #1609638
<b0stik> hi all
<b0stik> i just wonder where to find detailed reference for cloud-init directives
<b0stik> in particular system_info
<nrezinorn> hey everyone:  https://bugs.launchpad.net/cloud-init/+bug/1470433  can this be updated to be confirmed lol   gonna toss a patch for it soon
<nrezinorn> above bug is fixed with https://code.launchpad.net/~nrezinorn/cloud-init/+git/cloud-init/+merge/306653
<nrezinorn> looks like there is another bug with service names for some azure thing, i can prob get that one too, but no way to test it
<harlowja> nrezinorn i can update it
<harlowja> i'm here to help u help me help u
<harlowja> oh nm, smoser updated it
<harlowja> smoser did u get my CLA nigerian letter?
<harlowja> lol
<harlowja> *nigerian prince letter
<harlowja> nrezinorn cool, so other idea for that
<harlowja> nrezinorn so other idea is
<harlowja> https://git.launchpad.net/cloud-init/tree/config/cloud.cfg#n83
<harlowja> this is where that setting comes from
<harlowja> and if u look at https://git.launchpad.net/cloud-init/tree/config
<harlowja> there is already a cloud.cfg-freebsd
<harlowja> so perhaps we just need a cloud.cfg-rhel
<harlowja> *and then using it
<harlowja> *thats one idea
<harlowja> its prob a good one imho, because cloud.cfg imho should really be cloud.cfg-ubuntu (or debian)
<harlowja> smoser would u be strongly opposed to renaming cloud.cfg -> cloud.cfg-debian or ubuntu
<harlowja> ya, idk anything about the azure stuff, smoser i think does
<harlowja> i only know openstack
<harlowja> lol
<smoser> ah. i'm not opposd to trunk having the cloud.cfg for different oses. and ubuntu being "degraded" to just another os in that scenario
<smoser> i think that is what you're asking
<nrezinorn> well then we need to pick one way or the other, because arch adn gentoo use the same method my patch uses
<nrezinorn> lol
<nrezinorn> which is why i just did it that way
<harlowja> yup
<harlowja> afaik the fedora/rhel package, they already added there own
<harlowja> just making it offical would be cool
<harlowja> primarily http://pkgs.fedoraproject.org/cgit/rpms/cloud-init.git/tree/
<harlowja> see http://pkgs.fedoraproject.org/cgit/rpms/cloud-init.git/tree/cloud-init-fedora.cfg in that
<harlowja> i'd rather just degrade ubuntu (if that's what u want to call it)
<harlowja> damn, think i just found new patches in http://pkgs.fedoraproject.org/cgit/rpms/cloud-init.git/tree/ wtf
<harlowja> lol
<harlowja> oh maybe not, nm
<harlowja> so +1 to degrade
<gopenshaw_> if cloud init is provided multiple datasources, is there a way to guarantee the order they are applied?
<gopenshaw_> i'll ask next week :)
<nrezinorn> lol - someone was like "it's friday - PEACE OUT!"
<nrezinorn> its easy to use the -fedora.cfg file when you build via rpmbuild and not brpm or whatever ;)
<nrezinorn> mostly because im not sure how to make it pick the proper config file in the current method lol
<x58> Are there any good tools to build custom images for Linux for the cloud in a qcow2/raw container format?
<nrezinorn> any specific distro?
<x58> CentOS/Ubuntu mainly.
<nrezinorn> i use Oz to build the centos6/7 images
<powersj> x58, http://docs.openstack.org/image-guide/create-images-automatically.html
<x58> Just found that :-)
<powersj> dib was big at my last job
<nrezinorn> yeah pick one from that page lol
<nrezinorn> :)
<x58> Going to automate building images with the changes required for my cloud.
<nrezinorn> it is quite glorious when you get it working - we build daily.  but only push to stage/prod once a month, so if something goofy occurs we know immediately (and can fix)
<x58> Then along with some gitlab work flows, hopefully I can build images more easily as things are updated.
<nrezinorn> and out of cycle production pushes are just a button in jenkins lol
<x58> That's my goal ;-)
<nrezinorn> that was last years goal lol
<nrezinorn> BASH + OZ + serverspec (to verify what we expect is there)
<x58> Some of us just got our private cloud up and running...
<nrezinorn> x58: lmk if you get stuck anywhere, it was a learning experience.  do you have the bonus of not having to build windows base imageS?  windows images are :'(
<x58> nrezinorn: Windows images are coming :/
<x58> Sadly.
<nrezinorn> any idea what versions?  2k12r2?  the new cloud-y one that i dont think it out yet?
<nrezinorn> it takes ~5 hours to install windows lol - because its applying ~220 patches
<nrezinorn> it builds overnight, so my 9AM its ready to go lol
#cloud-init 2017-09-18
<blackboxsw> hey cloud-init, meeting will probably start a couple mins late
<ajorg> ok
<smoser> blackboxsw, how do i make the bot start logging ?
<dpb1> o/
<blackboxsw> #  meetingology startmeeting
<blackboxsw> https://wiki.ubuntu.com/meetingology
<blackboxsw> # #startmeeting cloud-init status meeting
<smoser> #startmeeting cloud-init status meeting
<meetingology> Meeting started Mon Sep 18 16:04:28 2017 UTC.  The chair is smoser. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
<meetingology> Available commands: action commands idea info link nick
<blackboxsw> hiya o/
<rharper> o/
<smoser> #link https://public.etherpad-mozilla.org/p/cloud-init-meeting
<dpb1> hello
<smoser> so that contains a agenda that we'd been following
<smoser> #link https://lists.launchpad.net/cloud-init/msg00100.html
<smoser> sorry for slow going here.
* smoser changed the topic of #cloud-init to: Release Preparation
<smoser> shoot
<smoser> #topic Release Preparation
<smoser> #subtopic Release Preparation
<smoser> hm..
<smoser> anyway
<smoser>  https://lists.launchpad.net/cloud-init/msg00100.html
<smoser> that is the primariy point of this meeting for today.
<smoser> The goal is to make a release named 17.1 on Thursday of this week.
<blackboxsw> strange
<smoser> We have landed several branches in the last few days, and have 2 more to land at least.
<smoser> from http://bit.ly/ci-reviews
<dpb1> meetingology â Available commands: action commands idea info link nick
<dpb1> something to look into. :)
<blackboxsw> #link http://bit.ly/ci-reviews
<smoser> the two we consider needs to be integrated are
<smoser>  https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330875
<smoser>   https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/330880
<smoser> and Chad's (the first) is the one we're looking at now.
<smoser> or that is the one that is non-trivial
<smoser> does anyone here wnat to raise other bugs or MP that they believe should be pulled ?
<smoser> ...
<smoser> ajorg, i think its best to hold off on some of yours until after release.
<smoser> anyone have anything else ?
<blackboxsw> msaika (vmware) pinged on a branch a couple days ago.  but I think there is a little of discussion/re-work that's needed there before we land
<ajorg> smoser: I assume you mean the instance identity one, in which case I agree.
<ajorg> (in either case I'd agree)
<smoser> and simpletable
<smoser> yeah
<smoser> ok. so that is where we are. we are still targettin a release on Thursday.
<smoser> other than that, i thinkw e move on to open discussion or office hours
<smoser> #topic Open Discussion
<smoser> anyone have anything for this ?
<robjo> Customer has run into an issue where we run out of threads during the user script phase, should generally set TasksMax in the service file, i.e. is that of interest upstream?
<robjo> we are still testing, but that appears to be the solution for this particular problem
<smoser> robjo, i dont have an immediate objection to that idea.
<smoser> could you open a bug ?
<robjo> sure
<smoser> i dont think that we'd want to pull that change in right now, but it otherwise seems reasonable at frist b lush.
<smoser> anything else ?
<robjo> Agreed, do't think this is sufficiently urgent for Thursday's release, it can wait until the next one
<ajorg> were there other bugs that we should consider high enough prio to deserve landing a fix in the release?
<smoser> ajorg, the only 2 that i'm aware of are linked in those mp. i can dig the numbers
<smoser>  * bug 1717598
<ubot5> bug 1717598 in cloud-init (Ubuntu) "Traceback when passing user-data on GCE" [High,In progress] https://launchpad.net/bugs/1717598
<smoser>  * bug 1717627
<ubot5> bug 1717627 in cloud-init "permission denied when executing dhclient in Ec2 datasource" [High,In progress] https://launchpad.net/bugs/1717627
<ajorg> ok
<smoser> Anything else? other wise i'll call this meeting done and hang around for office hours for the next 30 m inutes at least.
<smoser> thanks for attending / feedback, robjo and ajorg
<blackboxsw> #link  https://launchpad.net/bugs/1717598
<ubot5> Ubuntu bug 1717598 in cloud-init (Ubuntu) "Traceback when passing user-data on GCE" [High,In progress]
<blackboxsw> #linkhttps://launchpad.net/bugs/1717627
<ubot5> Ubuntu bug 1717627 in cloud-init "permission denied when executing dhclient in Ec2 datasource" [High,In progress]
<blackboxsw> just in case
<blackboxsw> duno whats up with the bot.... again
<smoser> ok. well, lets move on
<smoser> #endmeeting
<meetingology> Meeting ended Mon Sep 18 16:27:32 2017 UTC.
<meetingology> Minutes:        http://ubottu.com/meetingology/logs/cloud-init/2017/cloud-init.2017-09-18-16.04.moin.txt
<smoser> office hours now if you have anything
<dpb1> blackboxsw: bot looks ok, but... it doesn't understand topics
<dpb1> blackboxsw: it never echos on links or infos, that is normal
<ajorg> I'm playing with making simpletable output look like prettytable output, even though I think simpletable is prettier
<blackboxsw> published minutes will show up at https://cloud-init.github.io/from now on
<blackboxsw> heh ajorg  nice
<rharper> ajorg: lol
<ajorg> smoser: I did want you to take another look at the instance identity thing. I think you had your resolution comments backwards
<smoser> ajorg, ok.
<smoser> i have sense looked at that and the dns is wierd.
<smoser> $ host why-does-this-exist.ec2.archive.ubuntu.com
<smoser> why-does-this-exist.ec2.archive.ubuntu.com has address 91.189.92.201
<smoser> why-does-this-exist.ec2.archive.ubuntu.com has IPv6 address 2001:67c:1360:8c01::19
<smoser> i think you showed (and i see elsewhere)
<smoser> $ host why-does-this-exist.ec2.archive.ubuntu.com
<smoser> Host why-does-this-exist.ec2.archive.ubuntu.com not found: 3(NXDOMAIN)
<ajorg> meaning it's different depending on where we're calling from?
<smoser> yeah. somewhat. but mostly i was just giving you bad information
<smoser> i had a canonical VPN up.
<smoser> and was getting a dns response over that . i just figured this out... shut that down and i get the does not exist
<rharper> smoser: you route everything through the VPN right ?
<smoser> not everything.
<smoser> but apparently dns was going there for at lease *canonical
<smoser> er.. *ubuntu
<rharper> strange, I know I'm on a diff setup; but same vpn, and I've never had that host resolve; it always failed to resolve for me (with vpn up)
<ajorg> ah. so in The Real World it probably works the way I think it does?
<smoser> ajorg, right.
<smoser> at least for now :)
<smoser> rharper, i'm also using artful. so its possible network manager tied better into systemd-networkd or osmething and got *.ubuntu.com passed to vpn dns
<smoser> i dont know
<rharper> yeah;  on an internal system, I can see the wildcard working as you saw
<smoser> ajorg, so we have more thinking to do there. but i unfortunately can't prioritize said thinking right now.
<ajorg> okay. It can wait. I do hope you'll get back on it soon, because it's one of the more important patches we're carrying.
<ajorg> well, that wasn't too hard: http://paste.ubuntu.com/25566740/
<ajorg> I'll write a unit test or three this week and resubmit that with PrettyTable ripped out completely.
<ajorg> Do I recall correctly that we're encouraging putting tests next to modules now?
<blackboxsw> ajorg: yes please
<ajorg> thanks, will do
<blackboxsw> if there isn't a tests/ subdir next to your module just add one. post this release I think we'll overhaul the tests in one mega branch and put them all in the 'proper' location
<ajorg> cool
<blackboxsw> smoser: commented on https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/330880
<blackboxsw> approve with a nit
 * blackboxsw thought about adding a more significant unit test for main, but I'd ultimately like that to go away anyway with a standardized datasource.crawl_metadata() method (and cloud-init devel crawl subcommand)
<blackboxsw> so I figure why spend much time on covering that main() method w/ tests
<blackboxsw> since I'd like to see it dropped in a couple weeks
<smoser> the mains are debug only anyway
<smoser> devloper tools
<smoser> i took your suggested test cahnge
<smoser> and committed and pushed
<smoser> runnign tox here
<blackboxsw> +1
<Sachith> Hi
<Sachith> anyone here?
 * dpb1 hides
<rharper> dpb1: huh. maybe you shouldn't have hid
<dpb1> LOL
<paulmey> smoser: Re https://code.launchpad.net/~paul-meyer/cloud-init/+git/cloud-init/+merge/330925
<paulmey> From history we know that some infrastructure updates take at least 5 minutes and may induce transitional failures in the wire server...
<paulmey> Of course, that's unacceptable in the provisioning path, which we'll work on separately...
<paulmey> I'd like to set the timeout at 900 to be on the safe side of 300+...
<paulmey> Is that within the realm of possibilities?
<msaikia> hi, can anyone take a look at my review request. https://code.launchpad.net/~msaikia/cloud-init/+git/cloud-init/+merge/330105
<cj> hey folks
<cj> can I get help for cloudbase-init here?
<cj> I'm trying to kick off a jenkins jnlp windows slave, but can't seem to get the user-data I specify with Jenkins' Cloud plugin to execute
<powersj> cj: do you have access to the logs to see if your user-data had any issues?
<cj> I think I do.  I'm pretty sure I specified where the logs should live...
<cj> "msiexec /i C:\\packer\\CloudbaseInitSetup_Stable_x64.msi /qn /l*v c:\\packer\\cbi-log.txt"
<cj> from line 72 of https://gerrit.iotivity.org/gerrit/#/c/21639/41/packer/templates/baseline-win.json
<cj> or is this just the installation log?
<powersj> cj: tbh not sure, as I am not 100% certain 1) how much different cloudbase init is from cloud-init and 2) how things get run on windows ;)
<smoser> paulmey, i'm willing to defer to you.
<paulmey> Alrighty, mp updated
<dpb1> cj: well
<smoser> blackboxsw, subp_blob_in_tempfile will need updating
<dpb1> cj: it's not in tree, and not supported by people here directly.
<smoser> to /var/tmp/
<blackboxsw> smoser: yeah I wasn't sure if we wanted that in this branch or not. We could just change temp_utils.tempdir()
<blackboxsw> then we don't have to change it in all call-sites in the future
<smoser> or put a define somewhere.
<smoser> TEMP_EXE_DIR
<smoser> or something
<smoser> and then use that from both?
<blackboxsw> why would we not want to use it in all cases?
<blackboxsw> _ROOT_TMPDIR == '/var/tmp'
<blackboxsw> one place to fix, instead of the caller having to know that they want exec perms
<dpb1> cj: I'd say look here: https://ask.cloudbase.it/questions/, it seems to be their preferred method of Q/A
<smoser> blackboxsw, i'm not opposed... but it seems good to identify the places that do need exe
<smoser> but yeah, you're right in that we'd need also a non root value
<blackboxsw> ?
<blackboxsw> oops typo
<blackboxsw> smoser: is it worth a param to temp_dir(exec=True) ?
<blackboxsw> which then gets sorted
<smoser> another option would be to have a kwarg 'needs_exe' or something.
<blackboxsw> yea
<blackboxsw> I'm all for needs_exe, it'll make it explicit for the caller.  and they won't have to deal w/ imports of TEMP_EXE_DIR etc
<smoser> if you can do that in short term, next hour or so i'm good with that . but for the moment i'm also ok with everything being /var/tmp.  assuming we're reasonably confident that /var/tmp isnt going to be cleaned on boot
<cj> thanks, dpb1
<blackboxsw> smoser: shall we put unpriveleged users of temp_dir(needs_exec=True) in /var/tmp/cloud-init  or just root
<smoser> blackboxsw, i think its fine to put unpriv users tehre. we do set the perms so they can , i think. on creation
<blackboxsw>         os.chmod(tdir, 0o1777)
<blackboxsw> yeah
<blackboxsw> ok will do
<blackboxsw> I only need another 20mins I think  on this.
<blackboxsw> will have a fix
<blackboxsw> adding a couple unit tests now
<smoser> k. thanks.
<smoser> blackboxsw, what do you think about https://code.launchpad.net/~paul-meyer/cloud-init/+git/cloud-init/+merge/330925
<paulmey> rharper: remind me again where vendor data comes from?
<rharper> the cloud metadata service
<paulmey> via the cloud-specific data source?
<rharper> cloud-init fetches 'user-data' key from the metadata service, it will/can also look for 'vendor-data' ; yeah inside the DataSource
<paulmey> ok, I don't think Azure currently has that...
<paulmey> but it would be good to think about that...
<smoser> azure does not provide vendor data at the moment.
<rharper> yeah
<paulmey> I'll put it on my wish list
<blackboxsw> smoser: I guess we're dropping the maxwait default in the Azure data source, so it's a required param now per paul's branch. I only see two callers of that function, so sure.  I'm +1 https://code.launchpad.net/~paul-meyer/cloud-init/+git/cloud-init/+merge/330925
<blackboxsw> It'd be nice if we had a way to introspect the walinux agent health on this  instead waiting. But, maybe that's the approach we take for now and we improve upon it as time permits
<paulmey> it wouldn't look very healthy...
<rharper> lol
<rharper> the health check would block forever ?
<paulmey> showing 503/404 for requests to the wireserver/metadata server
<paulmey> or just timing out there...
<blackboxsw> I suppose that's what paulmey means by "I'm working on a fix for this issue." :)
<paulmey> the issue is on the host
<paulmey> :-) sure
<paulmey> I should probably have said "I have other teams working on a fix for this issue"
<blackboxsw> hah
<paulmey> silent provisioning failures lead to customer support calls
<paulmey> long provisioning times and failures just yield bad rep
<blackboxsw> s/bad rep/bad bug reports/
<rharper> or both
<paulmey> yeah
<smoser> bad rep leads to mean tweets
<smoser> mean tweets lead to executive calls
<blackboxsw> and mean tweets lead to smoser losing sleep
<paulmey> lol
<blackboxsw> so paulmey if we block waiting on certs and does walinux agent ever fix itself in these cases?
<paulmey> Yes, it should. Eventually the wire server should be up again.
<rharper> hrm, we could add a smoser reads mean tweets to the cloud-init meeting agenda
<paulmey> This situation mainly occurs when the wireserver itself is being updated
<smoser> rharper, once for uds or uos (online) we had the security team read mean tweets
<rharper> hehe
<smoser> https://youtu.be/4fB94_DndL8
<smoser> blackboxsw, https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330875
<smoser> any status ?
<smoser> i've got to be running.
<smoser> what i'll do right now is upload trunk to artful
<smoser> that will get us the gce fix
<smoser> and then i'll come back in ~ 4 hours and if you have merged paulmey and your branch i'll grab and upload those too
<blackboxsw> smoser: yep just finished a patch
<blackboxsw> took a while to setup the unit tests
<blackboxsw> wrap_and_call issues on my sid
<blackboxsw> finally pushed to the right branch https://git.launchpad.net/~chad.smith/cloud-init/commit/?id=a3a33b43c3c62b82c30050a78c69a11a5eca8c40
<blackboxsw> will await CI and run an AWS deploy test on this latest
<blackboxsw> paulmey: merged your branch and updated the commit message. it should be in our next release (which we are cutting this week)
<blackboxsw> https://code.launchpad.net/~paul-meyer/cloud-init/+git/cloud-init/+merge/330925
<blackboxsw> thanks again
<paulmey> blackboxsw: thanks!
#cloud-init 2017-09-19
<blackboxsw> thank smoser
<smoser> blackboxsw, tox && git pushing now
<smoser> then i'll upload again
<smoser> blackboxsw, ok. that is uploaded. i go away for the night now.
<smoser> blackboxsw, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/330954
<smoser> and now i really *am* gone.
<smoser> later.
<smoser> uploaded
<rharper> smoser: do we know if any public cloud where we can test DataSourceOVF ?
<smoser> well you can test it locally
<smoser> by providing an ovf iso. doc shows how to create one
<rharper> ok
<smoser> sankaradita has one
<rharper> ok
<rharper> https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/330995    this is what I'm going to test;  I think it's a good cleanup w.r.t reducing the number of devices we actually probe (and re-using blkid cache)
<dpb1> rharper: for a workaround for the case?  or a medium term fix?
<rharper> long term
<dpb1> k
<rharper> it's only looking for devices with iso9660 filesystem, blkid can do that quite quickly (and cloud-init respects the blkid cache); ds-identify will probe this early
<rharper> so, this should speed up OVF probing dramatically for systems with more than just one block device
<smoser> rharper, i'm pretty sure the mounting we're doing in the default case is doing a mount -t iso9660
<smoser> which i'd think would fail unless youhad iso9660
<smoser> so i'm not sure how this would cause races with other mounts.
<smoser> not sure why we're failing https://code.launchpad.net/~cloud-init-dev/+recipe/cloud-init-daily-xenial
<blackboxsw> test_openstack_on_non_intel_is_maybe   hmm
<blackboxsw> checking maybe it's a leaked mock thing?
<blackboxsw> or unmocked leak I mean
<smoser> yeah. thats all i can think of.
<smoser> i can't reproduce it on the branch (ubuntu/xenial) though
<smoser> even with patches applied
<smoser> so trying in a container
<smoser> blackboxsw, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/331012
<seven-eleven> hi
<seven-eleven> how can i run something after cloud-init is done?
<seven-eleven> basically i want my custom systemd unit file "openvpn2.service" run after cloud-init setup the hostname
<blackboxsw> smoser: approved
<seven-eleven> what i tried and didn't work: (1) appending After=cloud-init.target to openvpn2.service, (2) supplying user-data via droplets **user-data kwargs "systemctl daemon-reload; systemctl restart openvpn2"
<blackboxsw> seven-eleven: :n
<blackboxsw> seven-eleven: sorry I'm looking now. I thought systemd chaining would do that for you. but I'm checking which service you need to be after
<seven-eleven> maybe i can tell cloud-init in /etc/cloud/cloud.cfg   @ cloud_final_modules:  to restart my openvpn2, so it grabs the new hostname :]
<seven-eleven> blackboxsw, let me paste my service
<seven-eleven> blackboxsw, my customized openvpn2.service unit http://dpaste.com/1YF2AV0 which uses %H for hostname
<seven-eleven> it's actually starting my service fine, but when i want to restart the service i have to run `systemctl daemon-reload`, else it fails. i can leave it like that, but it's not perfect :-)
<seven-eleven> it fails with the error "Failed to start OpenVPN connection to bus1." and bus1 is the hostname of the vm i created the snapshot from
<rharper> smoser: re mount -t iso9660, we passed mtype to the util.mount_cb, all this does is change the list of candidate devices from os.listdir(dev)  to blkid -odevice -tTYPE=iso9660;  we still pass mtype into mount_cb;
<blackboxsw> seven-eleven: not sure if this helps, but what about After=cloud-final.service
<blackboxsw> that's the last stage to run
<blackboxsw> http://cloudinit.readthedocs.io/en/latest/topics/boot.html#final
<blackboxsw> seven-eleven: you could also add a runcmd: ['systemctl daemon-restart <yourservice>']   and that should be run in final stage
<smoser> rharper, right.
<smoser> but two things i was suggesting
<seven-eleven> blackboxsw, let me try. i was also thinking to maybe add Before= to cloud-final
<smoser> a.) you can/should further limit th results through the regex that was already being done. otherwise you risk extending the breadth inadvertantly.
<smoser> b.) the fact that we were passing '-t iso9660' makes me think that this was not actually the bug
<smoser> because there is basically no way that 'mount -t iso9660' was going to work on any of the entries in the fstab provided.
<smoser> so i dont know how it would have caused issues.
<rharper> a) already limits to block devices with iso9660; the regex is likely too narrow but practically I see you point;  if there were possibly a block device that's outside of the regex, OVF claims to not want to look at it even if it has an iso9660;  that's debatable but certainly increased the "scope" however unlikely
<rharper> b) is more interesting;  I wonder if mount's -t opens and peeks at the filesystem type
<rharper> if that's done via exclusive-open, it could race with a .mount unit
<rharper> which expects an exclusive-open
<smoser> right. so we should just avoid inadvertantly extending the scope.
<smoser> (and by doing so actually further *limit* this questionable search, which is good)
<rharper> smoser: yes; I can add the regex check back into the search;
<rharper> pratically it won't matter but it's certainly possible
<smoser> so yeah, it could be doing a exclusive open and a read-check for the fs magic
<smoser> but that'd seem *so* fast
<smoser> that i cant imagine we'd actually hit the issue
<smoser> open; read(4096); check; close()
<smoser> that is really really really fast
<rharper> races are races, however small
<rharper> the block device can be slow to respond
<smoser> but you woudlnt have seen someone report this
<smoser> or if you did, they couldnt reproduce
<smoser> thats my feeling.
<rharper> right, and
<smoser> i dont deny that it *could* happen
<rharper> its mostly only reproducible with large sets of mounts
<rharper> agreed
<smoser> just very unlikely that it is the source of the bug if i understand it right.
<rharper> the data from wolsen was that after disabling OVF datasource, some multi-hundred reboots never reproduces, where with OVF in, it reproduces every 3 or so
<rharper> this on a contrived instrance with like 26 ebs volumes
<smoser> that sounds plausible
<smoser> rharper, also limiting the mount-callbac-umount
<smoser> we are also passing -o ro
<rharper> it was also doing util.peek
<rharper> which does an open and a read
<rharper> well, filtering the set of devices we poke to zero (unless they have an iso9660) will surely work
<rharper> I generally expect that OVF run much faster that we're not checking all that regex devices, opening/read and also mount_cb each one
<smoser> rharper, well, blkid is going to also do an open
<smoser> fwiw
<rharper> no
<rharper> it's cached
<rharper> ds-identify runs blkid first
<rharper> then we just read the cache
<smoser> i'm not sure that is the case
<rharper> why do you think not?
<smoser> we do call find_devs_with with no_cache=False. but i have never actually been able to determine what blkid does wrt its cache.
<smoser> when it determines something is valid and when not
<smoser> run:
<smoser> sudo strace -o /tmp/out blkid
<smoser> and then run it again
<rharper> with -c/dev/null
<smoser> we only pass -c/dev/null if no_cache == true
<smoser> ie, thats supposed to be the "re-read everything"
<smoser> it should re-use its cache if *not* given that
<rharper> right, I just meant to compare the syscalls
<rharper> so you can see what the cache buys you
<rharper> w.r.t the cost;  for the instance with 26 volumes or so,  we have a total of 1.5 seconds time spent do 1) read /proc/mounts 2) regex match 3) peek file  4) mount_cb;
<rharper> thats about 60 ms per device;  the filtering helps eliminate all of those;  that's pretty nice;
<smoser> so i just never really understood what -c/dev/null does
<smoser> it seems to still do stuff
<rharper> -c means to ignore the cache; I suspect it depends on the query to some degree,
<rharper> smoser: around?  have time for a ho on the mount race?  I've got some questions I wanted to bounce off you if you're available
<smoser> quick. yeah. i got 10 minutes
<rharper> k
<rharper> smoser: https://hangouts.google.com/hangouts/_/canonical.com/cloud-init?authuser=1
<smoser> in
<seven-eleven> blackboxsw, sorry i was afk and could test only now. After=cloud-final.service didn't help. but by using run_cmd it worked. I used digitalocean's API and provided run_cmd through the user_data attribute: http://dpaste.com/29JDA2Z
<akik> firewall-cmd --reload on centos 7 eats DOCKER-USER iptables chain. is that a problem?
<akik> sorry wrong channel
<blackboxsw> :)
* blackboxsw changed the topic of #cloud-init to: Reviews: http://bit.ly/ci-reviews | Next status meeting: Monday 10/2 14:00 UTC
* blackboxsw changed the topic of #cloud-init to: Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/Ts2k8t |  Next status meeting: Monday 10/2 14:00 UTC
* blackboxsw changed the topic of #cloud-init to: Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj |  Next status meeting: Monday 10/2 14:00 UTC
<blackboxsw> seven-eleven: good work!
#cloud-init 2017-09-20
<elurkki> Hi. Having problem with 0.7.9 where NoCloud (meta-data) files network-interfaces: | gateway won't be set when defined. address and netmask seems to be working. The system is using sysconfig.py as a renderer in my CentOS 7.4 and seems that gateway parameter is not reached in that renderer. Would there be some way in meta-data to by pass this problem in CentOS 7.4 case? Thanks a lot!
<elurkki> Managed to get it working with network-config file.
<smoser> elurkki, what did you figure out ?
<smoser> basically, the 'network-interfaces' is a legacy input
<smoser>  /etc/network/interfaces format is a poor and incomplete mechanism for defining network configuration.  the network-config is the right way.
<robjo> smoser: Where is the variant set for the templating engine? For some reason I am getting "linux"
<rharper> the distro variant
<rharper> from the sysinfo, one sec
<rharper> robjo: see cloudinit/util.py:system_info()
<rharper> looks like we'd need another elif linux_disk in <suse names>
<robjo> rharper: thanks, yes, but I think with complications due to build service .. investigating
<robjo> OK, got that sorted out, but still failing with "KeyError: 'user'" from the templater, when processing the config template :(
<robjo> https://pastebin.com/hLrLKBU0
<smoser> man. it just hurts my eyes to look at pastebin.com
<smoser> robjo, you have no other changes ?
<robjo> Well I am adding patches to try to get the templater to behave, so I added a variant for suse in the config template
<robjo> I can push what I have to the build service if that would help
<smoser> i think you dont have jinja
<smoser> which clearly should fail in a more obvious way
<robjo> bingo
<robjo> and yes, when I set up the py3 build I have a BuildRequires for python3-Jinja2, but clearly I neglected this for the python2 build :(
<robjo> thanks
<smoser> robjo, did you see my response about python2 and argparse ?
<smoser> i dont understand how you all hit this (nor do i understand the weird things i've seen on centos wrt argparse)
<robjo> I did only skim your response since I am in the middle of getting the current version packaged
<robjo> btw, setup.py still says 0.7.9, I was expecting 2017.1 ;)
<smoser> well, it wont say that until the release.
<smoser> but i think at this point we're one commit away.  basically updating that.
<robjo> I should have a bunch of patches by the end of the week basically more integration work do you want bugs for those?
<smoser> robjo, yes pleas.e
<robjo> OK
<robjo> the get_data_file() test is failing in base.py , self.data is None, but I cannot find where self.data is supposed to be set??
<smoser> robjo, that isnt a test... it shouldnt be run
<smoser> tests start with test_
<robjo> duh, the test triggering the issue is test_no_stages_errors
<smoser> robjo, wow. that test has probably been broken for a while.
<smoser> it wont get run because the file doesnt start with test_
<robjo> During package build I need to run with "python3 -m testtools.run"
<smoser> oh
<smoser> rigth
<smoser> those arent for nose
<smoser> those are c-i tests (continuous integration)
<robjo> I guess I can just remove base.py
<smoser> why ?
<smoser> i'm sorry.
<smoser> my first response was wrong
<smoser> what is testtools.run ?
<smoser> we don't use testtools we use nose.
<smoser> its possible that both could be made to have the same behavior, but what *does* work is nose.
<robjo> https://testtools.readthedocs.io/en/latest/
<smoser> blackboxsw,   when we added cloudinit/ to the dirs that get tested, we did not add it in Makefile unittest3
<smoser> robjo, thats not how we run our tests.
<smoser> its not to say its not valid or useful, just not what is used.
<blackboxsw> smoser: ahh right you are
<blackboxsw> it's under the makefile "unittest" target, ! unittest3
<robjo> I know, need to get the package build to resemble upstream testing, it's a time issue, as always....
<smoser> robjo, so what i'd suggest is you install nose and run 'make test'
<smoser> what was it that is forcing you to run testtools.run ?
<robjo> I'll try nose and see what falls over
<blackboxsw> smoser: we also have cloudinit module in tox -e py3 but not py27
<blackboxsw> probably need it added everywhere
<robjo> I forgot why I am running testtools, bad notes in the spec file :(
<smoser> blackboxsw, i guess.
<blackboxsw> 	http://paste.ubuntu.com/25582039/
<blackboxsw> running tests now
<blackboxsw> green for me: make unittest; tox.      red: make unittest3
<smoser> hm.
<smoser> blackboxsw, well make sure youd' have all deps installed
<smoser> since you're not using tox
<smoser> but i have to run. i'll be back in a few hours to check.
<blackboxsw> smoser: yeah I have make ci-deps-ubuntu :)
<blackboxsw> but it's a mock issue.
<blackboxsw> assert_called_once() DNE
<blackboxsw> ï¿¼cloudinit/net/tests/test_dhcp.py
<blackboxsw> need to use assert_called_once_with()
<blackboxsw> making the change
<blackboxsw> or assertEqual(1, mock_obj.call_count)
<robjo> Well better than before, meaning I remove fewer tests :) those will have to wait for another day
<blackboxsw> ok tweaks to master work for me on ubuntu to include unit tests under cloudinit for make targets and tox envs
<blackboxsw> http://paste.ubuntu.com/25582096/
<blackboxsw> will put up a trivial branch
<blackboxsw> https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/331099
<robjo> Cloud:Tools:Next cloud-init in OBS is building :)
<robjo> will test in AWS tomorrow.....
<powersj> woohoo \o/
<blackboxsw> msaikia: sorry for the delay review posted on your branch
<blackboxsw> just merged just merged smoser/fix/makefile-not-find-bin
<smoser> ok. thanks.
<smoser> blackboxsw, https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/331099
<smoser> is that just an improvement on the test ?
<blackboxsw> smoser: yep, it just usess mocked_obj.call_count instead of assert_called_once (which isn't defined on py3 unittests)
<smoser> hmm.
<blackboxsw> and it adds a failure message of 'dhcp_discovery not called once' instead of 0 != 1
<blackboxsw> if the assert were to fail
<blackboxsw> I'll check centos w/ that to see if that jives
<smoser> yeah.
<smoser> i think that it just did nothing
<smoser> hm..
<smoser> wonder how it worked
<smoser> it is "new in version 3.6"
<smoser> https://docs.python.org/3/library/unittest.mock.html
<blackboxsw> I thought we added it in tests/helpers too
<blackboxsw> for earlier versions of unittest
<smoser> we added assert_not_called
<smoser> yeah. i commented in there
<smoser> when you ran not in tox you did not have python3-mock installed
<smoser> so you got the builtin version of python3.5's mock
<smoser> and that didnt have  that function
<smoser> i think
<smoser> anyway. i ack'd
<smoser> please merge that.
<smoser> and i have to go afk again . thanks blackboxsw
<blackboxsw> cheers
<blackboxsw> will do
<smoser> robjo, you have a link to a build ?
<blackboxsw> yeah I've got kid's soccer practice to head to in 30. I think I can beat them this time
<robjo> https://build.opensuse.org/project/show/Cloud:Tools:Next
<robjo> smoser: ^^^^
<blackboxsw> excellent robjo care if we add a link to that in the weekly cloud-init status (credited to you of course)?
<blackboxsw> example of the syndicated content https://insights.ubuntu.com/2017/09/19/ubuntu-server-development-summary-19-sep-2017/ for weekly status is here <
<blackboxsw> tested on cent7 and cent6
<blackboxsw> merged
#cloud-init 2017-09-21
<elurkki> smoser: CentOS 7.4 & cloud-init 0.7.9 didn't set the gateway when used earlier user-data and meta-data files with NoCloud. Managed to figure out the network-config syntax and now all the network configurations are set correctly.
<mtangaro> Dear experts, I've a problem with Cloud-init or at least I think so
<mtangaro> can you please help me?
<smoser> robjo, around ?
<smoser> responded to both your mps
<robjo> on the phone, will check after, thanks
<smoser> k
<robjo> smoser: https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/331142 should hopefully meet the needs
<smoser> robjo, are you ok if i change the indent on 'Copyright' ?
<robjo> sure
<smoser> robjo, ok. and is it ok if i just drop the copyright ?
<smoser> i think they're 99% largely harlowja work anyway.
<robjo> Yeah, I guess, this is going away in 2019 anyway when SLE 11 dies, I can't wait.....
<smoser>  http://paste.ubuntu.com/25587157/
<smoser> thats the diff from redhat/* to suse/*
<robjo> yup, just off enough to be annoying, I know
<smoser> i think then we shoudl just do a straight copy and then make the changes that matter
<smoser> is that ok ?
<smoser> we could do some integration or templating, but sysvinit in suse and redhat are both really EOL
<smoser> so that doesn't make much sense.
<kszarlej> hey guys, I have a Libvirt VMs that are using cloud-init provided by the 'config drive'. Is this possible using cloud init to setup the eth0 interface when booting the instance? I use in my network only static addressing and I dont want to add DHCP server
<robjo> Yes copy and then changing the bits where it matters also works for me
<kszarlej> So instance starts with no configured interfaces -> cloud init configures statically the eth0
<smoser> robjo, http://paste.ubuntu.com/25587206/
<smoser> thats what i would do on top of your branch
<smoser> which leaves me with
<smoser>  http://paste.ubuntu.com/25587211/
<smoser> as (redhat -> suse differences)
<smoser> robjo, i'm sorry that i didnt realize at the start that you had copied the redhat/ scripts
<smoser> i assumed they'd come from your packaging or something
<smoser> sorry
<smoser> kszarlej, not statically
<smoser> oh. sorry. wait.
<smoser> kszarlej, "config drive"
<kszarlej> ye
<smoser> is that meaning "OpenStack config drive" ?
<smoser> or meaning "configuration disk" more generically
<kszarlej> I am creating .iso with 'user-data' and 'meta-data'
<smoser> right.  NoCloud then.
<kszarlej> and I mount it to VM as 'cdrom' device
<kszarlej> and the VM recognizes it just fine
<smoser> 'config drive' is a term that means something specifically to openstack.
<smoser> sorry. i was just trying to clarify what you were asking
<robjo> smoser: http://paste.ubuntu.com/25587206/ looks fine, do not recall the origins os the sysvinit stuff it's been too long, been carrying those patches in cloud-init package forever
<kszarlej> I have problem setting up the dnsmasq in my network infrastructure that would be leasing DHCPs to VMs.. and I am thinking if it would be possible to
<smoser> ok. then robjo i'll pull those in with my changes.
<kszarlej> configure the interfaces using cloud-init instead of DHCP
<robjo> thanks
<smoser> kszarlej, so on your disk with 'user-data' and 'meta-data'.
<smoser> you need to
<robjo> also replied on the other proposal
<smoser> a.) set the label to be 'cidata'
<smoser> b.) provide 'network-config' file in addition to user-data and meta-data
<smoser> the network-config is described http://cloudinit.readthedocs.io/en/latest/topics/network-config-format-v1.html#network-config-v1
<kszarlej> so like that https://gist.github.com/Informatic/0b6b24374b54d09c77b9d25595cdbd47
<kszarlej> ??
<smoser> or http://cloudinit.readthedocs.io/en/latest/topics/network-config-format-v2.html#network-config-v2
<kszarlej> ok smoser
<kszarlej> thanks!!
<kszarlej> I was looking for "cloud init config drive network configuration"
<smoser> you do not need dsmode: local now
<smoser> i'm 99% sure
<kszarlej> and tried the network-interfaces: in meta-data
<kszarlej> but I am failed :)
<kszarlej> will try the network-config :)
<smoser> network-interfaces is legacy
<smoser> (as described in that do up there)
<kszarlej> thanks!
<paulmey> I have a case of vanishing tempdir... Any idea what might cause this? http://paste.ubuntu.com/25587235/
<smoser> kszarlej, if you're able, its best to provide the mac address of the nic
<smoser> that way cluod-init can
<smoser> a.) rename the nic to whatever you want
<paulmey> Is systemd mounting tmpfs on /tmp while cloud-init is in the middle of something?
<smoser> b.) actually identify it, rather than you guessing the name that the kernel will give it.
<smoser> paulmey, yes
<smoser> sort of
<smoser> paulmey, https://bugs.launchpad.net/cloud-init/+bug/1707222
<ubot5> Ubuntu bug 1707222 in cloud-init "usage of /tmp during boot is not safe due to systemd-tmpfiles-clean" [High,Confirmed]
<smoser> not "mounting", but deleting
<smoser> robjo, i'm going to pull the sysvinit changes for you
<paulmey> hmmm... nasty
<paulmey> smoser: thanks. That will be in the upcoming release, I assume?
<smoser> yeah, i just marked it fix-committed.
<paulmey> awesome thanks
<smoser> note, it had fallout though
<smoser> we changed to using /run/cloud-init/tmp
<smoser> so that systemd wouldnt think it should just delete things for us
<smoser> and that broke because /run is (typcially) mounted with 'noexec'
<smoser> and at times we need exec
<smoser> fun
<paulmey> sigh... sorry for your pain...
<smoser> i find the idea of deleting files under almost any criteria by a central daemon questionable.
<paulmey> but getting into earlier boot races is a good condition to be in... ;-)
<smoser> ie, there is stuff that will delete your files in /tmp if they're 30 days old
<robjo> smoser: thanks
<smoser> well... sometimes i put stuff there :)
<smoser> i dont plan ahead of time how long i'll want to keep something for
<paulmey> yeah... systemd is getting power hungry...
<harlowja> smoser wut
<harlowja> lol
<smoser> harlowja, cloud-init misses you
<harlowja> ya
<harlowja> i miss her to
<harlowja> lol
 * dpb1 hands smoser /home
<smoser> dpb1, i have ~/t/ for tmp files in /home that i think i might need later.
<smoser> but sometimes i just dont know.
<dpb1> smoser: I have a ~/tmp/
<dpb1> for same
 * smoser is 66% more effective than dpb1, but has to think about typing 'affective' or 'effective'
<dpb1> hahaha @ affective
<smoser> (googled that for kids last week. RAVEN: Remember Affect Verb Effect Noun)
<smoser> rharper, what do you think...
<smoser> https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/331140
<smoser> robjo's merge above
<smoser> we can drop the duplicated 'Default user' section (line 31-44) with this change instead
<smoser> http://paste.ubuntu.com/25587352/
<smoser> ie, the only change that robjo put in there was different default groups
<smoser> blackboxsw, what do you think ?
<blackboxsw> was looking it over
<smoser> basically i'm asking about whether to "nested loop" or "copy block"
<smoser> er...
<smoser> "nested if/else"
<blackboxsw> smoser: it looks like it's the same content as centos/rhel/fedora too
<smoser> ?
<smoser> my diff should make it clear what is different
<smoser>  http://paste.ubuntu.com/25587416/
<blackboxsw> ahh didn't know your were talking about a pastebin. I was on robjo's branch
<blackboxsw> yeah looking now
<smoser> my pastebin there is on top of robjo's branch. (and *this* pastebin doesn't use undefined variables)
<smoser> which is a nice feature i think (the last one spelled variant wrong)
<blackboxsw> +1 on that pastebin as I was thinking kinda the same, we didn't want to copy a separate block for suse as it's nearly the same as rhel etc
<blackboxsw> yeah I like that fix
<blackboxsw> s/fix/patch
<blackboxsw> gotta go for kid pickup
<smoser> ok. then i am going to pull that also.
<smoser> with my change there.
<smoser> robjo, are you there ?
<smoser> i do have some questions though on that mp.
<robjo> smoser:  on the phone again
<smoser> welll, please when you get done take a look at my comments.
<smatzek> Does multiple interface configuration work for anyone using the OpenStack datasource on Xenial using 0.7.9?
<smoser> i'd expect it to work.
<smoser> ah.
<smoser> with config drive
<smatzek> it works with config drive but not the OpenStack datasource.
<smoser> yeah
<smoser> er.. yes. that is what i would expect.
<smatzek> why is that?  A bug? I've dug through the code and have some ideas.
<smatzek> cloud-init successfully reads the network json: 2017-09-21 13:07:58,001 - url_helper.py[DEBUG]: Read from http://169.254.169.254/openstack/latest/network_data.json (200, 626b) after 1 attempts
<robjo> smoser: I did respond on the inline comments, hmm not sure what I screwed up that they didn't arrive on your end
<robjo> basic.target vs. sysinit.target
<smatzek> but then says this: stages.py[INFO]: Applying network configuration from fallback bringup=True
<smatzek> likely because the OpenStack datasource doesn't have the network_config() property in 0.7.9 and doesn't override it from the base class at master in github either
<smatzek> https://git.launchpad.net/cloud-init/tree/cloudinit/stages.py#n623
<smoser> robjo, maybe you didn't hit 'submit
<smoser> r... 'Save comment'
<smoser> i dont see them. i'm sorry
<robjo> probably :(
<robjo> I bet I cannot get back there now, darn.....
<smoser> smatzek, right now, only "local" datasources can provide network data.
<smoser> this is because the point at which cloud-init.service (non-local) runs is too late in boot to declaritively set up networking.
<smatzek> ah, I suppose because you don't want to depend on the networking to come up to allow metadata service gets to then configure networking....
<smoser> and we don't currently have any configuration of hotplugged networking
<smatzek> thanks!
<robjo> smoser: do you see the comments now?
<robjo> They were not lost that's nice
<smoser> the real problem woudl be that at cloud-init.service point in boot osmehting could already be using the network (and rightfully so, as network.target is reached)
<smoser> so taking it down to fix it would be difficult and possibly break things
<smoser> so it is as it is right now, smatzek but we do have intent on fixing feature to address it
<robjo> Well, that's not intuitive, I comment in-line but hen have to click "Save-Comment" for an empty text field....
<robjo> something learned
<smoser> robjo, yeah, launchpad has saved me there before :)
<rharper> smoser: +1 on the smaller delta to the config template for suse
<smoser> robjo, ok. if you're ok with my suggested changes, then i'll make some small comments in the template based on your responses and pull it
<smoser> my only concern is that you're never going to figure out why basic.target v sysinit.target so i'm accepting this technical debt when what i should tell you to do is properly justify it
<robjo> smoser: sounds good thanks
<robjo> I will put a card on my trello board to re-visit this topic before the end of the year
<blackboxsw> back
<smoser> robjo, http://paste.ubuntu.com/25587623/
<smoser> that is my total diff against your current branch (73acbf4a00741f8)
<smoser> seem sane ?
<robjo> smoser: yes, thanks
<blackboxsw> my heart goes pitter-patter when I hear others use trello. /me â¥'s agile w/ trello.
<blackboxsw> ... yeah I don't get out much
<rharper> lol
<smoser> but look at the mastery of unicode usage. ð
<blackboxsw> heh
<robjo> the world was much simpler without .decode() ;)
<rharper> f = 'ð''
<rharper>   File "<stdin>", line 1
<rharper>     f = 'ð''
<rharper> SyntaxError: EOL while scanning string literal
<rharper> so masterful
<smoser> blackboxsw, rharper robjo i'm looking at
<smoser>  https://cloud.google.com/compute/docs/storing-retrieving-metadata#querying
<smoser> and then planning on making sure that goes thorugh an integration run and then change versions and tag
<rharper> smoser: did you mean a code link not gce metadata ?
<robjo> yes, puzzled...
<rharper> smoser is having cut-n-paste fun after switching main browsers
<robjo> but did I tell you about gcemetadata ;)  https://github.com/SUSE/Enceladus/tree/master/gcemetadata
<blackboxsw> I recall from the summit, thanks for the link!
<smoser> i meant
<smoser> https://code.launchpad.net/~arnd-arndnet/cloud-init/+git/cloud-init/+merge/331148
<blackboxsw> btw /run/cloud-init/instance-data.json is queued for review an will land post this release
 * blackboxsw wants to get a branch up for DataSource.crawl_metadata ++ cloud-init devel crawl-metadata 
<smoser> powersj,
<powersj> yo
<rharper> smoser: sounds like a plan
<smoser> once i push above, i'd like to push a branch candidate/17.1
<smoser> then i'd like to run through integration tests on that
<smoser> and also as much other stuff as possible
<smoser> then just pull that.
<powersj> ok
<smoser> what all do you recommend we run and how can we do that ?
<powersj> honestly we should be doing all the tests we say we will run for an SRU exception: https://wiki.ubuntu.com/CloudinitUpdates
<powersj> so touching each of the data sources
<powersj> for integration tests, run through all tests on LTS + devel via lxd
<blackboxsw> I could spin up a couple instances on azure, gce and aws to test too
<smoser> hm..
<powersj> `python3 -m tests.cloud_tests run --verbose --platform --os-name xenial --os-name artful`  should do the lxd side
<powersj> woops forogt the lxd part
<powersj> python3 -m tests.cloud_tests run --verbose --platform lxd --os-name xenial --os-name artful`
<smoser> powersj, well, i'd like for jenkins to run that for me
<smoser> then there are some thing sthat ideally we'd test that dont (currently) take a branch
<powersj> you are looking for a release testing job that runs that based on a given branch?
<smoser> ie, testing xenial is actually much more valid to test the daily ppa
<smoser> which builds from trunk
<smoser> powersj, so yes, sort of :)
<smoser> what you said
<smoser> this doesnt have to be perfect now
<powersj> You know our current daily integration tests build for each distro and then run using the built deb right?
<smoser> well, do they ?
<powersj> yes
<powersj> https://github.com/canonical-server/jenkins-jobs/blob/master/cloud-init/integration.yaml
<smoser> do they use the packaging branch ?
<powersj> no they use trunk
<powersj> I sbuild each
<powersj> I do not have direct testing on your release branches, only master
<powersj> sounds like a test gap
<smoser> right.
 * powersj adds a card for release branch testing
<smoser> it'd be best for us to test those daily ppas or some other way build trunk + ubuntu packaging
<smoser> and test the result.
<powersj> I think I am testing the latter option, right?
<robjo> Our image build system has been terribly slow the last couple of days. I had hoped to have some rudimentary test results on SLES in EC2 by now but well, no images, no testing.....
<robjo> however rather than pulling master and pushing more on the long queue I'll just wait until this round goes through that should be a reasonably close approximation
<smoser> powersj, i suspect you're not testing ubuntu packaging
<smoser> i think its just using packages/bddeb
<smoser> which builds trunk's packaging
<powersj> can you define what you mean by that... to mean ubuntu packaging == debian directory
<smoser> which is missing some patches
<smoser> yeah, the debian/ directory in the ubuntu/xenial branch
<powersj> so I'm not testing the specific release patch sets
<smoser> right.
<powersj> ok table that for next week?
<smoser> from a "we are upstream" perspective, its arguable that that is fine.
<smoser> yeah
<powersj> more immediately you want a jenkins job to run lxd backed integration tests that i proposed above?
<smoser> powersj, yes
<powersj> ok
<smoser> but it will just do that
<smoser> right ?
<smoser> if i push a branch for review
<powersj> The review ill run 2-3 integration test cases on xenial only
<smoser> powersj, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/331161
 * smoser wonders if that will pass check_version in c-i
<blackboxsw> smoser is that using log2dch?
<powersj> heh we will find out shortly :)
<smoser> blackboxsw, it did, yeah
<blackboxsw> I just pushed that into our cloud-init/qa-scripts repo
<blackboxsw> so we have a copy of it in non-gist format :)
<smoser> i un-indented
<blackboxsw> figured we'd collect any tools we are using for our varios dev practice in powers
<blackboxsw> figured we'd collect any tools we are using for our varios dev practice in powers' shared repo
<blackboxsw> various even
<blackboxsw> bak 2 typoe skool
<powersj> :D
<powersj> ok created two jenkins jobs for release based testing, given a repo + branch it will run all LXD tests, one for xenial, one for artful.
<powersj> smoser: build passed
<powersj> install deb: "cloud-init_17.1-0-gbf6456f-1~bddeb_all.deb" into target
 * blackboxsw runs that branch on aws 
<powersj> smoser: are we really still including the git tag?
<smoser> powersj, ?
<powersj> when you release will the version string have "-gbf6456f" or some revision string in it and not just "17.1"?
<smoser> the version string in the debian package will have that.
<smoser> hm..
<powersj> CI passed
<powersj> running integration tests
<blackboxsw> smoser: python-jsonschema needs to not be required by the package on that branch
<blackboxsw> dpkg: dependency problems prevent configuration of cloud-init:
<blackboxsw>  cloud-init depends on python3-jsonschema; however:
<blackboxsw>   Package python3-jsonschema is not installed.
<kszarlej> Hey. I am having an exception when trying to use `network-config` with NoCloud cloudinit that is suppose to provision my libvirt instances. I tried both version1 and version2 of the network-config. Details can be found in http://paste.ubuntu.com/25588011/ (both configurations and an error)
<smoser> blackboxsw, so the debian packages built from trunk will get -0-gXXXX the ones that go into ubuntu will have
<smoser>  17.1-0ubuntu1
<smoser> but then the first time we grab a snapshot again we'll get
<smoser>  17.1-0-gXXXXX-0ubuntu1
<smoser> blackboxsw, "package on that branch" ?
<smoser> blackboxsw, trunk requires python3-jsonschema
<smoser> xenial patches that dependency out
<blackboxsw> sorry, just a reminder that we carry a patch for no python3-jsonschema
<blackboxsw> yeah
<blackboxsw> ok good good
<smoser> this is one of the things that testing the daily builds from the ubuntu/xenial + trunk build would cover
<smoser> https://git.launchpad.net/cloud-init/tree/debian/patches/stable-release-no-jsonschema-dep.patch?h=ubuntu/xenial
<smoser> ist hte patch
<powersj> xenial lxd tests passed https://jenkins.ubuntu.com/server/view/cloud-init/job/cloud-init-integration-release-lxd-xenial/2/console
<blackboxsw> 2017-09-21 19:31:43,001 - util.py[DEBUG]: Cloud-init v. 17.1 running 'init-local' at Thu, 21 Sep 2017 19:31:42 +0000. Up 5.54 seconds.
<blackboxsw> 2017-09-21 19:31:43,523 - handlers.py[DEBUG]: finish: init-local/search-Ec2Local: SUCCESS: found local data from DataSourceEc2Local
<blackboxsw> ok looking good aws
<rharper> kszarlej: I've seen that error before; I can't find the bug but it should be fixed in trunk;  if possible you can test/try the daily cloud-init rpm https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/
<smoser> kszarlej, please do try trunk. i can't reproduce the error there.
<smoser> i suspect you're on centos/rhel based on python2.7
<kszarlej> yep
<kszarlej> 2 bad
<kszarlej> I have to modify the
<kszarlej> original centos cloud image
<smoser> http://paste.ubuntu.com/25588132/
<smoser> kszarlej, ^ shows how you can use a tool in trunk to more quickly iterate
<smoser> which probably would have been useful to you earlier
<smoser> :)
<blackboxsw> 2017-09-21 19:45:36,396 - util.py[DEBUG]: Cloud-init v. 17.1 running 'init-local' at Thu, 21 Sep 2017 19:45:36 +000
<blackboxsw> 0. Up 9.36 seconds.2017-09-21 19:45:39,401 - handlers.py[DEBUG]: finish: init-network/search-GCE: SUCCESS: found network data from Dat
<blackboxsw> aSourceGCE
<blackboxsw> https://www.irccloud.com/pastebin/7XrX4aVw/
<blackboxsw> gce looking good
<blackboxsw> and now to remember my azure creds
<rharper> 7a3nf07sf
<rharper> something like that =)
<powersj> blackboxsw: created card and added to you
<powersj> blackboxsw: can you add links to your pastebin results there?
<blackboxsw> I'm pretty good w/ ctrl-v. let's see :)
<blackboxsw> thx powersj
<powersj> https://trello.com/c/sd5qvgJx/428-171-release
<rharper> blackboxsw: maybe you can help smoser; there was some ctrl-v/ctrl-p trouble earlier
<blackboxsw> hahaa
<smoser> in firefox with pentadactyl i could just hit 'y' rather than ctrl-v.
<smoser> but cvim doesn't seem to work as well. thats supposed to work i think, but doenst sometimes. in chrome.
<rharper> vimium!
<dpb1> vimperator
<rharper> did that get ported to chrome ?
<rharper> it used to be ff only (*years* ago )
<dpb1> idk, actually
<dpb1> ya
<dpb1> my cube neighbor at hp was hardcore vimperator user
<rharper> sadly, it's not as complete as vimperator was (input boxes) so I've added also wasavi (which is inputbox only vim support)
<rharper> but the link/tab highlighting is super cool in vimium
<smoser> pentadactyl was well better follow-on to vimperator
<smoser> but is kind of dead.
<smoser> https://github.com/5digits/dactyl/issues/99
<smoser> google seemed to think that cvim was better than other options for chrome
<smoser> and wasavi is pretty nice. but can't read or write to files.
<rharper> yeah
<powersj> artful lxd tests passed https://jenkins.ubuntu.com/server/view/cloud-init/job/cloud-init-integration-release-lxd-artful/2/console
<powersj> running nocloud kvm
<blackboxsw> azure clean boot test completed
<blackboxsw> ok, looks like that'll cover it w/ powersj' test
 * blackboxsw remebers to teardown my azure resource group
<powersj> smoser: ok got coverage on ec2, azure, aws, nocloud, lxd
<powersj> +build +ci
<powersj> +MAAS Compatability Testing
<blackboxsw> I use wasavi too. ... though I keep accidentally triggering it
<dpb1> same
<dpb1> it's also not the same
<smoser> https://public.etherpad-mozilla.org/p/cloud-init-17.1
<kszarlej> smoser: I upgraded to newest from daily rpm builds
<kszarlej> (v0.7.9)
<kszarlej> and still same error
<kszarlej> the exact version is cloud-init-0.7.9+286.gd3a8777-1.el7.centos.noarch
<smoser> kszarlej, hm..
<smoser> ah
<smoser> wait
<smoser> kszarlej, so...
<smoser> first thing
<smoser> is that 'network:' should not be present in the network-config file
<smoser> as its already namespaced to 'network'
<smoser> clearly better error would be ncie.
<smoser> but try out-denting that and dropping network:
<smoser> the 'version: 2.... i'm not sure
<rharper> https://bugs.launchpad.net/cloud-init/+bug/1708255
<ubot5> Ubuntu bug 1708255 in cloud-init "empty or invalid network config dictionaries are not handled well" [Undecided,New]
<kszarlej> ok trying to out-dent
<kszarlej> trying with version 1 synta
<kszarlej> x
<smoser> oh. and kszarlej you didn't have a mac_address in your v1
<kszarlej> smoser: I have
<kszarlej> network:
<kszarlej>   version: 1
<kszarlej>   config:
<kszarlej>   - type: physical
<kszarlej>     name: eth0
<kszarlej>     mac_address: 0e:a5:95:fd:a3:b2
<kszarlej>     subnets:
<kszarlej>       - control: auto
<kszarlej>         type: static
<kszarlej>         address: 10.10.1.120/24
<kszarlej>         gateway: 10.10.1.3
<kszarlej>         dns_nameservers:
<kszarlej>           - 8.8.8.8
<smoser> yeah, in your paste you didn't have {{ mac address }}
<kszarlej>           - 8.8.4.4y
<kszarlej> here is the exact config. (the y on the end is by accident)
<kszarlej> sry I should have used paste
<kszarlej> thats the exaact config I used now and it failed
<smoser> well you should not have 'network'
<smoser> remove l ine 1
<smoser> out-dent 2 spaces
<kszarlej> yep
<kszarlej> I will try that now
<kszarlej> is it possible to
<kszarlej> run cloud-init on host once again
<kszarlej> or I have to reproviosion it?
<kszarlej> (I will have to dd the original image to the VM disk and it takes time)
<rharper> if you have console, you should be able to restart the service (cloud-init-local)   you may need to remove /var/lib/cloud/instance/obj.pkl;  and of course update the the on-disk network-config file;
<blackboxsw> wouldn't sudo rm -rf  /var/log/cloud*log /var/lib/cloud; sudo reboot work too?
<smoser> blackboxsw, bah
<dpb1> blackboxsw: we have 'cloud-init clean' or something on our ideas, right?
<smoser> look at your /tmp
<smoser> see if you have ci-FakeExtendedTempFile*
<powersj> ci-FakeExtendedTempFile.b5pmLb
<powersj> ci-FakeExtendedTempFile.oqisp_15
<powersj> ci-FakeExtendedTempFile.pkggbqro
<powersj> ci-FakeExtendedTempFile.q9aq1dpv
<powersj> ci-FakeExtendedTempFile.uIPh1b
<powersj> ci-FakeExtendedTempFile.vq9_ldsb
<kszarlej> smoser: rharper finally
<kszarlej> it works
<kszarlej>  :)
<smoser> blackboxsw, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+ref/fix/fake-extended-tempfile-cleanup
<blackboxsw> bah /tmp
<blackboxsw> oops bah blackboxsw
<blackboxsw> not cleaning up my own tmpfiles
<smoser> blackboxsw, i might have regressed that for you
<smoser> i took that duplicated class out
<smoser> and put it at the top
<smoser> you probably had it reerencing  self.tmp_dir
<smoser> as i remember
<smoser> https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/331168
* smoser changed the topic of #cloud-init to: is Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj |  Next status meeting: Monday 10/2 14:00 UTC | cloud-init 17.1 released
<rharper> kszarlej: \o/
<smoser> kszarlej, so version 1 worked.
<smoser> version 2 should work also
<smoser> but if it works, it works.
<kszarlej> yup the out-dent helped
<kszarlej> :)
<kszarlej> thanks
<blackboxsw> smoser: still seeing the /tmp leaks
<blackboxsw> with the patch
<blackboxsw> smoser: nevermind. PEBKAC
<blackboxsw> approved
<blackboxsw> dpb1: I didn't know if we committed on that 'cloud-init clean'. We can have a branch up today if we want it. I think it sounds like a good idea, and it's simple to implement a basic approach. (We might end up tweaking it a bit with the re-rentrant cloud-init features). Certainly if all it is is wiping the cloud-init logs/lib dirs
<dpb1> blackboxsw: no, just curious
<dpb1> blackboxsw: it's a common testing pattern
<dpb1> would be nice to codify it
<blackboxsw> agreed
<robjo> blackboxsw: Could you take a look at https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/331149, need some help filling in the rest of the schema data, thanks
<blackboxsw> robjo: you made my week
<blackboxsw> I was just looking over schema stuff last night and thought that this is going to be a lot of work for the 50 remaining cloud-config modules
<blackboxsw> :)
<robjo> Well this is net new, so I am not really helping you get that work done, sorry. But at least I am not creating more ;)
<blackboxsw> +1 to that.
<kszarlej> ok there is one more problem. Cloud-init in NoCloud doesnt manage the resolv.conf. Both when setting in network-config for subnet as dns_nameservers: ['8.8.8.8'] and in meta-data
<kszarlej> manage_resolv_conf: true
<kszarlej> resolv_conf:
<kszarlej>   nameservers:
<kszarlej>     - '8.8.8.8'
<kszarlej>     - '8.8.4.4'
<kszarlej> in first case it should set the DNS1=8.8.8.8 in eth0 script but it doesnt actually.
<kszarlej> seems like the module is not activated on centos
<robjo> 17.1 is building in Cloud:Tools:Next smoser thanks for including all the new patches and for the help, hopefully I'll get to do some testing tomorrow.
<robjo> the https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/331149 will change the config format for SUSE repos, thus I am negotiating with the internal user of this feature whethere we will add the patch I am working on to the 17.1 build or keep the old patch and switch the next time
<robjo> Things are looking up I am 17 patches down from the previous package :D
<rharper> kszarlej: it looks like the sysconfig renderer doesn't handle the dns_nameserver entry under the subnets (eni does);  you can specify it as a top-level type: nameserver
<rharper>         - type: nameserver
<rharper>           address:
<rharper>           - 8.8.8.8
<rharper> kszarlej: if you have time, please file a bug; sysconfig render should handle both
<kszarlej> rharper: as top level in networking section
<kszarlej> or in cloud-config 'meta-data' ?
<rharper> at the same level as the type: physical
<rharper> under config: array
<kszarlej> btw. in centos7 /etc/cloud/cloud.cfg 'resolvconf' module is not listed under any of the 'stages'
<rharper> that's likely a but too w.r.t the which modules are run by default;
<kszarlej> w.r.t ?
<rharper> sorry, with respect to
<rharper> s/but/bug
<kszarlej> pl thx
<kszarlej> ok* ;p
<kszarlej> I will create bug reports tomorrow. Time to sleep. Gn8
<powersj> smoser: I usually run the integration tests by doing `python3 -m ...` versus tox... well since I added simplestreams as a dependency that broke tox. I don't see simplestreams in pypi.
<powersj> Becuase tox will only work with the lxd backend, my thought was to remove references to tox in the docs and remove the tox support. However, since you were pretty happy with that ability to run I wanted your thoughts.
#cloud-init 2017-09-22
<stl_> Hi all, does anyone know why cloud-init in Ubuntu 16 makes static network interface configuration instead of dhcp like in Ubuntu 14? Thanks
<stl_> Hi all, does anyone know why cloud-init in Ubuntu 16 makes static network interface configuration instead of dhcp like in Ubuntu 14? Thanks
<telling> I have both package_update and package_upgrade set to true in my cloud-config, now I expect it to update before it upgrades. But thats not the behaviour i'm seeing..Is this right?
<smoser> stl_, in 16.04, it will render network configuration that came from the cloud provider
<smoser> in 14.04, most likely you just got "dhcp on the first network device".
<smoser> telling, package_upgrade implies package_update at least on ubuntu. can you share a config and where you're running this ?
<smoser> also /var/log/cloud-init.log would be useful if you can paste that.
<smoser> powersj, sorry, responding to your last night comment.
<smoser> tox in my mind is somewhat broken for c-i due to lxd having c (and just the pain in building that way)
<smoser> wrt simplestreams, we can i think fix that. i have to check.
<stl_> @smoser yes, that's correct. I figure it out from interfaces.d/50-cloud-init.cfg that whole configuration is created. I would like to have it like in Ubuntu 14 -> dhcp on interfaces. I have tried to specify dhcp for network interfaces (version 1 and also version 2) http://cloudinit.readthedocs.io/en/latest/topics/network-config.html#network-configuration-sources
<stl_> @smoser i have upstream ubuntu images. Running OpenStack Ocata in our company
<stl_> smoser: and my main problem with cloud-init in Ubuntu 16 is that I have two interfaces (two diferent subnet) and I am getting two default routes configured ny cloud-init. With dhcp OS will install only one default route (that's correct)
<smoser> well, its impossible to install 2 default routes.
<smoser> so in 14.04, cloud-init woudl pick a interface via the hard coded name 'eth0'
<smoser> actually, cloud-init didn't do that.
<smoser> your image had to already be configured to do that. cloud-init didn't do any networking.
<smoser> so cloud-init in 14.04 completely ignored any information  your provider gave it.
<smoser> in 16.04, it listens to the provider's desired network configuration and tries to apply it.
<smoser> it is able to do that on OpenStack with ConfigDrive
<smoser> it will fall back to "dhcp on eth0" like behavior on OpenStack if there is no config drive.
<smoser> so, lets try to identify the problem with the routes
<smoser> then we can fix that and cloud-init will be working for you again (i think) in 16.04
<smoser> that make sense ?
<stl_> i will share ip r and 50-cloud-init.cfg in a few minutes.
<smoser> can you pastebin /var/log/cloud-init.log ? pastebinit /var/log/cloud-init.log
<stl_> smoser: cloud-init.log https://pastebin.com/yBx8Nq14  ;  ip r output after boot https://ibb.co/k0E7g5  ;  /etc/network/interfaces.d/50-cloud-init.cfg https://ibb.co/h2v2EQ  As you can see two defaults defined by cloud-init. Both networks are usual OpenStack virtual networks. First network advertise route 172.16.20.0/24.  Thank you a lot!
<smoser> stl_, you could have used 'pastebinit'
<smoser> :)
<smoser> literally type: pastebinit /var/log/cloud-init.log :)
<stl_> if i flush interfaces and run dhclient, then i get https://pastebin.com/Gh9kPdLx which is what i would expect. I have tried following cloud configs, but without success https://pastebin.com/7zb3tb1B
<stl_> ok :) i will type
<stl_> cloud-init.log http://paste.ubuntu.com/25592505/ :) sorry i didnt know about pastebinit
<smoser> its a very useful command. it can paste to other pastebins, but uses ubuntu by default. ubuntu is ad-free which is nice, but requires you to log in to get "raw" download (to avoid abuse)
<smoser> anyway... after that advertisement for pastebinit, i'm reading your pastes :)
<smoser> stl_, would you be able to get me the network_data.json file that is on the attached config drive ?
<smoser> cloud-inti doesnt log it, just what it converted it to.
<stl_> smoser: sure, could you please navigate me where it is stored?
<smoser> stl_, wow. i actuall thought that kernel/ip would fail if you tried to write two default routes (as in 'route exists')
<smoser> hm..
<smoser> hm... so there is a disk attached.
<smoser> run
<smoser> mount /dev/disk/by-label/config-v2 /mnt
<smoser> maybe..
<rharper> smoser:  while it's not in json format, looking at the v1 is easy enough to see: http://paste.ubuntu.com/25592551/
<smoser> rharper, yeah, but i wanted to get the real thing
<rharper> there are two nics, each one has a cateway, both are set as "Default"
<rharper> gateway
<smoser> in case we did a bad job converting
<rharper> that's fair
<smoser> i really thought you coudl'nt do 'ip route add default' more than once
<smoser> so https://ibb.co/k0E7g5 confused me :)
<rharper> we render eni here, though
<rharper> we have |: in those post-ups to deal with the artibrary order of the hook scripts
<rharper> interesting
<smoser> rharper, yeah, but his 'ip r' shows them
<rharper> well, maybe ip let's you shoot your self in the foot
<smoser> i didnt think the kernel would let you
<smoser> ie, i thought 'route exists'
<rharper> it's just a table
<rharper> that's not the kernel, that's the tool saying that
<smoser> i thought ip tried to add a route to table and got error
<smoser> thus "kernel"
<smoser> anyway.
<smoser> stl_, we really need the original network_data.json
<rharper> I think we can get it from the obj.pkl; lemme see on an openstack instance
<smoser> and if it says that both of those nics should contain the "default" route, then .... we're doing as close to what we we were told to do, and we'll probably need to file a bug in openstack.
<smoser> well, it shoudl be easily mountable too though. stl_ my 'mount' command above work ?
<rharper> y
<smoser> rharper, in your pasted config..
<smoser> we could have make a hueristic decision there
<smoser> as the first had a default route *and* a netmak'd route
<stl_> smoser: here's paste http://paste.ubuntu.com/25592580/ it's from configdrive openstack/latest/network_data.json
<smoser> oh wierd. but that .20
<smoser> wierd
<smoser> stl_, tyats what we needed. thanks
<stl_> smoser: great! thank you
<rharper> smoser: heh
<rharper> we just copy in the routes provided
<smoser> http://paste.ubuntu.com/25592599/
<smoser> ^ is just prettier formated of stl's paste.
<rharper> I'm not sure at conversion time we can make a choice;   I suspect the second network/nic need not have a gateway, but we don't know how the subnet in neutron was configured
<smoser> yeah. it is buggy.
<smoser> openstack is.
<smoser> they tell us there are two networks and give a default route on both. there isnt really a way to pick.
<smoser> hm..
<smoser> stl_, do you have access to the hypervisor ?
<stl_> smoser: yes I have
<smoser> it'd be good for you to file a bug with 'ubuntu-bug nova-compute'
<smoser> and then i can fill in some othoer info
<smoser> you open it, and then i'll fill in the rest.
<smoser> but basically they're giving us incomplete information
<smoser> cloud-init is doing what they said to do :)
<smoser> stl_, interstingly i think that if you ran dhclient on boht of those interfaces, you'd probably only get one route as default
<smoser> and that would be random based on which got done first
<stl_> actualy our hypervisors don't have access to Internet if 'ubuntu-bug nova-compute' is Ubuntu command
<smoser> unless they have that path somehow fixed and one of the dhcp servers would *not* give you that default route
<smoser> stl_, it is a command. it'd just collect some info on versions and things.
<stl_> smoser: yes, if i run dhclient i will get default only from second interfaces (that's what i want)
<smoser> "second"
<smoser> as in "second that your run" ? or as in "always network1"
<smoser> stl_, i'm kind of curious to see that. if you ran 'dhclient eth0' and 'dhclient eth1' and got the leases and the output of each. i'd be interested to see.
<stl_> smoser: I was always runing just "sudo dhclient"
<stl_> but i can try dhclient ens3 and then dhclient ens4
<smoser> stl_, you have a launchpad id ?
<smoser> i'll copy you on the bug.
<smoser> stl_, if you wanted... dhclient -v
<smoser> woudl be sufficient
<smoser> the grab the /var/lib/dhcp/dhclient.leases file
<stl_> smoser: https://ibb.co/bWHA15 got same default. Looks like if no route is advertised then default is installed.
<stl_> smoser: I have launchpad account
<smoser> stl_, i suspect that one of the two dhcp servers advertised a default route
<smoser> could you
<smoser>  dhclient -r
<smoser> then
<smoser> dhclient -v
<smoser> and then get /var/lib/dhcp/dhclient.leases ?
<stl_> sure
<stl_> console, dhclient -r, dhclient -v https://ibb.co/mZYTok  here's dhclient.leases http://paste.ubuntu.com/25592685/
<frickler> there is a bug against neutron that wants to allow not having a default route for networks: https://bugs.launchpad.net/neutron/+bug/1717560
<ubot5> Ubuntu bug 1717560 in neutron "allow to have no default route in DHCP host routes" [Undecided,New]
<rharper> interesting
<frickler> stl_: not sure if I missed that in the scrollback, could you also pastebin the output of "openstack network/subnet show" for your (sub)networks?
<stl_> frickler: will do
<rharper> smoser: looking at the leases, only ens4 has 'routers' option;  and there is 'classless-static-routes' for both;  I'm not sure how those relate to  setting routes on the system, but I wouldn't expose those to become a default route
<smoser> frickler, ! welcome . i was trying to think of who i knew that had some openstack networking chops
<smoser> and he stepped right into the room!
 * rharper is awed by the summoning powers of smoser 
<frickler> smoser: been lurking here for long :D
<frickler> just took a view after seeing stl_'s question over in #openstack ;)
<smoser> oh. ok. i'm opening a bug.
<frickler> also about to leave for the weekend, but I hope I can take another look later
<smoser> it does appear from his dhclient leases that the servers "know"
<smoser> frickler, you leave work to early (and i'm not accepting any excuse that includes the word 'timezone')
<smoser> :)
<stl_> here's openstack network and subnet pastes http://paste.ubuntu.com/25592742/
<stl_> smoser: if you need i can open bug on launchpad
<smoser> https://bugs.launchpad.net/nova/+bug/1718954
<ubot5> Ubuntu bug 1718954 in OpenStack Compute (nova) "network_data.json contains default routes for 2 interfaces" [Undecided,New]
<stl_> thanks!
<smoser> stl_, who are you there?
<smoser> Lukas or Dr. Jens ?
<stl_> smoser: Lukas :)
<smoser> stl_, if you're looking for a hack / workaround
<smoser> the easiest thing is probably to just delete the route
<smoser> ip route delete ...
<smoser> of the bad one
<stl_> smoser: yes, that's something i have came up also. I was hoping that cloud-config will work (set interfaces to dhcp), but no success from my side.
<smoser> stl_, well, you can't really affect networking confguration in user-data
<smoser> its just kind of "too late"
<stl_> :)
<smoser> stl_, you could provide some script to run that would re-write it, so it'd be fixed on reboot
<smoser> but you could just as well add a post ifup.d
<smoser> that took out that default route
<stl_> thanks for help and that bug report! Have a nice weekend.
<kszarlej> hi guys. I have that configuration for NoCloud drive. It is used for my LibVirt KVM VMs. The mount (vdb) is not mounted. The device is available at /dev/sdb and in /etc/cloud/cloud.cfg the mounts module is listed in init modules. There is no error in logs. Any suggestions?
<kszarlej> oh the configuration is here, forgot the link http://paste.ubuntu.com/25592915/
<kszarlej> the /data directory actually doesnt exist when booting but I assume that cloud-init should create it? (Creating it using runcmd is pointless as it runs after the mounts module)
<powersj> kszarlej: can you please look the two lcoud-init logs under: /var/log/cloud-init*
<powersj> smoser: rharper: https://bugs.launchpad.net/cloud-init/+bug/1718959 looks like this guy ran into an odd unicode character. Is there a good suggestion to help him debug his user-config?
<ubot5> Ubuntu bug 1718959 in cloud-init "user-data ignored in DataSourceNoCloud" [Undecided,New]
<kszarlej> powersj: found that
<kszarlej> http://paste.ubuntu.com/25593105/
<kszarlej> in cloud-init.log
<kszarlej> looks like the ci cant see the mount I have in config
<kszarlej> btw I use centos and a daily build from cloud init devel repo (because the upstream from centos repos is broken :P)
<kszarlej> rest of the stuff specified in meta-data was configured fine
<powersj> kszarlej: was the daily build from: https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/
<kszarlej> yes
<kszarlej> the one that is in centos 7 repos
<kszarlej> was unable to set the networking
<kszarlej> been consulting that yesterday evening here :P
<powersj> kszarlej: If you could, can you file a bug please.
<blackboxsw> I can't wait for us to get 17.1 into xenial so we can have  "ubuntu file-bug cloud-init".
<blackboxsw> it works currently on artful
<blackboxsw> I mean ubunt-bug cloud-init
<blackboxsw> I mean ubuntu-bug cloud-init
<blackboxsw> third time's the charm
* blackboxsw changed the topic of #cloud-init to: is Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj |  Next status meeting: Monday 10/2 14:00 UTC | cloud-init 17.1 released (artful)
* blackboxsw changed the topic of #cloud-init to: is Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj |  Next status meeting: Monday 10/2 14:00 UTC | cloud-init 17.1 released
<blackboxsw> I know that won't help us on centos, but it's a fun improvement for ubuntu
<blackboxsw> kszarlej: you can run cloud-init collect-logs to collect a tarfile of logs that can be attached to the bug
<blackboxsw> or even 'cloud-init collect-logs --include-userdata
<smoser> blackboxsw, well, kszarlej is using trunk
<smoser> so he can
<blackboxsw> trunk centos  right?
<smoser> trunk on centos
<smoser> (copr build)
<blackboxsw> he can "collect-logs --include-userdata" :)
<smoser> right
<smoser> ah. ubuntu-bug . i see . you were saying that wont work there.
<smoser> which is treu
<smoser> true even
<blackboxsw> glad someone has has a case of typos this morning
<smoser> kszarlej, is that you -> https://bugs.launchpad.net/cloud-init/+bug/1718959 ?
<ubot5> Ubuntu bug 1718959 in cloud-init "user-data ignored in DataSourceNoCloud" [Undecided,New]
<powersj> smoser: if you make a request of someone can you mark the bug incomplete?
<dpb1> yes
<dpb1> absolutely
<kszarlej> smoser: no
<kszarlej> I didnt create the bug yet
<kszarlej> trying to make a workaround first
<kszarlej> need to have that working today
<dpb1> powersj: there is room for judgement if the request isn't material to the working on the bug, but normally, you don't want to action a bug with incomplete information.
<powersj> dpb1: agreed: I just like to see new bugs that we respond to either moved to incomplete, invalid, or confirmed.
<kszarlej> ok now I lost my mind
<smoser> done
<dpb1> powersj: yes, it's a sane policy that we should try to stick to.  (or wont fix)
<kszarlej> http://paste.ubuntu.com/25593417/ and in log: 2017-09-22 16:32:00,901 - cc_runcmd.py[DEBUG]: Skipping module named runcmd, no 'runcmd' key in configuration
<ajorg> gah. why haven't I prepared https://git.launchpad.net/~ajorgens/cloud-init/commit/?id=aaa626dd8aecc078210d3a48ff39faf3fd30cbdc for merging yet. It seemed pretty important when I wrote it!
<blackboxsw> it's that "day job" thing... it just keeps getting in the way of good fun
<ajorg> seriously
<cesarjorge_> Hi
<cesarjorge_> I need help with cloud-init in an OS image of Centos 7.4 that I build
<ajorg> cesarjorge_: best to cut right to the chase and describe the problem you ran into
<cesarjorge_> I need to create this image to working without extra changes in the cloud or not cloud environments:
<cesarjorge_> I need that my image instantiate correctly if I launch it, for example, inside OpenStack, or, as a simple instance in local, I have virtualbox
<cesarjorge_> Then, if I launch the image in OpenStack, works correctly. But if launch the image in virtualbox (as a simple instance), the cloud-init does:
<cesarjorge_> Break my configured specific user with password, and takes many time to start
<cesarjorge_> And does other things
<cesarjorge_> Then, I need create a image that, if not launched in cloud, then do nothing and not interfere. If launch in a cloud, then execute as normal
<cesarjorge_> How can I do it?
<blackboxsw> ok schema suggestion for https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/331149 added robjo_away
<ajorg> cesarjorge_: you've tried, and it doesn't work? Or you're assuming it won't work?
<ajorg> IIRC for VirtualBox cloud-init will try to use the NoCloud data source: https://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html
<smoser> cesarjorge_, ajorg is right. cloud-init goes looking for a "datasource". and if it doesnt find one can be kind of annoying. you can provide it one in virtualbox.
<smoser> you can configure the datasource_list in the image to only consider OpenStack (or ConfigDrive) and NoCloud
<smoser> then it wont end up polling for the ec2 metadata service
<cesarjorge_> I have try, but dont work. Using datasources will work?
<smoser> and will fallback to the None datasource faster.
<smoser> well, "don't work" isnt much information.
<smoser> NoCloud will work in virtualbox for sure.
<smoser> https://asciinema.org/a/132013
<smoser> theres a ubuntu cloud image demo of doing this in qemu-system-x86_64 but the same basic thing will work on virtualbox.
<ajorg> "Break my configured specific user with password" you can probably configure it not to do, with a NoCloud source. "takes many time to start" sounds mysterious
<ajorg> https://cloudinit.readthedocs.io/en/latest/topics/modules.html#users-and-groups
<cesarjorge_> Then, I configure datasource_list: [ NoCloud ] ?
<cesarjorge_> In file cloud.cfg?
<ajorg> oh, you probably mean it take *much* time to start. That's fixed by limiting it to the data sources you know it might use, as smoser said.
<smoser> well, if you want it to work on penstack you'll need to put either ConfigDrive or OpenStack in there.
<smoser> (ajorg, i assume he's polling for the ec2 metadata service is what takes long)
<ajorg> smoser: I thought that was fixed by checking the system uuid?
<smoser> hm...... yeah.
<ajorg> but if he's on CentOS he's got an older version, probably. CentOS and RHEL are not aligned for cloud-init last I checked.
<smoser> you're right.
<smoser> yeah
<cesarjorge_> Version 0.7.9 for centos 7.4
<ajorg> Oh, maybe they fixed that. They used to be on 0.7.5.
<ajorg> I assume it checks for lots of other clouds though, and any of those takes time?
<cesarjorge_> I just tried using a image with line datasource_list: [ OpenStack ]. Inside virtualbox works good. But inside OpenStack fail
<cesarjorge_> Only if I boot the image without using no cloud task. Simply starting VM
<cesarjorge_> Question: What tasks does the None datasource do?
<cesarjorge_> This datasource is executed if no find any?
<ajorg> https://cloudinit.readthedocs.io/en/latest/topics/datasources/fallback.html
<smoser> none datasource doesnt do much. yes. basically generates ssh keys. will probably add the default user. and might lock its password (you could change that in config)
<smoser> frickler, just for your later reference
<smoser>  https://bugs.launchpad.net/nova/+bug/1718954
<ubot5> Ubuntu bug 1718954 in OpenStack Compute (nova) "network_data.json contains default routes for 2 interfaces" [Undecided,New]
<cesarjorge_> If I lock password, in config, then when I launch in openstack not work with ssh keys
<smoser> ?
<smoser> if you want a user that is already in the image and will always be accessible, just create one.  you can put ssh keys in it or not and set a password or not. cloud-init will not make any changes to that user.
<smoser> name it 'superdude' or something.
<cesarjorge_> If I launck in openstack, cloud-init modify the file /etc/ssh/sshd_config removing all passwordauthetication yes
<cesarjorge_> For all users
<cesarjorge_> If insert a specific line Match ssh user passwordauthentication no, then, does same thing, remove this line
<smoser> i'm going to go through fix-commnitted bugs now and mark fix-released for all in 17.1
<smoser> cesarjorge_, you can set that.
<smoser> i am not aware of ssh user passwordauthentication. you're saying that  is a sshd_config setting that you can set per-user stuff ? i didnt know and tis quite possible that cloud-init gets confused by that.
<smoser> you can tell cloud-init to either always enable password auth or not mess with it.
<smoser> ssh_pwauth: <yes/no/unchanged>
<cesarjorge_> In summary: I try to use same image for a cloud environment or isolate environment. In cloud environment I use cloud-init with ssh keys. In my isolate environment, i use one user with password, dont need the work for configurate extra things. Then, when the image works in cloud, dont work in isolate environment, and backwards
<cesarjorge_> The result in isolate env when I do ssh: no authentication method available, because cloud-init change this passwordauthentication in sshd config
<cesarjorge_> But, for openstack, I need this variable to do ssh public key for all users
<cesarjorge_> I can configure virtualbox for use ssh public key, but complicate the configuration for users than only need work with an isolate VM, using passwords, without extra configuration
<smoser> cesarjorge_, so you'd like for it to leave password authentication unchanged when running on a cloud
<smoser> but turn it off when running local
<smoser> er.. leave unchanged (off) when running on a cloud. but when run locally enable it.
<smoser> you can kind of do this i think if you backed config for NoCloud into the image that had its own config there.
<cesarjorge_> Passwordauthentication no, when cloud, passwordauthentication unchanged when no cloud
<smoser> wait.
<smoser> right
<cesarjorge_> Or, better, not execute nothing nothing if no cloud
<smoser> yeah. so you could do this by provding inside the image NoCloud config that would end up getting always found (in the event that it was not on a cloud)
<cesarjorge_> Better write: not change nothing in the instance if no cloud
<smoser> i think what you want is mostly how Ubuntu in 17.04 and later is set to work
<smoser> if cloud-init finds its not in a cloud, it disables itself entirely.
<smoser> so it should be acheivable with centos too. but i'm not sure how well the systemd-generator stuff is integrated tehre.
<smoser> cesarjorge_, i'm sorry, i can't spend any more time on this now.  i think what you're after is either doable or not far off.
<cesarjorge_> Workaround for centos? The RH/centos take a lot of tima in upgrade official versions
<smoser> cesarjorge_, you definitely need newer cloud-init. from trunk-ish
<smoser> we have copr builds
<smoser>  https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/
<smoser> those are of trunk
<cesarjorge_> Thanks, I will try this trunk version. If you encounter a workaround for 0.7.9,, i would be grateful
<robjo> blackboxsw: thanks
<blackboxsw> np robjo
<robjo> on the schema, I guess the basic question would be do we want to validate everything
<robjo> that would imply  that yes we'd have to document everything at https://en.opensuse.org/openSUSE:Standards_RepoInfo
<robjo> but that appears overkill to me
<robjo> and of course total duplication
<robjo> blackboxsw: ^^^^^
<blackboxsw> robjo: yeah it also forces you into a high-maintenance world where cloud-init was out of date as format moves on
<robjo> yes, thus it would be much nicer if the validation can somehow throw up it's hands and say "Ok, I know you need "baseurl" for everything else I don't care"
<robjo> is that an option?
<blackboxsw> as you saw with the config example, the permissive approach would be the following:                 'config': {
<blackboxsw>                     'type': 'object',
<blackboxsw>                     'description': 'Any supported zypo.conf key is written to /etc/zypp/zypp.conf'
<blackboxsw>                 }
<blackboxsw> yeah per the repos structure it could be the following if all you care about is baseurl
<blackboxsw> one sec, will paste
<robjo> And then the "id" is required becauase we cannot validate "any_random_name:" right?
<blackboxsw> http://pastebin.ubuntu.com/25594124/
<blackboxsw> yeah like this robjo
<blackboxsw> yeah otherwise jsonschema can't describe the subschema under <any_random_name>: {...}
<blackboxsw> I will take an action as I keep adding more schema though to see if there is a sensible way to describe that in jsonschema
<blackboxsw> and if so, to send a patch and ping you... admitedly I'm still a newbie at jsonschema
<robjo> Well we need multiple ids though such that a user can add as many repos as they want
<robjo> so repos is really repos :{ [id, baseurl,....],[id baseurl,...],[id, baseurl,....]}
<robjo> I am OK with calling it id, as this is an incompatible change we just need to nail it down and then leave it, whatever it is
<robjo> So how do we express the list behavior?
<blackboxsw> robjo: oops type in the schema.. one fixing the paste
<blackboxsw> will account for that
<robjo> and what does it look like in yaml?
<ajorg> robjo: I'm missing some context (sorry for jumping in here). What are you working on?
<robjo> Trying to figure out what the repo definition shoud look like, https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/331149
<blackboxsw> http://paste.ubuntu.com/25594201/
<blackboxsw> ok I'm looking over the following ^ as a potential it defines repos as a list w/ min items 1
<blackboxsw> and declares that each item must have baseurl and id in it.
<blackboxsw> other attributes are allowed
<ajorg> robjo: interesting. I thought the spec was identical for zypp and for yum?
<ajorg> just the file location was different?
<robjo> blackboxsw: that works for the schema I don't think this can be expresed in yaml
<robjo> ajorg: the repo description is different, and there is the desire to change things, i.e. th ecurrent implementation for RHEl is not what we want to do
<ajorg> okay, cool
<robjo> blackboxsw: That works for schema validation, but cannot be expressed in YAML, I think
<ajorg> reminds me i have a patch that helps make sure yum config is correct, but it's currently buried in another patch, need to tease that out
<blackboxsw> robjo: drawing up an example and testing w/ your branch
<robjo> The point of the <arbitrary_name>: is to give yaml a separator and each is unique thus the result is a dict with {arbitrary_name1:{....], arbitrary_name2:{....}}
<robjo> but if we say items: then there's no way to differentiate for the yaml parser and the result is undetermined
<blackboxsw> robjo: like this? http://pastebin.ubuntu.com/25594260/
<blackboxsw> can't your module validate 'id' on each object is unique?
<blackboxsw> or this http://pastebin.ubuntu.com/25594275/
<robjo> yaml bitches at that, looking at the error
<blackboxsw> hmm here's on my side http://pastebin.ubuntu.com/25594296/
<robjo> yes, without the {} I got it to work as well
<blackboxsw> ahh checking the {} again to see if I can get that working
<robjo> But that works, so I'll run with that
<blackboxsw> robjo: yeah less characters than the other way
<blackboxsw> here's the other http://pastebin.ubuntu.com/25594320/
<blackboxsw> had to fix a bunch
<robjo> Thanks, I like the - better I'll use that as example ;)
<blackboxsw> I'm still trying to get the suggestion for your schema to work... I iterate w/ python3 -m 'cloudinit.cmd.main devel schema --doc' and ... devel schema -c <sample-cloud-config>
<blackboxsw> and initial patch  for schema http://paste.ubuntu.com/25594361/
<blackboxsw> robjo: ^
<blackboxsw> you'll also need to git fetch; and get rebase origin master as we put in a docs fix related to schema yesterday
<blackboxsw> that helps rendering
<robjo> blackboxsw: great thanks, I'll get that included and hopefully will get the test written yet his afternoon, for another full review
<blackboxsw> +1
<robjo> next week is SUSECon I'll have little to no time....
<blackboxsw> I'd imagine, we're heading for a sprint next week too so response times may be slow
<smoser> blackboxsw, welll.. comments on https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/330115
<smoser> robjo, i think my invitation to SUSECon got lost in the mail.
<robjo> smoser: and Prague is such a nice place
<smoser> yeah.
<smoser> blackboxsw, i commented on your json file stuff
<blackboxsw> thanks smoser will look at what we think should be done there
<blackboxsw> smoser: good points, yeah I stopped work on this when we were trying to get 17.1 stuff in. Forgot that I didn't get back to work out the standardized top-level keys vs datasource blobs
<blackboxsw> I had debated about iterating on standard/nornmalized metadata keys in instance-data.json on a followup branch. But, I guess we can implement low-hanging fruit in this branch. The problem with normalized/general keys is that we may have to have a ds.normalized_metadata  function that each datasource may have to implement to surface the common named keys if they don't already have it in meta/user-data
<smoser> agreed
<robjo> how do I run just one test? while I am working on the tests for zypper_add_repo I do not really want to run the whole test suite over and over again
<robjo> and next question how do I populate the tmp location with a file that can be read by a test?
<smoser> robjo, fastest thing is to use ./tools/tox-venv
<smoser> $ ./tools/tox-venv py3 python3 -m nose tests/unittests/test_util.py
<smoser> if you just want to run your tests and dont want to use tox-venv or it didnt work for you
<smoser> $ tox -e py3 tests/unittests/test_util.py
<smoser> tox-venv skips the "install" phase which is slow.
<smoser> to populate a tmp location...
<smoser> probably easiest is, if youre subclass of CiTestCase
<smoser>  tmp = self.tmp_dir()
<robjo> right now, i.e. in the pending MR it is: class TestConfig(helpers.FilesystemMockingTestCase):
<smoser>  helpers.populate_dir(tmp, {'etc/passwd': content_for_that, 'usr/file1': more_stuff})
<robjo> great, thanks
<smoser> you want to do the populating before you patchUtils()
<smoser> or it gets in the way
<smoser> err... rather than patchUtils
<smoser> probably patch.reRoot() is what youw ant
<smoser> but do your pouplation of things *before* that
<blackboxsw> meh all systemd dhcp lease parsing content/handlers are defined in internal header files.
<blackboxsw> private dhcp options are parsed and added to internal lease objects
<blackboxsw> but I don't yet see a path to expose that content
<blackboxsw> FilesystemMockingTestCase subclasses from CiTestCase so self.tmp_dir() tmp_path are available there too
<blackboxsw> hrm per systemd: sd_dhcp_client_get_lease might be what I want
<robjo> smoser: is there already a utility function in the test infrastructure to confirm that warning messages get written or do I need to do that via @patch?
<blackboxsw> smoser: rharper  :) http://pastebin.ubuntu.com/25594875/
<blackboxsw> I think we can get to systemd custom dhcp options
<blackboxsw> busctl FTW
<blackboxsw> checking other instance types
<rharper> blackboxsw: whoa
<rharper> busctl needs a --json *so* bad
<rharper> I've not looked at it yet, but there is python systemd module
<rharper> maybe there's some easy access to the systemd dbus
<blackboxsw> yeah it's nasty
<blackboxsw> with custom types etc
<rharper> oh, that's NetworkManager
<rharper> org.freedesktop.systemd1
<blackboxsw> yeah unfortunately /org/freedesktop/network1 gives us nothing of use. it describes the systemd network files used, but nothing as far as dhcp or network related config settings
<smoser> blackboxsw, but that comes from networkmanager
<smoser> not systemd
<smoser> systemd-netwokrd
<blackboxsw> yeah, looking at extending systemd-networkd to expose options
<blackboxsw> yeah, looking at extending systemd-networkd to expose dhcp-options
<blackboxsw> just like NetworkManager does. it's a bit of a lift.... and my C is rusty
<smoser> horay for change for the sake of change!
<blackboxsw> heh
<smoser> we'd all be out of work otherwise.
<smoser> what would i have accomplished from 14.04 to 16.04 if not for systemd.
<smoser> nothing
<blackboxsw> heh
<blackboxsw> gotta stir up some dinner
<blackboxsw> have  a good weekend folks
<robjo> smoser: https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/331149 ready for another review.
<robjo> Have a nice weekend everyone
#cloud-init 2017-09-24
<ejb1123> Hello, I'm wondering if there is any way I can run my own metadata server?
<ejb1123> @smoser: Do you have any suggestion for how I can run my own metadata server.
#cloud-init 2019-09-16
<matthiasbaur> Hey powersj, may I talk to you in private regarding a CLA of the company I'm working for?
<Odd_Bloke> matthiasbaur: FYI, the cloud-init team (including Josh) are at a sprint this week, so he might take a while to respond here.  (You might be better off emailing him at josh.powers@canonical.com.)
<matthiasbaur> Sounds good! Thanks, Odd_Bloke! :)
<Odd_Bloke> Happy I could help! :)
<smoser> oops
<smoser> funny. that oops was actually an oops. it was in a buffer, and then i hit ctrl-m
<DavidGamba> Noob question, how can I clean cloud-init when creating an AWS AMI of Ubuntu 14.04. The `cloud-init clean` command is not available. I tried deleting `/var/lib/cloud` and it didn't work. modules:final didn't run.
<ahosmanMSFT> If your trying to clean the instance try
<ahosmanMSFT> rm -rf /var/log/cloud-init.log /var/log/cloud-init-output.log \ /var/lib/cloud/ /run/cloud-init/ /var/log/syslog
<ahosmanMSFT> delete the \ and it should work, I was using it with python and it line breaks '=D
<DavidGamba> I tried and `modules:final` still didn't run, had to run `cloud-init modules -m final`.
<DavidGamba> But I want to make sure it runs on its own
<DavidGamba> Haven't tried deleting syslog, will try that.
<ahosmanMSFT> Let me know how it goes. By the way, I believe most of the cloud-init team's in Paris. They'll be out this and some of next week.
<DavidGamba> Thanks for the help, it didn't work, it might be that the version of cloud-init on ubuntu 14.04 is too old. I don't have this issue with many boxes, just the one pet I can't upgrade!
<DavidGamba> I'll add a script to /etc/rc.local to run `cloud-init modules -m final`, hopefully that does it for this guy
<amansi26> I tried running the command cloud-init init --local and it fails with stages.py[ERROR]: Unable to render networking. Network config is likely broken: No available network renderers found. Searched through list: ['eni', 'sysconfig', 'netplan'] . Is there a fix for this?
#cloud-init 2019-09-17
<blackboxsw> rharper: minor suggestion https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/372828
<j4nd3r53n> Hi, I'm a total beginner with cloud-init, I'll probably ask some really elementary questions, just to warn you
<rharper> ask away
<j4nd3r53n> here's the first one: using terraform, I've set up an EC2 instance in AWS running Debian; it comes with cloud-init 0.7.9 installed. I have also uploaded a simple script to /var/lib/cloud/scripts/per-once, but it doesn't seem to get run at all (I've looked in the logs)
<j4nd3r53n> a
<j4nd3r53n> oops, big fingers it seem
<j4nd3r53n> is tehre a way to write code in this IRC?
<rharper> we use paste services, https://paste.ubuntu.com
<rharper> j4nd3r53n: typically we want to see the contents of /var/log/cloud-init.log ;  0.7.9 is quite old, you may be interested in seeing if you can upgrade to newer cloud-init in stable,
<rharper> cloud-init | 18.3-6         | stable       | source, all
<rharper> cloud-init | 19.2-1         | testing      | source, all
<rharper> cloud-init | 19.2-1         | unstable     | source, all
<rharper> per-once scripts run via runparts, so it usually only needs to be marked executable (chmod 0755 /var/lib/cloud/scripts/per-once/my-script-file
<j4nd3r53n> OK - I'll have to recreate the instance, I'm still working on getting this running
<j4nd3r53n> ah, that may be the problem
<j4nd3r53n> but it is a #cloud-config script? I'll make it runnable
<j4nd3r53n> I put it in pastebin: https://paste.ubuntu.com/p/jhGg26yKVg/
<rharper> ok
<rharper> ah, that's regular user-data; it needs to be provided to cloud-init via the platform's user-data include,  on ec2 you can include this via the launch too
<rharper> when using awscli, you pass it in like this --user-data file://my_script.txt
<rharper> or in the web panel, you can paste in your user-data
<rharper> https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html
<j4nd3r53n> yeah, I got as far as getting the file uploaded to the right place via terraform
<j4nd3r53n> but cloud-init ignored it - I'll try changing the permissions and see if it works; I'll let you know
<j4nd3r53n> yep, that worked like a charm! Thanks for helping me, really appreciate it
<rharper> j4nd3r53n: great!
<Odd_Bloke> blackboxsw: I've Approve'd https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/372828, do you want to re-review and Approved it if you're happy?
<blackboxsw> thx Odd_Bloke, clicky clicked all the approves
<rharper> Odd_Bloke: blackboxsw: thanks
<Odd_Bloke> rharper: blackboxsw: https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/372864
<smoser> DavidGamba:
<smoser> bah. sorry.
<blackboxsw> Odd_Bloke: rharper  new SRU branches https://trello.com/c/yqnRLAbY/1131-sru-cloud-init-19236-xenial-bionic-disco please review
<smoser> ugh.
<smoser> really need to make public jenkins public
<smoser> https://code.launchpad.net/~ruansx/cloud-init/+git/cloud-init/+merge/372445
<blackboxsw> smoser: +1 looking
<smoser> blackboxsw: i got it. and repllied
<blackboxsw> sorry smoser mid-sru upload and tripped on vpn connectivity
<blackboxsw> thx
#cloud-init 2019-09-18
<blackboxsw> 19.2-36-g059d049c-0ubuntu1~16.04.1
<blackboxsw> is what we are SRUing for cloud-init
<blackboxsw> trivial branch to fixup 'make clean' target
<blackboxsw> https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/372932
<blackboxsw> if anyone wants to weigh in
<andrein> hello, does cloud-init support some form of dry-run?
<andrein> I'm looking for a way to debug my network configuration(it's a baremetal node provisioned via ironic, with a configdrive, and every iteration takes... forever)
<smoser> andrein: network config can be rendered with
<smoser>  python3 -m cloudinit.cmd.main
<smoser> python3 -m cloudinit.cmd.main devel net-convert
<smoser> not the friendliest of all command lines
<smoser> but you're going to feed it '--network-data=some-file.json' --kind=network_data.json --directory=your.output.dir --distro=some-distro --output-kind=sysconfig
<smoser> or whatever values are appropriate
<andrein> thanks, smoser, that's exactly what I need
<andrein> cloud-init devel net-convert -p /mnt/config/openstack/latest/network_data.json -k network_data.json -d /tmp/ -D centos -O sysconfig
#cloud-init 2019-09-19
<blackboxsw> rharper: https://github.com/cloud-init/ubuntu-sru/pull/50
<rharper> thx
<andrein> hello everyone, i have the following network_data.json: https://www.irccloud.com/pastebin/fYPE11ed/
<andrein> the problem is, cloud-init doesn't configure my bond+vlan on first boot. however, using `cloud-init devel net-convert -p /mnt/config/openstack/latest/network_data.json -k network_data.json -d / -D centos -O sysconfig` the network configuration is applied correctly
<andrein> the one thing that stands out in cloud-init.log (https://seashells.io/p/pS8KbbuB) is `Applying network configuration from fallback bringup=True: {'version': 1, 'config': [{'subnets': [{'type': 'dhcp'}], 'type': 'physical', 'name': 'enp130s0f0', 'mac_address': '0c:c4:7a:1e:63:8e'}]}`
<smoser> andrein: cloud-init is not finding your config drive.
<smoser> it is falling back to datasourcenone
<smoser> not sure why, but some configuration is telling it that its datasource_list is just (i think) None
<smoser> possibly ds-identify did that... notsure.
<smoser> you can post a 'cloud-init collect-logs' tarball somewhere it will contain a lot more info
<andrein> smoser: it's complaining that it can't read `/run/cloud-init/result.json`. looks like a broken symlink
<andrein> i've ran cloud-init clean / clound-init init multiple times here, maybe i did something wrong at some point?
<andrein> is cloud-init supposed to read it's config drive directly from /mnt/config? how can it not find my config drive, it's right there :)
<smoser> andrein: can you just touch /run/cloud-init/result.json and then rujn it ?
<smoser> its supposed to be a symlink into /var/lib/cloud i think. but not much matter here.
<smoser> i'd like to see what that collect-logs says.
<andrein> I've noticed cloud-init status says it's still running, although I don't see any processes matching cloud
<smoser> if you want, you can let me in and i can poke a round a bit. if thats possible.
<smoser> where did you get this image?
<andrein> that's going to be complicated, unfortunately. but we can arrange a skype call or something and I can show you around if that's ok with you
<andrein> kolla-ansible built it for me usind dib
<smoser> well, try to get collect-logs to run
<andrein> i just got it running, gotta configure networking and i'll send it off
<smoser> or, in liue of that, i guess if you collect
<smoser>  /var/lib/cloud/
<smoser>   /run/cloud-init
<smoser>  /etc/cloud
<smoser> and
<smoser>  systemctl list-units | grep cloud
<andrein> is there something like paste.openstack.org for tarballs?
<smoser> you can base64 encode
<smoser> andrein: make sense ?
<andrein> smoser: sent you the link in private
<smooth_mess> is bugs where i submit a feature request?
<smooth_mess> there is a small chance that (in aws) attaching a volume to an instance will cause cloud-int "fs_setup" to fail; so a missing feature would be that there is no "wait" or "retry" functionality. So, to get around it, adding to 'bootcmd' the following: counter=0; while [ ! -b /dev/xvdf ]; do counter=$((counter+1)); printf "%s not found, tried %ds out of 300\n" "/dev/xvdf" $counter; if [ $counter -ge 300 ]; then printf "Did not find
<smooth_mess> %s" "/dev/xvdf"; exit 1; else sleep 1; fi; done
<smoser> smooth_mess: yes, submit a bug
#cloud-init 2019-09-20
<pager_lemon> Hi, I am trying to create an OVF template that will use 'cloud-init' to do a simple hostname change and run a script on a centos7 VM. The OVF/OVA will be deployed to VmWare ESXi from an Ansible playbook, which will supply the variables. Can someone help me with identifying what my cloud-init configuration should look like? (This is for a homelab
<pager_lemon> learning project...)
#cloud-init 2019-09-22
<nils_> Looking at the documentation, there seems to be no chapter about installing / upgrading cloud-init?
<nils_> also not a very googleable topic since most of it is about upgrading or installing things USING cloud-init.
