#cloud-init 2014-07-21
<smoser> harlowja, i'm gonna pull the templates change now
<smoser> thanks
<harlowja> smoser np boss
<harlowja> sounds good
<smoser> regarding the default emplater...
<smoser> thats interesting thoguht.
<smoser> oh. i guess one other thing for the fallback template
<smoser> would be to rip out '##" lines
<smoser> err. '##' lines
<smoser> i think jinja2 makes sense. since resolv.conf does have compex logic in it
<smoser> harlowja, one other thing is to replace the build dependency on cheetah
<harlowja> hmmmm, ya, replace ye-olde build dependency
<harlowja> smoser ya, like most of the template files should be fine with just the crappy templater
<harlowja> but can be changed later i guess, if we want
<smoser> harlowja, i'd like to have just one renderer used in the shipped templates
<smoser> ie, dont want to mix and match there. that just seems unnecessarily complex
<smoser> and since the built in wont work for resolv.conf...
<harlowja> kk, jinja2 it is then
<mwhudson> harmw: hello!
<mwhudson> i asked smoser about cirros on arm64 and he implicated you :)
<smoser> mwhudson, i have to run, but feel free to update me.
<smoser> amd harmw's work may not have been 64 bit. but https://code.launchpad.net/~harmw/cirros/cirros-buildroot-2014.02 might be relevant.
<mwhudson> yeah, i think an update to newer versions of things is basically what i want
<mwhudson> and while an official release would be nice, anything that behaves a bit more like a regular cloud image than the weird franken-thing i'm currently using would be nice...
#cloud-init 2014-07-22
<harmw> mwhudson: that updated buildroot branch is tested on x86_64 only, the arm bits aren't yet in place
<harmw> still need to get my ARM device to load up cirros in the first place
<Pursuit> anyone have experience with the chef cloud-config module? Having some trouble getting it to run right. the last bit of the traceback is "status, context = matchpathcon(path, mode)#012OSError: [Errno 2] No such file or directory"
<smoser> Pursuit, hm.. you can file a bug with info.
<mwhudson> harmw: hm, can i help somehow?
<mwhudson> harmw: e.g. can you tell me some commands to run that might make something arm64ish?
<mwhudson> i find buildroot pretty confusing
<mwhudson> smoser: maybe you could talk me through how cirros builds are supposed to work?
<smoser> mwhudson, well, to get there you'd have to update the buildroot to newer version
<smoser> and then get a config. 
<mwhudson> smoser: which is what harmw has done i think?
<smoser> well, he did update to newer buildroot, yes.
<mwhudson> i got it to the point of it asking me _lots_ of questions about the new config yesterday
<smoser> i dont know if that has arm64 or not
<smoser> https://code.launchpad.net/~smoser/cirros/powerpc
<smoser> that has some commits that got powerpc mostly functional
<smoser> so you could watch that.
<mwhudson> ah ok
<mwhudson> smoser: do the config questions require thought or can you just say "no" to almost everything?
<smoser> try 'make br-menuconfig'
<smoser> switch as few things as you tihnk you can get away with
<smoser> to make it 'arm64'y
<smoser> and then ... you'll have conf/buildroot-$(ARCH).config
<smoser> so ...
<smoser> make br-menuconfig ARCH=aarm64
<smoser> i'm not sure whta to call the arch
<mwhudson> that says things like
<mwhudson> /bin/sh: 1: cannot open /home/mwhudson/canonical/repos/cirros/cirros-buildroot-2014.02/conf/buildroot-arm64.config: No such file
<mwhudson> make: *** No rule to make target `/home/mwhudson/canonical/repos/cirros/cirros-buildroot-2014.02/conf/buildroot-arm64.config', needed by `/home/mwhudson/canonical/repos/cirros/cirros-buildroot-2014.02/output/arm64/buildroot/.config'.  Stop.
<mwhudson> i guess i need to read up on buildroot and stop flailing around
<mwhudson> smoser: calling the arch aarm64 would be a good troll :)
#cloud-init 2014-07-23
<KaaK> Can I bake some user-data into an image (EC2 AMI) such that I don't have to pass instances user-data when i instantiate them? I.E. does cloud-init also look for configuration on the local machine? If so, where?
<KaaK> I'm trying to take as much of the edge off of instantiating images for the rest of my team, and I really only need to take care of a small, unchanging set of cloud-init tasks
<guilload> Hi, I experimented a strange bug today with cloud init
<guilload> Stdin is redirected to /var/log/cloud-init-output.log via tee
<guilload> When using SaltStack salt
<guilload> and after a few minutes configuring my machines (AWS Ubuntu 14.04)
<guilload> all the processes stop working
<guilload> except for "tee"
<guilload> which keeps running at 100%
<guilload> for 10 long minutes
<guilload> and then everything goes normal again
<guilload> When I disable the redirection via tee to /var/log/cloud-init-output.log
<guilload> It works ok.
<CatKiller> smoser: Hi! Sorry I had been sidetracked on adding a "grow" feature to mount-image-callback
<smoser> guilload, hm..
<CatKiller> smoser: I've added it (well I've mainly added support to growpart and reuse growpart and resize-part-image from mount-image-callback, but I'm having some issues running chrooted commands using mount-image-callback
<smoser> guilload, i've never seen that. but it doesn't seem impossible.
<guilload> I've just fixed my problem by redirecting stdout and stderr to /dev/null
<agy> i'm adding metadata (not userdata) to an openstack instance that i've created using a config-drive. does cloud-init save this metadata somewhere? all i can find is a pickle file
<CatKiller> smoser: I'm just not sure how to run commands, if I follow the example: /mount-image-callback -v trusty-server-cloudimg-amd64-disk1.img -- chroot _MOUNTPOINT_ /bin/sh
<CatKiller> and add /mount-image-callback -v trusty-server-cloudimg-amd64-disk1.img -- chroot _MOUNTPOINT_ /bin/sh -c "ls /"
<CatKiller> it won't work (it'll complain about "-c")
<guilload> but I've taken a look at how the redirection is done to tee and it does look a little clumsy
<CatKiller> any idea on how to properly issue the chrooted command?
<smoser> agy, no. 
<smoser> i dont think so. maybe in the pickle.
<smoser> but not in a good format.
<agy> smoser: ok, that's what i thought. thanks
<smoser> the goal is to have have that sort of stuff in /run somewhere
<agy> it's in the pickle fwiw
<agy> that's where i expected it to be ;-)
<smoser> the goal is to put a /run/instance-data.json and /run/instance-data-private.json
<smoser> with the -private having things are known priviledged info
<agy> that would be ideal
<smoser> patches are welcome.
<smoser> in fact, you canprobably just ask harlowja_away to do it for you
<smoser> :)
 * agy was waiting for that
<agy> ^^ the "patches" comment
<smoser> guilload, how clumsy ?
<smoser> it might be, but its been used that way for quite a while. the default did change in 14.04 to redirect. but prior to that i always used that output definition and so did many other usres.
<smoser> i'm intereted in fixing it. can you file a bug with a reproducible example ?
<guilload> yes
<guilload> First, I will try to reproduce the bug :)
<smoser> CatKiller, hm.. example of chrooted commands issues ?
<smoser> CatKiller, hm.. it should be fine like that.
<smoser> you do need that '--' but mount-image-callback should respect it.
<CatKiller> smoser: Ah I wasn't sure I needed it
<CatKiller> smoser: In fact I think it's my bad, I have a 64bit image and I'm on a 32bit system....
<CatKiller> smoser: I should be OK once I get the 32bit one :p
<smoser> wow. you have a 32 bit system ?
<smoser> is your atari 2600 sitting on top of it ?
<CatKiller> smoser: Well in fact it's a 64bit machine I think, but it's one of the first AMD ever produced so it behaves better in 32bit ;)
<CatKiller> smoser: But yes the hardware we have at work here is laughable. I recently got an upgrade to 2GB of RAM from 512M
<CatKiller> :p
 * smoser sends CatKiller 512M dimm 
<CatKiller> I'll have to refuse them: all my DIMM slots are now full! :P
<CatKiller> My phone is more powerful than my dev machine now...
<CatKiller> actually not quite, I haven't upgraded to an iPhone 5 but if I did it would!
<CatKiller> smoser: Would there be an easy way to pass the contents of a script to the chroot command instead of relying on running /bin/sh in mount-image-callback?
<harlowja> smoser whats this, what code, lol
<harlowja> do waht, lol
<harlowja> i volunteer SpamapS  to do whatever it is, ha
<SpamapS> harlowja: challenge accepted. First step: rewrite it in go.
<harlowja> lol
<harlowja> damn it
<harlowja> SpamapS https://github.com/coreos/coreos-cloudinit u already did that, lol
<harlowja> smoser have u seen ^ ;)
<harlowja> looks similar right (but in go, haha)
<harlowja> except its to much hipster for me
<harlowja> lol
<smoser> yeah, i saw that a while ago.
<smoser> i'm having fun with that same thing right now
<smoser> that same thing being "doing the same thing only different"
<smoser> as i try to get cloud-init and systemd working on ubuntu
<smoser> horray for useful exercises in time
<smoser> </sarcasm>
<harlowja> :)
<harlowja> smoser are u just trying to convert the upstart stuff to systemd or is it more extensive than that?
<smoser> well..
<smoser>  https://code.launchpad.net/~smoser/cloud-init/systemd-enable/
<smoser> but in order to get to systemd functional, i had to do some other things... and then xnox helped and clean up some of hte packaging
<smoser> so its kind of snowballed.
<harlowja> :-/
<harlowja> fun fun
<smoser> harlowja, https://code.launchpad.net/~xnox/cloud-init/fix-systemd-install-paths/+merge/227918
<smoser> do you know why those were put in /etc ?
<harlowja> no idea
 * harlowja i haven't done much with systemd first 
<harlowja> waiting till u figure it all out ;)
<smoser> well, it blames to you
<harlowja> hmmm
<harlowja> odd
<harlowja> lol
<smoser> and your fix-everything-smoser-did branch
<smoser> oh well.
<harlowja> hmmm, lies
<harlowja> haha
<harlowja> fix-everything-smoser-did branch, haha
<harlowja> probably i didn't do it right, blame accepted
#cloud-init 2014-07-24
<harlowja> smoser btw, it seems like some parts of openstack are moving away from httpretty to https://pypi.python.org/pypi/httmock/
 * harlowja might have to dig into why to see if cloud-init should or shouldn't
<harmw> mwhudson: I think I have time for aarch64 (or whatever the correct name is) somewhere in and/or after the weekend
<mwhudson> harmw: yay
<mwhudson> i am on leave for a week so no real hurry:)
<harmw> oh lol
<harmw> you've got the original arm image working btw?
<mwhudson> haven't tried
<harmw> what kind of hardware do you run anyway, since it's aarm64 (which isn't widespread in the embedded/small world just yet)?
<mwhudson> i keep poking people about things and then forgetting that i'm away next week
<harmw> seamicro stuff?
<harmw> :P
<mwhudson> harmw: i work for canonical / linaro so have access to preprod stuff
<harmw> how nice
<harmw> :)
<mwhudson> yeah it's kinda fun
<mwhudson> can't really talk about it though, which is a bit annoying :)
<harmw> haha np
<smoser> harlowja, i do not understand something
<smoser> (big surprise)
<harlowja> ?
<harlowja> let me help you child
<harlowja> how may i help u child
<harlowja> j/k
<smoser> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/794/cloudinit/util.py#L148
<smoser> if stat.ST_MODE in stats
<harlowja> k
<harlowja> ^ u just not want that statement?
<smoser> what is it for?
<smoser> it doesn't seem tomake sense.
<smoser> stat.ST_MODE is 0
<smoser> (a constant)
<smoser> why would you say:
<smoser> if 0 in os.stat("/etc/passwd")
<smoser> what would that even mean?
<harlowja> agreed, wonder why i did that
<smoser> ok. thanks.
<smoser> that test was failing in sbuild
<smoser> where /etc/hosts didn't exist
<smoser> so i was making a temp file
<smoser> and then the tem pfile for whatever reason did not have a '0' in os.stat()
<harlowja> ya, seems like just do stats[stat.ST_MODE] 
<smoser> yeah, i think that is what youmeant 
<harlowja> must of been smoking crack when doing that
<smoser> but dont even know what that would do
<harlowja> ya, odd
<smoser> i have to run.
<harlowja> all the cracks fault
<smoser> for now i will just take that code out
<smoser> later
<harlowja> kk
<smoser> harlowja, can you verify the os.path.realpath and os.path.expanduser seem odd at http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/util.py#L148
<harlowja> don't seem so odd if u want the actual path?
<smoser> but why didn't they give you the actual path in the first place.
<smoser> what if they explicitly gave you a path with a '~' in it that should not have been expanded.
<smoser> i guess if you need the full path it makes sense.
<smoser> but the expanduser seems like it could false positive
<smoser> harlowja, http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/revision/987
<smoser> thanks for your help.
<harlowja> smoser ya, likely could just remove expanduser
<harlowja> smoser cool, selinux will i guess work better now
#cloud-init 2014-07-25
<mgagne> smoser: oh btw, I managed to backport cloud-init to precise without much problem. thanks to your help =)
<smoser> mgagne, you want to push a branch somewhere ?
<smoser> if you're interested in pursuing it, we can make a branch that we try to keep working there.
<mgagne> smoser: where would you like it to be hosted? I don't know bazaar and we were already planning on hosting it somewhere internally in git or publicly on github
#cloud-init 2015-07-20
<openstackgerrit> Daniel Watkins proposed stackforge/cloud-init: Use a registry to configure reporting handlers.  https://review.openstack.org/203093
<openstackgerrit> Daniel Watkins proposed stackforge/cloud-init: Implement a DictRegistry.  https://review.openstack.org/203094
<openstackgerrit> Daniel Watkins proposed stackforge/cloud-init: Make reporting handlers configurable.  https://review.openstack.org/203533
<Odd_Bloke> smoser: How do I build a package from lp:cloud-init (so I can check if my udev installation is working as expected)?
<openstackgerrit> Scott Moser proposed stackforge/cloud-init: improve test coverage  https://review.openstack.org/203249
<smoser> Odd_Bloke, ./packages/bddeb
<Odd_Bloke> smoser: http://paste.ubuntu.com/11909327/ is what I see when I try to use it.
<smoser> Unmet build dependencies: python3-configobj\ndpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting\ndpkg-buildpackage: warning: (Use -d flag to override.)\n'
<Odd_Bloke> Argh, I misread and thought it was complaining about UNRELEASED.
<smoser> it is a terrible error message
<smoser> agreeed
<smoser> if you have sbuild, i would normally do:
<smoser>  ./packages/bddeb -S -us -uc
<smoser>  sbuild ...
<smoser> but building on your system outside sbuild shoudl work also
<Odd_Bloke> https://code.launchpad.net/~daniel-thewatkins/cloud-init/fix-gce-az/+merge/263907 <-- smoser, review of that would be appreciated; I have another review that depends on it
<smoser> Odd_Bloke, partition rather than split ?
<smoser> thats all i have. and they're pretty much equivalent here.
<smoser> good enoug
<smoser> i think you can merge, right
<smoser> please update change log on merge
<openstackgerrit> Scott Moser proposed stackforge/cloud-init: improve test coverage  https://review.openstack.org/203249
<openstackgerrit> Merged stackforge/cloud-init: improve test coverage  https://review.openstack.org/203249
<harlowja> whats this launchpad thing, ha :-P
#cloud-init 2015-07-21
<Odd_Bloke> smoser: I've added the things you wanted in https://code.launchpad.net/~daniel-thewatkins/cloud-init/lp1411582/+merge/264831
<smoser> Odd_Bloke, ok. reading now.
<smoser> first hthing :
<smoser>  LOG.debug('Ensuring that we have a real device, not a symbolic link')
<smoser> that is pretty useless tatement.
<smoser> i think, no? it doesn't tell you anything.
<Odd_Bloke> smoser: Well, it tells you that (a) you have reached a particular point in the code, and (b) that if you are seeing a discrepancy between what you have configured and what the following log messages say that there might be a reason for it.
<Odd_Bloke> smoser: But I'm happy to improve it.
<smoser> Odd_Bloke, i'll merge, if you can do me a favor
<smoser> :)
<smoser> tox -e py26 is busted on 0.7 trunk right now.
<smoser> i believe due to mock changes.
<smoser> Odd_Bloke, i've fixed tests at https://code.launchpad.net/~smoser/cloud-init/fix-mock-tests/+merge/265410
<smoser> harlowja, or Odd_Bloke a review of my sanity there would be good.
<smoser> it really seems like upstream mock fixed a bug and now our tests that should not have passed actually do not pass :)
<harlowja> smoser u have been determined to not be sane
<harlowja> sanity review completed
<harlowja> smoser looks ok to me, i had similar mock fixups in taskflow
<smoser> you think your tests were bad and incorrectly passing also ?
<harlowja> smoser i know it :)
<harlowja> http://lists.openstack.org/pipermail/openstack-dev/2015-July/069156.html
<harlowja> smoser i even got taskflow mentioned in ^
<harlowja> lol
<harlowja> 'One of the improvements in mock is to fail when a bad method is called.'
<smoser> nice.
#cloud-init 2015-07-22
<Odd_Bloke> smoser: More review for you: https://code.launchpad.net/~daniel-thewatkins/cloud-init/fix-gce-az/+merge/265509 :)
<Odd_Bloke> smoser: That SSH key solution looks broadly good; would like it if X_2_Y could be renamed X_TO_Y; I think we can afford the extra character. ;)
<Odd_Bloke> smoser: But other than that, I think I'm happy with it.
<smoser> Odd_Bloke, thats fine. i started down several paths of re-writing that code entirely.
<smoser> but settled on only fixing the one that neededfixed.
<Odd_Bloke> Yeah, fair enough.
<smoser> i'm ok with your 2 fix though. i'll add that.
<smoser> Odd_Bloke, just read lp:~daniel-thewatkins/cloud-init/fix-gce-az
<smoser> that is really nice work.
<gamename> hi guys. for some reason, when I run cloud-init on my ami, the authorized keys I want go to '/root/.ssh/authorized_keys' instead of the '/home/foo/.ssh/authorized_keys' file.  How do I force that to happen?
<smoser> what OS ?
<gamename> CentOS
<smoser> i suspect that the default user is set wrong.
<gamename> 7.1.(mumble)
<smoser> err that it is set to root.
<gamename> smoser: That's what it is looking like.  I'm going to zap the /etc/cloud/cloud.cfg file. Will set the "name: centos" in the default to "name: foo".
<gamename> foo is the new default id.
<gamename> make sense, or am I on the wrong path?
<smoser> yeah, you want to set up like http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/config/cloud.cfg
<smoser> i'd take the packaged version, amek sure that it has default_user name set
<smoser> its possible the user-add is failing
<smoser> maybe youhave something in /var/log/cloud-init-*.log
<smoser> look for WARN
<gamename> smoser: ok.  Will verify in the log(s) as well.
<gamename> smoser: Other than the default user, are there other values in cloud.cfg that are required to change, or will just the name change be good enough in general?
<smoser> the mirror values should be adjusted and possibly ssh_svcname.
<smoser> mostly i'd go with what the distro provided you with and change it
<gamename> smoser: got it. Thanks;
<Odd_Bloke> smoser: Glad you were happy with it!
#cloud-init 2015-07-23
<patcable> question -- if I want to start a particular service before cloud-init in upstart, is there one good service to pick for that purpose?
<patcable> there's lots of options. should I just or them all?
<Odd_Bloke> patcable: cloud-init-local should do the trick, I think.
<NerdyBiker> Hey guys, getting an issue when using the cloud-init stuff on EC2, finding that the user running the script doens't have knowledge of their HOME dir... not sure if that's expected or I'm missing somehting.
<Odd_Bloke> NerdyBiker: cloud-init has a few different ways of running a script; could you paste the user-data you're passing?
<NerdyBiker> I'll get it now. The main bit is a chef run and that's the bit failing. somewhere in that process it requires the "HOME" environment varibale but that gets output as blank.
<Odd_Bloke> chef itself requires it, or your chef recipes?
<Odd_Bloke> NerdyBiker: (A copy of the error you're seeing would be good too :)
<NerdyBiker> Odd_Bloke one of the chef recipes requires the HOME to be set, here's a gist with the parts and brief explaination that should help https://gist.github.com/MattDevUK/7d4a7a52775a30706719
<Odd_Bloke> NerdyBiker: So my first reaction is that this is a bug in your Chef scripts rather than in cloud-init; but that might just be because then I don't have to do anything to fix it. ;)
<NerdyBiker> Odd_Bloke that might be the case, but this is the first time using cloud-init but that recipe is used on fresh boxes using the same AMI as this machine all the time. the echo at the beginning of the user-data outputs "home is set to ''" :(
<Odd_Bloke> NerdyBiker: I'm not especially familiar with this part of cloud-init, let me have a dig through the source and see what I find. :)
<NerdyBiker> it works fine if you log into the machine manually and run the chef-client. but then I'd expect if it was a cloud-init issue, it would've cropped up all the time before as I'd expect a lot of people to require the HOME dir when performing script sons tart up :P
<NerdyBiker> scripts on startup*
<Odd_Bloke> NerdyBiker: So it looks to me like cloud-init just runs scripts with the environment that it starts in itself.
<Odd_Bloke> NerdyBiker: So the problem you're hitting is that (upstart|systemd|...) doesn't set HOME in the environment of things that it runs.
<Odd_Bloke> s/problem/problem I think/
<Odd_Bloke> NerdyBiker: See http://askubuntu.com/a/394330/141343 for a bit more info.
<NerdyBiker> Odd_Bloke so it seems the issue is the user that executes the cloud-init hasn't tehcnically "Logged in" so their home would not be set? If I'm reading that right
<Odd_Bloke> NerdyBiker: Yep, precisely.
<Odd_Bloke> NerdyBiker: So I think you'll have to set HOME yourself; changing cloud-init to set it is probably opening a bag of worms.
<Odd_Bloke> Or perhaps I mean a bag of bugs. :p
<NerdyBiker> Odd_Bloke That's as I thought. so either we code around that, or possibly ask the aws guys how/why it's not being logged in properly.
<NerdyBiker> :P
<Odd_Bloke> NerdyBiker: It shouldn't be logged-in; it's run at startup by the init system.
<NerdyBiker> Ah yes,
<Odd_Bloke> (I assume; obviously I don't know what AMI you're using :p)
<NerdyBiker> yeah that sounds right, I thought it did some from of log in tuil I asked in ##aws today and they said it's just a stratup script that pulls the user-data in.
<NerdyBiker> Ah well, we'll work around it then :D Thank you for your help.
<Odd_Bloke> No worries, happy to help.
<Odd_Bloke> NerdyBiker: It is worth noting that you might be able to do what you want with the more complex form of user-data.
<NerdyBiker> Odd_Bloke oh? I'm listening :D
<Odd_Bloke> NerdyBiker: So I don't know a great deal about the Chef side of it, but http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config-chef.txt is an example user-data which does cheffy things.
<NerdyBiker> Odd_Bloke new to cloud-init so not familair with it all yet, will take a look. Thanks!
<Odd_Bloke> NerdyBiker: And a bit more info at http://cloudinit.readthedocs.org/en/latest/topics/modules.html#chef
<NerdyBiker> Odd_Bloke Ah, saw the first one before, but thought it was a config file due to the yaml. Didn't realise I could pass a yaml file as user-data.
<Odd_Bloke> NerdyBiker: Yeah, the key is the #cloud-config at the start; that tells cloud-init to process it as config rather than a shell script.
<Odd_Bloke> You can also do multi-part data, which would allow you to do the YAML config for some stuff and still pass in a script to be executed.
<NerdyBiker> Odd_Bloke Awesome, think I've got enough to go on from now, you've been super helpful :D
<Odd_Bloke> :)
<NerdyBiker> Odd_Bloke looking at the cc_chef module, would you know if it's possible to define which version of Chef to install? I can't seem to find anything referencing a version number anywhere
<NerdyBiker> well, apart from the gem method
<Odd_Bloke> NerdyBiker: I don't think it is; which method were you looking to use?
<NerdyBiker> omnibus ideally
<Odd_Bloke> NerdyBiker: https://bugs.launchpad.net/cloud-init/+bug/1462693
<NerdyBiker> Odd_Bloke awesome :D Should've looked there first I guess
<Odd_Bloke> NerdyBiker: That looks like it'd be fairly easy to fix, if you wanted to have a go. ;)
<NerdyBiker> Odd_Bloke Would be a way to get my head around how some it works and stop asking silly questions for sure. :P
<Odd_Bloke> NerdyBiker: You haven't asked a single silly question thus far!
<NerdyBiker> Odd_Bloke that's good
<patcable> Odd_Bloke: hmm, thanks. I did try to stick it before cloud-init-local but something else was going on. I may need to make sure I have the same start condition (I started on filesystem and cloud-init-local starting, but maybe i need to do runlevel & cloud-init-local)
<Odd_Bloke> patcable: My head is too much in systemd-land for me to be much more help.
<Odd_Bloke> patcable: smoser might be able to help you out, he's good like that.
<patcable> Odd_Bloke: that makes sense; a demo of something I'm doing is using 14.10 right now. just bumping from upstart to systemd at least wont be as bad
<Odd_Bloke> patcable: Utopic's EOL is today (https://wiki.ubuntu.com/Releases), so that's probably worth doing. :p
<patcable> ha
<patcable> we basically wrote a way to send encrypted data to cloud init so you could put private data into a user script, but it relies on a key being present somewhere before cloud-init is invoked
<Odd_Bloke> Oh, cool.
<Odd_Bloke> patcable: So how do you get the key?
<NerdyBiker> Odd_Bloke just to let you know, changed our user-data to use the cc_chef module, its cleaner but still stops at the same spot with the HOME dir, so going to have to manually set that variable as thought before :P
<Odd_Bloke> NerdyBiker: Great!  (Except for the HOME bit :p)
<patcable> Odd_Bloke: it's complicated. there's a process that is able to pull half the key from a verifier (that ensures the state of a trusted platform module is a okay) and the other half from openstack. The crypto folk I work with have a much better explanation, heh
<Odd_Bloke> patcable: Fair enough. :)
<smoser> patcable, running upstart 'start on starting cloud-init-local' should get you run before cloud-init's first job
<smoser> but you're basically guaranteed to not have network then
<smoser> which is probably not what you want
<smoser> start on starting cloud-init
<smoser> would guarantee you network, and in ec2 is what actually does things (as there is no local datasource for ec2, theirs requires metadta network accesss)
<patcable> ahh ok
<smoser> and above, you're not "basically guaranteed", you shoudl be guaranteed to have no network.
<patcable> gotcha
<patcable> smoser: though, we're using configdrive since we're in an openstack environment
<patcable> smoser: so I imagine we could still do what we want before cloud-init starts, though we'd have to work through the details later if we're going to port this to an EC2 like environment
#cloud-init 2015-07-24
<smoser> harlowja, around ?
<harlowja> sup
<smoser> in curtin, i'm wanting / needing to use pyparted
<smoser> there is no (to my knowledge) pipi option for it.
<harlowja> u are in a curtain
<smoser> how would you use such a thing in tox ?
<harlowja> hmmm, get it published to pypi :-/
<harlowja> otherwise u just have to get it installed as a system package, and then use it
<harlowja> push redhat to get it published on pypi?
<harlowja> they required the same push to get libvirt python stuff on pypi sadly
<smoser> how do i tell tox to tell virtualenv --system-site-packages ?
<smoser> bah. sitepackages=True
<smoser> oh for petes sake
<smoser> adding 'sitepackages = True' to tox.ini:
<smoser>  http://paste.ubuntu.com/11928050/
<smoser> bah. harlowja https://bugs.launchpad.net/curtin/+bug/1477795
<smoser> problem described there.
<harlowja> ah, cool
<smoser> what a pain.
<smoser> but at least some reasonably sane work around.
<harlowja> def
<smoser> have a nice night
<harlowja> u tooo
<cbolt> is there a way to disable cloud-init from checking for meta-data?
<cbolt> i am running it in a VM where meta-data isnt available. the log shows it takes ~4 mins for all of the meta-data calls to timeout
<cbolt> is it possible to disable cloud-init from attempting to retrieve meta-data?
<cbolt> i am running it in a vm where meta-data isnt available and it is taking ~4mins of bootup time
<devicenull> cbolt: you can disable metadata sources via a config file, it's usually somewhere like /etc/cloud.cfg.d/
#cloud-init 2015-07-25
<Odd_Bloke> cbolt: Yeah, you'll want to configure the list of datasources that cloud-init uses; have a look in /etc/cloud/cloud.cfg for the default list, and override it by dropping an extra config snippet in a file in /etc/cloud/cloud.cfg.d/.
<Odd_Bloke> cbolt: Alternatively, if you're using the Ubuntu (and, I assume, Debian) packaging, `dpkg-reconfigure cloud-init` should prompt you to select the ones you want.
#cloud-init 2016-07-25
<harlowja_at_home> smoser, so git is ready??
<harlowja_at_home> and larsks is fixing up brpm
<harlowja_at_home> even better :-P
<harlowja_at_home> whats git btw
<harlowja_at_home> i only know bzr
<harlowja_at_home> lol
<larsks> harlowja_at_home: well, trying to.
<harlowja_at_home> and then redhat will use brpm
<harlowja_at_home> even better
<harlowja_at_home> hja
<harlowja_at_home> *ha
<harlowja_at_home> damn think i lost my other open bzr pulls
<larsks> Yeah, note likely.  My official recommendation is "delete brpm and bddep and leave that to the packaging people", but smoser says these things are actually being used...
<harlowja_at_home> smoser, did those go away?
<harlowja_at_home> larsks, yes, i use them
<larsks> Yeah :)
<harlowja_at_home> leave that to the packaging people == i'm always stuck with some ancient version of cloudinit == not happy me
<smoser> i use bddeb all the time.
<smoser> larsks, honest question, if you're developing an open source project, how would *you* install that thing on a system?
<smoser> i suspect you'd use build a package
<smoser> (and if you wouldn't, how would you deal with the already-installed package or dependencies...)
<larsks> Yeah.  But I would probably start with, say, the existing Fedora rpm, and then insert the new sources.
<larsks> And I would maintain the spec file directly, rather than generating it.
<harlowja_at_home> ah, the hand-curated spec file stuff
<larsks> One of the problems with the existing brpm script is that it is super fragile...
<harlowja_at_home> its worked :-P
<larsks> It will never work for me because it doesn't deal well if your rpm configuration redefines things like _rpmdir, _specdir, etc.
<smoser> larsks, but if you're workign on trunk, you commit to trunk, you wan tot type some command have have somtehing to install
<smoser> thats my need.
<harlowja_at_home> larsks, well we can adjust that, just code ;)
<smoser> if that can be done in some other way, then i'm not opposed to some other way.
<larsks> For most packages I work with, packaging is handled "downstream" (by the distribution) rather than by the application developer (which is nice, because it means an application developer doesn't have to keep up with packaging standards, shifting names, etc).
<larsks> But no worries, I will make brpm work :)
<larsks> I will probably need a hand with bddeb, because it's been a long time since I worked on debian style packages.
<smoser> larsks, yeah, that is the common way "upstream doesnt care"
<smoser> but that just seems odd to me
<smoser> i want to install something on a n operating system.  and then remove it cleanly.
<smoser> there are tools for that :)
<smoser> larsks, i'll help with bddeb.
<larsks> Yay!
<harlowja_at_home> smoser, sooo all the other open branches i had on the bzr codebase, i guess i gotta find those and reopen? recreate?
<harlowja_at_home> did they diseapper in the ether?
<harlowja_at_home> ah, nm, i found them @ https://code.launchpad.net/cloud-init/+branches
<harlowja_at_home> i guess we have to resubmit?
<smoser> harlowja_at_home, yeah, i think thats best.
<harlowja_at_home> k
<harlowja_at_home> https://yahoo.tumblr.com/post/147941303269/verizon-to-acquire-yahoos-operating-business
<harlowja_at_home> and that's over with ^^
<harlowja_at_home> smoser, https://review.openstack.org/#/c/317739/ also relevant to us
<harlowja_at_home> don't think it changes much, but haven't reviewed in depth :-P
<larsks> harlowja_at_home: I am getting a keyerror from the templater when trying to generate the spec file.  Does this make any sense to you? https://gist.github.com/larsks/2728dcf1cd0036e19472e0e1d802ed69
<harlowja_at_home> larsks, will get back to u on that, does seem sorta odd there
<harlowja> so smoser what should we do with https://github.com/openstack/cloud-init ??
<harlowja> that ones an odd duckling now, lol
<harlowja> but the master has (?) something useful?
<harlowja> lots of windows stuff that i don't understand in that, lol
<smoser> harlowja, its a 2.0 branch that is currently shelved
<larsks> harlowja: the templating failure was because it was a cheetah template...but brpm explicitly ignores a venv python, so ended up using the basic rendered, which failed on those expansions.
<harlowja> ah
<harlowja> ya, wonder why it explicitly ignroes that
<harlowja> who wrote that crap
<harlowja> lol
<harlowja> so smoser i'm gonna get openstack-infra to drop/delete https://github.com/openstack/cloud-init
<harlowja> ok with u?
<harlowja> *or try to get them to
<harlowja> lol
<smoser> lets put a copy into launchpad first. but yeah.
<harlowja> k
<harlowja> smoser  do u have that ability, or do i also (to push branches?)
<smoser> you do, but i'll do it
<harlowja> k
<smoser> harlowja, ok. it is there at https://code.launchpad.net/~cloud-init-dev/cloud-init/+git/cloud-init-v2
<harlowja> cool
<wraithm> I'm trying to make an EC2 AMI with a centralized authorized keys file (ie. in sshd_config: AuthorizedKeysFile /etc/ssh/authorized_keys/%u). I see in the cloud-init source-code, cloudinit/ssh_util.py, the function setup_user_keys asserts the parent folder to be mode 700. Those permissions don't work with this AuthorizedKeysFile value. Having /etc/ssh/authorized_keys be mode 755 is what works. Is there a way around this?
<larsks> wraithm: you can run arbitrary shell commands via cloud-init, e.g., using the 'runcmd' directive.  Maybe try that?
<larsks> harlowja: when you have a moment, I've updated https://code.launchpad.net/~larsks/cloud-init/+git/cloud-init/+merge/300953 with a working brpm and I would appreciate your eyes on it.
<wraithm> like runcmd: [ 'chmod 755 <myauthkeysfolder>'] ?
<harlowja> cool
<larsks> wraithm: something like that, yes.
<larsks> harlowja: ...but I'm just about to take off for the day, so no rush :)
<harlowja> cools
<wraithm> The ssh module should probably be split up into one module for generating host box keys and one for inserting user keys
<wraithm> It's really annoying that it does both at the same time
<wraithm> I also had the idea of not running the ssh module, generating the ssh keys with a bootcmd or something (so that the keys still get printed to the console for my known_hosts). Do you think that would work?
<wraithm> The runcmd chmod worked :) Thanks!
#cloud-init 2016-07-26
<harlowja> hmmm, are these new errors
<harlowja> https://gist.github.com/harlowja/e4fc7bd6092cf7444a8d02155878e2fc
<harlowja> (running on a mac)
<harlowja> more things not mocked?
<smoser> harlowja, :-(
<dgarstang> Three year old bug... https://bugs.launchpad.net/cloud-init/+bug/1404060 ... would be nice if it was fixed
<smoser> dgarstang, were you the one that mentioned the related permissions issue on created directories ?
<dgarstang> smoser: I don't think that was me
<smoser> that was wraithm
<smoser> 07/25/16 16:11:31 <wraithm>   I'm trying to make an EC2 AMI with a centralized authorized keys file (ie. in sshd_config: AuthorizedKeysFile /etc/ssh/authorized_keys/%u). I see in the cloud-init source-code, cloudinit/ssh_util.py, the function setup_user_keys asserts the parent folder to be mode 700. Those permissions don't work with this AuthorizedKeysFile value. Having /etc/ssh/authorized_keys be mode 755 is what works. Is there a w
<smoser> ay around this?
<hans__> hi folks, i had a build question. i am trying to create a local build of cloud-init using the latest revision, but I see pep8 errors. is this a known issue? seems strange because the launchpad build is passing
<dgarstang> smoser: similar, but different
<smoser> hans__, pep8 is an obnoxiously moving target
<hans___> smoser - any recommendations for getting a successful local build?
<smoser> you're seeing errors on tests/unittests/test_handler/test_handler_mcollective.py ?
<hans___> yes, exactly. 4 errors
<smoser> i'll fix that. :-(
<smoser> where did you grab cloud-init from ?
<hans___> thank you. lp:cloud-init
<dgarstang> smoser: me?
<smoser> hans___, git ?
<smoser> dgarstang, sorry, that was for dgarstang
<dgarstang> smoser: oh, it's just the latest canonical upstream ami
<smoser> dgarstang, sorry, the question about cloud-init was for hans___  :)
<hans___> smoser, using bzr-builder, since it looked like the git recipe packaging hasn't landed yet?
<dgarstang> smoser: take upstream ami, rebundle into a new ami with packer. that new rebundle modifies /etc/ssh/sshd_config to add second location of ssh keys. When 2nd instance boots, cloud-init reads the sshd config file, and puts the packer key in the wrong place
<dgarstang> smoser: D'oh. Ok
<smoser> hans___, ok.
<smoser> hans___, i'll get the flake8 issue fixed in git.
<smoser> not sure what i want to do about the other.
<wraithm> smoser: What do you think about splitting the ssh module into a host ssh-keygen module and an inserting user authorized keys module?
<smoser> wraithm, that does seem like a reasonable idea.
<harlowja> shouldn't be to hard either wraithm
<harlowja> its pretty seperated in code as is
<smoser> so your issue, wraithm , what would you think the right thing to do is there?
<smoser> ok. so lets get git working
<wraithm> Yeah! The code was very easy to read.
<wraithm> So, the runcmd: - 'chmod 755 /etc/ssh/authorized_keys' suggestion worked for me
<wraithm> but what would be ideal is having a module for only generating host ssh keys
<wraithm> I was also very tempted by not running the ssh module and just using runcmd/bootcmd to run ssh-keygen
<smoser> so the format of /etc/ssh/authorized_keys is the same as ~/.ssh/authorized_keys ?
<wraithm> Not quite. It's a folder with a bunch of authorized_keys files in it
<smoser> oh.
<smoser> directory
<smoser> ok. yeah.
<wraithm> in sshd_config I have a line: AuthorizedKeysFile /etc/ssh/authorized_keys/%u
<smoser> but even if not split out, its only seting the permissions because generally it has to.
<wraithm> where %u is the username of the person attempting to log in
<smoser> if you set ~/.ssh/ to world writable, ssh will ignore the keys
<wraithm> Yes
<wraithm> But /etc/ssh/authorized_keys needs to be world readable
<wraithm> in my case
<wraithm> ie 755 instead of 700, which I think is what ~/.ssh/ gets chmodded to in ssh_util.py
<wraithm> Another possible solution is changing the code to do different modes depending on the location of the file
<smoser> right.
<smoser> that seems to be the right thing, trying to think of what the right rules for permissions are
<wraithm> I don't know know what sshd's permissions logic is :/ It's not really mentioned anywhere in the man pages.
<smoser> it seems like 700 is overkill actually
<smoser> my desktop has 755 on ~/.ssh/
<wraithm> Potentially
<wraithm> What about ~/.ssh/authorized_keys?
<wraithm> oh, nevermind
<wraithm> So, 755 works for ~/.ssh? Maybe that's the solution.
<smoser> seems 777 is even ok
<smoser> wait
<smoser> no
<smoser> seems 775 is ok
<smoser> just can't have other writable
<smoser> this is testing i'm doing on yakkety, so different ssh's could have different behavior
<smoser> but maybe
<wraithm> Interesting
<wraithm> Though, now I'm thinking that I still feel like generating host keys should be a different operation from inserting user authorized_keys. You could imagine people with some single-sign-on scheme (ldap/kerberos or whatever) baked into a cloud image. Authorized_keys is irrelevant to them.
<smoser> wraithm, but it only inserts them if keys were provided.
<smoser> so, then providing keys to the system is "irrelevant to them"
<smoser> this seems sane for my local quick testing:
<smoser>  http://paste.ubuntu.com/21032867/
<wraithm> Oh really? I look at the code and it looks like the root user always gets their key inserted.
<smoser> well, only if there was akey to insert
<smoser> if theres no keys, then it shouldnt do anything. if it does (and it breaks things) then yeah, we need to fix that too
<wraithm> Oh, okay. So, what if I wanted to create some default users, but I still don't want the keys to be inserted?
<wraithm> Which is actually sorta what I'm doing.
<smoser> it only inserts the key in to the default user i think
<wraithm> Right, and that breaks my thing
<smoser> the one that gets identified as 'default'
<smoser> not to all users.
<wraithm> Because ssh_util.py asserts /etc/ssh/authorized_keys to 700
<wraithm> which is a directory in my case analogous to ~/.ssh
<smoser> right. read that paste above though... i think that logic would work for you
<smoser> i'm not opposed to your suggestion of splitting them, but even if they're split, the key  insertion is doing the wrong thing for the case you provided.
<wraithm> 755 doesn't have world write permissions
<wraithm> oh
<wraithm> nvm
<wraithm> I see
<wraithm> Yes
<wraithm> That code looks good
<wraithm> :)
<hans___> smoser, i made the pep8 fixes in a private branch, but it looks like i see some test failures as well. checked that these are present in lp revision <1258 as well. there are 8 failures, all about virtual routers. do i need to set this up locally to run the tests?
<smoser> hans___, hm.. probably similar to what harlowja complained about above.
<harlowja> ya, probably
<smoser> :-(
<smoser> we really need some infrastgructure to test on non-ubuntu
<smoser> hans___, what is your system you're running on ?
<smoser> and are you running tox ?
<hans___> smoser - can't see the history, unfortunately, since i'm in webchat at the moment. i'm running pbuilder in a xenial container. ran tox successfully yesterday, haven't tried today
<hans___> smoser, just ran tox again successfully
<smoser> hm.
<smoser> this sucks
<smoser> the import of lp:cloud-init from bzr to git
<smoser> a.) "flattened" all history. making log of trunk show lots of people's in-branch commits that in bzr are nicely hidden , and only shown when bzr log --include-merged
<smoser> not a huge deal, but means that 'git log' is full of stuff like:
<smoser>  Remerge against head/master
<harlowja> lol
<smoser>  merge from trunk
<harlowja> durn
<harlowja> poo-crap-commits
<smoser> right, which bzr handles so nicely.  it allows you to accept your human.
<smoser> and that you suck
<smoser> other thing is that we lose the '--fixes' metadata
<harlowja> :(
<harlowja> durn
<harlowja> the feature i used every now and then
<harlowja> if i remembered to
<harlowja> lol
<smoser> i'm gonna look see if there is any obvious way to fix those things
<smoser> i'd be ok if it onverted the '--fixes' metadata into (Fixes LP: #XXXXX)
<smoser> and i think i'd even be ok if it just purged the merged
<smoser> i'd prefer that to a bunch of "fix spelling" and "fix tox i never thought someone would see this" commits
<smoser> https://www.fusonic.net/en/blog/migrating-from-bazaar-to-git/
<rharper> was someone else hacking on package/brpm ?  I'd really like to build my branch as an rpm so I can test a cross-distro feature;  I've got most of the bzr to git stuff fixed up (just a few more naming things w.r.t using git archive instead of bzr export); I saw some mention of spec files and such a few days ago
<smoser> larsks, ^
<smoser> rharper, ^
<rharper> cool
<larsks> rharper: yeah, me...there's a proposed merge that should work.
<larsks> And if it doesn't work, let me know :)
<rharper> ok
<smoser> that link for fusionic... c#
<smoser> really ?
<rharper> also noticed the basic renderer doesn't like the spec file and python-cheetah was a requirement;
<harlowja> smoser  u're probably gonna have to do that soon, and squash those commits into the one that actually matters
<harlowja> and then force update the repo
<smoser> yeah
<harlowja> its really basic
<harlowja> lol
<rharper> right, I mean, the spec file didn't include the # template: cheetah
<harlowja> oops
<rharper> which it should if it requires cheetah
<rharper> which it does
<harlowja> agreed
<harlowja> who wrote that crap
<harlowja> crappp
<harlowja> lol
<harlowja> probably that scott guy
<larsks> rharper: the spec file is correctly identified as a cheetah template in any case, but yeah you need cheetah installed for that to work :)
<larsks> I ran into the same thing a day or two ago.
<larsks> I sort of think the templater should blow up instead of falling back to basic...
<harlowja> ya, stupid rendering crap
<harlowja> who wrote that
<harlowja> lol
<rharper> it has a "## this is a cheetah template"  but the type matcher, wants '##  template: cheetah' ;
<rharper> TYPE_MATCHER = re.compile(r"##\s*template:(.*)", re.I)
<rharper> that said, even if the template was marked correctly, it would do the fallback
<rharper> though log that it could't find cheetah
<larsks> rharper: I updated the merge request to include the appropriate template: line in both (redhat + suse) spec files.
<rharper> cool
<rharper> larsks: do you have a link to your merge ?
 * rharper is looking for what you did around oauthlib 
<larsks> rharper: sure.  uh...did nothing arounmd oauthlib, though.  https://code.launchpad.net/~larsks/cloud-init/+git/cloud-init/+merge/300953
<rharper> and the rpm installs on centos 7 ?  I didn't find a python-oauthlib package available
<larsks> I haven't made any changes to requirements.  I've tested the rpm building on fedora.  Let me take a look at centos.
<larsks> CentOS only ships 0.7.5, which means that requirements introduced sinc then aren't available.  You can get python-oauthlib through EPEL, though.
<larsks> I am working on getting a newer version of cloud-init into RHEL, which means it should eventually show up in centos.  We may bundle all the requirements into a single cloud-init-deps package, at least in the short term.
<larsks> rharper: ^^^
<rharper> cool
<rharper> larsks: you're branch just worked =)  thanks
<larsks> That's what I like to hear :)
<rharper> I added epel-release, and the installed the cloud-init package created; +1
<rharper> so, something that we should look at is the default /etc/cloud/cloud.cfg sets the distro to ubuntu
<rharper> smoser: not sure how we 'd want to template that in the default config file
<larsks> rharper: one option would be to produce cloud.cfg for the packages are part of the brpm/bddeb scripts.
<larsks> *as part of...
<rharper> yeah
<rharper> hrm, centos7 has fun wit python2.6 and 2.7 and argparse ..
<larsks> rharper: but centos 7 ships with python 2.7...
<larsks> where does 2.6 come into things?
<rharper> you're right;
<smoser> i'm fine with requiring distro to ship something that changes the default user.
<rharper> I saw some thread mention 2.6 didn't have argparse package
<smoser> and fine carrying 'clouduser' as default even.
<smoser> it doesnt have it builtin
<rharper> pkg_resources doesn't find 'argparse' package which is required for cloud-init; but not expressed in the spec file
<smoser> oh yeahy, and something was really wierd there for rh.
<rharper> smoser: ah; I didn't find the obvious python-argparse package in epel or otherwise ;
<rharper> python2.7 says it provides python-argparse
<smoser> harlowja, had more info on that.
<rharper> but pgk_resources does't find it
<harlowja> right
<smoser> i think i'm going to have to push overwrite git history.
<smoser> i dont have it yet, but i really dont want to lose fixes metadata
<rharper> smoser: also, on the distro front; in lxd images: there is an opensuse image, but no sles that I can find (like no rhel) ; would you be ok to add 'opensuse' to the distro classes ?
<smoser> i woudl have thoguht something worked there. but yeah.
<larsks> rharper: yeah, you shouldn't need python-argparse, and I thought there was some logic in there to avoid listing the requirement for python 2.7 and later...
<larsks> rharper: still an issue for the package?
<rharper> larsks: let me merge in trunk
<rharper> but I think I'm pretty recent (from Friday I think)
<rharper> just running /usr/bin/cloud-init --help on centos7 after package/brpm from your branch shows me this argparse pkg thingy
<larsks> rharper: let me take a look, it's possible I lost some logic while fiddling with brpm.
<rharper> sure;  http://paste.ubuntu.com/21048423/
<rharper> that's what I see
<rharper> fedora24 fails in the same way;
<larsks> Yeah, that's because I seem to have dropped some code that handled this in brpm. Sorry!  Fixing.
<rharper> I see this https://bugzilla.redhat.com/show_bug.cgi?id=700596#c2
<rharper> so sounds like we should drop it from the setup requirements
<rharper> python2.7 -c 'import argparse' does indeed work on c7 and f27
<rharper> f24
<smoser> larsks, i saw a change to finding topdir that used git, right?
<larsks> smoser: yeah.
<smoser> i'd prefer to not rely on git, as the tools ideally would work without it.
<smoser> at least read-requirements and such.
<smoser> at least it seems sane to me that an environment variable can be set to say "this is where the top is" and then it use that.
<larsks> Okay.  I wanted it to work without having to set environment variables :)
<smoser> sure. it shoudl do that too
<smoser> it did at least.
<larsks> I will fix it up.
<smoser> again.. just so you're aware, i'm going to try to get a re-import done
<larsks> Sure.  Just ping me when you do that and I will fix the merge request.
<larsks> smoser: rharper: i've just pushed new changes to that merge request.  (a) this squashes everything down to a single commit, and (b) this correctly excludes argparse from the requirements for python >= 2.7 and (c) this will use CLOUD_INIT_TOP_D in read-dependencies if it is set.
<harlowja> larsks do u know how many people use spacewalk
<harlowja> we have this code that i may just make into a real-clout-init-top-level-module
<harlowja> https://gist.github.com/harlowja/e0b5a98844554d459338e3d72bbf7b7d
<larsks> harlowja: I have no idea...
<harlowja> damn, redhat people supposed to know
<harlowja> lol
<larsks> harlowja: not my bailiwick :) Ask me about openstack...
<harlowja> whats openstack
<harlowja> sounds weird
<harlowja> lol
<harlowja> pancakestack
<larsks> Mmmm, pancakes...
<harlowja> ok, i'll submit a spacewalk module
<harlowja> i convinced myself
<harlowja> lol
<harlowja> smoser did u push up to git with squished things
<harlowja> probably shouldn't do much with git until u do that
<harlowja> otherwise git be screwy when i `git pull`
#cloud-init 2016-07-27
<jgrimm> smoser, thoughts on -> https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1600766
<jgrimm> hmm.. maybe we've fixed already but we just haven't done a release with it? https://bugs.launchpad.net/cloud-init/+bug/1246485
<smoser> jgrimm, this is fixable by lxc templates. i've discussed it with stgraber.
<smoser> it is now fixable in cloud-init with some thought ... the most recent changes would allow for it when previously there wasn't really a  point in boot where you could do that.
<smoser> but a rose by any other name would still smell as sweet
<smoser> ie, why does it matter?
<jgrimm> ubuntu-core folks have use case
<smoser> probably not good ones.
<smoser> the host platform (lxd in this case) declares that the hostname of the system is XXXX
<smoser> why do you care what the system calls itself ?
<smoser> the system says "I want to be named bob"
<smoser> lxd says "his name is robert"
<smoser> who cares what the system says. if you ask the platform (dns) his name is robert.
<smoser> but, as i said, 2 ways to fix it.
<smoser> does that make sense ?
<smoser> why do you care what the container calls itself.
<jgrimm> This is a problem when using the dnsmasq for local dns resolving for *.lxd, which is the standard way of doing host dns for containers, as new containers are not dns addressable with a restart or renew.
<smoser> lxd should fix that.
<smoser> if lxd wants to insist that you can ask dnsmasq for dns resolution
<smoser> then it can't rely on the system behaving correctly
<smoser> jgrimm, stgraber i commented in bug
<jgrimm> smoser, sorry.. laptop locked up.   looking now
<smoser> rharper, harlowja larsks
<rharper> here
<smoser> i'm going to delete lp:cloud-init
<smoser> which means http://paste.ubuntu.com/21155351/
<larsks> howdy
<larsks> I've only got one in that list and I can resubmit...
<rharper> ditto
<smoser> your repos wont be deleted, but yeah. they  have to be re-done
<smoser> k
<smoser> larsks, and harlowja since i know you're such high bzr and launchpad users, you might sometime need the solution i came up with at
<smoser>  https://bugs.launchpad.net/bzr-fastimport/+bug/1606973
<smoser> powersj, rharper might actually use that at some point (curtin and other things)
<rharper> smoser: nice
<rharper> that'll be needed for curtin move to git as well
<smoser> i wasted a lot of time trying to use another solution that just didnt work with cloud-init's bzr export
<powersj> this is cool
<smoser> this way was much easier.
<rharper> y
<smoser> larsks, rharper https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init harlowja
<smoser> that is up now.
<smoser> ready for merge proposals
<rharper> thx
 * larsks looks...
<larsks> re-propsed: https://code.launchpad.net/~larsks/cloud-init/+git/cloud-init/+merge/301310
<smoser> larsks, what does git push -u do for me ?
<larsks> smoser: sets the upstream tracking branch so that in the future you can just "git push" instead of "git push myremote mybranch"
<larsks> Otherwise you get: http://chunk.io/f/6378b82ea0c647cd888de1af0d912bb8
<larsks> (-u is short for --set-upstream)
<smoser> larsks, do you want to stick with with cheetah in that tempatle ?
<larsks> You mean the spec file?  Not really!  I just didn't want to make too many changes at once that weren't directly related to the task at hand...
<smoser> ok
<smoser> thats fine,  want to ditch cheetah
<smoser> but that is good to not do that here.
<larsks> I will make that a subsequent merge proposal.
<larsks> (converting the redhat and suse spec files to jinja templates)
<smoser> although you did do arbitrary white space (line 534)
<smoser> :)
<larsks> That hurt my brain.
<smoser> i think keep the 'find_root' around line 635
<smoser> same reason as before... would like to set those
<smoser> rather than relying on git
<smoser> alghouth i guess less improtant there
<smoser> as that is explicitly a git operation
<larsks> Even make-tarball? That seems like we really want that to come from version control.
<larsks> Yeah.
<smoser> line 681 in that diff
<smoser> isthat set -o pipefailed ?
<smoser> i dont think so.
<smoser> so the gzip is unlikely to fail and thus unlikely to error
<smoser> (if the left sizde of that | (git archive) fails, then the command will succeed)
<smoser> read-depednecies... there is some reason we cant' use egrep
<smoser> let me log
<smoser> commit e26ac6b63072f3217de2fc9214584e61682cd211
<larsks> I'll update the script to set pipefail.
 * larsks looks at that commit.
<smoser> pipefail is bash
<smoser> silly non-linux unixes
<larsks> Ah, right, and we can't assume bash.
<smoser> and their silly 1986 shells
<larsks> That basically means "freebsd" at this point?
<smoser> yeah.
<smoser> those changes came in with freebsd
<smoser> it makes sense to use python though
<smoser> as we're pretty dependent on python working
<smoser> :)
<larsks> I guess.  It seemed like a ridiculous amount of logic, but that's okay, I'll revert.
<smoser> larsks, my mom reminds me that i didnt' say thank you. this is good, thank you.
<larsks> smoser: thanks! just pushed a restored (mostly) read-dependencies; fixing up make-tarball now...
<larsks> ...and there's a fixed up make-tarball
#cloud-init 2016-07-28
<kevinbenton> smoser: hi Scott. harlowja sent me your way. :) can you take a look at https://bugs.launchpad.net/cirros/+bug/1605832 and let me know if its feasible?
<harlowja> mr.cirros
<harlowja> lol
<harlowja> cubswin
<harlowja> lol
<kevinbenton> cubswin:)
<harlowja> ha
<smoser> kevinbenton, feel free to join #cirros
<smoser> powersj, hey... very simple "does cloud-init work in ubuntu" shown http://paste.ubuntu.com/21303132/
<smoser> script is http://paste.ubuntu.com/21303074/
<smoser> (testing lxd)
<powersj> oh sweet!
<powersj> I'll look this over this afternoon - thx
<rharper> smoser: any reason to not use lxc file pull ?
<smoser> i dont know.
<rharper> mm, lxc file pull <container/path - writes to stdout;  fancy
<rharper> I do with file pull/push had path completion though
<smoser> rharper, one thing woudl ble that you can't tell the difference between no file exists and something went horribly wrong
<smoser> $ lxc file pull x1/etc/asdf -; echo $?
<smoser> error: not found
<smoser> 1
<smoser> $ lxc file pull nocontainer/dd -; echo $?
<smoser> error: not found
<smoser> 1
<smoser> (non-obvious above, but x1 does exist)
<rharper> well, you can wait on x1 to be read before your x1
<rharper> or file pull
<rharper> my typical idion was launch and wait;  I don't see that exposed in the cli though
<rharper> actually, IIRC, launch blocks until it's running
<smoser> rharper, but you can't tell if it shut itself down or communication from client to daemon is dead...
<smoser> you just know something failed.
<smoser> which is fine. in that the wait would eventually catch it anyway.
<smoser> but the way i did it, exiting '9' indicated that every thing was happy, the file just didn't exist.
<rharper> yeah
<rharper> hrm, I wonder why they don't let you file pull from stopped containers (I'm guessing zfs isn't mounted)
<rharper> oh
<rharper> nm
<rharper> that works
<rharper> that's prolly worth an issue (missing file vs. unknown container name)
<rharper> for example, exec throwns the not found for non=container name, but 255 on exec failure
<rharper> smoser: did we have a bug filed for the systemd-udev interface rename where it won't rename a second time with .link files ?
<smoser> yeah, it works.
<smoser> pull and push from non-started.
<smoser> there is one... let me find
<smoser> https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1579130
<rharper> smoser: thanks!
#cloud-init 2016-07-29
<powersj> smoser, jenkins URL is now externally available at: https://jenkins.ubuntu.com/server/
<powersj> I am working this morning to update the URLs generated by the CI bot and to get venonat online for the vmtests.
<smoser> powersj, awesome!
<smoser> venonat is offline
<smoser> (just i saw that there, fyi)
<smoser> powersj, is it possible to hide things from non-logged in users ?
<smoser> i could see doing something stupid in a slave config
<powersj> yeah sounds like IS doesn't have access to it, so I have to manually update it's URL for Jenkins to the new one.
<smoser> such as exposing a credential or something in its log
<powersj> smoser, agreed, let me look into that
<powersj> in fact I am going to blow away the slave-config logs we have
<rharper> powersj: nice!
<smoser> harlowja, aorund ?
<harlowja> sup
<smoser> we used bzr revno before
<smoser> for a ever-increasing number
<smoser> i think pbr has a way of doing a similar thing
<smoser> that is failrly standard.
<smoser> and i think you have at some point given sane way of getting that.
<rharper> smoser: don't you love the random last 7 chars of git hash ?
<smoser> well they dont always go up
<smoser> thats what i dont like.
<rharper> I know
<rharper> =)
<rharper> well, git describe --tags --long
<smoser> maybe we can just go with that until we're unlucky
<smoser> :)
<rharper> does prepend a number of commits since tag
<smoser> yea, ok.
<rharper> so you get 0.7.6, then 0.7.6-N-hash, and then 0.7.5-N+M-hash
<rharper> that sorta helps I think
<rharper> I need to play with that to confirm
<smoser> what is -N+M ?
<smoser> whats the M
<rharper> M may be more than one commit since the previous number
<rharper> similar to revno I guess then
<rharper> it's offset in numbver of commits since tag
<rharper> http://paste.ubuntu.com/21428257/
<rharper> master reports as 1021 commits since  0.7.6 was tagged
<rharper> =)
<smoser> but why did you say '-M'
<smoser> er.. +M
<rharper> I just meant that it may be more than 1 since the last time you ran describe
<smoser> ah.
<rharper> I'm sure I confused you
<smoser> ok.
<smoser> its easy to do
<smoser> :)
<rharper> no, head to keyboard transfer isn't always perfect
<rharper> usually PBCAC for me
<smoser> heres hwat sucks.
<smoser> ew can't use 0.7.6-XXX
<smoser> as we've already released 0.7.7~bzrX
<smoser> which is > 0.7.6-
<smoser> rharper, so. ..
<smoser> $ git describe --tags
<smoser> 0.7.6-1022-g36e92d3
<smoser> what is g36e92d3 ?
<smoser> ah. the g . ii forgot about that.
<rharper> git describe --tags --long
<rharper> the g is hardcoded
<rharper> for whatever reason
<rharper> g + 7 chars of the hash
<rharper> smoser: need to tag 0.7.7
<smoser> yeah.
<rharper> I rant into trouble with git since changlog has 0.7.7 but it's not bumped in the repo
<rharper> but since larsks branch drops the changelog building, it was no longer an issue;  that said, we should still tag it for a new deb build
<harlowja> so ya, pbr us one solution here
<harlowja> there is another one that can also help figure out the version to
<harlowja> https://pypi.python.org/pypi/setuptools_scm is one
<harlowja> there are a few more
<harlowja> i'm ok with any of them really
<harlowja> https://pypi.python.org/pypi/setuptools_scm#default-versioning-scheme
<harlowja> is that fine with folks?
<harlowja> that one seems to be the 'the blessed package to manage your versions by scm tags'
<harlowja> https://github.com/pypa/setuptools_scm/
<harlowja> i guess from the pip folks?
<smoser> harlowja, thanks.
<smoser> rharper, or larsks is it ok for me to annotate tags and push them overwrite ?
<larsks> smoser: it is okay to overwrite tags, sure.
<smoser> rharper, i think the easiest answer to ~ or + is to just use + and to release an 0.7.7
<rharper> y
<smoser> and then use + always fom here out.
<larsks> people who have checked out the repository will probably need to do an explicit get pull --tags to get the updated tags.
<rharper> y
<smoser> 0.7.7+1-ghash
<smoser> larsks, yeah.
<smoser> how can i get the commit hash that a signed/annotated tag points to ?
<harlowja> $ git checkout <tag> && git log -n 1
<harlowja> ?
<harlowja> prob a better way, ha
<smoser> $ git show 0.7.6 | awk '$1 == "commit" { print $2 ; exit(0); }'
<smoser> yeah
<smoser> ok
<harlowja> so https://pypi.python.org/pypi/setuptools_scm#default-versioning-scheme what we gona go with?
<harlowja> i'd prefer that one over pbr (although its similar)
<harlowja> hmmmm, ok, let me know when thats done
<harlowja> i'll have to redo my fork and reviews i think
<harlowja> then i can get my jenkins stuff thats internally building cloud-init rpm back into shape
<harlowja> (using new jenkins way)
<harlowja> *git way
<smoser> harlowja, it shoudlnt hur your checkout
<smoser> it just changes the tags
<harlowja> kk
<smoser> http://paste.ubuntu.com/21439291/
<smoser> that is 'sign-tag' and me running it.
<smoser> i'm going to push now to master
<harlowja> proof its u?
<harlowja> how do i really know
<harlowja> u could be the fake-scott
<smoser> harlowja, its the same key that signs important things.
<smoser> like cirros builds
<harlowja> it better be
<harlowja> or i'm calling the police
<harlowja> lol
<smoser> larsks, hey still arond ?
<larsks> yessir.
<larsks> smoser: ^^^
<smoser> i have some changes here to get bddeb to build
<smoser> on top of your branch.
<smoser> in bzr world, i'd just commit on yours, push and then request merge. of those
<smoser> how would i do this in a sane way.
<smoser> i dont care really about the annotation just want to get somethign to you i guess.
<larsks> I think I would approximately the same thing: I would start a new branch based on mine, push it, and request merge.
<larsks> And then hope that launchpad isn't crazy.
<larsks> And then merge your changes after mine merge...
<smoser> ok.
<larsks> (honestly, I just wish everyone in the world used gerrit for handling code submissions, because it has great handling of changeset dependencies)
 * larsks also wants a pony.
<smoser> the one thing that bzr did better.
<smoser> it basically does --first-parent by default
<smoser> so peopple chan have all these little silly commits, you can merge them into yours, then i can merge them into master
<smoser> and if you git log on master, then you only see the one commit
<smoser> as if you had squashed them all before i merged or pulled
<smoser> thats why you see allthat nonsense little commits in git log now
<larsks> True.  Although you can also adopt a policy of "one commit per change please", and reject patches that have all those little commits.
<smoser> right, but thats just squashing
<smoser> and yes, i would adopt such a thing. :)
<smoser> i probaly use revision control more than i should
<smoser> but i really like reading history
<smoser> and put a lot of stuff in it
<smoser> and often would bzr log --include-merged
<smoser> whenever i would say WTH WAS I THINKING!
<smoser> anywahy
<smoser> https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/301547
<smoser>  ./packages/bddeb  -S -v
<smoser> that works in my branch
<smoser> larsks, ^ thanks.
<smoser> thats the last reference it hink to 'bzr' in the repo
<smoser> that uses bzr at least
<smoser> audios all
<rharper> smoser: later
<harlowja> what's bzr
<harlowja> lol
 * harlowja purges all knowledge of bzr
<harlowja> sorry canonical
<harlowja> lol
#cloud-init 2016-07-30
<harlowja> smoser u mirrored the 2.0 stuff into a branch on https://git.launchpad.net/cloud-init/ right?
<harlowja> i only see master, maybe it got overwritten?
<harlowja> btw, i sent out http://lists.openstack.org/pipermail/openstack-dev/2016-July/100476.html
<harlowja> (which is apparently part of that whole process to make sure people know)
<smoser> i thought i did
<smoser> harlowja, odd. this works:
<smoser> git clone git+ssh://smoser@git.launchpad.net/~cloud-init-dev/cloud-init/+git/cloud-init-v2
<smoser> there it is.
<smoser> https://code.launchpad.net/~cloud-init-dev/+git
<smoser> https://code.launchpad.net/~cloud-init-dev/cloud-init/+git/cloud-init-v2
#cloud-init 2017-07-25
<powersj> blackboxsw: your ci run hung it appears. I just relaunched it
<blackboxsw> thanks powersj . yeah I just double checked to make sure i didn't leave a pdb.set_trace() anywhere
<blackboxsw> running tox locally seems to run fine.
<powersj> well remember CI is much more than tox ;)
<dgarstang> So, it's been a couple of months since I was here. At the time, cloud-init didn't seem to support fs_setup on CentOS. Seems that's still the case
<dgarstang> Sure wish you guys would fix that
<smoser> dgarstang, it is probably much closer.
<smoser> and we have trunk rpms that you can test
<smoser> but i dont think we've explicitly fixed it
<blackboxsw> dgarstang: is there a bug open about this? I see we've got a released bug https://bugs.launchpad.net/cloud-init/+bug/1687712 that helped a bit
<ubot5> Ubuntu bug 1687712 in cloud-init "cc_disk_setup: fs_setup with cmd doesn't work" [Medium,Confirmed]
<dgarstang> smoser: last time I tried that it broke my instance, it wouldn't bot
<smoser> https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/
<dgarstang> boot even
<dgarstang> I also need this to work with CentOS 6, not just 7
<smoser> rpms there for both
<smoser> but probalby not fixed. :-(
<dgarstang> :(
<gbenhaim> Hi All, not sure if this is the correct place to ask, but I'll try. I wanted to know if "merging user-data sections" is supported when using "NoCloud" datasource ?
<blackboxsw> smoser: approved https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/327532
<smoser> blackboxsw, had some questions https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/327827
<blackboxsw> smoser: per your review there, you mentioned we want EphemeralIPv4Network to validate and setup networking to match the requested ip addr. Do we have existing funcitons in net somewhere that look at the existing interfaces IP?   I found netdev_info in cloudinit/netinfo.py. Any other utility function I should be using?
<blackboxsw>  
<blackboxsw> I didn't want to rewrite something if we already have a utility function
<smoser> blackboxsw, i dont have anything :-(
<smoser> i'm open to comments like "can't we do that later".
<smoser> blackboxsw, a low cost/tech path might be this
<smoser> root@z1:~# ip -family inet addr add 192.168.1.2/27 dev eth0
<smoser> root@z1:~# ip -family inet addr add 192.168.1.2/27 dev eth0
<smoser> RTNETLINK answers: File exists
<smoser> root@z1:~# echo $?
<smoser> 2
<smoser> so if its already present then it will exit 2 and 'File exists'. (if we pay attention to stderr/stdout then we need to LANG=C)
<blackboxsw> yeah I was leaning toward trying to use ip cmds instead of ifconfig (as netdev_info uses) just to be more forward looking
<smoser> for sure use ip
<blackboxsw> smoser: yeah I think we can do something simple for this branch. No need to put off content as tech debt if it's a sensible approach and easy implementation.
#cloud-init 2017-07-26
<smoser> powersj, or blackboxsw https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/328098
<powersj> smoser: how did master break?
<smoser> https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/328101
<smoser> master broke centso 7 build
<smoser> because of the systemd-dropin fix
<smoser> ^ fixes that faliure
<blackboxsw> grabbing that branch smoser
<smoser> its being tested at https://jenkins.ubuntu.com/server/job/cloud-init-ci/96/console right now
<smoser> and going through the centos build
<blackboxsw> tjx
<blackboxsw> thx
<blackboxsw> approved
<smoser> powersj, i can just push rebuild for
<smoser>  https://jenkins.ubuntu.com/server/view/Cloud-init/job/cloud-init-ci-nightly/
<smoser> right ?
<smoser> well, just pushed 'build now' and no prompt, like i expected there to be, so its building.
<powersj> yeah it will just run against master
<smoser> powersj, is 'Delete Pipeline' the same as the 'x' ?
<smoser> ie, if i just want to kill that build
<smoser> https://jenkins.ubuntu.com/server/view/Cloud-init/job/cloud-init-ci-nightly/45/ that one
<smoser> and then restart one as i just pushed the locale fix to trunk
<powersj> Delete pipeline will delete the entire job
<powersj> all builds and all runs
<powersj> You want to click into the run and there should be an X at the top left corner
<powersj> top right corner*
<smoser> found the 'x' on the job.
<smoser> yeah.
<smoser> thanks
<smoser> that pipeline thing is really cool
<smoser>  https://jenkins.ubuntu.com/server/job/cloud-init-ci-nightly/44/changes
<powersj> yeah
<smoser> blackboxsw, were you going to launch a zesty on azure ?
<smoser> i can do it real quick.
<blackboxsw> smoser: sure please do if you are free. I was trying to fixup the unit tests my branch to get that out of the way. I probably should've prioritized the other order
<blackboxsw> smoser: got my azure account setup again. Spinning up a zesty instance now.
<blackboxsw> unless you;'re there
#cloud-init 2017-07-27
<smoser> blackboxsw, https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/327827
<smoser> did you test ?
<smoser> dont' you have to 'ip link set up dev eth0' ?
<blackboxsw> smoser I had until last rev of changes. I just pulled it into the init-local aws branch to test now
<blackboxsw> and we do ip link set up dev eth0
<blackboxsw>             # Address creation success, bring up device and queue cleanup
<blackboxsw>             util.subp(
<blackboxsw>                 ['ip', '-family', 'inet', 'link', 'set', 'dev', self.interface,
<blackboxsw>                  'up'], capture=True)
<blackboxsw> oops
<smoser> oh. yes you do
<smoser> sorry
<smoser> ignore me
<blackboxsw> no worries. I'll repoet in the MP after this test finished
<blackboxsw> no worries. I'll repoet in the MP after this test finishes
<blackboxsw> I had my popcorn for your #server discussion
<blackboxsw> smoser: just tested success dhclient run on ubuntu aws Local. one minor fix about cleanup cmd ordering I just pushed.
#cloud-init 2017-07-28
<fritchie> has anyone else seen cloud-init scripts fail to run on stock centos cloud images while the same cloud-init scripts work fine on ubuntu cloud image? it looks like the centos build doesn't even attempt to execute anything even though all the files in /var/lib/cloud look fine
<smoser> fritchie, can you get /var/log/cloud-init.log ?
<smoser> you will probably have to make a modification to get goot logs there.
<smoser> good logs
<fritchie> smoser, sure ... I updated cloud-init to 0.7.9 from 0.7.4 and the script runs fine on centos ... I am using the "nocloud" data source
<fritchie> smoser:
<fritchie> https://thepasteb.in/p/58hgNG6ZJoNuv
<smoser> fritchie, in /etc/cloud/cloud.cfg.d/ you probably have a file named 05_logging.cfg
<smoser> if it has a line like:
<smoser> - [ *log_base, *log_syslog ]
<smoser> then either remove that or comment ('#') it out
<smoser> then you should get more logs there.
<smoser> there should be more log there...
<smoser> blackboxsw,
<smoser> http://paste.ubuntu.com/25191361/
<blackboxsw> hah oops
<blackboxsw> didn't mean to rage quite
<blackboxsw> didn't mean to rage quit
<blackboxsw> smoser: you said you'll get me a couple of something
<smoser> fritchie, 0.7.9... did that work for you?
<blackboxsw> then I fatfingered the close hangout window
<smoser> now orries.
<smoser> worries
<blackboxsw> smoser: dhclient works on centos AWS without -q.  one win
<fritchie> smoser, i'll try in a bit, thx
<fritchie> smoser: more logs now
<fritchie> https://thepasteb.in/p/zmh8V2wO3oquZ
<blackboxsw> and works w/out dhclient -I on  ubuntu. ok. the world is simpler than I thought.
<smoser> fritchie, so it looks like cloud-init-final isn't running. this is centos 6 ? or centos 7?
<smoser> either systemctl status cloud-init-final
<smoser> or
<fritchie> smoser, centos 6 ... but it looks like the scripts are running now ... it was a late night of troubleshooting and maybe I made a mistake
<smoser> k
<powersj> blackboxsw: what's the fastest way to install build-deps for cloud-init now? didn't you have a script?
<blackboxsw> make ci-deps-ubuntu or make ci-deps-centos
<blackboxsw> powersj: ^
<powersj> blackboxsw: thanks!
<powersj> blackboxsw: http://paste.ubuntu.com/25192636/
<blackboxsw> oops, I must've botched something on a previous run. will propose a trivial patch in a  couple mins powersj
<powersj> blackboxsw: yep http://paste.ubuntu.com/25192658/
<powersj> I think just the extra -
<blackboxsw> patch accepted. sorry had to wrap up another branch 1st.
<blackboxsw> powersj: pushed to master
<blackboxsw> 80bf98b..664a220
<powersj> blackboxsw: thx
<smoser> blackboxsw, your'e good to merge your other branch
<smoser> https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/327827
<blackboxsw> sweet smoser  thx
<blackboxsw> errr...  sweet! smoser, thanks.
<blackboxsw> punctuation makes all the difference
<powersj> bah so bddeb expects you to be in the root directory
<powersj> which has broken the tree-run of the integration tests
<blackboxsw> ok so review pass one on vmware branch is done. I'll have to get to the other tonight.  family just arrived a bit early
<blackboxsw> powersj: so we need to tweak bddeb to be more flexible? or can integration tests cd to that root dir before running it
<blackboxsw> yeah those script don't really *need* to be in the root directory. they just do an initial status check to see if any setup.py exists right?
<powersj> blackboxsw: I'm making integration tests figure it out, but running into missing dependencies now :\
<powersj>                 make[2]: nosetests3: Command not found
<powersj> which is not in requirements.txt or test-requirements.txt
<blackboxsw> lame
<powersj> I'll propose my workarounds tonight and you can look on monday
#cloud-init 2018-07-23
<ijw> #startmeeting networking-vpp
<meetingology> Meeting started Mon Jul 23 16:05:02 2018 UTC.  The chair is ijw. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
<meetingology> Available commands: action commands idea info link nick
<ijw> OK, my punishment for that was crashing my client...
<smoser> :)
<smoser> penance paid
<rharper> lol
<smoser> mgerdts: you missed one set notation
<smoser>  https://jenkins.ubuntu.com/server/job/cloud-init-ci/172/consoleFull
<smoser> cloudinit/sources/tests/test_init.py
<smoser> rharper: a review of https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/349359 would be good. its redhat specific and at redhat request, and i agree with it :)
<rharper> smoser: looking
<mgerdts> thanks smoser.  I'll get to that over the next day or so.  Just about to pack up to hop on a plane.
<smoser> filed https://bugs.launchpad.net/cloud-init/+bug/1783198
<ubot5> Ubuntu bug 1783198 in cloud-init "transient failures during lxc test during shutdown" [Medium,Confirmed]
<smoser> looking at it.
#cloud-init 2018-07-24
<dubuc> hello guys! :)
<dubuc> is this the correct place if we have some questions regarding cloud-init?
<dubuc> or there is a better medium?
<rharper> yes
<rharper> dubuc: ask away
<dubuc> thanks! :)
<dubuc> well, I am trying to change some mount points on an Azure VM running OpenLogic CentOS 7.4 (we manually install cloud-init with packer and save the Azure VM Image). When I specify the mount location of the ephemeral disk of that machine, it generates an /etc/fstab that does not match the location Azure mounts the ephemeral disk (/mnt/resource). This causes the mnt.mount to fail because of the wrong /etc/fstab
<dubuc> https://gist.github.com/dubuc/2db3dc3efc27750b1d43e8b28eabc64a this is the cloud-init.txt I pass when I create the VM with Azure CLI (az vm create --image ... --custom-data cloud-init.txt ...)
<rharper> dubuc: sorry for the delay;  had some meetings, let's see what's going on
<dubuc> yeah well rharper, I read the docs
<dubuc> apparently cloud-init does not support changing the mount point of the resource disk (ephemeral) on azure, and the waagent is supposed to handle that. could you confirm?
<dubuc> http://cloudinit.readthedocs.io/en/latest/topics/datasources/azure.html#waagent-conf-config
<rharper> do you have the cloud-init.log from the failure ? and can you confirm what cloud-init version you're running ?  I know we've seen some issues w.r.t empheral storage config so it may be resolved in master or might be a new issue
<dubuc> yes
<dubuc> give me a sec
<dubuc> but I did find the fix for my issue of failed mnt.mount service by following the `/mnt` location instead of the default waagent.conf `/mnt/resource`.
<dubuc> cloud-init 0.7.9
<rharper> ok, there's definitely newer cloud-init that has had some changes w.r.t azure ephmeral devices
<rharper> https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/
<rharper> you could try from the cloud-init-dev repo; that's a daily build from master
<rharper> if you can't then I suggest filing a bug with the cloud-init logs (/var/log/cloud-init* /var/lib/cloud/*) and your cloud-config you already pasted;
<dubuc> will try that
<dubuc> thank you rharper
<rharper> dubuc: sure
#cloud-init 2018-07-25
<nspmangalore> Hi all
<nspmangalore> I went through the cloud-init documentation
<nspmangalore> I've ported an AWS EC2 instance into my local xenserver setup
<nspmangalore> Now looking to modify cloud-init to work on this setup. So I wrote a small custom datasource on the ported VM
<nspmangalore> After reboot of the VM, I'm expecting that at least the VM's hostname should change. But that is not happening
<nspmangalore> First of all, I'd like to understand if what I'm expecting is valid
<nspmangalore> Is it?
<smoser> ugh.
<smoser> i'm am at a loss for this transient failure https://bugs.launchpad.net/cloud-init/+bug/1783198
<ubot5> Ubuntu bug 1783198 in cloud-init "transient failures during lxc test during shutdown" [Medium,Confirmed]
<smoser> i was sure that we were calling LXDInstance.destroy() multiple times or referencing a LXDInstance.pylxdcontainer after it'd been deleted
<smoser> bt i sure can't reproduce that
<rharper> smoser: when all else fails, blame systemd
#cloud-init 2018-07-26
<m00c0w> Hi! I'm new to using cloud-init and I have success with CentOS 7.  But when I try to spin up Ubuntu 18.04 using the CLoudImage I end up with KVM console saying that there is no boot partition. Is anything special required to do cloud-init with kvm/Ubuntu Bionic other than converting the img file to qcow2?
<smoser> m00c0w: that seems unlikely related to cloud-init
<smoser> i suspect you have an entry in /etc/fstab that is not right.
<smoser> fwiw, i would always using offiicial downloadable images from cloud-images.ubuntu.com
<smoser> if you have to modify them, I'd suggest modifying the images rather than building yourself elsewhere.
<smoser> caribou: do you need anything from me ?
<m00c0w> smoser: I figured it out. There are many different cloud images available for Ubuntu and while the filename was the exact same for the image I had, it was found in the wrong folder on the mirror site. Took a while to figure out
<m00c0w> m00c0w: https://paste.ee/p/4tHJ9    Confusing, right? ;)
<smoser> powersj: i want to run https://jenkins.ubuntu.com/server/job/cloud-init-integration-lxd-c
<smoser> against https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/351371
<smoser> in order to catch bug 1783198
<ubot5> bug 1783198 in cloud-init "transient failures during lxc test during shutdown" [Medium,Confirmed] https://launchpad.net/bugs/1783198
<smoser> any thoughts on how best to do that ?
<powersj> https://github.com/CanonicalLtd/server-jenkins-jobs/blob/master/cloud-init/integration-lxd.yaml#L56
<smoser> powersj: well, i want to run it on on c-i hardware
<smoser> i've not caught the failure here.
<powersj> you have ssh to torkoal ;)
<smoser> thats fine
<powersj> would you prefer a debug job? like we do for vmtest
<smoser> i guess that'd be nice. but i'm ok for now to just run on taht system.
#cloud-init 2019-07-22
<rharper> Odd_Bloke: thanks for the reminder about the upcoming meeting
 * rharper goes and gets the meetingology commands 
<robjo> I am going to have to miss the meeting today, sorry
<robjo> rharper: I'll catch up to the comments in the next couple of days
<rharper> robjo: np
<rharper> #startmeeting Cloud-init bi-weekly status
<meetingology> Meeting started Mon Jul 22 16:15:03 2019 UTC.  The chair is rharper. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
<meetingology> Available commands: action commands idea info link nick
<rharper> \o/
<rharper> #chair Odd_Bloke
<meetingology> Current chairs: Odd_Bloke rharper
<rharper> hi folks, welcome to another cloud-init community status meeting. All discussions and interjections welcome.
<rharper> cloud-init upstream uses this meeting as a platform for community updates, feature/bug discussions, and an opportunity to get some extra input on current development.
<rharper> our format is the following topics: Previous Actions, Recent Changes, In-progress Development, Office Hours
<rharper> anyone is welcome to participate, interject, make suggestions or ask questions
<rharper> we host the meeting every two weeks at the date and time  indicated in the IRC channel topic  ^
<rharper> #topic Previous Actions
<Odd_Bloke> o/
<rharper> We had a few previous action items to look at
<rharper> https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1832381
<ubot5> Ubuntu bug 1832381 in cloud-init (Ubuntu) "vm fails to boot due to conflicting network configuration when user switches from netplan to eni" [Undecided,Incomplete]
<rharper> AnhVoMSFT was looking to collect logs from this scenario;
<Goneri> hey!
<rharper> it appears that getting an instance where the MAC address changes is harder so fewer folks trip over this; however, we agreed that cloud-init can track which renderer it used and if it switches it can clean up the config it wrote;
<rharper> #action rharper to update https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1832381 status
<meetingology> ACTION: rharper to update https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1832381 status
<ubot5> Ubuntu bug 1832381 in cloud-init (Ubuntu) "vm fails to boot due to conflicting network configuration when user switches from netplan to eni" [Undecided,Incomplete]
<rharper> The other action was to review https://code.launchpad.net/~vtqanh/cloud-init/+git/cloud-init/+merge/369785
<rharper> This was completed the other week while we worked toward the 19.2 release; that branch is currently work-in-progress, awaiting feedback/changes from submitter
<rharper> that's all of the action items from previous meeting
<rharper> previous meeting status found here:
<rharper> #link http://ubottu.com/meetingology/logs/cloud-init/2019/cloud-init.2019-07-08-16.16.html
<rharper> normally at the cloud-init.github.io status page; looks like we didn't push the logs there.
<rharper> #action rharper to followup with blackboxsw on pushing status minutes up to cloud-init.github.io page
<meetingology> ACTION: rharper to followup with blackboxsw on pushing status minutes up to cloud-init.github.io page
<rharper> #topic Recent Changes
<rharper> % git log --oneline  --since 2019-07-08
<rharper> a02c0c9 (HEAD -> master, origin/master, origin/HEAD) cloud_tests: updates and fixes
<rharper> 5498107 Fix bug rendering MTU on bond or vlan when input was netplan.
<rharper> b3a87fc net: update net sequence, include wait on netdevs, opensuse netrules path
<rharper> 060b1a1 (tag: 19.2, raharper/release/19.2, release/19.2, fix/fs_setup_custom_command_lp1801790) Release 19.2
<rharper> 07b1723 net: add rfc3442 (classless static routes) to EphemeralDHCP
<rharper> 1404817 templates/ntp.conf.debian.tmpl: fix missing newline for pools
<rharper> a785462 Support netplan renderer in Arch Linux
<rharper> a066ccd Fix typo in publicly viewable documentation.
<rharper> d9769c4 Add a cdrom size checker for OVF ds to ds-identify
<rharper> 9c47c68 VMWare: Trigger the post customization script via cc_scripts module.
<rharper> a24550a Cloud-init analyze module: Added ability to analyze boot events.
<rharper> a6faf3a Update debian eni network configuration location, retain Ubuntu setting
<rharper> e5f5421 net: skip bond interfaces in get_interfaces
<rharper> 217c893 Fix a couple of issues raised by a coverity scan
<rharper> biggest item in there is the 19.2 release
<rharper> #link https://discourse.ubuntu.com/t/cloud-init-19-2-release/11873
<rharper> a big thank you from the cloud-init team to everyone who helped contribute to the release
<Goneri> do you have time for a little BSD update?
<rharper> Yes, let's talk about In progress developement
<rharper> #topic In Progress Development
<rharper> Goneri: go ahead
<Goneri> #topic FreeBSD/NetBSD status
<Goneri> so there is two active branches, the first one is: https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/368507
<rharper> #link https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/368507
<Goneri> it has started with a tiny fix to address a configuration difference with FreeBSD (there is no chpasswd there)
<Goneri> and it's now a slightly bigger refactoring now, I believe it clarify the code base and I would like to land it like that.
<Goneri> a discussion is ongoing with rharper on the PR
<Goneri> second PR is https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/365641
<Goneri> #link https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/365641
<Goneri> this one is much bigger, and I've just addressed the last comment from rharper, I test it often and it works fine for me
<Goneri> if you want to give it a try, I pushed some pre-built images here: http://bsd-cloud-image.org/
<rharper> nice!
<Goneri> I test it with OpenStack and NoCloud, a friend who maintains CBSD also test it on Bhyve (FreeBSD)
<Goneri> finally, the last one is https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/368508
<Goneri> #link https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/368508
<Goneri> No active merge request yet because it depends on the two actives PR that I just mentioned
<Goneri> this patch brings NetBSD support (7 and 8)
<Goneri> I would like to work on OpenBSD later, but it's still a low priority
<rharper> Goneri: thanks for the update
<Goneri> finally, I've a bunch of scripts that I use to build my images
<Goneri> it's still rather raw, but I would like to integrate that at some point with your CI
<Goneri> that's all
<rharper> thanks
<rharper> #topic In Progress Development
<rharper> #link https://code.launchpad.net/~tribaal/cloud-init/+git/cloud-init/+merge/369516
<rharper> Adding a new datasource for Exoscale
<rharper> #link https://code.launchpad.net/~xiaofengw/cloud-init/+git/cloud-init/+merge/367889
<rharper> vmware user-defined-scripts
<rharper> #link https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/369783
<rharper> Allow datasources to configure the order of network-config sources
<rharper> #link https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+ref/feature/stage_threadpool
<Odd_Bloke> I'm expecting to have a response to Ryan's review comments on that today, and whatever conclusion we reach shouldn't be too much work to implement.
<rharper> definitely
<Odd_Bloke> And then I'll have a follow-up to split apart the "cmdline" network data source in to "cmdline" and "initramfs", which are currently conflated.
<Odd_Bloke> (Neither of these should cause behavioural changes, they're just setting us up for some data source work down the line.)
<rharper> The threadpool branch is a more general approach to handle running modules async from the mainthread;  there were some limitations depending on systemd;  and there is a desire for more than just disk_setup to run async;  this branch I'm working on would allow modules to be tagged async and they run in a separate thread allowing the next module to proceed; we then join at the end of stage to ensure completion of threads;
<rharper> any other upstream development I'm missing?
<rharper> ok, I think that's it then;
<rharper> #topic  Office Hours (next ~30 mins)
<rharper> feel free to ask for help, reviews, discussions on any cloud-init items you're looking at.
<metsuke> Is Ubuntu the recommended, or most maintained, distribution of cloud-init?
<rharper> metsuke: hi; cloud-init in Ubuntu is the most-up-to-date as we're both the upstream maintainers (working for Canonical) and handle getting the latest upstream into Ubuntu images
<rharper> metsuke: we also help produce daily rpm builds for RedHat/Centos/Fedora in our copr repo
<rharper> #link https://copr.fedorainfracloud.org/coprs/g/cloud-init/cloud-init-dev/
<rharper> #link https://download.opensuse.org/repositories/Cloud:/Tools/
<rharper> suse's cloud:Tools keeps a really recent cloud-init as well
<metsuke> great, thanks for the info!
<rharper> sure
<metsuke> I'm looking to distribute standardized VMs to 100+ sites running ESXi so I'm trying to find the best way to do that =)
<Odd_Bloke> metsuke: If you have any questions, please don't hesitate to ask in here; we're generally around US working hours, for reference.
<rharper> yes, cloud-init can help keep your base image generic, allowing customization to happen at boot time;
<metsuke> thanks, I'm trying to do some preliminary investigation so I have an intelligent question to ask =P
<rharper> #endmeeting
<meetingology> Meeting ended Mon Jul 22 17:15:54 2019 UTC.
<meetingology> Minutes:        http://ubottu.com/meetingology/logs/cloud-init/2019/cloud-init.2019-07-22-16.15.moin.txt
<rharper> Thanks everyone !
<Odd_Bloke> Yes, thanks folks!
<Odd_Bloke> rharper: https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/369783 <-- I have hopefully responded properly to your comment here, let me know if you have any questions!
<rharper> Odd_Bloke: thanks!
<AhosmanMSFT> Hi I'm Ahmed from Microsoft. I'd like to get more information on cloud testing with cloud init?
<meena> AhosmanMSFT: testing, what, exactly?
<AhosmanMSFT> upstream cloud tests
<AhosmanMSFT> https://cloudinit.readthedocs.io/en/latest/topics/tests.html
<rharper> AhosmanMSFT: hi
<rharper> AhosmanMSFT: I think the platforms is the most interesting part
<rharper> if you look at tests/cloud_tests/platforms/ec2/platofrm.py ; for Azure, you'd need a platform subclass which talks Azure API to implemeant each of the platform methods
<rharper> that's mostly image related and then handling the instance creation/destruction in instance.py for the platform.
<AhosmanMSFT> rharper I see, so the base code is there it just needs to be inherited by us
<rharper> AhosmanMSFT: yeah; the structure of the tests framework is there, you'd be adding a new platform,
<rharper> the basic flow is 1) get an image 2) boot it  3) copy in the cloud-init package to be installed/upgraded  4) install test cloud-init package 5) clean-up  6) shutdown and capture image for boot;
<rharper> this is saved as a snapshot image; and then the collect phase boots the snapshot for each test-case we have; and then waits to ssh'in and collect output;
<rharper> and the verify stage runs the unittests against the collected data;
<AhosmanMSFT> rharper: Thanks, trying this now. Can I contact you for assistance?
<Odd_Bloke> AhosmanMSFT: Feel free to ping rharper or myself if you need anything.
<AhosmanMSFT> Odd_Bloke: ð
<metsuke> does cloud-init directly compare to hashicorp's Packer for its base functionality?  I'm trying to get a sense of scope of features and use case.
<rharper> https://events.linuxfoundation.org/wp-content/uploads/2017/12/cloud-init-The-cross-cloud-Magic-Sauce-Scott-Moser-Chad-Smith-Canonical.pdf
<rharper> metsuke: ^^
<rharper> metsuke: I would say that packer creates custom _images_ where as cloud-init customizes _instances_ ;  starting with a generic image, at _runtime_ you pass in user-data/configuration and cloud-init helps customize that instances consistently (allowing a common image and config to run the same on many platforms)
<metsuke> Interesting. So does cloud-init need access to the platform that the images are being spun up on?
<metsuke> or rather, are you executing cloud-init in the same network in which your VM platform resides (including cloud)?
<Odd_Bloke> metsuke: cloud-init runs within the instance.
<Odd_Bloke> So during boot, it will run, determine which platform it is on, gather configuration from various sources, and then apply that configuration.
<metsuke> hm, that's kind of backwards from the other model.  Cool
<AhosmanMSFT> For the files __init__.py, image, instance, platform and snapshot.py in cloud_tests. Should I work on these in any specific order, since some of the files depend on others. Also after implementing the files, what would be the next steps?
<Odd_Bloke> AhosmanMSFT: So I haven't written a cloud test platform myself, so take this with a grain of salt, but I think you're going to need to work on them all together.  That said, I would do this by working out a cloud_tests command line that works, changing the platform to "azure" and then iterating on the missing pieces reported by the framework.  (I _expect_ that would go in the order platform, image,
<Odd_Bloke> snapshot, instance based on reading cloud_tests/collect.py.)
<Odd_Bloke> So, for example, `tox -e citest -- tree_run --verbose --os-name bionic --preserve-data --data-dir ~/scratch/citest --platform lxd --test-config tests/cloud_tests/testcases/modules/ntp.yaml` runs the NTP tests in lxd for me.  So I would change "lxd" to "azure" and see what happens, and then iterate.
<Odd_Bloke> AhosmanMSFT: Is that a useful answer?  If not, I'll ping the folks with more domain knowledge. :)
<AhosmanMSFT> That helps. It would be great to get some more insight from them too.
<Odd_Bloke> rharper: powersj: Perhaps you can flesh out my answer to help AhosmanMSFT get started on Azure cloud tests?
<rharper> Odd_Bloke: that's exactly where'd I'd start as well.
<powersj> AhosmanMSFT, I would start with platform. That is how you setup how we interact with Azure. Any work that is needed before an instance is done goes there
<powersj> After that, go after instance. There all the basic operations of start stop and getting the instance log
<powersj> then go to snapshot to generate a... snapshot of the instance :)
<powersj> and finally image
 * powersj disappears for a while
#cloud-init 2019-07-23
<reith> hi, is it possible to include a cloud-config (and make cloud-init process it) is vendordata?
<reith> i read vendordata docs (https://cloudinit.readthedocs.io/en/latest/topics/vendordata.html) like ten times and i see it's said "vendordata is handled exactly like user-data" so I conclude it can have cloud-config like user-data, correct?
<reith> but in only samples that i found from openstack, it seems to be just arbitary key-values (https://web.archive.org/web/20180110021506/http://www.stillhq.com/openstack/000022.html)
<reith> i managed to make cloud-init make vendordata-cloud-config.txt but it won't be executed :( user-data does not exclude vendordata execution, should I do anything to make it run?
<reith> s/vendordata-cloud-config.txt/vendor-cloud-config.txt/
<reith> it will be ran if user-data does not exist
<reith> https://bugs.launchpad.net/cloud-init/+bug/1837530
<ubot5> Ubuntu bug 1837530 in cloud-init "cloud-config in vendordata overriden by user-data" [Undecided,New]
<Odd_Bloke> rharper: I believe https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/369783 is ready for re-review.
<rharper> cool
<Odd_Bloke> rharper: Unit test added.
<rharper> Odd_Bloke: thanks!
<Odd_Bloke> rharper: Thanks for the reviews!  The next piece of work is up at https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/370526
#cloud-init 2019-07-24
<Odd_Bloke> rharper: I've added some thoughts to https://bugs.launchpad.net/cloud-init/+bug/1834875; could you have a look and see what you think of my hypothesis?
<ubot5> Ubuntu bug 1834875 in cloud-init "cloud-init growpart race with udev" [Medium,Confirmed]
<Odd_Bloke> (No rush.)
<Odd_Bloke> rharper: I double checked; no comments on https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/370526 as yet.
<Odd_Bloke> rharper: Really helpful comment on that growpart bug, thanks!
<rharper> Odd_Bloke: sure;  I was chasing symlink removal/creation for curtin's dname and had done something like this before
<rharper> Odd_Bloke: when you have time, https://code.launchpad.net/~wrigri/cloud-init/+git/cloud-init/+merge/370348
<mroutis> is it possible to run a cloud-config file once the server is already configured? (if so, any guidance is  appreciated)
<Nick_A> perhaps https://cloudinit.readthedocs.io/en/latest/topics/capabilities.html#cloud-init-modules with the config mode
<mroutis> Nick_A: thanks
<mroutis> where can I find a list of supported keys?
#cloud-init 2019-07-25
<chillysurfer> hey, all! what's the best way to test out data source changes without having to reprovision a machine in the cloud provider for every code change iteration?
<chillysurfer> for instance, to change vendordata in the datasource, is the only way to test this out by patching cloud-init and then reprovision a machine with the patch?
<chillysurfer> seems to make for a very long dev iteration loop
<rharper> chillysurfer: you'll likely want 'cloud-init clean --logs'  and possible add --reboot ; that will reset most of the cloud-init state and re-run like firstboot
<chillysurfer> rharper: ah cool! so basically just run that command and then reboot the machine?
<rharper> chillysurfer: yes
<chillysurfer> rharper: great thanks! i'll give it a try!
<rharper> chillysurfer: you also don't _have_ to reboot; it depends on what your testing; you can just call cloud-init like boot does;  cloud-init init --local; cloud-init init, cloud-init modules --mode=config; cloud-init modules --final
<chillysurfer> rharper: i'm really just testing out injecting vendordata
<chillysurfer> i'm not sure exactly what part of cloud-init that we crawl metadata (and therefore get vendordata i think..?) and then handle and execute vendordata
<rharper> chillysurfer: local
<rharper>  https://cloudinit.readthedocs.io/en/latest/topics/boot.html
<rharper> chillysurfer: depending on the datasource, we like to create main functions on the DataSource.py which just crawls the metadata and dumps what had (and you could merge it)
<chillysurfer> rharper: i'm working with the azure datasource
<rharper> chillysurfer: if you look at cloudinit/sources/DataSourcesGCE.py for example
<chillysurfer> i'll check out the gce esample
<rharper> chillysurfer:  so you can use the crawl_metadata()
<chillysurfer> *example
<chillysurfer> that would be a good way to see what i end up with for sure
<chillysurfer> rharper: but to see the actual metadata (in this case, vendordata) applied to the machine then i should be running the entire local stage i think right?
<rharper> chillysurfer: right
<chillysurfer> perfect
<chillysurfer> thanks so much! going to play around with this
<rharper> well, and dpending on what you put into the config; some things happen at different stages
<chillysurfer> yep i'm just injecting some 'hello world' in vendodata runcmd
<rharper> ok, runcmds will happen at final time
<chillysurfer> ah i see
<chillysurfer> so the final stage then?
<rharper> also, generally vendoring runcmds is going to be problematic since users likely want to include those, and then merging rules apply
<chillysurfer> yep totally understand about merging and that sort
<rharper> chillysurfer: you'd need to run local, init, and final
<chillysurfer> i need to do some research on the default merging of vendordata + userdata
<rharper> https://cloudinit.readthedocs.io/en/latest/topics/merging.html
<rharper> should help
<rharper> particuarlly at the bottom; exaples on merging multiple run commands
<chillysurfer> awesome
<rharper> it's somewhat awkward to allow things to merge "the way you want"  which may be different than how users want
<Odd_Bloke> https://bugs.launchpad.net/cloud-init/+bug/1837927 <-- this bug stems from a broken OpenStack metadata service.  On xenial we end up using Ec2, so instances are at least usable, but on bionic we don't (because we correctly detect we're on OpenStack, so only use that DS).  Is there anything the user can do when launching an OpenStack instance to work around the broken IMDS by selecting the Ec2 DS?
<ubot5> Ubuntu bug 1837927 in cloud-init "Missing mandatory path: http://169.254.169.254/openstack/2018-08-27/meta_data.json" [Undecided,New]
<Odd_Bloke> Or will they need to bake their own image to select the working DS?
<Odd_Bloke> (I think this bug is Invalid regardless, would just like to give them a bit more direction than "go bug your cloud admin" if possible.)
<rharper> Odd_Bloke: huh
<rharper> Odd_Bloke: how would we know that Ec2 would work? I guess you're suggesting we could try ...
<rharper> Odd_Bloke: IIC, on Xenial, the _get_data() returns False, but because we _search_ all known datasources, we'll try Ec2 last
<rharper> on bionic, since ds-identify says, just OpenStack, then it won't try anything else
<rharper> we would sort of want to allow cloud-init to try a list of maybe datasources
<rharper> the user could modify ds-identify, with found=all, like so:  ci.di.poliyc=search,found=all,maybe=none,notfound=disabled;  which is equivalent to the Xenial defaults
<rharper> s/ds-identify/ds-identify-policy
<Odd_Bloke> Yes, on xenial we try OpenStack and it fails so we move on, eventually to Ec2.
<Odd_Bloke> rharper: That should be notfound=enabled, right?
<rharper> no
<rharper> oh, well for fully xenial equivalent yes
<rharper> but the goal is to fallback, not emulate xenial behavior
<rharper> so, if we don't find any datasources, then we shouldn't enable
<Odd_Bloke> But in this case, we aren't going to "find" the Ec2 DS, right?
<Odd_Bloke> xenial also has mode=report.
<Odd_Bloke> Which is I think why it works, because that means we try everything anyway?
<rharper> we will find OpenStack
<Odd_Bloke> Right, but OpenStack is broken.
<rharper> and di-identify returns a datasource_list like: ['OpenStack', None]
<rharper> I think it will list all network sources as maybe since we're not in strict mode
<Odd_Bloke> I think the problem is going to be that it will identify OpenStack as definitely found.
<Odd_Bloke> So we might just want ci.di.policy=enabled, with OpenStack,Ec2 configured as the data sources in /etc/cloud/... ?
<rharper> yeah; I think you're right;   I think we'd need to either disable ds-identify via the report mode
<rharper> right
<rharper> well, via command line for images booted on this broken openstack
<rharper> either both configs on the kernel command line or in the image itself if someone is making modifications
<Odd_Bloke> Oh, can you change the command-line when launching via Nova?  Or are you saying that the cloud admin could work around their broken cloud by changing the default command line that's used?
<rharper> I was looking for image attributes
<rharper> and I thought one could pass things through but I think I'm not right about that; the iamge has a boot loader
<Odd_Bloke> Oh, looks like ci.di.datasource=OpenStack,Ec2 should DTRT.
<Odd_Bloke> enabled/disabled would mean we couldn't configure _anything_ via ds-identify.
<Odd_Bloke> But ci.di.datasource returns before the report/search behaviour is used.
<rharper> yes, if you specify them; then we don't bother detecting them
<Odd_Bloke> Right, which is what we want here.
<Odd_Bloke> Right?
<rharper> yes
<rharper> effectively setting the list to OpenStack and then Ec2
<rharper> though in this case they shoud just set Ec2
<rharper> no point in trying a broken openstack
<Odd_Bloke> Yeah; I put that in there so that they'll switch to OpenStack if the cloud is fixed.
<Odd_Bloke> I'll include a note in my reply explaining that.
<rharper> that can just use non-modified image  to verify
<rharper> but yeah
<rharper> so Ec2 doesn't do a check-instance id
<rharper> which means that if we use Ec2, on next boot if it found OpenStack, I think it would use that;  I wonder about the transition on upgrade
<rharper> with such a setting
<Odd_Bloke> OK, I'll just put Ec2 in there then.
<Odd_Bloke> My guess is they'll either fix the cloud soon or never.
<rharper> right
<chillysurfer> anybody know how to hit a breakpoint with nosetest3? doing `import pdb; pdb.set_trace()` and then running nosetest3 doesn't seem to do it
<chillysurfer> nor does the `--pdb` option (which breaks on failure and error, but not breakpoing)
<chillysurfer> and the manpages for it don't mention a thing about breakpoints
<rharper> chillysurfer: you want to pass -s
<rharper> so you get standard input/output to console
<rharper> well, I've not used breakpoints, but I've dropped an import pdb; pdb.set_trace() into various parts of code or tests;then used -s on the nose commandline
<chillysurfer> rharper: that did it!!
<chillysurfer> thanks so much!
<rharper> \o/
#cloud-init 2019-07-26
<factor> Did'nt know this existed until now.
<factor> Can I use git clone inside a runcmd ?
<factor> and use said repo
<nacc> is it expected that 'selinux_user' in a cloud config on ubuntu would cause an exception ("useradd: -Z requires SELinux enabled kernel")
<Odd_Bloke> factor: o/ There shouldn't be any problem doing that; are you seeing issues?
<Odd_Bloke> nacc: \o I'm not sure, TBH, but a bug would be appreciated!
* Odd_Bloke changed the topic of #cloud-init to: Reviews: http://bit.ly/ci-reviews | Meeting minutes: https://goo.gl/mrHdaj | Next status meeting July 22 16:15 UTC | cloud-init v 19.2 (07/17) | https://bugs.launchpad.net/cloud-init/+filebug
<nacc> Odd_Bloke: ack will get it filed
<Odd_Bloke> Thanks!
<Odd_Bloke> rharper: Do you know why cloud-init doesn't emit its logs to the journal?  When trying to debug interactions between multiple units, it's quite annoying to have to keep two log files open and mentally interleave them.
<Odd_Bloke> rharper: I filed https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1838032, so we can carry out the conversation there.
<ubot5> Ubuntu bug 1838032 in cloud-init (Ubuntu) "cloud-init should emit its logs to the systemd journal" [Undecided,New]
<rharper> Odd_Bloke: it does log to the journal... journalctl -u cloud-init-local.service  ?
 * rharper reads bug 
<Odd_Bloke> rharper: Clarifying in there now.#
<rharper> Odd_Bloke: I understand now
<rharper> and we can discuss the defaults in upstream, in ubuntu, and I've added a comment to make a one line config change to get what you want
<rharper> the latter which is immediately useful to you
<Odd_Bloke> rharper: Ack, thanks; I've clarified that I mean _all_ logs.
<factor> <Odd_Bloke> Thanks for the response. Going to test it , have not yet. So I can git a puppet script to run as part of cloud-init.
<Odd_Bloke> factor: You can definitely `git clone` the puppet script from a runcmd, yes.  And if you're asking about using the puppet cloud-init module, then that runs after runcmd so whatever you do should be on the system by the time it runs.
<factor> Odd_Bloke, thanks
<Odd_Bloke> rharper: So it looks like for that Azure growpart bug the udev event is missing information: https://bugs.launchpad.net/cloud-init/+bug/1834875/comments/24
<ubot5> Ubuntu bug 1834875 in cloud-init "cloud-init growpart race with udev" [Medium,In progress]
<Odd_Bloke> rharper: Do you have any suggestions as to where I should go looking next?
<rharper> yes, so if you look at growpart, you can see what it's doing with sgdisk,
<rharper> which is updating the table and setting values
<rharper> it looks like whatever is watching the disk (systemd) sees the write/close on the device, and fires an uevent where the partition data isn't present,
<rharper> Odd_Bloke: look at growpart at line 476
<rharper> Odd_Bloke: so I wonder if the version of sgdisk matters here
<rharper> it
<rharper> is effecitvely deleted pt table, re writing it, then calling partx --update
<rharper> to which, I think one *could* add a udevadm settle --exit-if-exists /path/to/partition
<rharper> but I thought your change added a settle right before the check on new size value;
<Odd_Bloke> So --exit-if-exists only short-circuits the regular behaviour, AIUI.
<rharper> if update was triggered (partx --update) then a settle after that certainly should have already re-read pt table
<Odd_Bloke> So if the queue is empty, it won't _wait_ for that file.
<rharper> Odd_Bloke: right; it just means to not wait the default 120 seconds if the path already exists
<rharper> it checks if it exists, then checks if the queue has anything in it
<rharper> and then will wait up to 120 seconds for queue to empty
<Odd_Bloke> Right, which means `udevadm settle` will do whatever it would do; and it didn't address the issue.  That said, I don't think tobijk used my patched cloud-init for these latest tests.
<rharper> effectively, the sgdisk write new table info, partx --update will relese the syscall to re-read partition table on the disk
<rharper> if we then call settle; the updated pt will generate change events
<rharper> and *those* should have the updated partition data in the event
<Odd_Bloke> Why would settle cause more events to be generated?
<rharper> otherwise I don't see why a change event would occur;  though when the partition is _deleted_ via sgdisk command, that will create a change event
<rharper> settle won't cause any events
<rharper> only sgdisk command will generate events
<rharper> the partx --update is a kick to ask the kernel to reread the partition table data (and then the kernel may emit update events if the table has changed since it last read it)
<rharper> so, a settle after those two commands have run, should ensure that once a CHANGE event for the updated ptable is emitted, and rules have run, that the symlink would be present
<rharper> it's not clear to me if the captured change event you;'re seeing is from when sgdisk is "clearing things out" or after it's written the new data and called partx --update
<rharper> Odd_Bloke: I can hangout if you want
<Odd_Bloke> On both failing and succeeding instances, we see the same number of sda1 udev events: one add when the system comes up, one change when we resize, and one change later in boot (I think when systemd is reloaded?).  After that second "change", both systems are in a good state.
<Odd_Bloke> The order in which the sda{1,14,15} udev events are logged is different between the two systems; sda1 is first on the failing system, and last on the good system.
<Odd_Bloke> So my theory is that the events are prompted by the clear-out, and we're then racing against the partition table being rewritten.
<rharper> and the timing of the second change from passing to failing ? timestamp wise; does this occur before or after the read ?
<Odd_Bloke> In both cases the second change is 20-30s after the resize has happened.
<rharper> oh my
<rharper> so, let's test this if we can'  in the check_size path; invoke 'partx ---update' or blockdev --rereadpt /dev/sda;  then udevadm settle
<rharper> this will *force* a reread of the table before we check size
<rharper> and settle should enforce rules complete if update detected a table change
<rharper> if that works, then it's likely that the partx --update after sgdisk returns isn't effictive due to the processing of the change of removal
<Odd_Bloke> As an aside, those second change events come after "systemd[1]: Reloading."
<chillysurfer> it's not jumping out at me by looking at bddeb, but is there a way to tell the build to dump debs and other distribution artifacts in a `dist` dir?
<chillysurfer> i see that it is gitignored so wondering if i'm just not seeing something
<Odd_Bloke> rharper: Further confirmation that we aren't seeing the event due to the partx command; the growpart call specifies a single partition, and we're seeing events for all partitions.
<Odd_Bloke> rharper: I'm heading to lunch; could you take a look at https://bugs.launchpad.net/cloud-init/+bug/1834875/comments/25 and let me know if that matches your understanding?
<ubot5> Ubuntu bug 1834875 in cloud-init "cloud-init growpart race with udev" [Medium,In progress]
<rharper> Odd_Bloke: that looks right;  reading partx source-code (https://github.com/karelzak/util-linux/blob/master/disk-utils/partx.c) I wonder if what we really want instead of partx --update /dev/sda 1 ; is just blockdev --rereadpt /dev/sda ;
<rharper> partx --update looks like it does more than just asking the kernel to reread the partition table on the disk.
<Odd_Bloke> rharper: When I tried a `sudo blockdev --rereadpt /dev/sda` on a running system, I get "blockdev: ioctl error on BLKRRPART: Device or resource busy"
<Odd_Bloke> So I don't know if we _can_ use that.
<rharper> right
<rharper> that's why they used partx; uses a different syscall
<rharper> Odd_Bloke: I think it would be interesting to see partx --show /dev/sda  ; before the --update and after
<Odd_Bloke> OK, will include that.
<Odd_Bloke> chillysurfer: Sorry, skipped over your question before.  If I had to guess, I think dist/ is ignored for Python packaging reasons, not because that's where we expect deb/rpm packages to end up.
<Odd_Bloke> chillysurfer: I think adding an option for output location would probably be a reasonable proposal, though, defaulting to the current behaviour.
<chillysurfer> Odd_Bloke: yep that makes sense. i have a one-liner git and xargs command that just deletes those untracked files anyways
<chillysurfer> so not a huge deal (as long as the build files are my only untracked ones)
<bitfehler> i was wondering - is there any "official" definition of what constitutes a variant "other" in the cloud.cfg template?
<bitfehler> there are several uses like `{% if variant in ["ubuntu", "unknown", "debian"] %}` or even `{% if variant in ["ubuntu", "unknown"] %}`
<bitfehler> e.g. enabling the apt-related modules
<bitfehler> which seems odd, i figured "other" would be a distro... well, _other_ than the ones that have specific values
<Odd_Bloke> Ubuntu and Debian aren't the only distros that use debs (e.g. Mint).
<bitfehler> true, but still seems a far-reaching assumption? also, i though mint was in the list as well?
<bitfehler> ah, not quite, but it seems mint is mapped to ubuntu already
<bitfehler> system_info() has something like
<bitfehler>         elif linux_dist in ('ubuntu', 'linuxmint', 'mint'):
<bitfehler>             var = 'ubuntu'
<Odd_Bloke> Fair enough.
<bitfehler> i mean, i am also not even trying to prove anything, i was just wondering. the apt thing was just an example. if it really is meant to be "anything else" that's file
<bitfehler> fine :)
<rharper> bitfehler: no strict definition; I would say, other  is something not currently defined
<rharper> we're opt-in here; the default being Ubuntu; and as distros and variants of a particular distro have need to deviate; those get introduced/added
<Odd_Bloke> I guess the thinking is that if we know that we _don't_ know what distro we're on, we shouldn't remove things from the list.
<bitfehler> interesting point :)
<Odd_Bloke> So `{% if variant in ["ubuntu", "unknown", "debian"] %}` is more like "we shouldn't run this if we know definitively we aren't on Ubuntu or Debian"
<bitfehler> that is good guidance for me, thanks. i guess it's clear that "other" shouldn't be anything to rely on anyways, so i am just trying to understand how to best go about supporting another distro there
<rharper> bitfehler: is the distro a variant of existing ones? or something altogether different ?
<Odd_Bloke> Yeah, my first reaction to that is that we should be defining this distro in cloud-init. :)
<rharper> I think the best way to answer that is looking at the distro classes and their implementation of things like package install, and default file locations
<bitfehler> as a fun fact, the variant of my distro (arch) as returned by system_info is actually "linux", which is not a valid input when using --distro= for ./setup.py
<bitfehler> however, when not setting anything, that's what gets passed to the template
<rharper> arch doesn't have an os-release file ?
<rharper> https://gist.github.com/natefoo/814c5bf936922dad97ff
<bitfehler> it does, and that is reflected somewhere in system_info result, but the variant is "linux"
<rharper> the python 'dist' info just says linux then ?
<rharper> if there is extra info in either os-release or another file then we can update util.py:get_linux_distro() to parse that bit out
<bitfehler> no, dist is set correctly, but the variant is determined by checking for a few know values and e/t else is "linux"
<bitfehler>     if system == "linux":
<bitfehler>         linux_dist = info['dist'][0].lower()
<bitfehler>         if linux_dist in ('centos', 'debian', 'fedora', 'rhel', 'suse'):
<bitfehler>             var = linux_dist
<bitfehler>         elif linux_dist in ('ubuntu', 'linuxmint', 'mint'):
<bitfehler>             var = 'ubuntu'
<bitfehler>         elif linux_dist == 'redhat':
<bitfehler>             var = 'rhel'
<bitfehler>         elif linux_dist in (
<bitfehler>                 'opensuse', 'opensuse-tumbleweed', 'opensuse-leap', 'sles'):
<bitfehler>             var = 'suse'
<bitfehler>         else:
<bitfehler>             var = 'linux'
<rharper> bitfehler: we'd certaily accept a patch to help set the variant to arch
<bitfehler> cool, can do
<rharper> if that's found with get_linux_distro() method;
<rharper> which it sounds like it does
<bitfehler> totally
<bitfehler> which leads me to my last question :) i figured it would be good to add some arch-specifics into the template generation as well then
<rharper> for sure
<bitfehler> and i was wondering what the approach should be there? run every module that can be reasonably expected to work on the distro?
<rharper> and likely revist the distros value in the config modules
<rharper> bitfehler: yeah, you can force all modules to run or you could run each one via cloud-init single --force --name
<rharper> https://wiki.archlinux.org/index.php/Cloud-init
<rharper> even mentions using unverified_modules
<bitfehler> yeah, i guess it would be nice to get to the point where that is no longer neccessary. i wasn't aware that the modules declare this as well, thanks for pointing that out
<bitfehler> cool, thanks a lot for all that, i'll send a few patches your way soon :)
#cloud-init 2019-07-27
<seven-eleven> whats the difference between "ssh_import_id" and "ssh_authorized_keys:"
#cloud-init 2020-07-20
<ftarasenko> Hi! Team, can someone point me to right VLAN configuration for Centos 8 by cloud-init 18.5? Currently, i get TYPE=Ethernet instead of TYPE=Vlan in ifcfg-bond0.3996 file.
<ftarasenko> Is this a bug or expected behavior? Tnx!
<meena> ftarasenko: can you try the latest, and see if it's the same issue?
<ftarasenko> meena: I see that there is unfixed bug with this problem, so latest won't help. https://bugs.launchpad.net/cloud-init/+bug/1788915
<ubot5> Ubuntu bug 1788915 in cloud-init "sysconfig renders vlan with TYPE=Ethernet" [Medium,Triaged]
<meena> ftarasenko: yeah, looking up on the bug-tracker, that's an option, tooâ¦ whichever is faster, lol
<rharper> powersj ... files a bug and isn't here ...    https://bugs.launchpad.net/bugs/1888298
<ubot5> Ubuntu bug 1888298 in cloud-init "cc_timezone fails on Ubuntu Bionic and Xenial minimal" [Undecided,New]
<rharper> what of Focal?    does that fail as well, if so why not ?
#cloud-init 2020-07-21
<rharper> otubo: is RHEL8  NetworkManager only now?
<soupyy> hi, I'm trying to build a cloud init image from a cloud config file, but I'm on Gentoo and portage doesn't seem to have cloud-image-utils, is there a way to do this without cloud-localds?
<smoser> soupyy: just use it outsie of a package.
<smoser> https://github.com/canonical/cloud-utils/blob/master/bin/cloud-localds
<rharper> soupyy: yeah; it's just a directory turned into an iso image; you can likely pull down the source code and just install the required binaries,
<smoser> it uses genisoimage and qemu-img
<soupyy> oh ok awesome, thanks!
<rharper> smoser: does anyone put up simplestreams for centos kvm images?    lxd has streams and centos vm images ...  can't seem to get sstream-query to read from images.linuxcontainers.org:8443 ;  have you queried them directly before ?
<smoser> looking
<smoser> rharper: sstream-query --max=1 --no-verify --pretty https://images.linuxcontainers.org/streams/v1/images.json os=Centos
<rharper> smoser: thanks , I was pointing at 8443 port (which cloud_tests does)
<smoser> are you sure that there are vm images there ?
<rharper> also works ... hrm
<rharper> for centos7/8 on amd64
<rharper> I see, I had the wrong path  (needed streams/images.json)
<smoser> there are.
<smoser>  sstream-query --max=1 --pretty https://images.linuxcontainers.org/streams/v1/images.json os=Centos ftype=disk-kvm.img
<smoser> so... the reason you aveh to pass the images.json
<smoser> is that by default it hits 'streams/v1/index.sjson' (sjson == signed json)
<rharper> ah
<smoser> but on linuxcontainers.org the signed json points at an *unsigned* streams/v1/images.json
<smoser> i just raised that in lxc-dev
<rharper> I'm -> <-> close on getting a cloud_test for this chronyd service name change on centos8;  but it's been uphill;   centos8 has NM only (AFACIT) so we render sysconfig files with NM_CONTROLLED=no;  so no networking comes up in centos8 (cloud container images);
<smoser> you could also just go to the https://images.linuxcontainers.org/streams/v1/images.json
<smoser> rharper: i would not put much stock in anything from linuxcontainers.org
<rharper> cpaelzer wrote a nice wrapper around chronyd.service to check if it can adjust time inside a container (awkwardly unpriv containers in lxd expose cap_time_sys but it's actually denied )
<rharper> smoser: image-wise; yes .. it's inconsistent
<rharper> but nothing else indexes those images AFAICT  ...  and cloud_tests doesn't have infra for just wget'ing kvm images from various URLs
<smoser> right. whether it has wicked or sysconfig or network manager or 'ip scripts' doesn't mean anything really.
<smoser> i'd suggest that http://cloud.centos.org/centos/8/x86_64/images/ are official
<rharper> yeah, that's where I've downloaded images before;  I don't think they have any sort of stream publishing though, or even daily/8/amd64/current URL pointer
<smoser> they dont.
<smoser> ugh
#cloud-init 2020-07-22
<meena> is it just me, or has it been quite â¦ quiet in here?
#cloud-init 2020-07-23
<Odd_Bloke> uebera||: Can I confirm your issue from a few days ago: changing `500` to `"500"` in your config is what fixed your UID issue?  Or did you end up finding something else going on?
<Odd_Bloke> meena: I've been lying on the floor with back pain all of this week so far, which has probably contributed to the silence. :p
<uebera||> Odd_Bloke: Yes, just using a string "500" instead of an integer 500 solved my problem with respect to UIDs.
<uebera||> If only there was a way to specify GIDs (as far as I understand, a certain patch was not applied). I used runcmd (along with sed) to change the GID of a created account (requires a reboot), but that's really, really dirty.
<Odd_Bloke> uebera||: Would you be interested in contributing a gid change?
<uebera||> Odd_Bloke: If this --> https://bugs.launchpad.net/cloud-init/+bug/1396362/comments/5 does not apply cleanly anymore, I'd be interested in a modification (have not tested this yet).
<ubot5> Ubuntu bug 1396362 in cloud-init "Support UID and GID specification in user and group definitions" [Low,Triaged]
<uebera||> Given that the VM in question is now up and running, I should have time to look at the above during the weekend.
<Odd_Bloke> uebera||: Given that we do now support `uid`, I don't think that patch will cleanly apply (because it includes changes to support `uid`).  If you do end up having time, we'd love the contribution; the getting started guide is here: https://cloudinit.readthedocs.io/en/latest/topics/hacking.html#submitting-your-first-pull-request
<Odd_Bloke> (If you hit any hurdles, we're happy to help in here, but note that most of the committers only monitor this channel in ~North American weekday timezones so assistance over the weekend may be sparse. :)
<meena> Odd_Bloke: obviiously, the solution to that then is that you should complain about your back pain, whenever that happens. a lot.
<uebera||> Odd_Bloke: Thanks for the link, bookmarked. ;)
<meena> i cannot believe i was just removed from the Canonical github organisation!
<meena> why was i on it again??
<powersj> meena, yeah I couldn't recall and all I saw was read-only
<powersj> so read-only on a public project didn't make sense
<meena> it's just jokey fake outrage
<meena> Odd_Bloke: added me like that, so y'all could request me as reviewer
<meena> but of course, that probably wasn't documented, and anyway, most authorization UIs don't really have a space for "why", or?
#cloud-init 2020-07-24
<meena> I've been invited to the canonical org again!
<Odd_Bloke> meena: powersj: Yep, it was so we could request you as a reviewer; we're adding you back ATM.
<Odd_Bloke> (Our IS department are introducing automation to sync LDAP and GitHub to ease starter/leaver tasks; you just need adding to an allowlist. :)
<Odd_Bloke> https://github.com/canonical/cloud-init/pull/493 is no longer WIP, and is ready for review/landing.
<Odd_Bloke> smoser: rharper: Given ds-identify should mean that we only use the OpenStack DS when we should expect a response, https://github.com/canonical/cloud-init/pull/501 seems like a reasonable change; any thoughts?
<smoser> so current behavior is it will try once, with a socket timeout of 10 seconds ?
<smoser> before ditching it?
<smoser> non-intel is an issue
<rharper> Odd_Bloke: I wonder if we should order the 90_dpkg.cfg datsource_list by increasing timeout?   for non-intel, they are trying all ds ; NoCloud, ConfigDrive, OpenNebula, DO, Azure, AltCld, OVF, MAAS, GCE before openstack ;   for intel they'll head right for it (and not trying anything else);
<Odd_Bloke> Oh, because non-Intel don't (necessarily) have DMI data?
<rharper> correct
<rharper> so it's a maybe and you don't get the short list
<rharper> beyond optimization; for non-ds id paths;  the change seems fine; though going from 10 to 120 is a big jump for timeouts;    but generally I don't think there's anything wrong with the change;
<smoser> its hard... because the truth is, if your metadata service isn't there, its because you have a failed cloud.
<smoser> in the past 10 years, i have never seen ec2 metadata service down.
<smoser> but, imo because of bad design, every openstack has issues with terribly slow metadata service.
<smoser> so one par tof me says "sure, change cloud-init to work better on your little toy"
<smoser> and the other part says "fix your silly cloud."
<smoser> but we do not want a larrge timeout on non-intel unless there is only one datasource in the list.
<rharper> smoser: we had a similar discussion around ipv6 imds for openstack ...   without a non-networky way to know whether it's worth waiting or not; you just end up waiting;
#cloud-init 2020-07-25
<meena> 22:46 <@smoser> but, imo because of bad design, every openstack has issues with terribly slow metadata service. â¬ï¸ do you mean bad design of OpenStack, or bad design of people's OpenStack installations?
