[01:22] <harlowja> hmmm, are these new errors
[01:22] <harlowja> https://gist.github.com/harlowja/e4fc7bd6092cf7444a8d02155878e2fc
[01:22] <harlowja> (running on a mac)
[01:22] <harlowja> more things not mocked?
[11:34] <smoser> harlowja, :-(
[16:55] <dgarstang> Three year old bug... https://bugs.launchpad.net/cloud-init/+bug/1404060 ... would be nice if it was fixed
[17:24] <smoser> dgarstang, were you the one that mentioned the related permissions issue on created directories ?
[17:24] <dgarstang> smoser: I don't think that was me
[17:25] <smoser> that was wraithm
[17:25] <smoser> 07/25/16 16:11:31 <wraithm>   I'm trying to make an EC2 AMI with a centralized authorized keys file (ie. in sshd_config: AuthorizedKeysFile /etc/ssh/authorized_keys/%u). I see in the cloud-init source-code, cloudinit/ssh_util.py, the function setup_user_keys asserts the parent folder to be mode 700. Those permissions don't work with this AuthorizedKeysFile value. Having /etc/ssh/authorized_keys be mode 755 is what works. Is there a w
[17:25] <smoser> ay around this?
[17:25] <hans__> hi folks, i had a build question. i am trying to create a local build of cloud-init using the latest revision, but I see pep8 errors. is this a known issue? seems strange because the launchpad build is passing
[17:25] <dgarstang> smoser: similar, but different
[17:26] <smoser> hans__, pep8 is an obnoxiously moving target
[17:28] <hans___> smoser - any recommendations for getting a successful local build?
[17:29] <smoser> you're seeing errors on tests/unittests/test_handler/test_handler_mcollective.py ?
[17:29] <hans___> yes, exactly. 4 errors
[17:29] <smoser> i'll fix that. :-(
[17:29] <smoser> where did you grab cloud-init from ?
[17:29] <hans___> thank you. lp:cloud-init
[17:30] <dgarstang> smoser: me?
[17:31] <smoser> hans___, git ?
[17:31] <smoser> dgarstang, sorry, that was for dgarstang
[17:31] <dgarstang> smoser: oh, it's just the latest canonical upstream ami
[17:32] <smoser> dgarstang, sorry, the question about cloud-init was for hans___  :)
[17:32] <hans___> smoser, using bzr-builder, since it looked like the git recipe packaging hasn't landed yet?
[17:32] <dgarstang> smoser: take upstream ami, rebundle into a new ami with packer. that new rebundle modifies /etc/ssh/sshd_config to add second location of ssh keys. When 2nd instance boots, cloud-init reads the sshd config file, and puts the packer key in the wrong place
[17:32] <dgarstang> smoser: D'oh. Ok
[17:33] <smoser> hans___, ok.
[17:39] <smoser> hans___, i'll get the flake8 issue fixed in git.
[17:39] <smoser> not sure what i want to do about the other.
[17:39] <wraithm> smoser: What do you think about splitting the ssh module into a host ssh-keygen module and an inserting user authorized keys module?
[17:40] <smoser> wraithm, that does seem like a reasonable idea.
[17:40] <harlowja> shouldn't be to hard either wraithm
[17:40] <harlowja> its pretty seperated in code as is
[17:41] <smoser> so your issue, wraithm , what would you think the right thing to do is there?
[17:42] <smoser> ok. so lets get git working
[17:42] <wraithm> Yeah! The code was very easy to read.
[17:43] <wraithm> So, the runcmd: - 'chmod 755 /etc/ssh/authorized_keys' suggestion worked for me
[17:43] <wraithm> but what would be ideal is having a module for only generating host ssh keys
[17:45] <wraithm> I was also very tempted by not running the ssh module and just using runcmd/bootcmd to run ssh-keygen
[17:45] <smoser> so the format of /etc/ssh/authorized_keys is the same as ~/.ssh/authorized_keys ?
[17:46] <wraithm> Not quite. It's a folder with a bunch of authorized_keys files in it
[17:46] <smoser> oh.
[17:46] <smoser> directory
[17:46] <smoser> ok. yeah.
[17:46] <wraithm> in sshd_config I have a line: AuthorizedKeysFile /etc/ssh/authorized_keys/%u
[17:46] <smoser> but even if not split out, its only seting the permissions because generally it has to.
[17:46] <wraithm> where %u is the username of the person attempting to log in
[17:46] <smoser> if you set ~/.ssh/ to world writable, ssh will ignore the keys
[17:47] <wraithm> Yes
[17:47] <wraithm> But /etc/ssh/authorized_keys needs to be world readable
[17:47] <wraithm> in my case
[17:48] <wraithm> ie 755 instead of 700, which I think is what ~/.ssh/ gets chmodded to in ssh_util.py
[17:48] <wraithm> Another possible solution is changing the code to do different modes depending on the location of the file
[17:51] <smoser> right.
[17:51] <smoser> that seems to be the right thing, trying to think of what the right rules for permissions are
[17:52] <wraithm> I don't know know what sshd's permissions logic is :/ It's not really mentioned anywhere in the man pages.
[17:52] <smoser> it seems like 700 is overkill actually
[17:52] <smoser> my desktop has 755 on ~/.ssh/
[17:52] <wraithm> Potentially
[17:52] <wraithm> What about ~/.ssh/authorized_keys?
[17:53] <wraithm> oh, nevermind
[17:53] <wraithm> So, 755 works for ~/.ssh? Maybe that's the solution.
[17:55] <smoser> seems 777 is even ok
[17:55] <smoser> wait
[17:55] <smoser> no
[17:55] <smoser> seems 775 is ok
[17:55] <smoser> just can't have other writable
[17:55] <smoser> this is testing i'm doing on yakkety, so different ssh's could have different behavior
[17:55] <smoser> but maybe
[17:56] <wraithm> Interesting
[17:57] <wraithm> Though, now I'm thinking that I still feel like generating host keys should be a different operation from inserting user authorized_keys. You could imagine people with some single-sign-on scheme (ldap/kerberos or whatever) baked into a cloud image. Authorized_keys is irrelevant to them.
[17:59] <smoser> wraithm, but it only inserts them if keys were provided.
[17:59] <smoser> so, then providing keys to the system is "irrelevant to them"
[18:00] <smoser> this seems sane for my local quick testing:
[18:00] <smoser>  http://paste.ubuntu.com/21032867/
[18:00] <wraithm> Oh really? I look at the code and it looks like the root user always gets their key inserted.
[18:00] <smoser> well, only if there was akey to insert
[18:00] <smoser> if theres no keys, then it shouldnt do anything. if it does (and it breaks things) then yeah, we need to fix that too
[18:01] <wraithm> Oh, okay. So, what if I wanted to create some default users, but I still don't want the keys to be inserted?
[18:01] <wraithm> Which is actually sorta what I'm doing.
[18:01] <smoser> it only inserts the key in to the default user i think
[18:01] <wraithm> Right, and that breaks my thing
[18:01] <smoser> the one that gets identified as 'default'
[18:02] <smoser> not to all users.
[18:02] <wraithm> Because ssh_util.py asserts /etc/ssh/authorized_keys to 700
[18:02] <wraithm> which is a directory in my case analogous to ~/.ssh
[18:02] <smoser> right. read that paste above though... i think that logic would work for you
[18:03] <smoser> i'm not opposed to your suggestion of splitting them, but even if they're split, the key  insertion is doing the wrong thing for the case you provided.
[18:04] <wraithm> 755 doesn't have world write permissions
[18:04] <wraithm> oh
[18:04] <wraithm> nvm
[18:04] <wraithm> I see
[18:04] <wraithm> Yes
[18:04] <wraithm> That code looks good
[18:04] <wraithm> :)
[18:11] <hans___> smoser, i made the pep8 fixes in a private branch, but it looks like i see some test failures as well. checked that these are present in lp revision <1258 as well. there are 8 failures, all about virtual routers. do i need to set this up locally to run the tests?
[18:14] <smoser> hans___, hm.. probably similar to what harlowja complained about above.
[18:14] <harlowja> ya, probably
[18:15] <smoser> :-(
[18:15] <smoser> we really need some infrastgructure to test on non-ubuntu
[18:15] <smoser> hans___, what is your system you're running on ?
[18:15] <smoser> and are you running tox ?
[18:16] <hans___> smoser - can't see the history, unfortunately, since i'm in webchat at the moment. i'm running pbuilder in a xenial container. ran tox successfully yesterday, haven't tried today
[18:20] <hans___> smoser, just ran tox again successfully
[18:44] <smoser> hm.
[18:44] <smoser> this sucks
[18:45] <smoser> the import of lp:cloud-init from bzr to git
[18:45] <smoser> a.) "flattened" all history. making log of trunk show lots of people's in-branch commits that in bzr are nicely hidden , and only shown when bzr log --include-merged
[18:46] <smoser> not a huge deal, but means that 'git log' is full of stuff like:
[18:46] <smoser>  Remerge against head/master
[18:46] <harlowja> lol
[18:46] <smoser>  merge from trunk
[18:46] <harlowja> durn
[18:46] <harlowja> poo-crap-commits
[18:47] <smoser> right, which bzr handles so nicely.  it allows you to accept your human.
[18:47] <smoser> and that you suck
[18:47] <smoser> other thing is that we lose the '--fixes' metadata
[18:47] <harlowja> :(
[18:47] <harlowja> durn
[18:47] <harlowja> the feature i used every now and then
[18:48] <harlowja> if i remembered to
[18:48] <harlowja> lol
[18:48] <smoser> i'm gonna look see if there is any obvious way to fix those things
[18:48] <smoser> i'd be ok if it onverted the '--fixes' metadata into (Fixes LP: #XXXXX)
[18:48] <smoser> and i think i'd even be ok if it just purged the merged
[18:49] <smoser> i'd prefer that to a bunch of "fix spelling" and "fix tox i never thought someone would see this" commits
[18:50] <smoser> https://www.fusonic.net/en/blog/migrating-from-bazaar-to-git/
[18:51] <rharper> was someone else hacking on package/brpm ?  I'd really like to build my branch as an rpm so I can test a cross-distro feature;  I've got most of the bzr to git stuff fixed up (just a few more naming things w.r.t using git archive instead of bzr export); I saw some mention of spec files and such a few days ago
[18:52] <smoser> larsks, ^
[18:52] <smoser> rharper, ^
[18:52] <rharper> cool
[18:53] <larsks> rharper: yeah, me...there's a proposed merge that should work.
[18:53] <larsks> And if it doesn't work, let me know :)
[18:53] <rharper> ok
[18:53] <smoser> that link for fusionic... c#
[18:53] <smoser> really ?
[18:53] <rharper> also noticed the basic renderer doesn't like the spec file and python-cheetah was a requirement;
[18:53] <harlowja> smoser  u're probably gonna have to do that soon, and squash those commits into the one that actually matters
[18:53] <harlowja> and then force update the repo
[18:53] <smoser> yeah
[18:54] <harlowja> its really basic
[18:54] <harlowja> lol
[18:54] <rharper> right, I mean, the spec file didn't include the # template: cheetah
[18:54] <harlowja> oops
[18:54] <rharper> which it should if it requires cheetah
[18:54] <rharper> which it does
[18:54] <harlowja> agreed
[18:54] <harlowja> who wrote that crap
[18:54] <harlowja> crappp
[18:54] <harlowja> lol
[18:54] <harlowja> probably that scott guy
[18:55] <larsks> rharper: the spec file is correctly identified as a cheetah template in any case, but yeah you need cheetah installed for that to work :)
[18:55] <larsks> I ran into the same thing a day or two ago.
[18:55] <larsks> I sort of think the templater should blow up instead of falling back to basic...
[18:56] <harlowja> ya, stupid rendering crap
[18:56] <harlowja> who wrote that
[18:56] <harlowja> lol
[18:59] <rharper> it has a "## this is a cheetah template"  but the type matcher, wants '##  template: cheetah' ;
[18:59] <rharper> TYPE_MATCHER = re.compile(r"##\s*template:(.*)", re.I)
[18:59] <rharper> that said, even if the template was marked correctly, it would do the fallback
[18:59] <rharper> though log that it could't find cheetah
[19:07] <larsks> rharper: I updated the merge request to include the appropriate template: line in both (redhat + suse) spec files.
[19:07] <rharper> cool
[19:12] <rharper> larsks: do you have a link to your merge ?
[19:13]  * rharper is looking for what you did around oauthlib 
[19:14] <larsks> rharper: sure.  uh...did nothing arounmd oauthlib, though.  https://code.launchpad.net/~larsks/cloud-init/+git/cloud-init/+merge/300953
[19:14] <rharper> and the rpm installs on centos 7 ?  I didn't find a python-oauthlib package available
[19:15] <larsks> I haven't made any changes to requirements.  I've tested the rpm building on fedora.  Let me take a look at centos.
[19:18] <larsks> CentOS only ships 0.7.5, which means that requirements introduced sinc then aren't available.  You can get python-oauthlib through EPEL, though.
[19:19] <larsks> I am working on getting a newer version of cloud-init into RHEL, which means it should eventually show up in centos.  We may bundle all the requirements into a single cloud-init-deps package, at least in the short term.
[19:19] <larsks> rharper: ^^^
[19:19] <rharper> cool
[19:40] <rharper> larsks: you're branch just worked =)  thanks
[19:41] <larsks> That's what I like to hear :)
[19:42] <rharper> I added epel-release, and the installed the cloud-init package created; +1
[19:43] <rharper> so, something that we should look at is the default /etc/cloud/cloud.cfg sets the distro to ubuntu
[19:44] <rharper> smoser: not sure how we 'd want to template that in the default config file
[19:49] <larsks> rharper: one option would be to produce cloud.cfg for the packages are part of the brpm/bddeb scripts.
[19:49] <larsks> *as part of...
[19:49] <rharper> yeah
[19:49] <rharper> hrm, centos7 has fun wit python2.6 and 2.7 and argparse ..
[19:50] <larsks> rharper: but centos 7 ships with python 2.7...
[19:50] <larsks> where does 2.6 come into things?
[19:50] <rharper> you're right;
[19:50] <smoser> i'm fine with requiring distro to ship something that changes the default user.
[19:50] <rharper> I saw some thread mention 2.6 didn't have argparse package
[19:50] <smoser> and fine carrying 'clouduser' as default even.
[19:50] <smoser> it doesnt have it builtin
[19:50] <rharper> pkg_resources doesn't find 'argparse' package which is required for cloud-init; but not expressed in the spec file
[19:51] <smoser> oh yeahy, and something was really wierd there for rh.
[19:51] <rharper> smoser: ah; I didn't find the obvious python-argparse package in epel or otherwise ;
[19:51] <rharper> python2.7 says it provides python-argparse
[19:51] <smoser> harlowja, had more info on that.
[19:51] <rharper> but pgk_resources does't find it
[19:52] <harlowja> right
[19:52] <smoser> i think i'm going to have to push overwrite git history.
[19:52] <smoser> i dont have it yet, but i really dont want to lose fixes metadata
[19:52] <rharper> smoser: also, on the distro front; in lxd images: there is an opensuse image, but no sles that I can find (like no rhel) ; would you be ok to add 'opensuse' to the distro classes ?
[19:53] <smoser> i woudl have thoguht something worked there. but yeah.
[19:53] <larsks> rharper: yeah, you shouldn't need python-argparse, and I thought there was some logic in there to avoid listing the requirement for python 2.7 and later...
[19:53] <larsks> rharper: still an issue for the package?
[19:54] <rharper> larsks: let me merge in trunk
[19:54] <rharper> but I think I'm pretty recent (from Friday I think)
[19:54] <rharper> just running /usr/bin/cloud-init --help on centos7 after package/brpm from your branch shows me this argparse pkg thingy
[19:55] <larsks> rharper: let me take a look, it's possible I lost some logic while fiddling with brpm.
[19:58] <rharper> sure;  http://paste.ubuntu.com/21048423/
[19:58] <rharper> that's what I see
[19:59] <rharper> fedora24 fails in the same way;
[20:01] <larsks> Yeah, that's because I seem to have dropped some code that handled this in brpm. Sorry!  Fixing.
[20:02] <rharper> I see this https://bugzilla.redhat.com/show_bug.cgi?id=700596#c2
[20:02] <rharper> so sounds like we should drop it from the setup requirements
[20:03] <rharper> python2.7 -c 'import argparse' does indeed work on c7 and f27
[20:03] <rharper> f24
[20:03] <smoser> larsks, i saw a change to finding topdir that used git, right?
[20:03] <larsks> smoser: yeah.
[20:03] <smoser> i'd prefer to not rely on git, as the tools ideally would work without it.
[20:03] <smoser> at least read-requirements and such.
[20:04] <smoser> at least it seems sane to me that an environment variable can be set to say "this is where the top is" and then it use that.
[20:04] <larsks> Okay.  I wanted it to work without having to set environment variables :)
[20:05] <smoser> sure. it shoudl do that too
[20:05] <smoser> it did at least.
[20:05] <larsks> I will fix it up.
[20:06] <smoser> again.. just so you're aware, i'm going to try to get a re-import done
[20:07] <larsks> Sure.  Just ping me when you do that and I will fix the merge request.
[20:18] <larsks> smoser: rharper: i've just pushed new changes to that merge request.  (a) this squashes everything down to a single commit, and (b) this correctly excludes argparse from the requirements for python >= 2.7 and (c) this will use CLOUD_INIT_TOP_D in read-dependencies if it is set.
[21:02] <harlowja> larsks do u know how many people use spacewalk
[21:03] <harlowja> we have this code that i may just make into a real-clout-init-top-level-module
[21:03] <harlowja> https://gist.github.com/harlowja/e0b5a98844554d459338e3d72bbf7b7d
[21:03] <larsks> harlowja: I have no idea...
[21:03] <harlowja> damn, redhat people supposed to know
[21:03] <harlowja> lol
[21:03] <larsks> harlowja: not my bailiwick :) Ask me about openstack...
[21:03] <harlowja> whats openstack
[21:03] <harlowja> sounds weird
[21:03] <harlowja> lol
[21:03] <harlowja> pancakestack
[21:05] <larsks> Mmmm, pancakes...
[21:05] <harlowja> ok, i'll submit a spacewalk module
[21:05] <harlowja> i convinced myself
[21:05] <harlowja> lol
[21:05] <harlowja> smoser did u push up to git with squished things
[21:06] <harlowja> probably shouldn't do much with git until u do that
[21:06] <harlowja> otherwise git be screwy when i `git pull`