[01:22] hmmm, are these new errors [01:22] https://gist.github.com/harlowja/e4fc7bd6092cf7444a8d02155878e2fc [01:22] (running on a mac) [01:22] more things not mocked? [11:34] harlowja, :-( === shardy is now known as shardy_lunch === shardy_lunch is now known as shardy [16:55] Three year old bug... https://bugs.launchpad.net/cloud-init/+bug/1404060 ... would be nice if it was fixed [17:24] dgarstang, were you the one that mentioned the related permissions issue on created directories ? [17:24] smoser: I don't think that was me [17:25] that was wraithm [17:25] 07/25/16 16:11:31 I'm trying to make an EC2 AMI with a centralized authorized keys file (ie. in sshd_config: AuthorizedKeysFile /etc/ssh/authorized_keys/%u). I see in the cloud-init source-code, cloudinit/ssh_util.py, the function setup_user_keys asserts the parent folder to be mode 700. Those permissions don't work with this AuthorizedKeysFile value. Having /etc/ssh/authorized_keys be mode 755 is what works. Is there a w [17:25] ay around this? [17:25] hi folks, i had a build question. i am trying to create a local build of cloud-init using the latest revision, but I see pep8 errors. is this a known issue? seems strange because the launchpad build is passing [17:25] smoser: similar, but different [17:26] hans__, pep8 is an obnoxiously moving target [17:28] smoser - any recommendations for getting a successful local build? [17:29] you're seeing errors on tests/unittests/test_handler/test_handler_mcollective.py ? [17:29] yes, exactly. 4 errors [17:29] i'll fix that. :-( [17:29] where did you grab cloud-init from ? [17:29] thank you. lp:cloud-init [17:30] smoser: me? [17:31] hans___, git ? [17:31] dgarstang, sorry, that was for dgarstang [17:31] smoser: oh, it's just the latest canonical upstream ami [17:32] dgarstang, sorry, the question about cloud-init was for hans___ :) [17:32] smoser, using bzr-builder, since it looked like the git recipe packaging hasn't landed yet? [17:32] smoser: take upstream ami, rebundle into a new ami with packer. that new rebundle modifies /etc/ssh/sshd_config to add second location of ssh keys. When 2nd instance boots, cloud-init reads the sshd config file, and puts the packer key in the wrong place [17:32] smoser: D'oh. Ok [17:33] hans___, ok. [17:39] hans___, i'll get the flake8 issue fixed in git. [17:39] not sure what i want to do about the other. [17:39] smoser: What do you think about splitting the ssh module into a host ssh-keygen module and an inserting user authorized keys module? [17:40] wraithm, that does seem like a reasonable idea. [17:40] shouldn't be to hard either wraithm [17:40] its pretty seperated in code as is [17:41] so your issue, wraithm , what would you think the right thing to do is there? [17:42] ok. so lets get git working [17:42] Yeah! The code was very easy to read. [17:43] So, the runcmd: - 'chmod 755 /etc/ssh/authorized_keys' suggestion worked for me [17:43] but what would be ideal is having a module for only generating host ssh keys === gfidente is now known as gfidente|afk [17:45] I was also very tempted by not running the ssh module and just using runcmd/bootcmd to run ssh-keygen [17:45] so the format of /etc/ssh/authorized_keys is the same as ~/.ssh/authorized_keys ? [17:46] Not quite. It's a folder with a bunch of authorized_keys files in it [17:46] oh. [17:46] directory [17:46] ok. yeah. [17:46] in sshd_config I have a line: AuthorizedKeysFile /etc/ssh/authorized_keys/%u [17:46] but even if not split out, its only seting the permissions because generally it has to. [17:46] where %u is the username of the person attempting to log in [17:46] if you set ~/.ssh/ to world writable, ssh will ignore the keys [17:47] Yes [17:47] But /etc/ssh/authorized_keys needs to be world readable [17:47] in my case [17:48] ie 755 instead of 700, which I think is what ~/.ssh/ gets chmodded to in ssh_util.py [17:48] Another possible solution is changing the code to do different modes depending on the location of the file [17:51] right. [17:51] that seems to be the right thing, trying to think of what the right rules for permissions are [17:52] I don't know know what sshd's permissions logic is :/ It's not really mentioned anywhere in the man pages. [17:52] it seems like 700 is overkill actually [17:52] my desktop has 755 on ~/.ssh/ [17:52] Potentially [17:52] What about ~/.ssh/authorized_keys? [17:53] oh, nevermind [17:53] So, 755 works for ~/.ssh? Maybe that's the solution. [17:55] seems 777 is even ok [17:55] wait [17:55] no [17:55] seems 775 is ok [17:55] just can't have other writable [17:55] this is testing i'm doing on yakkety, so different ssh's could have different behavior [17:55] but maybe [17:56] Interesting [17:57] Though, now I'm thinking that I still feel like generating host keys should be a different operation from inserting user authorized_keys. You could imagine people with some single-sign-on scheme (ldap/kerberos or whatever) baked into a cloud image. Authorized_keys is irrelevant to them. [17:59] wraithm, but it only inserts them if keys were provided. [17:59] so, then providing keys to the system is "irrelevant to them" [18:00] this seems sane for my local quick testing: [18:00] http://paste.ubuntu.com/21032867/ [18:00] Oh really? I look at the code and it looks like the root user always gets their key inserted. [18:00] well, only if there was akey to insert [18:00] if theres no keys, then it shouldnt do anything. if it does (and it breaks things) then yeah, we need to fix that too [18:01] Oh, okay. So, what if I wanted to create some default users, but I still don't want the keys to be inserted? [18:01] Which is actually sorta what I'm doing. [18:01] it only inserts the key in to the default user i think [18:01] Right, and that breaks my thing [18:01] the one that gets identified as 'default' [18:02] not to all users. [18:02] Because ssh_util.py asserts /etc/ssh/authorized_keys to 700 [18:02] which is a directory in my case analogous to ~/.ssh [18:02] right. read that paste above though... i think that logic would work for you [18:03] i'm not opposed to your suggestion of splitting them, but even if they're split, the key insertion is doing the wrong thing for the case you provided. [18:04] 755 doesn't have world write permissions [18:04] oh [18:04] nvm [18:04] I see [18:04] Yes [18:04] That code looks good [18:04] :) [18:11] smoser, i made the pep8 fixes in a private branch, but it looks like i see some test failures as well. checked that these are present in lp revision <1258 as well. there are 8 failures, all about virtual routers. do i need to set this up locally to run the tests? [18:14] hans___, hm.. probably similar to what harlowja complained about above. [18:14] ya, probably [18:15] :-( [18:15] we really need some infrastgructure to test on non-ubuntu [18:15] hans___, what is your system you're running on ? [18:15] and are you running tox ? [18:16] smoser - can't see the history, unfortunately, since i'm in webchat at the moment. i'm running pbuilder in a xenial container. ran tox successfully yesterday, haven't tried today [18:20] smoser, just ran tox again successfully [18:44] hm. [18:44] this sucks [18:45] the import of lp:cloud-init from bzr to git [18:45] a.) "flattened" all history. making log of trunk show lots of people's in-branch commits that in bzr are nicely hidden , and only shown when bzr log --include-merged [18:46] not a huge deal, but means that 'git log' is full of stuff like: [18:46] Remerge against head/master [18:46] lol [18:46] merge from trunk [18:46] durn [18:46] poo-crap-commits [18:47] right, which bzr handles so nicely. it allows you to accept your human. [18:47] and that you suck [18:47] other thing is that we lose the '--fixes' metadata [18:47] :( [18:47] durn [18:47] the feature i used every now and then [18:48] if i remembered to [18:48] lol [18:48] i'm gonna look see if there is any obvious way to fix those things [18:48] i'd be ok if it onverted the '--fixes' metadata into (Fixes LP: #XXXXX) [18:48] and i think i'd even be ok if it just purged the merged [18:49] i'd prefer that to a bunch of "fix spelling" and "fix tox i never thought someone would see this" commits [18:50] https://www.fusonic.net/en/blog/migrating-from-bazaar-to-git/ [18:51] was someone else hacking on package/brpm ? I'd really like to build my branch as an rpm so I can test a cross-distro feature; I've got most of the bzr to git stuff fixed up (just a few more naming things w.r.t using git archive instead of bzr export); I saw some mention of spec files and such a few days ago [18:52] larsks, ^ [18:52] rharper, ^ [18:52] cool [18:53] rharper: yeah, me...there's a proposed merge that should work. [18:53] And if it doesn't work, let me know :) [18:53] ok [18:53] that link for fusionic... c# [18:53] really ? [18:53] also noticed the basic renderer doesn't like the spec file and python-cheetah was a requirement; [18:53] smoser u're probably gonna have to do that soon, and squash those commits into the one that actually matters [18:53] and then force update the repo [18:53] yeah [18:54] its really basic [18:54] lol [18:54] right, I mean, the spec file didn't include the # template: cheetah [18:54] oops [18:54] which it should if it requires cheetah [18:54] which it does [18:54] agreed [18:54] who wrote that crap [18:54] crappp [18:54] lol [18:54] probably that scott guy [18:55] rharper: the spec file is correctly identified as a cheetah template in any case, but yeah you need cheetah installed for that to work :) [18:55] I ran into the same thing a day or two ago. [18:55] I sort of think the templater should blow up instead of falling back to basic... [18:56] ya, stupid rendering crap [18:56] who wrote that [18:56] lol [18:59] it has a "## this is a cheetah template" but the type matcher, wants '## template: cheetah' ; [18:59] TYPE_MATCHER = re.compile(r"##\s*template:(.*)", re.I) [18:59] that said, even if the template was marked correctly, it would do the fallback [18:59] though log that it could't find cheetah [19:07] rharper: I updated the merge request to include the appropriate template: line in both (redhat + suse) spec files. [19:07] cool [19:12] larsks: do you have a link to your merge ? [19:13] * rharper is looking for what you did around oauthlib [19:14] rharper: sure. uh...did nothing arounmd oauthlib, though. https://code.launchpad.net/~larsks/cloud-init/+git/cloud-init/+merge/300953 [19:14] and the rpm installs on centos 7 ? I didn't find a python-oauthlib package available [19:15] I haven't made any changes to requirements. I've tested the rpm building on fedora. Let me take a look at centos. [19:18] CentOS only ships 0.7.5, which means that requirements introduced sinc then aren't available. You can get python-oauthlib through EPEL, though. [19:19] I am working on getting a newer version of cloud-init into RHEL, which means it should eventually show up in centos. We may bundle all the requirements into a single cloud-init-deps package, at least in the short term. [19:19] rharper: ^^^ [19:19] cool [19:40] larsks: you're branch just worked =) thanks [19:41] That's what I like to hear :) [19:42] I added epel-release, and the installed the cloud-init package created; +1 [19:43] so, something that we should look at is the default /etc/cloud/cloud.cfg sets the distro to ubuntu [19:44] smoser: not sure how we 'd want to template that in the default config file [19:49] rharper: one option would be to produce cloud.cfg for the packages are part of the brpm/bddeb scripts. [19:49] *as part of... [19:49] yeah [19:49] hrm, centos7 has fun wit python2.6 and 2.7 and argparse .. [19:50] rharper: but centos 7 ships with python 2.7... [19:50] where does 2.6 come into things? [19:50] you're right; [19:50] i'm fine with requiring distro to ship something that changes the default user. [19:50] I saw some thread mention 2.6 didn't have argparse package [19:50] and fine carrying 'clouduser' as default even. [19:50] it doesnt have it builtin [19:50] pkg_resources doesn't find 'argparse' package which is required for cloud-init; but not expressed in the spec file [19:51] oh yeahy, and something was really wierd there for rh. [19:51] smoser: ah; I didn't find the obvious python-argparse package in epel or otherwise ; [19:51] python2.7 says it provides python-argparse [19:51] harlowja, had more info on that. [19:51] but pgk_resources does't find it [19:52] right [19:52] i think i'm going to have to push overwrite git history. [19:52] i dont have it yet, but i really dont want to lose fixes metadata [19:52] smoser: also, on the distro front; in lxd images: there is an opensuse image, but no sles that I can find (like no rhel) ; would you be ok to add 'opensuse' to the distro classes ? [19:53] i woudl have thoguht something worked there. but yeah. [19:53] rharper: yeah, you shouldn't need python-argparse, and I thought there was some logic in there to avoid listing the requirement for python 2.7 and later... [19:53] rharper: still an issue for the package? [19:54] larsks: let me merge in trunk [19:54] but I think I'm pretty recent (from Friday I think) [19:54] just running /usr/bin/cloud-init --help on centos7 after package/brpm from your branch shows me this argparse pkg thingy [19:55] rharper: let me take a look, it's possible I lost some logic while fiddling with brpm. [19:58] sure; http://paste.ubuntu.com/21048423/ [19:58] that's what I see [19:59] fedora24 fails in the same way; [20:01] Yeah, that's because I seem to have dropped some code that handled this in brpm. Sorry! Fixing. [20:02] I see this https://bugzilla.redhat.com/show_bug.cgi?id=700596#c2 [20:02] so sounds like we should drop it from the setup requirements [20:03] python2.7 -c 'import argparse' does indeed work on c7 and f27 [20:03] f24 [20:03] larsks, i saw a change to finding topdir that used git, right? [20:03] smoser: yeah. [20:03] i'd prefer to not rely on git, as the tools ideally would work without it. [20:03] at least read-requirements and such. [20:04] at least it seems sane to me that an environment variable can be set to say "this is where the top is" and then it use that. [20:04] Okay. I wanted it to work without having to set environment variables :) [20:05] sure. it shoudl do that too [20:05] it did at least. [20:05] I will fix it up. [20:06] again.. just so you're aware, i'm going to try to get a re-import done [20:07] Sure. Just ping me when you do that and I will fix the merge request. [20:18] smoser: rharper: i've just pushed new changes to that merge request. (a) this squashes everything down to a single commit, and (b) this correctly excludes argparse from the requirements for python >= 2.7 and (c) this will use CLOUD_INIT_TOP_D in read-dependencies if it is set. [21:02] larsks do u know how many people use spacewalk [21:03] we have this code that i may just make into a real-clout-init-top-level-module [21:03] https://gist.github.com/harlowja/e0b5a98844554d459338e3d72bbf7b7d [21:03] harlowja: I have no idea... [21:03] damn, redhat people supposed to know [21:03] lol [21:03] harlowja: not my bailiwick :) Ask me about openstack... [21:03] whats openstack [21:03] sounds weird [21:03] lol [21:03] pancakestack [21:05] Mmmm, pancakes... [21:05] ok, i'll submit a spacewalk module [21:05] i convinced myself [21:05] lol [21:05] smoser did u push up to git with squished things [21:06] probably shouldn't do much with git until u do that [21:06] otherwise git be screwy when i `git pull` === psauxww is now known as er1k757