/srv/irclogs.ubuntu.com/2014/05/14/#juju-dev.txt

wallyworldsinzui: just got back from vet, haven't looked at log yet. does the above scrollback mean your ppc64 tools issue is solved?00:01
sinzuiwallyworld, almost. The tools not found issue was because juju was swallowing a 40300:02
wallyworldwell that sucks, the error needs to be propagated00:02
sinzuiwallyworld, I need to get creative with /etc/hosts to get access to download the actual tool. the lab shouldn't need access to the tools00:03
sinzuiwallyworld, This is a cases where streams.canonical.com is the correct site to get tools., but ubuntu didn't build tools to put there00:04
wallyworldok00:05
wallyworldaxw: morning. standup?01:32
axwwallyworld: morning, otp with waigani01:32
wallyworldok, ping me when you're free01:33
axwwallyworld: free now01:42
waiganiwallyworld: sorry, for stealing axw, didn't realise it was your standup01:43
axwwallyworld: I need to get a proper webcam... I get that weird crackly feedback, but only on G+ for some reason01:57
wallyworldG+ maxes out my cpu also01:58
wallyworldso it's not my favourite01:58
axwwallyworld: https://bugs.launchpad.net/juju-core/+bug/1317197/comments/1402:08
_mup_Bug #1317197: juju deployed services to lxc containers stuck in pending <oil> <juju-core:Fix Committed by axwalk> <https://launchpad.net/bugs/1317197>02:08
axwI hope that's clear enough02:08
wallyworldyeah, looks good, thank you02:09
wallyworldwe do the the status stuff at some point cause if things take a while to complete that's useful02:10
axwyes absolutely02:10
axwjust didn't want ryan to think we were doing something that we weren't02:11
stokachuwallyworld: in order to speed up downloading of cloud images is there a way to specify a root image tarball to lxc-create via juju?02:26
stokachuits only 175M so not a huge deal but times like tonight its slow for some reason02:26
wallyworldnot that i am aware of directly. maybe you could put the tarball in the lxc cache dir02:26
stokachuwallyworld: ahh good idea02:27
wallyworldthe same place where lxc would put it02:27
stokachucan use juju scp or something02:27
wallyworldnot sure of exact details02:27
stokachuwallyworld: ill mess around with it and let you know02:27
wallyworldok02:27
stokachuwallyworld: it is so much faster already btw02:29
wallyworldgreat :-)02:29
wallyworlddemos going well?02:29
stokachuyea there is a contest going on in getting an openstack cloud deployed in under 30 minutes02:30
stokachui think i can make that happen with this cloning02:30
stokachuall in kvm's/lxcs02:30
stokachuthe quickest install is 33 minutes i think02:30
wallyworldare you deploying stuff to the kvm instance?02:31
wallyworldof just to the lxc containers?02:31
stokachuyea ive got like 8 kvm's running02:31
stokachu1 kvm with nested lxcs for cloud controller and other bits02:31
stokachu1 for compute02:31
stokachu3 for swift, and 3 for swift-proxy02:31
wallyworldso all charms are deployed to lxc02:32
stokachuso 8 kvms and 7 lxcs02:32
stokachuswift, swift-proxy, nova-compute are on separate kvms02:32
stokachubut everything else is on lxcs nested within 1 kvm02:32
wallyworldthere's a bug whereby if you deploy charms to kvm *and* start lxc containers within that kvm host it could fail02:33
wallyworldbut sounds like you are not affected02:33
stokachui manually bind the lxc bridge to eth0 in the kvm02:33
stokachuwith some juju scp magic02:33
stokachuand restart the kvm node before deploying the lxcs02:34
wallyworldok02:34
stokachuwallyworld: otherwise https://bugs.launchpad.net/juju-core/+bug/130453002:34
_mup_Bug #1304530: nested lxc's within a kvm machine are not accessible <addressability> <cloud-installer> <kvm> <local-provider> <lxc> <juju-core:Triaged> <https://launchpad.net/bugs/1304530>02:34
stokachuthat issue happens02:35
wallyworldyeah, that's being worked on this cycle02:35
stokachuok cool02:35
stokachunot a blocker since i can work around it02:35
wallyworldgood :-)02:35
stokachuyea i think if the nested lxc's would honor the network-bridge option in the environments file02:35
stokachuthat could work02:35
stokachubut im sure there is a better approach02:36
wallyworldi'm not sure tbh, but at this point any approach that works is good i reckon02:37
thumperdavecheney: do you have a list of the tests that currently fail on power?02:37
davecheneythumper: two secs02:38
davecheneydidn't I include that in my mail ?02:38
stokachuwallyworld: im sure people will want vlan support within those lxc's maybe02:38
wallyworldunderstandable02:38
stokachuthough it probably only matters with local provider02:39
stokachusince maas does vlan now02:39
thumperdavecheney: no, there was a link to the ppc64el bugs in juju-core, but no list of currently failing tests02:42
menn0davecheney: what does SGTM mean in a review comment? Sounds Good To Me or Silently Giggling To Myself? :)02:42
thumpermenn0: the frist02:42
menn0davecheney: thumper: I figured. Next question, does the landing bot treat SGTM the same as LGTM?02:43
thumperI don't think so02:43
menn0ok thanks for clarifying02:43
davecheneymenn0: sounds good to me02:44
menn0I saw davecheney use it and wasn't sure if it has special meaning to the landing bot02:44
thumperdavecheney: also, any idea why juju/errors tests fail with gccgo?02:44
thumperdavecheney: I can't see it myself02:44
thumperdavecheney: given that I had the ones in arrar passing with gccgo02:45
davecheneyon da phnoe02:47
thumperkk02:48
davecheneymenn0: the sgtnm/lgtm thing comes from code review02:49
davecheneylgtm (case insensitive) trigger the green success line in code review02:49
wallyworldthumper: we want to introduce a universally used base test suite to kill outgoing http egress, protect the user's home real dir, encapsulate loggingsuite etc - do you like the name "BaseSuite"?02:49
davecheneyeven if you say02:49
davecheneythere is no way in hell i would lgtm this in a millino years02:49
thumperwallyworld: sounds fine02:49
davecheneyit's still a thimbs up02:49
davecheneyso, sgtm is a way of saying i like it, but i'm not prepared to put my name to it02:49
davecheneythumper: i'll take a look a tthe errors tests now02:50
davecheneyi have trusty and gccgo02:50
thumperdavecheney: to avoid the juju/errors package02:50
thumperyou could just start with juju/errgo02:50
menn0davecheney: makes sense. thanks.02:50
thumperwhich also fails with gccgo02:50
thumperget the foundations right first I guess02:50
davecheneythumper: got time for a hangout02:52
thumperyup02:52
davecheneyi think i'm confused about which error packge is broken02:53
thumper:)02:53
davecheneythis being able to hangout with your boss thing is sorta neat02:53
davecheney... only taken 2 years02:53
thumperdavecheney: https://plus.google.com/hangouts/_/g36ayyi6golhypoflszins4leia?hl=en02:54
axwwallyworld: if you haven't thought about this already: the thread on signature wrapping reminds me that we should create a git pre-push hook, a la lbox.check03:03
axwwallyworld: also, would be nice to run the checks in the lander bot, if we don't already03:04
wallyworldagree with lander bot checks, hadn't thought of a pre push hook03:04
wallyworldsounds like a good idea03:04
wallyworldi was deferring thinking about it till next week when hopefully there'll be some progress on the jenkins side of things03:06
rick_h_wallyworld: we do that https://github.com/juju/juju-gui/blob/develop/HACKING.rst#hooks and then in our CI lander side http://ci.jujugui.org:8080/job/juju-gui-merge/03:08
wallyworldrick_h_: awesome, thanks, we'll look to do something similar03:09
rick_h_wallyworld: yea, let me know if you want a run through the lander stuff we setup when you get that far https://github.com/juju/jenkins-github-lander03:09
wallyworldrick_h_: will do. curtis was going to start looking at setting up juju-core's jenkins next week03:10
rick_h_wallyworld: cool03:10
wallyworldi already have a migrated repo to test with03:10
rick_h_very nice03:10
wallyworldgithub/wallyworld/core03:10
rick_h_when you get the migration script let me know. We want to move more stuff over but we didn't have it collapse commits so our history is messy03:11
wallyworldhas a pending pull request to merge and everything :-)03:11
rick_h_:)03:11
wallyworldi have the migration steps which i follow, and i think i collapsed history03:11
rick_h_cool maybe I can see if sinzui's team has the bandwidth to subordinate the lander code for jenkins03:11
rick_h_wallyworld: cool, yea when I use bzr fast import it didn't collapse so lots of old commits of "fix lint"03:11
wallyworldrick_h_:  i used these steps http://flexion.org/posts/2012-10-migrating-bzr-to-git.html03:12
wallyworldrick_h_: as a rule, i hate how git workflow seems to be to rebase out history03:13
wallyworldthrows away information03:13
wallyworldbzr's model is much nicer03:13
rick_h_yea, but it's usually useless information ime03:13
wallyworldhide it by default, but show it if needed03:13
rick_h_but I can appreciate both sides03:13
axwwallyworld: can you please update gomaasapi on the bot?03:13
wallyworldsure03:13
axwrick_h_: is there a way to see old comments in GH after you rebase?03:14
axwline comments03:14
rick_h_wallyworld: can you see https://bazaar.launchpad.net/~rharding/juju-gui-lander/trunk/files03:14
rick_h_axw: it leaves the comments, but marks them as "xxx commented on an outdated branch"03:14
axwhmm, when I have done PRs the comments always disappear after I rebase and push -f03:15
wallyworldrick_h_: yep03:15
rick_h_wallyworld: so might be useful to sinzui as well I don't know. That's the scripted setup we used to migrate03:16
rick_h_wallyworld: the git upload, the jenkins deploy/initial install/etc03:16
wallyworldrick_h_: ok, ta. i've sent curtis the link previously03:16
rick_h_wallyworld: there's some api creds and such in there which is why it's private03:16
rick_h_ah ok03:16
wallyworldwell, i think so03:16
sinzuithat doesn't look right03:17
sinzuibzr-git and git-bzr can preserve commit histoyr03:17
sinzuiI symlink my .bazaar/ignore to .gitignore03:17
rick_h_axw: https://github.com/juju/juju-gui/pull/303 (hatched commented on an outdated diff 2 days ago)03:18
wallyworldsinzui: i think the issue is we need to rebase to get a single revision for merging?03:18
rick_h_axw: and click on the '2 days ago' to get the comment in with a snippet03:18
axwrick_h_: cool, thanks03:18
davecheneyaaaaaaaaaaand, that's lunch03:19
wallyworldaxw: gomaasapi on rev 50 now03:19
axwwallyworld: thanks03:19
sinzuiwallyworld, Yeah,  git is like multi-inheritance  diamond parents. Just fucked. bazaar was smart in that respect. the left-hand ancestry is special and we can traverse that to learn the direct commits and choose to go deeper later03:20
sinzuiwallyworld, rebase is essentially a hack to fix a logging problem03:20
rick_h_heh, every commit is in teh reflog, just how easy can you get at it ;P03:20
* wallyworld will miss bzr :-(03:20
sinzuiwallyworld, since juju likes informative commit messages with links, it is essentially at odds with most projects that use git. They want all the details squashed to a single line03:22
wallyworldyeah, i hate that :-(03:23
rick_h_huh? I disagree with that03:24
sinzuiwallyworld, We might have a policy to never delete the branches from our public repos. They have all the commits. The merge bot will create a squashed commit using the pull request subject and description03:25
wallyworldmight be all we can do03:26
sinzuithe merge bot would include the link to the source branch with preserved history03:26
sinzuiNo one needs to rebase03:26
rick_h_maybe auto push to a second remote to track all commits?03:26
rick_h_it'll be a bit nuts to have all branches ever in github03:27
rick_h_and we force users to fork so they can keep their branches in their own fork if they want03:27
rick_h_so that the main repo looks clean03:27
rick_h_https://github.com/juju/juju-gui vs https://github.com/mitechie/juju-gui03:28
sinzuirick_h_, I assumed that developers would be using the triangle trade arrangement. The juju/juju just has stables a devels with the squashed revs. The commit message will have a link to the developer's repo with the squashed revs03:30
sinzuiBut we could have juju/juju-deep which is where the merge bot puts a copy of the unsquaded branch03:31
thumperjuju/juju?03:33
thumperare we not having juju/core?03:33
wallyworldnot, it's juju/core03:34
wallyworlds/not/no03:34
davecheneysgtm03:34
davecheneyjuju/sgtm03:34
wallyworldlol03:34
thumperotherwise we'd have "github.com/juju/juju/juju"03:34
davecheneyyeah, no03:34
wallyworldwe decided at the sprint it would be core03:34
thumper\o/03:34
thumperhmm...03:45
thumperI notice that the .jenv file has a user and password entry at the top of it03:45
thumperbut both are empty strings03:45
thumperwhy?03:45
* thumper steps away for a bit as heaps of calls later tonight03:46
* thumper needs to put on dinner03:46
=== axw is now known as axw-lunch
menn0can anyone confirm whether it's possible to use API and state port numbers other than the defaults?04:59
menn0I know they can't be changed once an env is bootstrap but could someone use a different value at bootstrap time05:01
axwmenn0: yes I have done this in the past05:06
axwit's necessary when using the local provider with multiple users on the same host05:06
menn0ok cool.05:06
menn0I was reviewing a change to "juju restore" and noticed it generates bash scripts with hardcoded port numbers.05:07
menn0I guess my review comments are valid concerns then :)05:07
axwcool05:07
menn0thanks05:07
axwwas just about to review that. now I have to stop procrastinating... :)05:08
jamperrito666: the journal thing is a parameter we have to pass on the Session/Write request.05:10
jamfwereade: ^^05:10
jamwallyworld: fwereade: IIRC the specifics of why we fallback was because we had the "released" vs "daily" streams, and it was searching the "daily" and not finding any "released" items and then stopping. And when we discussed it, it seemed better to keep searching.05:12
wallyworldjam: and also, people adding their own images etc as overrides05:13
wallyworldi'm ok with it, william seems twitchy about it05:14
jamwallyworld: pre-checks ala lbox propose, we *can* but if it is things like "go fmt is sad" I'd rather just have the box run go fmt and clean things up than stall someone to tell them to go run a command and propose again.05:16
wallyworldi don't mind either way tbh05:17
jamwallyworld: you don't have to rebase, and I'd rather we didn't have single commit for review05:17
jamgit does have "git log --follow-first-parent"05:18
wallyworldmy git foo is poor, i'm just going by what curtis etc said at the sprint. curtis will be doing the landing side of things pretty much so we can liaise with hin to et the model right05:19
jamI certainly don't have final say in any of this, but while I don't think we need all branches on master, I'd really rather we didn't rebase to land05:19
wallyworldi think it was being canvased to allow a single rev to be easily cherry picked etc05:20
wallyworldlike we do with bzr with backports etc05:20
jam wallyworld: for cherrypicking you can do "merge ABCD ~ ABCD^" IIIRC05:20
jamwallyworld: which is the same meaning as "bzr merge -c"05:20
jamjust different syntax05:20
jam$REV^ is the parent of REV05:20
jamaka, before:REV or REV-1 to bzr users05:21
wallyworldi'm totally ok with whatever folks who know git recommend05:21
jamwallyworld: well in the git world you can still do things in different ways, I'd like to push back on how the gui guys decide05:21
jamdecided05:21
jambecause I think you can use git in a form that allows easier exposure of specific detail05:21
wallyworldi don't like how git rebase discards history05:22
jamThough, TBH, if rebase still refers to the old commits in a sensible fashion, that might be ok, but I don't quite see how it works05:22
wallyworldif we can work aornd it, great05:22
jamvs giving you the detail only if you have the magic revs that aren't pulled in by default05:22
wallyworldi just want to it work like bzr :-)05:22
wallyworldso for dev workflow, i hope we just push up the raw branch, and let the landing bot sort out how to deal with the final merge05:23
wallyworldso we need to talk to curtis et al about the rules for that05:24
jamwallyworld: interestingly you can "git rebase --exec $RUNMYTESTSUITE" which is nice05:24
wallyworldwhat does that do? rebase and then run the tests?05:25
jamwallyworld: I believe it runs the test after each rebased commit05:28
wallyworldok05:29
jamwallyworld: conceptually I'm hesitant about throwing away history. Though having someone actually spend time to make sure there is logical meaning to commits is nice, having a policy of "throw away all the stuff you were thinking about at the time and only give me a summary" isn't great either.05:31
jamOften when spelunking I'd really like to understand why someone changed *this exact line* rather that "this was changed as part of doing foo"05:32
wallyworldjam: i'm +1 on that, hence i dislike rebase05:32
menn0wallyworld: jam: I get what you guys are saying. There are definite downsides to overuse of rebase but that's more a policy/process thing IMHO.05:35
jammenn0: gui's policy is that every commit gets squashed when merging to master05:35
jamwhich I disagree with05:35
wallyworldi guess they view commit history as noise?05:36
menn0wallyworld: jam: one thing I'm finding with bzr is that I'm committing less often because I know it's much harder to go back and tidy things up.05:36
jamwallyworld: by default "git log" shows everything in time-sorted order05:36
jamit doesn't have the indented logging that bzr has05:36
wallyworldwhich is poor :-(05:37
jamso you don't have "here is a summary commit, and here are the commits that made up that summary commit"05:37
wallyworldexactly05:37
menn0git log output has tons of configuration options05:37
jamwallyworld: yes, but we paid *a fuckton* of CPU for it that lots us a lot of hearts and we weren't quite good enough at explaining why05:37
jamit is actually quite expensive to compute05:37
jammenn0: but not that one05:37
jamit does have --follow-first-parent05:37
menn0brb05:37
jamwhich will get you "bzr log -n1"05:37
jamand it does have topological sorting05:37
jambut topo doesn't help when 2 people are doing concurrent work05:38
jamneither one is "before" the other05:38
jamthe sort bzr uses is "include a commit after the last commit that didn't have it, and before the one that does"05:38
jambut doing the "what commit doesn't have this" is expensive to compute05:39
wallyworldO(n) ? or O(what?)05:39
jcw4fwereade: https://codereview.appspot.com/9826004305:39
* menn0 is back from dealing with vomiting child...05:39
jamwallyworld: so to be able to compute lots of them fast, the algorithm isn't bad if you just walk the whole history, as you can push on a stack everything you've seen so far. So the one we use is O(N=all_history)05:41
jamto do it by walking just a bit of the graph05:41
jamit easily gets into N^2 or N^3 or thereabouts because of needing to do backtracking05:41
jamso N is smaller, but the order is higher05:41
wallyworldok05:42
jamwallyworld: so I still love it, and for histories the size of Juju it is actually still a wonderful view of the world (IMO)05:42
jambut for things like the kernel/mozilla/etc it made things slow05:42
jamand, of course, it is off by default in bzr, too with "bzr log -n1" being the default05:43
jcw4fwereade, bodie_ I made some changes after the discussion today and put in a new MR.  I didn't implement the Unit cleanup yet.  Some prerequisites still arent in.05:43
menn0jam, wallyworld: would it help if we followed a different policy than the GUI team in terms of squashing commits. it's not a given that we do that is it?05:44
jamwallyworld: "time bzr log -n1 -r -10..-1" vs "time bzr log -n0 -r -10..-1" is 85ms vs 250ms on juju today, which is noticeable05:44
jammenn0: no it isn't a given, but that's why I'm bringing it up05:44
wallyworldwe can decide how *we* want to do it05:44
jammenn0: we *are* switching to git, that has been enforced on us05:44
wallyworld:-(05:45
menn0in my experience that's not at all standard practice05:45
jammenn0: but the details are what we are hashing out, and I'm just participating in the discusison05:45
menn0with git projects05:45
wallyworldjam: i was going to experiment a bit once the test landing bot on my test repo got set up next week05:46
menn0on projects that use git that i've worked on, people might do some tidying up with rebase before pushing but rarely down to just one commit.05:46
jammenn0: so I think some cleaning up can be nice, but often I've wished people would have committed *more* detail because they aren't explaining the actual steps, just the global chunks05:47
menn0it's usually about making things more coherent and easier to follow for other developers (and yourself in 2 weeks)05:47
menn0jam: I get that.05:47
menn0different people use different approaches when committing and rebase makes it easier for people to hide detail05:48
menn0perhaps the best we can do is educate each other as part of the review process05:49
menn0if you feel like someone didn't break up their commits into small enough pieces then call them on it05:50
jammenn0: there seem to be projects in git that review your merge by the commits you did, so you have to clean it up into logical steps. Which can be good, but you can also lose some of the "why is it this way".05:50
jammenn0: I really like the default of looking at the global thing rather than the individual bits , we can have multiple branches for landing something that needs evolution05:50
jamI'm not sure the github default view, though.05:50
jamDoes it default to the global commit change or the individual commits?05:51
jammenn0: so if merge --squash or rebase had a way to still reference the old commits and just not show them by default in log, that would be ideal for me, but I haven't seen a tool that really separates the two05:52
menn0jam: yeah I've never heard of such a thing05:52
jammenn0: so looms was bzr's evolution towards that, and bzr-pipelines is a good approximation of it. Where you have several branches and you are iterating among them.05:53
menn0with git once you squash revs those old revs are effectively gone05:53
jamAnd the tips of each branch are the "patch for review" while the individual commits are still there.05:53
menn0jam: yeah i've read a little about looms05:53
jamand then in bzr when you merge a commit, by default you only see the merge commit, rather than the details, but the details are all there.05:53
jammenn0: and certainly things like stacked-git, or quilt, etc are also similar approaches.05:54
jamthough again, most of those are explicitly squashing the intermediates down to the final patch05:55
jammenn0: like I think stacked git is actually a great workflow, but I wish it actually kept the intermediates as an alternate view.05:57
menn0i've just been looking at the stacked git site and yeah it doesn't look it helps with keeping the intermediate commits05:58
jammenn0: looms/stacked-git/etc have lots of nice process built in for managing multiple branches that you want to eventually apply all together, but still keep things in logical chunks05:58
jamlike what I'm working on now for the API Versioning has about 4 logical steps05:58
jamstart doing the internals, start exposing it on the server, start consuming it in the client, etc.05:59
jamit is great to separate those out and still sync between them05:59
menn0jam: yep that makes sense05:59
jambzr pipelines lets you create a "pipeline" aka stack of branches, and then switch to the first one, and "pump" the changes through the pipeline.05:59
menn0jam: in the past I've done that kind of thing manually but it gets tricky when you get beyond a handful of concurrent branches.06:00
menn0jam: a tool to help would sometimes be nice06:00
jammenn0: yeah, I don't know state-of-the-art with git here. I bzr-pipelines are used the most on our team, and got the most love and attention. Looms had a potentially better data model, but were a bit bigger picture (you could merge looms at the conceptual loom level), and didn't actually get as much polish06:01
jamthe idea being that you could collaborate on the set of patches that are being tacked on06:01
jamwhich is stuff that OS people do06:02
jambecause they track UPSTREAM and have their local (deb) patches applied, and need to sync from what Debian is doing to how Ubuntu is doing, etc.06:02
axwwallyworld: are you fixing the TestConstraintsValidator network isolation? if not I will06:02
menn0jam: yeah I've maintained a linux kernel fork for a product before. I know the problem space06:02
wallyworldaxw: not explicitly but it will be done as part of the overall network isolation work06:03
wallyworldso yeah, i'll do it06:03
axwwallyworld: ok. it is calling out to cloud-images.ubuntu.com06:04
axwwallyworld: seems to be only provider/maas06:04
wallyworldaxw: i was leaving it till i go the framework in place to see that it failed :-)06:04
axwwallyworld: I just ran the tests with "unshare", as described in #123360106:05
_mup_Bug #1233601: juju test suite should be isolated and do 0 outbound network requests <test-failure> <testing> <juju-core:Triaged> <https://launchpad.net/bugs/1233601>06:05
axwthat was the only test that failed06:05
menn0wallyworld, jam: potentually useful for the basics: http://agateau.com/talks/2010/git-for-bzr-users_uds-natty/git-for-bzr-users.pdf06:05
axwwallyworld: eh sorry, only package. 3 tests failed in maas06:05
jammenn0: mercurial had "mq" or mercurial queues to work in the same process space, but it seems stacked-git is the only similar one for git06:05
wallyworldok, i thought there was one for store also06:06
axwwallyworld: *shrug*, did not fail in my run06:06
wallyworldaxw: you know the one i mean? it shows up in failure logs but i haven't looked in detail yet06:07
menn0anyway I've got to go... see you'll tomorrow06:08
menn0you all even06:09
axwwallyworld: I forget. I've seen store fail in the past, but not atm06:09
axwlater menn006:09
wallyworldaxw: i can't recall the specifics right atm. it's in the logs somewhere06:10
jamwallyworld: how did you end up doing the conversion?06:11
wallyworldjam: http://flexion.org/posts/2012-10-migrating-bzr-to-git.html06:11
wallyworldseemed to work fine06:11
wallyworldhtp://github/wallyworld/core06:12
wallyworldi didn't rename any authors though06:13
jamwallyworld: sounds about like how I'd do it06:18
wallyworldjam: good :-) must be right then :-)06:19
wallyworldit did work nicely06:19
jamAn interesting alias:     ll = !git --no-pager log --first-parent HEAD~10..HEAD --reverse --pretty=short06:19
jammatches pretty close to my default bzr output, though doesn't have *any* datestamp06:20
wallyworlda good start i guess06:20
jamI personally find dates like: Mon May 12 09:28:05 2014 +1000 to be a bit too complete06:21
jamvs just ISO dates06:21
wallyworldyep06:21
jamwallyworld: git log --date=iso06:21
jam:)06:21
wallyworldonce small hurdle solved06:26
wallyworldso many more to go06:26
jamgit log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit06:49
jamis quite pretty06:49
wallyworldniiiiiiiiice06:50
jamI still like --first-parent logs, but having a terminal graph is fun06:53
jamgit --no-pager log --first-parent --reverse --pretty=tformat:"%C(yellow)%h%Creset%<|(25) %an %ai%n        %B" -1006:55
jamis the closest I could get to my "bzr log --short --forward -r-10..-1" alias06:55
jamI couldn't find a way to indent all lines of the body, though *maybe* %w()06:55
jamwallyworld:     ll = !git --no-pager log --first-parent --reverse --pretty=\"tformat:%C(yellow)%h%Creset%<(25) %an %ai%n%w(72,8,8)%s%+b%n\" -1007:07
jamthat looks really close to "bzr log --short" though the date is full instead of just YYMMDD, and it doesn't have [merge] markers07:08
jamfound it, instead of "%ai" for iso date, use "%ad" and --date=short07:11
jam[alias]07:12
jam    ll = !git --no-pager log --first-parent --reverse --pretty=\"tformat:%C(yellow)%h%Creset%<(30) %an %ad%n%w(72,8,8)%s%+b%n\" -10 --date=short07:12
jamwallyworld: so the reason people like squashing commits is because 'git log' and 'giggle' and 'gitk' look nowhere near as nice as "bzr qlog" on the same data.07:15
wallyworldwell that's not a goof excuse :-(07:16
jamyou can tell what commits are "mainline" because tarmac puts "[r=" in the header, but it only takes a few commits before all of those are lost07:16
wallyworldfix the tool, don't throw away data07:16
jamwallyworld: if you're tool can't show you good history, you clean up your history first07:16
wallyworldthat's arse about07:16
jamwallyworld: well, your preaching to the choir here, but if you try to look at the juju-core history with anything I've found it is a bit hard to figure out what is going on07:17
jamoutside of about 3 steps07:17
jamand if you do "gitk --first-parent" it shows you just the mainline07:18
jambut for some reason, it wants half of the merges to be from the right, and half from the left07:18
wallyworldjam i get a syntax error with git --no-pager log --first-parent --reverse --pretty=\"tformat:%C(yellow)%h%Creset%<(30) %an %ad%n%w(72,8,8)%s%+b%n\" -10 --date=short07:18
jamwallyworld: if you are pasting it into shell, you need to remove the "\"07:18
wallyworldsigh07:18
jamthat is the escape for putting it in ~/.gitconfig07:18
wallyworldyes07:18
jamwallyworld: it took me a while to find the right escape point for gitconfig07:19
wallyworldi didn't look too closely clearly07:19
jambecause without escaping " it treats the | as a shel lchar07:19
jamwell the <07:19
wallyworldhey i like that latest log output07:19
wallyworldi just wish the revs nums were monotonically increasing07:20
jamah, ffs, I figured out how log decides which commit comes first07:20
jamalphabetical sort of the hash...07:20
wallyworldi can't fathom the hashes07:20
wallyworldwhy not date?07:20
jamwallyworld: consider hashes arbitrary handles that you just have to copy & paste07:20
wallyworldsure07:20
wallyworldbut not user friendly07:20
jamwallyworld: sort of the hashes is because in the git world, everyone is "equal" there is no "mainline"07:20
jamso me merging you is equivalent to you merging me07:21
jamAIUI07:21
wallyworldwhich is a design flaw to me07:21
wallyworldnot how most projects work, except maybe the kernel07:21
jamwallyworld: anyway, that is why 'gitk' et al can't really do mainline, because they don't think about it, and they take the most arbitrary thing (the hash) and use it for sorting07:21
wallyworldand yet we've been mandated to use it07:22
jamit is a *stable* sort, I can give them that07:22
* wallyworld wishes we could use the best tool for the job07:22
jamwallyworld: so there are certainly things that show git's had more man-hours put into it. Like the configurability of log is better than what bzr gives you07:23
jamwe'll get stockholm syndrome eventually07:24
wallyworldhope not :-)07:24
wallyworldyou can pry bzr from my cold, dead hands07:24
jamwallyworld: you can always polish up your bzr-git and have it both ways07:27
wallyworldtrue, i need to look into that07:27
wallyworldi also need to figure out how to make working with github nice07:28
wallyworldfrom the cmd line etc07:28
jamwallyworld: I believe working with github is "clone core into your own namespace, and then push and pull from there"07:28
=== vladk|offline is now known as vladk
wallyworldsure, but all the bzr setup to allieviate typing i need to figure out - parent location based on repo etc etc07:29
jamused named branches (don't commit directly to master), push those names back to github, merge from named branches to trunk.07:29
wallyworldplus it keeps asking me for username and password all the time07:29
jamwallyworld: git's "origin" policy works decent here, I think you have to set it up 1 per "repo" but then it should continue to work for all branches.07:29
wallyworldyeah, that stuff i need to read07:29
wallyworldhaven't found time yet07:30
wallyworldi need a "github for bzr/launchpad refugees" document07:30
TheMuemorning07:30
jamwallyworld: if you set up an SSH key with github, you then push to the magic "git@github.com:wallyworld/core" and it will detect who you are from your SSH key fingerprint07:34
jamnote that "user@host:/path" is git's way of declaring an SSH path07:35
jamvs bzr's bzr+ssh://...07:35
wallyworldjam: ah, i've been pushing to just github.com/... i think07:35
wallyworldi'm also hoping to just git push and it knows where to push to like bzr does with lp07:36
wallyworldi don't like typing the full remote location each time07:36
axwwallyworld: you create "remotes" for that07:37
axwessentially aliases for a repo location07:37
wallyworldrightio, that's the github 101 stuff i haven't read yet07:37
jamwallyworld: https://help.github.com/articles/error-permission-denied-publickey is how to set up your ssh key stuff, and the default from github is to use "https://" which require username & password, but if you look at the "clone from here" there is a link for getting the SSH address.07:51
wallyworldgreat, thanks07:52
wallyworldsome weekend reading07:52
jamby default I believe "git clone $SOMETHING" will set that something as an origin so "git push" will push back to it07:52
jamwallyworld: I think working out this stuff/documenting it is all part of actually getting the team moved over07:52
wallyworldyes indeed07:52
* wallyworld -> dinner07:54
jamhttp://www.seamicro.com/node/313 "Up to 64 8-core processors with 64GB dram each" and up to 4TB of RAM for the 10U box.08:02
jamthat's a big box08:02
TheMuejam: nice, indeed. but also a nice power consumption. may also use it as heating system. :)08:05
jamTheMue: apparently it is about 2x as efficient as regular boxes of that size (6 racks of 40kW for normal machines gets replaced with 2 racks of 20kW)08:06
jamso it is definitely power-dense08:06
jambut maybe slightly more efficient per kW than othres.08:06
jamat least, the numbers they give put it as 1.5x more kW/rack space, but 3x the compute power08:06
TheMuejam: yeah, it’s fantastic. espcially when thinking back to the good old Sun E10K times.08:07
TheMuejam: we once ran three of those boxes, at that time absolutely top (around 2000).08:07
jamTheMue:  http://en.wikipedia.org/wiki/Sun_Enterprise#Enterprise_10000 quite a machine in 200008:09
TheMuejam: absolutely. our computer room has been under the roof of our building and they had to deliver it with a large crane. :D08:11
* TheMue thinks about quad-core mobiles or notebooks (with 8 hyperthreads and 16 gb ram) today compared to the good old Z80 he started with08:21
* axw punches peergrouper tests in the face08:23
voidspacemorning all08:27
jammorning voidspace08:51
=== vladk is now known as vladk|offline
perrito666morning09:53
voidspaceperrito666: morning09:59
wwitzel3morning10:09
voidspacewwitzel3: morning10:09
voidspacewwitzel3: did you pull out your tests yesterday, or just not push them?10:15
wwitzel3voidspace: never pushed them, never got them working10:15
voidspacewwitzel3: the changes look pretty complete in terms of replacing the EnvironConfig watcher with the Rsyslog watcher10:15
voidspacewwitzel3: :-/10:16
wwitzel3voidspace: I will push them now10:16
voidspacewwitzel3: cool10:16
voidspacewwitzel3: want to pair again? Shall I drive this time?10:18
wwitzel3voidspace: yep, that seemed to work well yesterday10:19
voidspacewwitzel3: yeah, when we finally got a screen resolution I could see :-)10:20
voidspacewwitzel3: it meant that for the first hour or two you were doing all the work and describing it to me as we went10:20
voidspacesorry about that10:20
voidspacewwitzel3: my branch landed by the way10:20
wwitzel3voidspace: ahh, good10:21
voidspaceyep10:21
voidspaceright, so coffee update then hangout10:21
wwitzel3voidspace: sounds good, doing the same10:21
=== vladk|offline is now known as vladk
vladkjam: hangout?10:49
voidspacewwitzel3: nearly 2pm here, so I'm going on lunch12:57
voidspacesee you back soon :-)12:57
wwitzel3voidspace: ok, sounds good12:59
bodie_fwereade / mgz -- working through some of this Action stuff in my head.  I think I got a much better picture of what we need to do! :)13:10
fwereadebodie_, cool, we can have another chat about it later if you think that'd be useful?13:10
bodie_fwereade, absolutely.  I was looking at the gojsonschema stuff.  should we be assuming that the args for an Action will be defined as a json-schema file?13:12
bodie_instead of the "actions.yaml" I was basing my thoughts on -- because if so, our work is literally cut out for us13:13
bodie_or does the schema define what a legit actions.json might look like?13:13
bodie_i.e. action, description, args, types13:13
fwereadebodie_, so ISTM that we unmarshal the contents of actions.yaml, and call NewJsonSchemaDocument with the params definition for each action; and then we call Validate on that doc, passing in the unmarshalled params we got from the user13:18
fwereadebodie_, I perceived you as talking about json *files* which I think is a slight derail13:18
bodie_won't the UI be generating json-schema?13:19
rick_h_the UI will be using the json schema document you send us over the api13:19
bodie_that is, the charm definition GUI13:19
rick_h_and we'll construct a UI, do client side validation, but juju must do the same13:19
rick_h_to jump in as I follow along :)13:19
fwereadebodie_, it will be using a json-schema document to generate the ui, and we will be using that same json-schema document to validate what they send to us, yeah13:19
bodie_I see, but the charm will define the action spec using yaml?13:20
rick_h_the GUI might wish to ask you what actions are available, what is the schema doc for a specific charm/action.13:20
fwereadebodie_, but the representation in the charm of that json-schema document will just be a bunch of maps/lists/values under a particular key in the yaml13:20
bodie_I thought there was some JSON-schema builder which was desired to be used to actually put the charm's actions spec together13:20
fwereadebodie_, we unmarshal that stuff and then pass the resulting interface{} into the gojsonschema funcs13:20
bodie_sorry, fwereade / rick_h_, talking about charm manufacture, not UX13:21
fwereadebodie_, that hadn't really crossed my mind -- a simple json-schema can be pretty simple, though, according to my reading of it13:22
fwereadebodie_, certainly no worse than a charm config schema13:22
bodie_right, it's very human-readable13:22
fwereadebodie_, but with more scope for complexity if it's warranted13:22
rick_h_yea, didn't realize there was talk of a charm building UI/tool. I think that's something of a follow up though to  actually having actions work13:22
fwereaderick_h_, +1 to that13:22
bodie_:)13:22
bodie_well, hopefully that will be coming soon now that I have the conceptual framework in place much more clearly13:23
fwereadebodie_, +1 also to that :)13:23
rick_h_sorry, didn't mean that to come across as snarky, just mean a UI tool to help you build a charm shouldn't influence how to write a charm without.13:23
rick_h_imo13:23
bodie_right, the question was more whether json-schema is going to replace YAML13:24
bodie_since that would obviously make validation / parsing really simple13:24
bodie_perhaps I'm not listening lol, rereading13:24
rick_h_since yaml is a superset of json, my understanding is that we'll update charm proof to validate actions yaml files to be valid json files.13:25
rick_h_charms will write yaml13:25
rick_h_and what they are internally once pulled into juju doesn't matter (so using json would be ok?)13:26
rick_h_especially since the GUI will be requesting json anyway13:26
rick_h_but asking charm devs to do part XX in json and part YY in yaml isn't a great experience for them13:26
bodie_Gotcha, so really the ideal interface takes either-or13:26
fwereaderick_h_, yeah, exactly -- although really the only parts that *have* to be valid json are the subsections of those files that will actually be piped into gojsonschema13:27
fwereaderick_h_, ie the params definitions13:27
rick_h_fwereade: true, I was under the actions.yaml perspective so not pondered the nested structure idea yet13:27
fwereaderick_h_, (I would be surprised if actions.yaml *actually* ended up containing anything that wasn't *representable* as json though)13:27
rick_h_fwereade: but definitely, we can scope the issue down easily enough.13:27
fwereaderick_h_, I wouldn't expect the gui to have to care about that fwiw13:28
bodie_:) well I think my work is cut out for me13:28
rick_h_fwereade: right, just that yaml allows some form of variables and such if I recall, which wouldn't dump to pure json correctly13:28
fwereaderick_h_, ask us about the charm over the api, we'll tell you the schemas for the actions13:28
fwereaderick_h_, yeah13:28
fwereaderick_h_, I'm not taking it off the table, just saying I'd be surprised13:28
rick_h_fwereade: right, that's cool. Easy to work with.13:28
fwereaderick_h_, cool13:28
bodie_rick_h_, what you said about the json-schema sent over the API13:29
rick_h_delivery truck, have to run for a few13:29
bodie_let me know when you're back :)13:29
bodie_never mind, seems simple enough.  just need to add a "get-schema" api endpoint13:30
bodie_fwereade, the question in my mind is still whether to simply store the params and spec as json or as an interface map13:31
fwereadebodie_, let me turn that round: what's the benefit of storing as json?13:31
bodie_heh :)13:32
bodie_makes it a simple gojsonschema call internally13:32
bodie_whereas storing as k/v makes it easy to send args to the charm13:32
bodie_is that something like what you're thinking13:33
bodie_?*13:33
fwereadebodie_, ISTM that the gojsonschema funcs actually work with interface{}, and supply convenience functions to turn json text into them13:33
bodie_ah13:33
bodie_didn't realize that13:33
fwereadebodie_, maybe I'm looking at the wrong gojsonschema, it's not immediately apparent which of xeipuuv's and tolsen's is better, if there's any difference at all13:34
bodie_fwereade, it appears xeipuuv's is much more maintained -- tolsen's is a fork of it from 8 months ago13:36
bodie_hm, it was actually merged into xeipuuv's a couple of months ago13:36
bodie_maybe I'm looking at the wrong numbers here13:37
bodie_https://github.com/tolsen/gojsonschema/network13:37
bodie_so, probably xeipuuv's is our best bet :)13:38
rick_h_sorry back13:39
fwereadebodie_, I think we'd likely want to vendor it anyway, I presume the license is tolerable13:40
fwereadebodie_, but maybe we just stick with godeps13:40
bodie_hi rick_h_, no worries.  Just wanted to reflect what you said about a get-schema API endpoint to make sure I'm on the same page :)13:41
bodie_fwereade, it doesn't appear to be licensed at all.  under Github's TOS, I believe that means if you fork it, it's yours :P under the most liberal interpretation, afaik13:42
fwereadebodie_, ha, fun :/13:42
rick_h_bodie_: I think we should file an issue to get a license in place and suggest we're willing to add one in a pull request that's of a version we'd like to see :)13:43
bodie_perfect13:43
fwereaderick_h_, sgtm; bodie_, would you handle that please?13:43
rick_h_explicit ftw, shoot, maybe even start with a pull request that's bsd or MIT and see if it gets the discussion going13:43
bodie_https://github.com/xeipuuv/gojsonschema/issues/16 -- is this good?13:53
bodie_it's an active repo and there are about 2 issues open13:53
bodie_didn't see your suggestion of opening the MR myself13:53
bodie_but, that of course would've been a good idea ;) and I can still go ahead and do that if you'd like13:54
rick_h_bodie_: oh, you'd have to fork it and then do a pull request against it?13:54
bodie_correct13:55
rick_h_oic, yea. That's cool, just might use pull request vs MR as MR is launchpad centric terminology13:55
rick_h_but nitpicking language there13:55
bodie_he he.  just caught that myself13:55
bodie_I wasn't sure if I ought to mention what project it's for13:58
rick_h_bodie_: if it comes up it comes up.14:03
wwitzel3voidspace: standup14:03
jcw4fwereade: (bodie_) I re-submitted the MR after addressing the issues in the review14:14
jcw4fwereade: I didn't start working on the Unit cleanup though... I figured that could be the next MR?14:14
bodie_jcw4, perfect :) I just broke several MR's out of that monolithic one from earlier14:16
jcw4bodie_: nice14:16
bodie_so, each of those should be pretty clean and easy to get in, we might even want to consolidate a couple of them14:17
fwereadejcw4, dammit, sorry, I started looking this morning and something distracted me14:18
jcw4fwereade: no worries...  I know you have a lot of demands :)  I appreciate whatever you're able to help us with14:19
fwereadejcw4, I'd rather like it if you did fix up the cleanup, I don't *think* it'd grow the CL too far, but feel free to start working on it in another branch -- you can always merge them into one to land14:19
jcw4fwereade: Yeah. I'll start that this morning14:19
fwereadejcw4, (or ofc demonstrate that I'm crazy, and it does/would grow it too far ;))14:19
jcw4fwereade: hehe will do14:19
bodie_fwereade, were you saying an Actions spec ought to be an interface{} or a map[string] interface{} ?14:55
voidspacefwereade: ping14:56
voidspacefwereade: we're implementing WatchForRsyslogChanges14:56
jam1void14:56
jam1voidspace: i'm around if fwereade isn't14:56
voidspacecool14:57
voidspacejam1:  fwereade: what we *really* want is a "MultiEntityWatcher"14:57
voidspacethat will watch two mongo collections and notify the client when *either* of them change14:57
jam1fwereade: le sigh… I can't find AllWatcher in a test suite to validate my changes...14:57
voidspaceis that a) something that already exists  or b) a feasible approach we just need to do it or c) a dumb idea14:57
jam1voidspace: I don't know that we've written the code, but AIUI NotifyWatchers can be trivially collapsed14:58
voidspaceNotifyWatcher is client side, right?14:58
jam1voidspace: it is both sides14:58
voidspaceWe'd rather collapse the EntityWatcher on the server14:58
voidspaceah, ok14:58
voidspacewe'll have a go at it then14:58
jam1voidspace: MuxNotifyWatcher ?14:59
voidspacewe just didn't want to burn time on it if it was an approach that was never going to work and we should just use two of them14:59
sinzuiHi Devs. If you look at juju-ci, don't panic. HP is very ill. IS knows about it and are in contact with HP.14:59
jam1voidspace: well IIRC the discussion from fwereade was that we really should treat the items in environconfig as immutable, in which case you don't need to watch them anymore.14:59
voidspacejam1: ah, so *just* watch APIHostPorts14:59
jam1sinzui: thanks for the heads up, IIRC azure was having some troubles14:59
voidspacejam1: that's certainly easier...15:00
jam1sinzui: was that just temporary?15:00
sinzuijam1, azure often resolves itself over 4 hours15:00
jam1voidspace: right I'm hesitant to just change the code that we've already had working15:00
jam1but fwereade seemed pretty confident that we can't just change things like ports and ca certs15:00
jam1voidspace: so as long as we have some sort of implementation of a WatchForRsyslogStuff15:01
jam1then we can make it trigger with whatever logic we converge on15:01
sinzuijam1 This HP outage is cause by bad archives or networks. Apt is crippled. This happen in March when I did the 1.17.4 release I think15:01
voidspacejam1: we have that done. It's currently just doing WatchForEnvironConfigChanges under the covers15:01
jam1voidspace: certainly you can *start* with just APIHostPorts15:01
jam1sinzui: did we ever get trusty in HP ?15:02
voidspacejam1: we can switch it to APIHostPorts and check that everything still works *plus* the parts that weren't working before15:02
voidspaceand if everything works then great15:02
sinzuijam1 it is listed :)15:02
voidspaceawesome15:02
jam1k, I thought for a while it wasn't available, but yeah, ISTR HP having archive routing issues in the past15:02
* jam1 is sad, I can delete the AllWatcher which GUI critically depends on and nothing in juju-core would notice (at least that I can find)15:04
natefinchjam1: well, the GUI is an important part of Juju, obviously.  Is there a way we could support them differently without the AllWatcher?15:06
voidspacepubhubsubpubhub!15:07
jam1natefinch: well I can just migrate the code over there, it isn't like I'm getting rid of it.15:09
jam1the *bug* is that we don't actually test that we don't break them all the time15:09
jam1If I mess up the permissions, or accidentally don't expose the API, nothing notices15:10
jam1we have some client code15:10
jam1but it isn't exercised (AFAICT15:10
jam1rick_h_: do you do regular testing with dev-releases or trunk ?15:11
rick_h_jam1: no, it's something we brought up with sinzui about getting into the CI/QA steps15:11
jam1wallyworld_: if you want to write some extra tests bug #131944115:12
_mup_Bug #1319441: AllWatcher API is untested in juju-core <tech-debt> <testing> <juju-core:Triaged> <https://launchpad.net/bugs/1319441>15:12
jam1rick_h_: k. this is mostly about "I should be able to refactor code and know that everything works"15:12
rick_h_jam1: agreed, it's a hole we hit with quickstart leading to 14.04 and we brought up in vegas to address15:13
jam1but AllWatcher just "exists" as an API. It looks like the underlying "megawatcher" code is reasonably tested, just not that we can actually get anything from it via the api15:13
jam1rick_h_: is it possible to drive the GUI from a script?15:13
jam1rick_h_: I guess you have quite a bit of JS tests written15:14
rick_h_jam1: we do have a few functional tests using selenium we run to drive the gui in a couple of ways15:14
sinzuijam1. rick_h_ and I agreed that QA needs to test the next juju against quickstart and gui. Run their tests suites that exercise juju too15:14
sinzuiperrito666, regarding the ha restore test. Do I expect to see just a master (users need to rerun ensure-ha) after the restore completes?15:17
perrito666sinzui: yes15:18
sinzuithank you.15:18
perrito666sinzui: btw, I found that in "restore_present_state" server you are masking possible juju-restore runtime errors15:21
sinzuioh, how do I fix that?15:22
perrito666well, you try to match the instance name to destroy the instance, if there is no match you raise a generic exception with a string, you might want to append the output from err in that string15:23
sinzuithank you perrito66615:28
perrito666sinzui: np15:30
voidspacewhen is TestStartStop called in a suite (how does it differ from SetUp)?15:30
jam1voidspace: it is just a test15:31
voidspaceah...15:31
voidspacethat explains why changing it isn't affecting the other tests :-)15:32
voidspacejam1: d'oh, thanks15:32
jam1voidspace: Ithink anything starting with Test* is just atest, the SetUpSuite et al start differently15:32
voidspaceyep, shoulda twigged15:32
wwitzel3:q15:41
wwitzel3irssi is the worst vim ever15:42
jcw4:)15:42
jcw4:w15:43
jcw4:w!15:43
jcw4:qa!15:43
perrito666wwitzel3: you should have seen my week trying to switch to emacs, I have never corrupted so many files with ESC^i and :w in my life15:45
_benoit_Outscale has a S3 endpoint in fact! so juju should work easily \°/15:50
fwereade_benoit_, sweet!15:56
jcw4fwereade: about cleaning up the unit.... 1) I assume the cleanup should go in func (u *Unit) Remove()16:02
jcw4fwereade: and 2) the same predicate used by the watcher to find Actions specific to the unit can be used to find and remove the Actions queued for the unit being Removed16:03
fwereadejcw4, look in cleanup.go, I think16:03
jcw4ok16:03
jcw4fwereade: thx16:03
fwereadejcw4, I think you need to look at service.removeUnitOps to find the stuff that'd definitely run when a unit is actually removed16:04
fwereadejcw4, do we have an opinion about running actions on units after someone's destroyed them, though?16:04
jcw4fwereade: good question16:05
jcw4fwereade: I'll review the draft doc to see if it's mentioned16:05
fwereadejcw4, because we might be in a position to actually mark them all failed at that point, I'm right now writing a cleanup op that's queued when someone calls Destroy on a unit16:05
fwereadejcw4, pretty sure it's not16:05
fwereadejcw4, btw, I'm afraid I need to be off pretty sharpish now, otherwise I'd pop in and talk to you16:05
fwereadejcw4, if I'm around later I will try to get you a fresh review16:06
jcw4fwereade: tx!16:06
jam2reasonably trivial code review: https://codereview.appspot.com/9825004416:12
=== jam2 is now known as jam1
_benoit_fwereade: http://paste.debian.net/99505/ how can I bypass this lord of the ring check ?16:16
voidspacenatefinch: ping16:33
voidspacenatefinch: worker/rsyslog/rsyslog_test.go TestNamespace - do you know anything about this test?16:34
voidspacenatefinch: being our resident rsyslog expert16:34
perrito666is anyone having problems with amazon taking forever to initialize machines?16:36
voidspacejam1: sooo... we have some of our tests that call the following and expect rsyslog to be restarted16:36
voidspaces.APIState.Client().EnvironmentSet(map[string]interface{}{"rsyslog-ca-cert": coretesting.CACert})16:37
voidspacejam1: I can put a workaround in the tests, but is it possible that the rsyslog cert will be set *after* APIHostPorts have changed?16:37
voidspacejam1: in our manual testing everything appears to work for all the relevant scenarios16:38
voidspacenon-HA, HA, deploy unit *then* ensure-availability16:38
jam1voidspace: so the argument from fwereade was that rsyslog-ca-cert could be treated as immutable, and if we can consider it thusly, then the above test is invalid16:38
jam1voidspace: rsyslog-ca-cert has to be set up on machine-0 before anything happens16:39
voidspacejam1: immutable except for *initial setting*, but would that *not* be done through the api16:39
voidspaceright16:39
voidspaceso in which case I will modify the test16:39
jam1the only bit you have to worry about is if something comes up right away before bootstrap actually sets up the cert16:39
voidspacethat's my specific question16:39
voidspacedo I have to worry about that16:39
jam1voidspace: so I'd rather get fwereade to confirm, but that is my understanding from our conversation16:40
jam1it should be considered immutable-once-set16:40
jam1though it would be best if we actually enforced that.16:40
voidspaceright16:40
jam1I'm not entirely sure why it is in environconfig16:40
voidspaceheh16:40
jam1(which lets user's touch it) other than we may not have a grab-bag for stuff we want to create and keep track of16:40
voidspaceit was probably just somewhere convenient16:41
voidspaceit certainly seems like *in practise* the cert is already set when we write out the initial rsyslog conf16:41
voidspacebut I'd like to be certain that *must* be the case, otherwise we'll have intermittent problems16:42
voidspaceand if we can't be certain then we need to watch environconfig changes at least until the cert is set16:42
voidspacewhich is extra code16:42
natefinchvoidspace: just got back16:43
voidspacenatefinch: hey, hi16:44
voidspacenatefinch: see the above conversation with jam1 from where I pinged you16:44
voidspacenatefinch: we're now only triggering rsyslog conf to be rewritten, and rsyslog to be restarted, when the set of APIHostPorts changes16:44
voidspacenatefinch: because we consider the rsyslog stuff in environconfig (cert and port) to be immutable16:45
natefinchvoidspace: sounds good16:45
voidspacenatefinch: the only danger is that this information is "somehow" set *after* the config has been written the first time16:45
voidspaceI'd like to assume that's impossible - and it certainly seems to work in practise16:45
jam1voidspace: so my personal view is I'd rather have the ability to change it and not use it than not allow it at all (especilaly if we don't enforce that it is immutable). However, since it is easier to implement and fwereade liked it being immutable we can go with it16:46
voidspacejam1: it's easy to change16:46
voidspacejam1: on the other hand EnvironConfig changes for a lot of irrelevant things, so the new way restarts rsyslog a lot less16:47
natefinchjam1, voidspace: my preference is for whatever makes the code simpler.  It sounds like assuming it's immutable will make the code simpler16:47
natefinchI don't want us to write and maintain code that only .001% of people will actually use16:48
natefinchhell, I don't want to write and maintain code that only 1% of people will actually use :)16:48
voidspaceat some point 1% becomes a lot of people16:49
natefinchDon't care, that means 99% is always way way more people16:50
natefinchmore complexity and more features will just slow us down for the features the 99% want16:50
fwereadejam1, I'm not even necessarily *against* having it mutable, but if we do we should be careful to make it sane; and we haven't, and nor have we scheduled mutability, so we should just save ourselves hassle by making it immutable16:53
fwereadejam1, and it's easy, and simple, and maybe one day there will be a compelling case for changing it, and that will be fine too16:53
* fwereade is done for the day (or at least for now) anyway16:55
jam1fwereade: night, have a good evening16:55
voidspacefwereade: night16:58
voidspacehmmm... my actual error now is a permissions error17:00
voidspacenatefinch: should all api endpoints do a permissions check?17:03
voidspacenatefinch: our initial implementation of WatchRsyslogChanges included this check17:04
voidspaceif api.authorizer.AuthOwner(agent.Tag)17:04
voidspacewhich caused a test to fail elsewhere17:04
voidspaceI wonder if it's needed17:04
voidspacebrb17:05
natefinchvoidspace: it seems like watching shouldn't need permissions, since you're not changing anything, but I'm actually not sure what the policy is.17:05
voidspacenatefinch: it looks to me like the WatchAPIHostPorts itself doesn't have this check17:05
natefinchvoidspace: that sounds like a good enough precedent to me17:06
voidspacenatefinch: and as this is *essentially* what WatchRsyslogChanges is doing it shouldn't need extra permissions I don't think17:06
voidspaceheh, cool17:06
voidspacewe probably just copied the permissions check from somewhere else anyway...17:06
jam1voidspace: natefinch: all API methods do permission checking17:24
jam1voidspace: especially when it was exposing the raw EnvironmentConfig before, but even now, it is only for use by Agents17:25
jam1probably you just had a wrong Authorizer set up17:25
jam1natefinch: voidspace: WatchAPIHostPorts is special, because *everyone* needs to watch that17:25
jam1but Clients shouldn't have WatchRsyslogChanges exposed to them.17:25
voidspaceah, dammit17:28
voidspaceat least I've confirmed the error even if my fix is wrong :-)17:28
voidspacealthough it seems like this new check must be more draconian than the previous one17:29
natefinchjam1: thanks for the clarification.  I guess I sort of assume that read-only stuff doesn't really need authentication because.... who cares?  But good to know the policy anyway.17:29
jam1natefinch: well readonly of your Environment credentials would be a bad thing, right?17:30
natefinchjam1: yes, true17:30
jam1natefinch: so while I could see "does security really matter on *this* function", it is more of a "lets not think about it, and just restrict everything"17:31
natefinchjam1: fair enough17:31
jam1because I can agree, why would leaking rsyslog needs changing be a security hole17:31
natefinchjam1: but then again, you never can tell where you might leak information in an indirect way that gives an attacker just enough advantage17:32
voidspacejam1: so the correct permissions check is17:34
voidspaceauthorizer.AuthMachineAgent() ?17:35
jam1voidspace: the UnitAgent doesn't run rsyslog, right?17:35
jam1then AuthMachineAgent17:35
* perrito666 wishes lbox didn 't h a n g t h e w h o lemachinewhilerunning17:36
natefinchperrito666: get a better machine ;)    lbox does do a lot of stuff, and Linux (Unity?  not sure the culprit) does have the unfortunate bug where if you're hammering the CPU, it tends to freeze up everything.17:37
voidspaceso, the test now passes...17:37
voidspacewith that change in place17:37
voidspacetime to run all the tests and go jogging17:37
perrito666natefinch: perhaps a less mobile processor, but it is hard to go back from an extremely lightweight machine17:38
=== circ-user-UuLvN is now known as mramm
mrammhey all!17:44
perrito666hey mramm17:44
mrammODS demo yesterday was #*(%ing amazing17:44
mrammgreat work everybody17:44
natefinchmramm: yeah it was.  I watched it this morning.  Damn awesome.17:45
mrammnatefinch: how hard would it be for you to do a sprint for a few days in a couple of weeks?   (I know it is both short notice, and close to the last juju sprint temporaly speaking)17:46
mrammnatefinch: and am I remembering right that you/your squad have Windows and CentOS on your work list for this cycle?17:46
natefinchmramm: yeah, we have non-ubuntu workloads17:47
natefinchmramm: as for a sprint, if it's within commuting distance (like at the Lexington, MA office), then it's cool.  Going elsewhere and my wife might kill me.17:47
mrammwell, let me talk to the cloudbase guys17:48
mrammand see if I can get them to come back to the US17:50
mrammIt's too late (I think) to keep them here after ODS17:51
mrammwho has the resources work on their schedule?17:55
lazyPowerhey, did we add a fsmount type of rpc_bindfs wrt bind mounting local filesystems on local provider?17:56
natefinchmramm: Onyx (Tim) has resources, looks like17:58
rick_h_mramm: I think that's thumpers team17:58
=== vladk is now known as vladk|offline
sinzuiThe HP archive issue is resolved. Juju is confirmed to be fine18:10
sinzuihttp://juju-ci.vapour.ws:8080/18:10
sinzui^ if we fixes the i386 and ppc unittests, every test would have passed18:10
natefinchsinzui: nice!18:11
voidspacewwitzel3: I fixed the auth issues18:17
voidspacewwitzel3: but there's a bunch of failing tests in state/api/rsyslog/rsyslog_test.go18:17
voidspaceno state/apiserver/rsyslog/rsyslog_test.go18:18
voidspacewwitzel3: they look like the same failure we've fixed before ("No state server machines found")18:18
voidspacewwitzel3: but I have to EOD18:18
voidspacewwitzel3: all my changes are pushedc18:19
voidspace*pushed18:19
voidspacewwitzel3: so have fun :-)18:19
voidspaceg'night everyone18:19
voidspaceEOD18:19
natefinchnight voidspace18:20
perrito666nite vds18:20
perrito666agh18:20
perrito666nick autocomplete should work after they left :)18:21
natefinchheh18:21
lazyPowernatefinch:  did we add a fsmount type of rpc_bindfs wrt bind mounting local filesystems on local provider?18:29
lazyPoweri ran into a corner case with apparmor not having the correct profile defition that blocked me from spinning up lxc containers this morning - working with ty helped me get it sorted.18:29
lazyPower*Definition18:29
natefinchlazyPower: interesting... we haven't twiddled with the local provider code very much recently.  I don't believe we've changed anything about the filesystem, or anything like that... though I know there's been some lxc/apparmor problems in the past18:33
lazyPowernatefinch: http://paste.ubuntu.com/7463805/18:34
lazyPowerthis is what i found - and it only happens when deploying local charms. if i specify CS charms i don't see the behavior.18:34
lazyPowerif we made some alterations to lxc without getting the app armor, i was requested to forward on teh bug so they can SRU the app armor fixes. just let me know either way and i'll put in the due dilligence.18:35
natefinchlazyPower: hmm... I really doubt we mucked with lxc lately.... can you try just spinning up a container manually and make sure it works without juju?18:37
lazyPowernatefinch: i only encountered the issue when using juju to deploy a local charm. otherwise it had no issues.18:38
lazyPoweri've created lxc containers while debugging, that didn't exhibit the issue18:38
natefinchlazyPower: weird.  Ok. There must be something we're doing with the local charms that is making app armor mad.... if you can make a bug for it, that would be great.  Obviously it's something we'll need to fix ASAP18:39
lazyPowerack. I'll add what i've got - which isn't much. but i can comment out the app armor patch and reproduce.18:39
lazyPowernatefinch: https://bugs.launchpad.net/juju-core/+bug/131952518:45
_mup_Bug #1319525: juju-local LXC containers hang due to App Armor Denial of rpc_fsbind request with local charms <local-provider> <lxc> <juju-core:New> <https://launchpad.net/bugs/1319525>18:45
natefinchlazyPower: awesome, thanks18:46
lazyPowerThanks for taking a look. I hope its just a corner case that I've found - and not something thats out there in the wild.18:47
perrito666if anyone has literally 2 seconds https://codereview.appspot.com/9737004418:51
=== vladk|offline is now known as vladk
natefinchneat, I may finally get to use the windows named pipe package that I wrote for Gustavo during(ish) my interview19:17
=== andreas__ is now known as ahasenack
menn0morning all21:04
=== vladk is now known as vladk|offline
waiganimorning all :)21:20
menn0waigani: howdy21:35
perrito666menn0: iirc, agent.conf from the old machine is put in place and services restarted in these ports21:35
perrito666menn0: the old agent.conf is ignored21:35
menn0perrito666: ok, so the agent.conf that awk is manipulating isn't the one from the backup?21:41
* menn0 pulls branch to have a closer look21:46
perrito666menn0: is the one from the backup21:46
perrito666what it currently does is it untars the backup over the fresh mashine21:47
perrito666machine*21:47
perrito666and then makes some surgery there21:47
perrito666:p21:47
menn0yep I get that21:47
menn0but if somehow the ports in the .jenv on the host that's running "juju restore" don't match what's in the backup then the user will never be able to connect to the env after it's bootstrapped21:48
menn0I know this should really never happen21:48
menn0sorry, I meant after it's restored, not after it's bootstrapped21:49
perrito666mmm, you mean if the port change between the backup and the new machine?21:49
menn0I'm thinking: backup is taken, someone monkeys with the ports, then restore is done21:50
menn0shouldn't happen I know21:51
perrito666menn0: mm,true, that is a story I am really not so happy to support for the moment21:51
menn0fair enough.21:51
menn0you have your LGTM anyway :)21:51
perrito666but, really not that far from where I am21:51
perrito666so I think I can give a shot at your idea until shinzui finishes the test :)21:51
menn0just refusing to restore with an error should be enough I would have thought21:52
menn0especially given that we currently don't support changing the API or state server ports once an env is bootstrapped21:52
menn0if the ports don't match between the .jenv and the backup then someone has done something they shouldn't have (or we have a bug)21:53
sinzuithumper, is this bug in juju-core, or does it belong  ubuntu/trusty/+source/lxc https://bugs.launchpad.net/bugs/131952522:04
_mup_Bug #1319525: juju-local LXC containers hang due to App Armor Denial of rpc_fsbind request with local charms <local-provider> <lxc> <juju-core:New> <https://launchpad.net/bugs/1319525>22:04
* thumper looks22:04
thumpersinzui: I'm unclear, but most likely lxc22:10
thumpersinzui: I asked lazyPower in the bug if he knows what caused the problems with the wordpress charm22:10
lazyPowerthumper: its any local charm. if i pull seafile and deploy it locally22:11
lazyPowerit has the same denial message22:11
lazyPoweri ran into this working the rev queue22:11
thumperah...22:11
thumperI don't know the local charm area very well at all22:11
thumperI don't think it should trigger anything with lxc22:12
thumpersinzui: it could be either juju or lxc...22:12
sinzuiIs this issue different from https://bugs.launchpad.net/juju-core/+bug/130528022:12
_mup_Bug #1305280: apparmor get_cgroup fails when creating lxc with juju local provider <apparmor> <armhf> <local-provider> <lxc> <packaging> <regression> <juju-core:Invalid> <apparmor (Ubuntu):Confirmed> <https://launchpad.net/bugs/1305280>22:12
lazyPowerit appears to be some kidn of fs mounting voodoo, because the error it threw was rpc_bindfs22:12
lazyPowerwhich i immediatly thought about that feature we talked about in vegas about bind mounting the local fs, but i didn't know if we did anything with that as of yet.22:12
thumpersinzui: the second bug there appears to be nested lxc failing22:13
thumperwhich is expected22:13
thumperlazyPower: we don't yet22:13
sinzuithumper, what make you think the cgoup bug is lxc in lxc?22:16
thumperbecause that is the failure you get if you try?22:17
* thumper looks at the bug22:17
thumpersinzui: ok, could just be apparmour on arm22:18
thumperdon't think it is us22:18
sinzuithumper, I am sure it isn't us. I just could get someone other than us to look at the issue22:19
thumperrick_h_: you there?22:30
* thumper calculates time zones22:30
rick_h_thumper: kinda22:30
thumperdinner time?22:30
rick_h_thumper: 6:30pm22:30
thumperrick_h_: got time for 10 min chat?22:30
rick_h_yea, dinner is in the oven, what's up?22:31
rick_h_sure thing22:31
thumpercheers22:31
thumperrick_h_: https://plus.google.com/hangouts/_/gsuud6kuvi6fowv6i3dfnpwsbqa?hl=en22:31
rick_h_lol23:01
rick_h_thumper: all good, let me know if you want to chat more, I want to make sure we get it right23:01
thumperrick_h_: I'll start documenting what we need and keep you in sync23:02
thumperwe are done for today23:02
rick_h_thumper: k, thanks for the chat23:05
rick_h_thumper: how dare you make me think about stuff :P23:27
thumperrick_h_: np... more coming23:38

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!