| === kadams54 is now known as kadams54-away | ||
| === kadams54 is now known as kadams54-away | ||
| === kadams54 is now known as kadams54-away | ||
| mup | Bug #1442493 was opened: Openstack services failing on 1 node while deploying using JUJU <juju-core:New> <https://launchpad.net/bugs/1442493> | 07:24 |
|---|---|---|
| dooferlad | TheMue: morning! o/ | 07:26 |
| TheMue | dooferlad: morning o/ | 07:27 |
| dooferlad | TheMue: Could you review this for me? http://reviews.vapour.ws/r/1406/ | 07:27 |
| TheMue | dooferlad: so the sapphire group is complete for today ;) | 07:28 |
| dooferlad | TheMue: yep! | 07:28 |
| TheMue | dooferlad: yes, will do | 07:28 |
| dooferlad | TheMue: thanks | 07:28 |
| TheMue | dooferlad: uff, reading the bug is longer than the fix :) | 07:38 |
| dooferlad | TheMue: the effort of testing it on EC2 was larger than that by many times. MAAS just worked. Our lack of support for when there is no default VPC really screwed me over | 07:39 |
| TheMue | dooferlad: do you know why this SNAT rule has been in there? so doesn't its removal crash something? | 07:39 |
| dooferlad | TheMue: not that I have found. | 07:39 |
| TheMue | dooferlad: hmm, ok. here I definitely need some good networking lessons to get more safe in understanding it | 07:41 |
| dooferlad | TheMue: I am sure I can arrange that :-) | 07:41 |
| TheMue | dooferlad: would be great. in my past I mostly have done typical business software, absolutely different area ;) | 07:42 |
| TheMue | dooferlad: you've got you ship it | 07:43 |
| dooferlad | TheMue: thanks! | 07:43 |
| TheMue | yw | 07:44 |
| * fwereade out for an hour or two | 08:13 | |
| dooferlad | TheMue: hangout? | 09:00 |
| TheMue | dooferlad: coming, just had phone | 09:01 |
| rogpeppe1 | this PR removes the testing deps from the production juju code: http://reviews.vapour.ws/r/1407/ | 09:05 |
| rogpeppe1 | reviews appreciated (it's pretty trivial) | 09:05 |
| TheMue | rogpeppe1: will do after hangout | 09:05 |
| rogpeppe1 | TheMue: ta! | 09:06 |
| dooferlad | TheMue: http://reviews.vapour.ws/r/1408/ is very similar to the last change :-) | 09:30 |
| TheMue | *click* | 09:30 |
| TheMue | dooferlad: thinking you're right, so ship it. alternatively %T is also valid, but no combination | 09:32 |
| natefinch | dooferlad, TheMue: there's nothing wrong with %#T | 09:33 |
| natefinch | http://play.golang.org/p/8XAOJVdeO2 | 09:33 |
| rogpeppe1 | natefinch: why would you use %#T - it's identical to %T | 09:34 |
| natefinch | rogpeppe1: because I always forget it's the same | 09:35 |
| TheMue | natefinch: rogpeppe1: %#T is not documented, and %#v and %T behave different in showing also the values with %#v | 09:35 |
| mup | Bug #1442541 was opened: hook name ommitted using juju-log <juju-core:New> <https://launchpad.net/bugs/1442541> | 09:36 |
| TheMue | hmm, does vet complain about it? | 09:36 |
| natefinch | TheMue: %T is the correct fix then. I just got done fixing that error message so it *didn't* use %v. It's doing type checking, so it should print out the type if it's incorrect, not the error's string, which doesn't tell you anything. | 09:37 |
| natefinch | dooferlad: ^ | 09:37 |
| TheMue | natefinch: ic, so dooferlad, fix it, then ship it ;) | 09:38 |
| natefinch | dooferlad: and thanks for fixing my go vet mistake | 09:38 |
| dooferlad | natefinch: no problem - shame I didn't ask you what your intention was before the first review :-) | 09:42 |
| natefinch | dooferlad: np :) | 09:45 |
| TheMue | dooferlad: $$merge$$, not %% | 09:51 |
| dooferlad | TheMue: darn it! | 09:52 |
| TheMue | dooferlad: no wonder, after all this %... discussion | 09:55 |
| rogpeppe1 | natefinch: looks like go vet doesn't like the %#T | 10:07 |
| rogpeppe1 | mgz: ping | 10:08 |
| rogpeppe1 | natefinch: i think in the place you've used it, %#v would be a better bet, as if the test fails you really want to see the actual value there not just the type | 10:15 |
| natefinch | rogpeppe1: very true | 10:15 |
| rogpeppe1 | natefinch: the test isn't doing type checking, it's doing value checking, BTW | 10:16 |
| rogpeppe1 | anyone have any idea how this build failure might be happening? http://juju-ci.vapour.ws:8080/job/github-merge-juju/2845/console | 10:16 |
| rogpeppe1 | Extant directories unknown: | 10:17 |
| rogpeppe1 | gopkg.in/juju/charm.v5 | 10:17 |
| natefinch | rogpeppe1: regardless, printing out the string value of the error is quite a bit less than useful | 10:17 |
| rogpeppe1 | i'm presuming that's the reason for the build failure, not the vet message: apiserver/server_test.go:88: unrecognized printf flag for verb 'T': '#' | 10:17 |
| rogpeppe1 | natefinch: that's why i'd use %#v | 10:17 |
| natefinch | rogpeppe1: yep | 10:17 |
| rogpeppe1 | natefinch: the string value is almost certainly going to be more useful than just the type though | 10:18 |
| rogpeppe1 | natefinch: which will usually be just *errors.Err | 10:18 |
| natefinch | rogpeppe1: let's just agree that %#v is the correct fix :) | 10:19 |
| rogpeppe1 | natefinch: :) | 10:19 |
| natefinch | dooferlad: is that code already committed? | 10:19 |
| dooferlad | natefinch: yes | 10:19 |
| dooferlad | natefinch: as %T | 10:20 |
| dooferlad | I did wonder about %T: %#v... | 10:20 |
| natefinch | dooferlad: %#v includes the type...%#v is basically %T with %v | 10:21 |
| natefinch | well.. no that;s a bad description | 10:21 |
| natefinch | dooferlad: %#v prints out the value as if it were go code to construct the value | 10:23 |
| natefinch | dooferlad: http://play.golang.org/p/6HKt-7atpL | 10:23 |
| natefinch | except of course, ignore the "type:" string on the beginning of each line, since that was just copied from the old text I had in there | 10:24 |
| natefinch | fixed example: http://play.golang.org/p/FtEYRp1dJS | 10:24 |
| natefinch | I need to stop trying to explain things before 7am | 10:24 |
| dooferlad | natefinch: no worries :-) | 10:25 |
| dooferlad | natefinch: actually the merge job failed, so I can easily switch back to %#v if you like | 10:25 |
| natefinch | dooferlad: yes please :) | 10:26 |
| natefinch | rogpeppe1: about that error in the console... note that error message is looking for gopkg.in/juju/charm.v5, not gopkg.in/juju/charm.v5-unstable | 10:31 |
| rogpeppe1 | natefinch: yes, i think i know what's going on now | 10:31 |
| rogpeppe1 | natefinch: i've updated that branch to change everything to use charm.v5 | 10:32 |
| rogpeppe1 | natefinch: which should fix the issue (although it makes that branch quite a bit bulkier) | 10:32 |
| rogpeppe1 | natefinch: i've also changed it to fix your %#T thing | 10:32 |
| natefinch | rogpeppe1: I think dooferlad is making the %#v fix, though I suppose it can't hurt | 10:33 |
| mgz | rogpeppe1: hey | 10:33 |
| rogpeppe1 | mgz: hiya | 10:33 |
| rogpeppe1 | mgz: too late! :) | 10:33 |
| mgz | you're just loving breaking deps at the moment | 10:33 |
| rogpeppe1 | mgz: it's one of my favourite activities | 10:34 |
| mgz | well, at least this one was an actual catch by the bot | 10:34 |
| rogpeppe1 | mgz: not really | 10:34 |
| rogpeppe1 | mgz: it's actually a bug in godeps | 10:34 |
| rogpeppe1 | mgz: because godeps uses go get to fetch new deps | 10:35 |
| rogpeppe1 | mgz: and there's no way to tell go get not to fetch recursively | 10:35 |
| mgz | yeah, which just pulls in everything | 10:35 |
| rogpeppe1 | mgz: so really i think godeps needs to fork the go get vcs fetch functionality | 10:35 |
| mgz | so, tells you to fix the imports, no? | 10:35 |
| rogpeppe1 | mgz: no, the imports are right | 10:35 |
| mgz | rogpeppe1: I agree with that though, we need to not go get really | 10:36 |
| rogpeppe1 | mgz: the problem is that the tip of the repo has a different set of deps from the dep we wanted to use | 10:36 |
| mgz | rogpeppe1: ah, that one again | 10:36 |
| mgz | er... i can sort that for now I guess | 10:36 |
| rogpeppe1 | mgz: i'm fixing it my updating to use the latest deps | 10:37 |
| mgz | I have a flip that just removes unkniwn deps rather than complaining, and assumes that if things build the godep stated deps were in fact the intended ones | 10:37 |
| rogpeppe1 | mgz: the branch was all about updating deps anyway | 10:37 |
| mgz | fair enough | 10:37 |
| rogpeppe1 | mgz: so here's the branch, updated (and now huge 'cos of all the import path changes): http://reviews.vapour.ws/r/1407 | 10:37 |
| rogpeppe1 | mgz: if i want to change the description of a PR, should I do it in reviews.vapour or in github ? | 10:39 |
| mgz | rogpeppe1: well, reviews is the one people read | 10:39 |
| rogpeppe1 | mgz: i care mostly about the commit log message | 10:39 |
| mgz | I guess change on github then | 10:40 |
| * rogpeppe1 $$merge$$s | 10:41 | |
| === rogpeppe1 is now known as rogpeppe | ||
| rogpeppe | mgz: https://go-review.googlesource.com/#/c/8725/ | 12:00 |
| rogpeppe | mgz: it won't fix it for now but sometime in the future, i hope | 12:00 |
| rogpeppe | mgz: (assuming it's accepted) | 12:00 |
| mattyw | fwereade, rogpeppe is one or both of you packing dominion? | 12:35 |
| fwereade | mattyw, I will surely pack a set or two | 12:36 |
| fwereade | mattyw, hopefully so will rogpeppe or dimitern as well | 12:36 |
| rogpeppe | fwereade: i definitely intend to bring some, probably seaside, prosperity, maybe intrigue too | 13:13 |
| fwereade | rogpeppe, intrigue is probably my favourite -- hopefully dimitern will bring his :) | 13:14 |
| fwereade | rogpeppe, any of mine you'd be particularly interested to see? | 13:14 |
| dooferlad | TheMue: Another review for you... http://reviews.vapour.ws/r/1400/ | 13:14 |
| rogpeppe | fwereade: i haven't played many real life games with alchemy, and dark ages is always good | 13:15 |
| fwereade | rogpeppe, I don;t have alchemy actually -- I'll probably go with hinterlands/dark ages | 13:15 |
| rogpeppe | fwereade: sgtm | 13:15 |
| rogpeppe | fwereade: BTW i came across your recent dependency engine thing - cool stuff! | 13:17 |
| fwereade | rogpeppe, glad you like it -- it borrows heavily from worker.Runner | 13:17 |
| rogpeppe | fwereade: i like that it fits in with the worker framework too | 13:17 |
| fwereade | rogpeppe, and I *suspect* will come closer still to it as I need to cover some of the more baroque cases I want in jujud | 13:18 |
| fwereade | rogpeppe, yeah, absolutely, the worker approach has been fantastic -- it's just organising them clearly where we've really fallen down | 13:18 |
| rogpeppe | fwereade: i still don't quite understand the overall motivation behind it mind | 13:18 |
| fwereade | rogpeppe, I want to run something to take over some of the uniter's responsibilities when certain resources (like an api conn) are not available | 13:19 |
| rogpeppe | fwereade: the output value thing could probably do with a little more documentation | 13:19 |
| fwereade | rogpeppe, the thought of coordinating two distinct workers that ran at such very different levels made me cry | 13:20 |
| fwereade | rogpeppe, noted, thanks | 13:20 |
| rogpeppe | fwereade: i found the term "manifold" somewhat opaque; was this the meaning you had in mind "a manifold is a topological space that resembles Euclidean space near each point. More precisely, each point of an n-dimensional manifold has a neighbourhood that is homeomorphic to the Euclidean space of dimension n" | 13:21 |
| rogpeppe | ? | 13:31 |
| fwereade | rogpeppe, no, it's a mechanism with pipes going in and oout of it | 13:32 |
| rogpeppe | fwereade: ah "a chamber having several outlets through which a liquid or gas is distributed or gathered." | 13:32 |
| rogpeppe | fwereade: not a term i'm familiar with | 13:32 |
| rogpeppe | fwereade: (which isn't to say that it's not entirely appropriate!) | 13:33 |
| fwereade | rogpeppe, I could swear I made a point of documenting that... | 13:33 |
| fwereade | / Manifold defines the behaviour of a node in an Engine's dependency graph. It's | 13:33 |
| fwereade | / named for the "device that connects multiple inputs or outputs" sense of the | 13:33 |
| fwereade | / word. | 13:33 |
| rogpeppe | fwereade: ha, i evidently skipped over that :) | 13:34 |
| fwereade | rogpeppe, ;p | 13:34 |
| sinzui | katco, dooferlad : the commit that reverted the container SNAT rule in 1.23 may be causing aws bundle tests to fail exactly like bug 1441319 | 13:45 |
| mup | Bug #1441319: intermittent: failed to retrieve the template to clone: template container juju-trusty-lxc-template did not stop <lxc> <oil> <juju-core:Incomplete by cox-katherine-e> <https://launchpad.net/bugs/1441319> | 13:45 |
| * sinzui tries to get container log | 13:45 | |
| dooferlad | sinzui: no, that is because we don't support container cloning. | 13:47 |
| sinzui | dooferlad, this test passed on the previous version on 1.23 and we know the landscape bundle we are deploying hs not changed | 13:47 |
| TheMue | dooferlad: just seen your message, reviewing it now | 13:48 |
| sinzui | this test passed on joyent and maas 1.7 and hp cloud | 13:48 |
| dooferlad | sinzui: it was also reported before the SNAT rule change was reverted | 13:48 |
| sinzui | dooferlad, not be CI and not with 1.23 | 13:49 |
| sinzui | dooferlad, Until your commit, CI has never seen this error. This test is voting, so 1.23 will not be blessed for a release. I need to get more information about what is wrong. | 13:50 |
| dooferlad | sinzui: OK, the only thing that should have changed in terms of what other machines see, is that traffic from containers comes from the containers IP address, not the host machine's address. This used to be the case before rev b584fcb85b9bcce3dadba97d32fe50e8f3680e40 | 13:57 |
| dooferlad | on that commit we added an SNAT rule to modify all traffic leaving a container host to look as if it came from that host | 13:57 |
| sinzui | dooferlad, yes, I agree and this test was happy when that change was made. I don't know why but would appear. could there have been another change made inconjuction for aws | 13:58 |
| * natefinch almost doesn't know what to do now that the HA code finally made it in. | 14:01 | |
| dooferlad | sinzui: so it worked OK on the 8th after the proxy change? That would seem more likely. | 14:03 |
| sinzui | dooferlad, yep | 14:03 |
| natefinch | sinzui: what would you say is the correct behavior for this bug? https://bugs.launchpad.net/juju-core/+bug/1441904 just failing? | 14:04 |
| mup | Bug #1441904: juju upgrade-juju goes into an infinite loop if apt-get fails for any reason <canonical-is> <upgrade-juju> <juju-core:Triaged> <https://launchpad.net/bugs/1441904> | 14:04 |
| sinzui | natefinch, yes, say it failed | 14:05 |
| natefinch | sinzui: wonder if we should retry a few times in case of network problems etc | 14:05 |
| sinzui | natefinch, I thought all apt calls were retried 3 times. dimitern fixed an issue a few months ago where juju wasn't retrying | 14:07 |
| natefinch | sinzui: hmm.. good question, I can check, I don't know offhand | 14:08 |
| sinzui | katco, I added the container.log you requested on https://bugs.launchpad.net/juju-core/+bug/1441319 | 14:15 |
| mup | Bug #1441319: intermittent: failed to retrieve the template to clone: template container juju-trusty-lxc-template did not stop <lxc> <oil> <juju-core:Incomplete by cox-katherine-e> <https://launchpad.net/bugs/1441319> | 14:15 |
| katco | sinzui: ty... that's from the machine that failed? | 14:16 |
| sinzui | katco, no larry's. the latest commit to 1.23 broken the aws deployer test in CI | 14:17 |
| katco | sinzui: confused... isn't larry the one who reported the failure? and this is the one we're thinking is intermittent? | 14:18 |
| sinzui | katco, yes, but now CI is affected. I am providing a log | 14:18 |
| katco | sinzui: ah ok. so we're sure it's the same issue? | 14:18 |
| sinzui | yep. we can reproduce it on command with 1.23 tip deploying the landscape bundle to aws | 14:19 |
| sinzui | I have 5 failures | 14:19 |
| katco | sinzui: i thought larry's issue was an lxc container problem... | 14:19 |
| * katco apologizes if she's being dense | 14:19 | |
| natefinch | rogpeppe, fwereade: FWIW, I have Alchemy, it's just not that great, so I kinda hesitate to lug it 3000 miles :) | 14:20 |
| rogpeppe | natefinch: i've quite enjoyed it on occasion in Androminion | 14:21 |
| sinzui | katco, oh, sorry, my error is not the same. I will report another bug. | 14:21 |
| katco | sinzui: k, thanks for walking me around the block on that one | 14:22 |
| dooferlad | it looks like that container is having problems because of apparmor. The peer has disconnected message is because https://github.com/dotcloud/lxc/blob/master/src/lxc/af_unix.c -> lxc_af_unix_rcv_credential is failing. | 14:22 |
| katco | sinzui: still working on some caffeien here :p | 14:22 |
| katco | caffeine even | 14:22 |
| sinzui | katco, thank you for questioning me. I am low on caffeine. | 14:22 |
| katco | sinzui: a toast! | 14:22 |
| natefinch | rogpeppe: at least it's small. I could probably leave the box at home and it would be less of a big deal. | 14:23 |
| katco | dooferlad: are you refering to 1441319? | 14:23 |
| dooferlad | katco: yes | 14:23 |
| katco | dooferlad: which log on that bug? | 14:24 |
| dooferlad | katco: https://bugs.launchpad.net/juju-core/+bug/1441319/+attachment/4371597/+files/container.log (the last attachment, from sinzui) | 14:24 |
| mup | Bug #1441319: intermittent: failed to retrieve the template to clone: template container juju-trusty-lxc-template did not stop <lxc> <oil> <juju-core:Incomplete by cox-katherine-e> <https://launchpad.net/bugs/1441319> | 14:24 |
| katco | dooferlad: ah ok. so that will be relavant to sinzui's new bug he's opening, not 1441319 | 14:25 |
| dooferlad | katco: yea, indeed | 14:25 |
| katco | relevant even | 14:25 |
| katco | jees with the spelling this morning | 14:25 |
| alexisb | dooferlad, can you work iwth sinzui on a bug as a result of your lastest commit | 14:29 |
| dooferlad | alexisb: sure | 14:33 |
| alexisb | and dooferlad I just finished reading the backscroll :) | 14:33 |
| sinzui | dooferlad, katco sorry, more caffeine and CI's error is the same as oil's 'failed to retrieve the template to clone: template container | 14:35 |
| sinzui | "juju-trusty-lxc-template" did not stop' | 14:35 |
| sinzui | instance-id: pending so https://bugs.launchpad.net/juju-core/+bug/1441319 is the issue | 14:35 |
| mup | Bug #1441319: intermittent: failed to retrieve the template to clone: template container juju-trusty-lxc-template did not stop <lxc> <oil> <juju-core:Incomplete by cox-katherine-e> <https://launchpad.net/bugs/1441319> | 14:35 |
| katco | sinzui: that's actually great! repeatable! | 14:37 |
| dooferlad | TheMue: thanks for that review. Fixes commited. | 14:49 |
| natefinch | anyone have an opinion on the best way to get a PR merged into master into the 1.23 branch? I could re-PR to 1.23, or I could cherry-pick from master to 1.23... not sure which was is "better" (or if there's a better third way) | 14:54 |
| natefinch | mgz: ^ any thoughts? | 14:56 |
| natefinch | (should say "to get a PR that was merged into master, into the 1.23 branch" | 14:56 |
| natefinch | sinzui: for the record, we try apt installs *30* times, with 10 second delays between each. Wowza. | 14:59 |
| sinzui | natefinch, well we tried. juju can admit it failed | 14:59 |
| katco | natefinch: i always like to cherry pick from master, but that's just my workflow | 14:59 |
| katco | natefinch: if it's important that it be done fast for 1.23, land it there first and forward port | 14:59 |
| natefinch | sinzui: yep, it shouldn't be a big deal to have the error from apt-get just tell the command to fail. If it doesn't work after trying for 5 minutes, it's not going to work. | 15:00 |
| dooferlad | katco: I thought forward porting was the new normal. Maybe just for Saphire. | 15:00 |
| natefinch | no, we're supposed to land on 1.23 first and forward port ,from what I remember... I just did the wrong thing | 15:00 |
| katco | dooferlad: could be? i don't think it really matters... isn't it personal preference? | 15:00 |
| natefinch | katco: I don't think it really matters, which is not the same as saying no one cares ;) | 15:01 |
| katco | natefinch: hey. you paint that shed blue dammit. | 15:01 |
| natefinch | I have an email from alexisb about making sure to land on stable and then forward port to master.... but it was only sent to me and wwitzel3, but I think that's The Way It's Supposed To Be™ | 15:03 |
| sinzui | natefinch, bug 1394755 says it is fixed in 1.23, but I don't see the commit. I do see the commit in master though | 15:17 |
| mup | Bug #1394755: juju ensure-availability should be able to target existing machines <cloud-installer> <ha> <landscape> <juju-core:Fix Committed by natefinch> <https://launchpad.net/bugs/1394755> | 15:17 |
| mup | Bug #1436863 was opened: test failure replicaset_test.go:52: SetUpTest.pN42_github_com_juju_juju_replicaset.MongoSuite <ci> <ppc64el> <regression> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1436863> | 15:22 |
| natefinch | sinzui: working on that, sorry | 15:31 |
| natefinch | katco: how do I cherry-pick the PR without listing out the 37 commits? Picking just the merge commit says I'm missing -m, and using -m says it expects a numerical value, but I have no idea what the number should be | 15:36 |
| natefinch | also probably doesn't help that I have merge commits from master into that branch... sigh | 15:37 |
| sinzui | mgz, can your review http://reviews.vapour.ws/r/1409/ | 16:04 |
| katco | natefinch: sorry i was in a meeting | 16:05 |
| katco | natefinch: i am the worst person to ask for advice on git, because i use an emacs mode which does it all for me | 16:06 |
| natefinch | katco: haha no problem | 16:06 |
| mgz | sinzui: on it | 16:06 |
| natefinch | katco: I think I'm getting it. I just ended up doing a cherry pick of all ~34 non-merge commits | 16:06 |
| mgz | sinzui: lgtm, hold till when you need it | 16:07 |
| sinzui | thank you mgz | 16:07 |
| alexisb | natefinch, "git help -a" should have what you are looking for | 16:13 |
| alexisb | though I must admit that it has been a long time since i have done a kernel patch and used git | 16:13 |
| natefinch | alexisb: git help doesn't always. | 16:15 |
| katco | alexisb: you used to work on the linux kernel? | 16:15 |
| alexisb | yes mam | 16:15 |
| katco | alexisb: that's awesome! | 16:15 |
| natefinch | alexisb: I thought i had it, but somehow my cherry pick left this branch in a bad state... some code referring to packages that don't exist. Sigh. | 16:16 |
| alexisb | it was fun, there are days I think about going back :) | 16:16 |
| katco | alexisb: nooo! we need you! | 16:16 |
| alexisb | katco, it has been long enough I would have to really work to get back into it, my last public commit was in 2008 I believe | 16:17 |
| natefinch | alexisb: I'm sure it's mostly the same ;) | 16:17 |
| sinzui | natefinch, I have paused CI because I have a change merging now, and I don't want CI to be busy when you change arrives | 16:18 |
| natefinch | sinzui: thanks, I guess. I don't know that my change will make it in terribly soon, given the troubles I'm having with git, and I need to go pick up my daughter from preschool in 20 minutes | 16:18 |
| sinzui | :( | 16:19 |
| katco | natefinch: is it checked into 1.23? | 16:19 |
| sinzui | natefinch, I assume CI will take 3 hours to test my version change. | 16:19 |
| natefinch | katco: no, it's in master, that's the problem... I developed it off of master, merged master into my branch a couple times, and then PR'd into master | 16:20 |
| katco | natefinch: so you're trying to backport? | 16:20 |
| natefinch | katco: yes | 16:20 |
| katco | natefinch: what's the commit hash? | 16:20 |
| natefinch | katco: b228e89dd3ef9a3fe0f14b958db123e87a68bc48 | 16:21 |
| natefinch | katco: I presume there's some magic command line that'll do the right thing... but I'll be damned if I can figure it out. | 16:22 |
| katco | natefinch: from what i can remember, it's not a single command | 16:22 |
| katco | natefinch: you set cherry head, and upstream | 16:22 |
| katco | natefinch: and then start picking commits | 16:22 |
| alexisb | natefinch, katco if we have someone that can help out natefinch on this that would be great | 16:25 |
| alexisb | we gotta get 1.23 out | 16:25 |
| katco | alexisb: i am backporting now | 16:25 |
| mup | Bug #1258485 changed: support leader election for charms <juju-core:Fix Released by jameinel> <cassandra (Juju Charms Collection):Triaged> <postgresql (Juju Charms Collection):Triaged> <https://launchpad.net/bugs/1258485> | 16:28 |
| katco | natefinch: ptal: http://reviews.vapour.ws/r/1410/ | 16:29 |
| katco | natefinch: especially make sure i didn't mess up the merge | 16:29 |
| natefinch | katco: hmm.... I think I must have given you the wrong hash.... that's not what needed to get ported... well, that may still need to get ported, but that's not all of it | 16:31 |
| katco | natefinch: is there more than one commit? | 16:31 |
| natefinch | katco: there's two different merges... one is the test fixes and one is the HA --to code | 16:31 |
| katco | natefinch: k i'll pull the other commit too | 16:32 |
| natefinch | katco: there's Merge pull request #1962 from natefinch/ha3 which contains 37 commits :/ | 16:33 |
| katco | natefinch: k tal | 16:33 |
| mgz | it is possible to charrypick an entire merged branch with git, I don't recall the spelling off the top of my head | 16:34 |
| natefinch | mgz: it's the spelling that I'm not sure of... plus the branch contains commits that are merges from master to that branch, which would contain changes that shouldn't go into 1.23 (presumably) | 16:35 |
| mgz | natefinch: right, but the diff from merged to master commit vs previous commit to master should be the right thing | 16:35 |
| mgz | so the dumb version is just take that diff and apply it, but cherry-pick does have a one-step of that | 16:37 |
| natefinch | mgz: I would think so, but I'm always disappointed in what vcs figures out on its own | 16:37 |
| mgz | git just doesn't record mainline, so talking about three-way merges in the commandline is more annoying | 16:38 |
| natefinch | unfortunately, I gotta run to pick up my daughter.... I'll be back in like an hour. That's about the best I can do. | 16:38 |
| katco | gosh... it's going to be hard for me to merge this much in without knowing anything about what i'm merging | 16:44 |
| mgz | katco: how conflicty is the `git cherry-pick -m <rev for parent> <merge rev>`? | 16:48 |
| sinzui | natefinch, mgz, I think I had to use patch or format-patch once | 16:56 |
| katco | mgz: 47 unmerged files | 16:58 |
| mup | Bug #1442719 was opened: juju sync-tools fails <sync-tools> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1442719> | 16:59 |
| katco | mgz: i (or rather emacs) ended up doing "git --no-pager -c core.preloadindex=true merge master~2 --no-commit" | 16:59 |
| mgz | hm, I wonder if that's actually right without specifying the mainline? | 17:01 |
| katco | mgz: i'm merging it into a branch off of 1.23 upstream | 17:01 |
| mgz | maybe the ~2 just happens to be resolving right on sha1 alpha order | 17:02 |
| katco | mgz: it is; that's emacs doing magic | 17:02 |
| katco | mgz: i found the merge master hash, hit m and it does the right thing | 17:02 |
| katco | mgz: i would hate doing all of this manually | 17:02 |
| katco | mgz: which is to say via the command line | 17:03 |
| mgz | well resolving conflicts is always going to need an editor | 17:03 |
| mgz | you can use an OS if you want :P | 17:03 |
| katco | mgz: haha | 17:03 |
| katco | mgz: i'll show you this workflow in nuremberg. it's quite nice | 17:03 |
| mgz | neat | 17:04 |
| katco | mgz: merge dumps me into a buffer with all merge conflicts. pressing "e" on an unmerged file dumps me into a visual diff | 17:04 |
| katco | mgz: and i hit "a" or "b" depending on which side i want. and can edit in 3rd pane at bottom | 17:04 |
| katco | alexisb: i can't perform this backport. i don't know enough about the patch to perform the requisite merges. | 17:05 |
| alexisb | katco, just fyi, the juju-core leadership team advocates the process of forward porting as juju has historically seen more issues with back porting then forward porting | 17:05 |
| === psivaa_ is now known as psivaa-afk | ||
| alexisb | katco, ack, we will have to wait for natefinch then | 17:05 |
| katco | alexisb: ok, duly noted. | 17:05 |
| alexisb | as wwitzel3 is traveling | 17:05 |
| alexisb | mgz, sinzui if need be we will have to add the HA working in a point release | 17:06 |
| alexisb | sinzui, mgz, dooferlad, where are we with the aws container failure? | 17:07 |
| sinzui | alexisb, If Ubuntu deem the patch to be a feature, they will not accept it. natefinch is the patch more like a bug than a feature | 17:08 |
| alexisb | sinzui, in many of the "bugs" we fix for stakeholders there is a fine line between feature and bug | 17:10 |
| alexisb | I see your point, but thought we had decided this was a bug | 17:11 |
| sinzui | alexisb, yes, and Ubuntu doesn't honour that line they have reject our streams change in 1.20 and our right refused to package the backup/restore changes to 1.16.4 | 17:12 |
| alexisb | katco, get with natefinch when he is back and lets get this merged | 17:14 |
| alexisb | sinzui, I see this impacting our ability to deliver by weds | 17:14 |
| alexisb | sinzui, where do we stand on the aws failure mentioned this morning? | 17:14 |
| sinzui | No progress. | 17:14 |
| sinzui | alexisb, and this bug was just moved to juju-core https://bugs.launchpad.net/juju-core/+bug/1439535 | 17:15 |
| mup | Bug #1439535: 1.23-beta2 websocket incompatibility <juju-core:New> <python-jujuclient:Incomplete> <https://launchpad.net/bugs/1439535> | 17:15 |
| sinzui | ^ There are contradictions about beta3 being affected | 17:16 |
| alexisb | so sinzui adding bugs is not helping get the release out :) | 17:16 |
| alexisb | so lets start with the bug from this morning, what does no progress mean? | 17:17 |
| alexisb | is there no one from core looking at it? is there something blocking us from debugging the issue? | 17:17 |
| sinzui | alexisb, neither katco or dooferlad have asked for more information from me after I provided the log that was requested. | 17:18 |
| sinzui | alexisb, I am at a critical moment in the release, so I cannot let myself get too distracted | 17:18 |
| alexisb | sinzui, that is fine, I understand | 17:19 |
| alexisb | dooferlad, are you still around? | 17:19 |
| dooferlad | alexisb: just. In the middle of feeding the baby | 17:19 |
| alexisb | :) that is a fun task | 17:20 |
| alexisb | I know it is close to your eod but have you had a chance to take a look at the logs sinzui provided | 17:20 |
| dooferlad | alexisb: As I said above, the failure seems to be to do with LXC and apparmor having a bad interaction. | 17:21 |
| dooferlad | alexisb: and I haven't touched code anywhere near that | 17:21 |
| alexisb | dooferlad, ack, katco is this similar to the issue you looked at earlier this week? | 17:22 |
| katco | alexisb: yeah... looks to be exactly the same | 17:22 |
| alexisb | which we determined was not a juju issue, correct? | 17:23 |
| sinzui | dooferlad, really. great. let me check to see if an apparmor change was delivered to trusty in the last day ( and see if the aws mirrors are our of sync | 17:23 |
| katco | alexisb: we were leaning that way because we hadn't touched the code in that area; however, sinzui indicated that a commit caused this to be repeatable? | 17:24 |
| katco | dooferlad: or was sinzui saying the change was your commit? | 17:24 |
| sinzui | dooferlad, katco. apparmor did change last week: | 17:24 |
| sinzui | 2.8.95~2430-0ubuntu5release (main)2014-04-04 | 17:24 |
| katco | sinzui: if we haven't done any commits that touch that area of code, i'm blaming that | 17:25 |
| sinzui | We have testing this week that says aws was fine, but let me see if I can find evidence that the mirrors are stale | 17:25 |
| katco | sinzui: i apologize, i still don't understand the connection between aws and an lxc issue? | 17:25 |
| sinzui | katco, I am sorry about the situation | 17:26 |
| katco | sinzui: i'm assuming that i'm missing something; nothing to be sorry for | 17:26 |
| dooferlad | OK all, see you in Germany! | 17:30 |
| katco | dooferlad: see you there! | 17:30 |
| natefinch | katco: back | 17:31 |
| rogpeppe | mgz: here's the start of the fix to godeps - copying a bunch of code from the go tool: https://codereview.appspot.com/223390043/ | 17:31 |
| katco | natefinch: hey, i can't backport that change... there are too many merge conflicts i don't understand | 17:32 |
| natefinch | katco: I was worried about that | 17:32 |
| mgz | rogpeppe: neat! I had a look at your get upstream change earlier, that seems like a good idea regardless | 17:33 |
| katco | natefinch: try "git --no-pager -c core.preloadindex=true merge master~2 --no-commit" but tweak the ~2 to point to the correct merge hash for your branch of master | 17:39 |
| natefinch | katco: man, how did I miss something so obvious? | 17:40 |
| katco | natefinch: (shrugs) it's easy to get blinders on that you have to cherry | 17:40 |
| perrito666 | natefinch: obviously you dont use emacs, there most likely is a shortcut for that, smth like: Ctrl-M gnpccptmmnc2 | 17:41 |
| katco | perrito666: lol i put my cursor over the merge hash and pressed "m" | 17:41 |
| katco | some of those command line flags are probably superfluous | 17:41 |
| * katco somewhat embarrassingly just realized natefinch was joking. | 17:42 | |
| natefinch | katco: how do I figure out the number after ~? That stuff confuses the hell out of me | 17:42 |
| natefinch | katco: lol yes | 17:42 |
| katco | natefinch: it's just an offset from head | 17:42 |
| katco | natefinch: so however many commits back from head that merge hash is for you | 17:43 |
| natefinch | katco: you say that, but sometimes ~2 brings in like 1000 changes if you've done a merge | 17:43 |
| katco | natefinch: i pulled from trunk recently so it is likely the same, but i would 2x check | 17:43 |
| alexisb | katco, I need to step out for a bit, but I will be back soon, can you please make sure that the latest summary gets captured here: https://bugs.launchpad.net/juju-core/+bug/1441319 | 17:48 |
| mup | Bug #1441319: intermittent: failed to retrieve the template to clone: template container juju-trusty-lxc-template did not stop <deployer> <lxc> <oil> <regression> <juju-core:Triaged by cox-katherine-e> <juju-core 1.23:Triaged> <https://launchpad.net/bugs/1441319> | 17:48 |
| alexisb | so that we don't loose the conversation that happened on irc | 17:49 |
| katco | alexisb: sure | 17:49 |
| alexisb | thanks | 17:49 |
| natefinch | katco: this keeps bringing in a ton of changes to files I know I haven't been anywhere near | 17:50 |
| katco | natefinch: ok, i'll see if i can wizard up another command for you | 17:51 |
| natefinch | katco: what I really need is to just whip up a patch file from the commit to master and then apply that to 1.23... | 17:51 |
| katco | natefinch: so see on the PR, the merge has 2 parents? https://github.com/juju/juju/commit/b228e89dd3ef9a3fe0f14b958db123e87a68bc48 | 17:55 |
| katco | natefinch: which is the parent you don't want to cherry? | 17:55 |
| * perrito666 has a ton of things to do and already lost 6hs on bureaucracy and now I just discovered that wile I was a away the mailman delivered mail... before the rain | 17:56 | |
| natefinch | katco: both of those should be fine... though I don;t know why those are parents per se... those are just the two previous commits I made | 17:57 |
| natefinch | katco: I think the problem is that I merged master into my branch, which is evidently screwing up git, because it thinks I then want to merge that into 1.23, which I don't. | 18:00 |
| cmars | natefinch, i've often found git diff start..merge | patch -p1 --merge works, when cherry-pick doesn't dwiw. maybe there's a way to tell cherry-pick how to pick a sensible parent, but i've not grasped it | 18:02 |
| cmars | that's kind of dirty, but works when branches have diverged quite a bit | 18:03 |
| katco | natefinch: so maybe do both: git cherry-pick b228e89dd3ef9a3fe0f14b958db123e87a68bc48 -m 1 | 18:03 |
| katco | natefinch: and then git cherry-pick b228e89dd3ef9a3fe0f14b958db123e87a68bc48 -m 2 | 18:04 |
| mgz | right, that's it | 18:07 |
| mgz | whether it's -m 1 or 2 depends on alpha order of the parent shas | 18:07 |
| mgz | you may want to squidge the commits anywhat if using cherrypick | 18:08 |
| cmars | git-squidge should totally be a thing | 18:08 |
| katco | cmars: lol | 18:08 |
| natefinch | katco: so far, the first one looks promising | 18:09 |
| natefinch | katco: and that second one brings in a bunch of crap that is totally wrong. So maybe just the first one is correct | 18:10 |
| natefinch | running tests... we'll see how it goes. It looks promising | 18:11 |
| katco | natefinch: are you sure that has all of the changes you need? | 18:18 |
| natefinch | katco: nope. I should probably throw up a diff and make sure it includes everything | 18:19 |
| natefinch | katco: looking good. Files look correct. NUmber of changes is correct | 18:29 |
| katco | natefinch: woohoo! | 18:30 |
| natefinch | well, it is missing my other test fixes, but that's just a single commit which hopefully is trivial to cherry-pick | 18:45 |
| katco | not bad | 18:45 |
| natefinch | they're not strictly part of HA, but they fail reliably on my machine... maybe they don't on the bot though | 18:45 |
| natefinch | I think the bot is running with GO_MAX_PROCS = 1, which is likely the difference | 18:46 |
| katco | ah yeah i run with that too... i learned quickly lol | 18:46 |
| natefinch | katco: I think that backport you produced is actually the other half of stuff I'd like to get in | 18:48 |
| katco | natefinch: lol i deleted the pr | 18:48 |
| natefinch | katco: here's the review for my backport (really, I should just call it your backport ;) | 18:49 |
| natefinch | http://reviews.vapour.ws/r/1414/ | 18:49 |
| katco | natefinch: ah jees... for the same reason i couldn't do the backport, this is going to be hard to review | 18:49 |
| katco | natefinch: how confident are you that you pulled the right changes? | 18:49 |
| mgz | I'm going to have a bunch of trivial changes to licence headers I'll need a rubber stamp on shortly | 18:50 |
| mgz | ..meh, and dep bumps that's a bit more painful | 18:50 |
| natefinch | katco: basically 100%. I did a visual side by side diff of the changes for the original PR: https://github.com/juju/juju/pull/1962/files vs. the backport PR: https://github.com/juju/juju/pull/2060/files | 18:51 |
| natefinch | katco: the diff in the review in this case is not really useful. You need to diff the two diffs :) | 18:51 |
| katco | natefinch: k i'm going to rubber stamp it then... | 18:51 |
| katco | natefinch: and tests are passing? | 18:52 |
| mgz | oh, but I'm being called for food first | 18:52 |
| natefinch | katco: they only fail in the way the ones on master fail for me... if that's any consolation. Some of that is what I fixed in that other PR | 18:53 |
| katco | k | 18:53 |
| katco | you have been rubber stamped | 18:53 |
| natefinch | katco: I promise to fall on the grenade if it blows up after merging. | 18:54 |
| katco | lol | 18:54 |
| natefinch | So, last night at 6pm, I realized that my last two pairs of jeans that were at all fit to wear had decided to simultaneously acquire large rips in the knees.... and given no time to actually go out shopping anywhere.. .I made my first purchase of jeans off amazon, to be delivered tomorrow, hopefully before I have to leave. | 18:57 |
| katco | lol that is brave | 18:57 |
| natefinch | I *think* they're a style that has fit well before... and men's sizes are usually regular enough that as long as I stay away from slim-fit and other nonsense, they'll just work. But yes, it is a bit of a gamble. | 18:58 |
| natefinch | I wouldn't have done it if they didn't explicitly say they have free returns. | 18:58 |
| natefinch | katco: btw..... just noticed this works: https://patch-diff.githubusercontent.com/raw/juju/juju/pull/1962.patch | 19:00 |
| sinzui | dooferlad, katco. I have confirmed that 1.22.1 and 1.23-beta4 can deploy the test bundle to aws, but the revision that changed reverted container networking cannot. I can report a separate bug if you think that will be less confusing. | 19:00 |
| natefinch | katco: I got that from seeing this tip at the bottom of the PR page: ProTip! Add .patch or .diff to the end of URLs for Git's plaintext views. | 19:01 |
| sinzui | dooferlad, The release notes for the container networking state it was only enabled for aws and maas. Could the revision be incomplete? Does something else need to change for aws? | 19:01 |
| natefinch | anyway, I gotta run some errands, I'll check back in later to see if this has blown up or not | 19:01 |
| natefinch | katco: I'll try to get you an email with info ASAP, but it might be late tonight. | 19:02 |
| natefinch | (info about team lead stuff) | 19:02 |
| katco | sinzui: can you explain the lxc + aws connection to me? | 19:03 |
| sinzui | katco, bundle is deploying to two containers. | 19:03 |
| katco | sinzui: ah ok, so it's deploying containers to an aws host? | 19:04 |
| sinzui | yes. Two app servers in a containers, and haproxy directly on the host to round-robin the work load | 19:05 |
| katco | sinzui: ah ok. thanks, that's the information i was missing | 19:05 |
| sinzui | katco, let me update the bug with the command line. I used any of us can run it. It isn't anything like the awkward CI tests | 19:05 |
| katco | sinzui: awesome | 19:06 |
| katco | sinzui: so to echo this back to make sure i understand | 19:06 |
| katco | sinzui: 1.23-b4 works? where was the commit that breaks things? trunk? | 19:07 |
| sinzui | katco, the dooferlad's commit is the only change from 1.23-beta4 released. We used cherylj's commit | 19:08 |
| katco | sinzui: ah so it was to 1.23-beta4, it just hasn't been packaged (released) yet? | 19:09 |
| sinzui | katco, it was released 20 minutes ago. | 19:09 |
| katco | sinzui: so i'm confused (sorry)... you said 1.23-beta4 is working, but 1.23-beta4 includes dooferlad's commit, which is not working? | 19:11 |
| sinzui | no | 19:11 |
| sinzui | katco, we selected cherylj's as the official 1.23-beta4 because tip was broken. | 19:12 |
| katco | ah ok | 19:13 |
| katco | i understand now, thanks | 19:13 |
| katco | and that commit repeatably works whereas dooferlad's commit repeatably fails | 19:13 |
| katco | sinzui: is that https://github.com/juju/juju/commit/7e7bc9d3ad436cf25fa725f54334527cea9cb938 ? | 19:14 |
| sinzui | katco, https://bugs.launchpad.net/juju-core/+bug/1441319/comments/11 | 19:14 |
| mup | Bug #1441319: intermittent: failed to retrieve the template to clone: template container juju-trusty-lxc-template did not stop <deployer> <lxc> <oil> <regression> <juju-core:Triaged by cox-katherine-e> <juju-core 1.23:Triaged> <https://launchpad.net/bugs/1441319> | 19:14 |
| sinzui | yes, that is the commit that breaks things. | 19:15 |
| katco | sinzui: well at first glance, it certainly looks to be directly related... | 19:15 |
| sinzui | katco, Since the release notes say only maas and aws were supported by SNAT, maybe another change was made that also needs to be reverted | 19:16 |
| lazyPower | Attention core developers - juju actions are friggin sweet. Thank you for this feature. that is all | 19:17 |
| katco | lazyPower: hey you are pretty awesome. | 19:17 |
| lazyPower | :D | 19:17 |
| lazyPower | I've been prototyping actions for a couple days now. i want to go through and hyperbole all the charms with actions now | 19:18 |
| katco | lazyPower: i want to work on a charm with you at the sprint | 19:18 |
| lazyPower | actions ALL THE THINGS | 19:18 |
| katco | lol | 19:18 |
| lazyPower | katco: i wont be at n.burg :( | 19:18 |
| katco | lazyPower: oh what the boo | 19:18 |
| lazyPower | only a handfull of eco peeps wil be there, we got split up | 19:18 |
| lazyPower | we're now under d. westervelt in the big division of juju teams | 19:18 |
| katco | yeah heard about that | 19:19 |
| katco | didn't realize you wouldn't be at the sprints though... for good? | 19:19 |
| lazyPower | hopefully not | 19:19 |
| lazyPower | but we'll see what the future brings | 19:19 |
| lazyPower | marco, cory and antonio will be there however | 19:19 |
| katco | cool | 19:19 |
| lazyPower | also - i'm *always* up for a review/pairing session | 19:20 |
| lazyPower | just hit me with a repo link and a timeblock and i can make time to pair | 19:20 |
| katco | cool | 19:20 |
| katco | wish i had more time to actually use juju and not just work on it :p | 19:20 |
| katco | just have a small installation on my home network | 19:20 |
| lazyPower | i've got coutn it - 8 environments running now | 19:20 |
| lazyPower | that i manage/assist | 19:21 |
| katco | wow | 19:21 |
| katco | that's pretty cool | 19:21 |
| lazyPower | :D | 19:21 |
| lazyPower | getting easier every day | 19:21 |
| katco | my cousin works at a zoo which is understaffed | 19:21 |
| lazyPower | keep landing great features and i'll keep spinning up the envs | 19:21 |
| katco | and they need to stand up a bunch of standard windows things | 19:21 |
| katco | pinged him to use juju | 19:21 |
| lazyPower | oh man | 19:21 |
| lazyPower | our windows support without maas/openstack is non-existant today | 19:21 |
| katco | so bring them to maas! ;) | 19:22 |
| lazyPower | +1 for that | 19:22 |
| katco | sinzui: so this kind of blows any theory that the apparmor update was at all responsible? | 19:26 |
| katco | sinzui: since cherylj's commit works fine | 19:26 |
| sinzui | katco, yes, I didn't see any issue with the aws mirrors to imply they are different from other mirrors too | 19:27 |
| katco | sinzui: ok. well, we'll have to ping alexisb for her opinion on what to do. i'm assuming this blocks v1.23 for the time being? | 19:31 |
| sinzui | katco, this is very awkward, because dooferlad was reverting to unblock. so we cannot revert his change. We need to find another change that was made related to the SNAT | 19:37 |
| katco | sinzui: do you know if anyone else was involved with that code? | 19:40 |
| sinzui | I don't. I can only think to use annotate and log to find other parts of the code that were changed for snat. | 19:41 |
| katco | sinzui: looks like possibly dimiter | 19:45 |
| alexisb | katco, whats up? | 19:59 |
| katco | alexisb: well, curtis has arguably proved that we have a commit in juju that is responsible for the failures | 20:00 |
| alexisb | lazyPower, make sure to thank jw4 for actions | 20:00 |
| katco | alexisb: it's made complicated by the fact that the commit was backing another commit out to fix another issue | 20:00 |
| katco | alexisb: we probably need dimiter+team to make further changes to fix tip | 20:01 |
| katco | 1.23 tip | 20:01 |
| alexisb | well that means we have to wait tell monday for those fixes | 20:03 |
| katco | alexisb: that is the issue we needed your input on | 20:03 |
| katco | alexisb: what does that mean for vivid? | 20:03 |
| alexisb | it means we release vivid with a broken juju | 20:04 |
| alexisb | so what "fix" was reverted that broke aws and why did we revert it? | 20:05 |
| katco | sorry had to blow my nose >.< | 20:06 |
| katco | sinzui: do you know the answer to that question? | 20:06 |
| alexisb | the issue is that we have a known issue that affects oil and any openstack bundle really, and the current 1.23 blessed version does not have a fix for that issue | 20:07 |
| katco | alexisb: here's the suspect commit: https://github.com/juju/juju/commit/7e7bc9d3ad436cf25fa725f54334527cea9cb938 | 20:07 |
| alexisb | which means the first 1.23.0 we are targeted to get released to vivid will break many of our stakeholders | 20:08 |
| alexisb | but if we dont release then juju will just not work on vivid | 20:08 |
| katco | alexisb: well that's not a good decision to have to make. | 20:09 |
| alexisb | katco, so that commit is actually a fix for the issue I am referring to | 20:09 |
| alexisb | I take it that is the one that is breaking aws | 20:09 |
| katco | alexisb: yep. | 20:09 |
| sinzui | alexisb, the bug doesn't really say we reverted the container network changes. It says we made a small change that address stakeholder conerns. the change just is not enough to restore AWS containers. maas containers are fixed | 20:09 |
| katco | alexisb: can oil work around the bug by manually tweaking the nat tables on containers? | 20:10 |
| alexisb | yeah I actually dont think it is reverting anything | 20:10 |
| mgz | hmm, who reported the aws container issues? | 20:10 |
| sinzui | alexisb, this is difficult. before the change this morning some charms were broken, now all aws containers are broken | 20:10 |
| alexisb | mgz, you guys did :) | 20:10 |
| alexisb | katco, I dont have answer to your question jamespage would have an answer | 20:11 |
| alexisb | sinzui, yeah it sucks :) | 20:11 |
| sinzui | mgz, https://bugs.launchpad.net/juju-core/+bug/1441319/comments/9 and beyond | 20:12 |
| mup | Bug #1441319: intermittent: failed to retrieve the template to clone: template container juju-trusty-lxc-template did not stop <deployer> <lxc> <oil> <regression> <juju-core:Triaged by cox-katherine-e> <juju-core 1.23:Triaged> <https://launchpad.net/bugs/1441319> | 20:12 |
| alexisb | so if we know that maas is fixed, is there a way to trace the aws code path and figure out what is different that would affect the issue | 20:12 |
| alexisb | katco, question for you ^^^ | 20:13 |
| katco | alexisb: it's possible, but not likely in the time we have with what resources are left | 20:13 |
| alexisb | katco, that is totally fair | 20:14 |
| alexisb | so the next question is, is it more important for the first release to have oil working or aws containers working? | 20:15 |
| alexisb | and katco I need to follow-up re your question on work around | 20:15 |
| mgz | so... I think containers on aws matters a fair bit for our testing, but I don't know of any customers deploying on aws | 20:16 |
| * alexisb goes to jason hobbs | 20:16 | |
| mgz | as in, big paying ones, lots of people use it | 20:16 |
| katco | alexisb: mgz: yeah, i'm thinking OIL is probably more important to get right out of the gate? | 20:18 |
| alexisb | yeah I am thinking oil working is going to be the priority | 20:18 |
| alexisb | katco, you beat me to it | 20:18 |
| katco | will oil be utilizing vivid right away? | 20:18 |
| mgz | almost certainly not | 20:18 |
| sinzui | katco, not likely | 20:18 |
| katco | hm | 20:19 |
| katco | so the "crumple zones" we have are: workarounds, and adoption rate | 20:19 |
| alexisb | but they do use the latest stable of juju | 20:19 |
| katco | if we can understand those better, we could probably make a better decision | 20:19 |
| sinzui | and they want leader elections | 20:19 |
| katco | yeah | 20:19 |
| katco | well, i guess i vote oil then. | 20:20 |
| alexisb | yeah so katco we do have a pretty good understanding of those, oil is the right choice | 20:20 |
| alexisb | it just sucks | 20:20 |
| alexisb | given there is an obvious failure we can see in our testing | 20:20 |
| katco | yeah =/ | 20:20 |
| alexisb | so sinzui .... | 20:21 |
| katco | another lesson learned: releases should be "done" well before sprints and everyone takes off | 20:21 |
| alexisb | lol | 20:21 |
| sinzui | alexisb, there are now 3 regressions are in https://bugs.launchpad.net/juju-core/+milestone/1.23.0 | 20:21 |
| alexisb | well we had planned for 3/27 but you know ho that goes | 20:21 |
| katco | lol yeah | 20:21 |
| alexisb | sinzui, you are referring to all the ones marked critical | 20:24 |
| sinzui | yes. | 20:25 |
| alexisb | so lp 1439535 just came to us today | 20:25 |
| alexisb | we think that 811 is fixed | 20:25 |
| sinzui | alexisb, and I think we need me to split one of them. because while the errors CI sees are the same as oil in one of the bugs, we know they were using a older juju | 20:25 |
| alexisb | and will not be able to verify thta without a release | 20:25 |
| sinzui | alexisb, we could say it is fixed, except the same charms listed in the bug are still broken on aws | 20:26 |
| alexisb | sinzui, we should open a new bug, as what I really want to know is if oil is fixed for that bug | 20:27 |
| alexisb | if it is then we need to figure out the path for aws | 20:27 |
| alexisb | if it is not then we need to rethink the fix altogether | 20:27 |
| sinzui | okay, I will shuffle the bugs | 20:27 |
| alexisb | sinzui, thank you for all you work helping us sort all these issues out | 20:30 |
| katco | sinzui: yeah curtis, you are amazing | 20:31 |
| davecheney | katco: anything customer facing will be on the LTS releases as that is all we well support on an all that our support wing will support customers using | 20:31 |
| davecheney | we don't have charms for the non LTS releases either | 20:31 |
| sinzui | alexisb, katco, My son doesn't think so. I forgot to pick him up from school 45 minutes ago. I suck | 20:31 |
| katco | sinzui: oh no :( | 20:32 |
| katco | davecheney: ah ok ty for the added info | 20:32 |
| davecheney | katco: and recently I found out that CTS doen't even start recommending the new LTS version for 4-6 months after release | 20:34 |
| alexisb | katco, davecheney is correct, however we do have a special circumstance in the case of 1.23 because the openstack team next released set of charms will leverage Leader Elections | 20:34 |
| davecheney | the first "stabilty release" or some other euphemism, stragenly coinciding with the point release | 20:34 |
| alexisb | so we will need 1.23 for that release at the end of the month | 20:34 |
| davecheney | alexisb: what does that have to do with V ? | 20:34 |
| katco | davecheney: i am familiar with that pattern from the MS world. maybe it spilled over somehow | 20:35 |
| alexisb | davecheney, point taken, actually nothing except the release is aligned | 20:35 |
| davecheney | alexisb: it's probably because to get 1.23 backported we always have to land a release in the current version | 20:37 |
| davecheney | that's the way backports work aparently | 20:37 |
| davecheney | if you'd like to know more | 20:37 |
| davecheney | i'll be crying into my beer all next week about this | 20:37 |
| davecheney | (try the fish) | 20:37 |
| katco | davecheney: lol, i'll need to pick your brain to understand the situation better | 20:38 |
| katco | davecheney: if you don't mind some annoying questions :) | 20:39 |
| davecheney | katco: oh it's a turbulet tail | 20:40 |
| katco | haha | 20:41 |
| davecheney | full of technical intriegue and skullduggery | 20:41 |
| davecheney | turbulet -- what kind of a word is that | 20:41 |
| alexisb | davecheney, lol | 20:41 |
| alexisb | davecheney, I like having you in this timezone | 20:41 |
| davecheney | how do I even adjective | 20:41 |
| katco | davecheney: i think you have a novel in you ;) | 20:41 |
| davecheney | it's weird working in this timezone | 20:41 |
| davecheney | i'm use to having the west coast available all morning | 20:42 |
| davecheney | right now, they don't come online til I get back to the hotel | 20:42 |
| mgz | katco: I am spamming reviewboard with licence header fixes for various juju subrepos at various versions of the branches - if you get a mo can you rubber stamp them? | 20:47 |
| katco | mgz: sure | 20:47 |
| mgz | cmd and testing*2 are there, charm*2 should be there shortly | 20:48 |
| mgz | then I can do the dep update and juju*2 | 20:48 |
| mgz | katco: hm, actually charm is not under reviewboard, those are on https://github.com/juju/charm/pulls only | 20:49 |
| mup | Bug #1442801 was opened: aws containers are broken in 1.23 <ci> <deployer> <ec2-provider> <lxc> <regression> <juju-core:Triaged> <juju-core 1.23:Triaged by cox-katherine-e> <https://launchpad.net/bugs/1442801> | 20:50 |
| katco | mgz: k tal | 20:50 |
| katco | mgz: i think i got all of them | 20:52 |
| mgz | katco: thanks! | 20:53 |
| katco | np | 20:53 |
| mgz | wtf is date 2015-03-30T-9:15:59Z in dependencies.tsv about... | 21:10 |
| mgz | I guess I'll just change that line as well | 21:10 |
| katco | mgz: i don't know what that's used for, so i always just change it | 21:12 |
| mgz | urk, there are non-trivial charm.v4 changes it seems | 21:12 |
| mgz | hm nope, git is just weird, everything is fine | 21:14 |
| katco | time to switch to my laptop to make sure everything is working properly | 21:14 |
| mgz | katco: last bit http://reviews.vapour.ws/r/1417 | 21:26 |
| mgz | I may also do trunk while I;m here but less urgency on that | 21:26 |
| katco | mgz: kathump | 21:27 |
| katco | (sound of a rubber stamp) | 21:27 |
| mgz | :D | 21:28 |
| alexisb | katco, did you assign yourself to lp1442801 | 21:32 |
| katco | alexisb: no | 21:34 |
| alexisb | katco, ok, I am going to go ahead and assign it to james | 21:34 |
| katco | alexisb: ok | 21:34 |
| mgz | if anyone has a mo, licence header changes for trunk too, http://reviews.vapour.ws/r/1418 | 21:59 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!