[00:20] <waigani> wallyworld: I've got a bug fix to land. Shall I target 1.24 first and after shipit land it in master?
[00:21] <wallyworld> waigani: which bug?
[00:21] <waigani> wallyworld: https://bugs.launchpad.net/juju-core/1.24/+bug/1384549
[00:21] <mup> Bug #1384549: Running Juju ensure-availability twice in a row adds extra machines <canonical-bootstack> <canonical-is> <ha> <maas-provider> <juju-core:Triaged> <juju-core 1.24:In Progress by waigani> <https://launchpad.net/bugs/1384549>
[00:21] <wallyworld> waigani: 1.24 please. master is blocked. we will look to merge 1.24 into master and get all the fixes in one go
[00:22] <waigani> wallyworld: okay, sounds like a plan - thanks
[00:22] <wallyworld> np, great that you have a fix for that bug
[00:23] <waigani> wallyworld: yeah, well see what you think...
[00:23] <waigani> ah, have to branch 1.24, I was working against master
[00:35] <waigani> thumper:  HA bug fix: http://reviews.vapour.ws/r/1570
[00:59] <wwitzel3> waigani: did you ever get your virtual maas setup?
[01:00] <waigani> wwitzel3: yes I did!
[01:00] <waigani> wwitzel3: then I read a comment from nate that the bug I was working on didn't need maas to be reproduced :/
[01:01] <waigani> wwitzel3: but I now have maas setup on my machine for future debugging
[01:06] <mup> Bug #1451626 was opened: Erroneous Juju user data on Windows for Juju version 1.23 <1.23> <juju> <juju-core:New> <https://launchpad.net/bugs/1451626>
[01:37] <axw> wallyworld: hey, sorry I missed the standup. kids kept waking up last night, so I slept in
[01:37] <wallyworld> axw: np. hope everything's ok
[01:38] <wallyworld> i've been trying to fix my farking internet
[01:38] <axw> wallyworld: yeah, just a bit cold and they keep rolling around and kicking their covers off...
[01:38] <wallyworld> got a new modem
[01:38] <axw> oh, finally :)
[01:38] <wallyworld> well, i didn't know what the issue was , so i'm trying swapping out components :-)
[01:39] <wallyworld> axw: we'll be releasing 1.24 alpha 1 tomorrow, and then will look to merge 1.24 into master to bulk forward port the fixes
[01:41] <anastasiamac> axw: o/
[01:41] <axw> anastasiamac: howdy
[01:41] <perrito666> wallyworld: same question as waigani
[01:41] <perrito666> I have the patch for the debug hooks issue
[01:41] <wallyworld> perrito666: great, merge into 1.24 first please
[01:42] <wallyworld> perrito666: but do it tomorrow as it's past your eod
[01:42] <anastasiamac> axw: m good :D i have a branch for u to review when u settle :D
[01:42] <axw> wallyworld: seems there's some issues with charm.v5? mgz proposed a revert of one of my branches which updates dependencies.tsv
[01:42] <anastasiamac> axw: storage stuff is beta than coffee, u know :)
[01:43] <perrito666> wallyworld: k just wondering if it was required before the cut
[01:43] <axw> anastasiamac: sure, I'd like to try and prevent a revert of my work first though
[01:43] <wallyworld> nah, that bug has been there for a while
[01:43] <anastasiamac> axw: sounds good
[01:44] <anastasiamac> axw: reverts r always painfull and 1.24 takes priority over my branch (against master) anyway
[01:44] <anastasiamac> axw: i looked at the revert and it looked like the commit u added for charm.v5 is more recent?..
[01:44] <perrito666> wallyworld: http://reviews.vapour.ws/r/1571/
[01:44] <perrito666> cheers people
[01:45] <wallyworld> perrito666: thanks, will look
[01:45] <axw> anastasiamac: commits were added to v6-unstable, and dependencies.tsv apparently had that rev listed. so it says v5, but the commit wasn't on even in that branch
[01:46] <anastasiamac> axw: what a mess :(
[01:46] <axw> indeed
[01:50] <wallyworld> axw: perrito666's change looks ok but i'd like a 2nd opinion, so after your charm.v5 adventures, would be great if you could look. no rush as the work will be landed tomorrow anyway
[01:53] <wallyworld> menn0: with that upgrade bug, i really think juju should by default be changed to upgarde to latest stable. this whole only 1 version at a time thing is a relic of when we didn't do upgrades very well
[01:54] <menn0> wallyworld: I agree
[01:55] <wallyworld> i think there's a 1.24 bug related to that
[01:55] <wallyworld> or at least a similar issue
[01:55] <menn0> wallyworld: related to what?
[01:55] <axw> wallyworld: https://github.com/juju/charm/pull/126  <- PTAL
[01:56] <wallyworld> an upgrade selected a surprising version
[01:56] <menn0> wallyworld: right
[01:56] <wallyworld> menn0: so doing upgrades right will sort of fix that implicitly
[01:56] <wallyworld> axw: looking
[01:56] <menn0> wallyworld: if we are going to support big upgrade jumps then we need to socialise that and have a CI test that upgrades from 1.18 to current stable
[01:57] <wallyworld> yes
[01:57] <menn0> wallyworld: I think i've found yet another upgrade bug going from 1.18 to 1.23
[01:57] <menn0> wallyworld: just figuring it out now
[01:57] <wallyworld> best find them now before we change behaviour :-)
[01:57] <wallyworld> axw: how was the wrong rev added to dependencies.tsv?
[01:57] <axw> wallyworld: fuck knows
[01:58] <axw> someone was lazy
[01:58]  * axw shrugs
[01:59] <wallyworld> axw: i added $$merge$$, can't recall if there's a bot
[01:59] <wallyworld> will merge manually if needed
[01:59] <axw> wallyworld: thanks, I think I can merge
[01:59] <wallyworld> tests all pass right?
[01:59] <wallyworld> ok
[01:59] <axw> what, run tests? I suppose I should, but it's pretty trivial
[02:00] <axw> all tests pass :)
[02:02] <wallyworld> \o/
[02:04] <wallyworld> axw: so when you proposed the juju core hooks change and updated the charm.v5 revision, did you just grab the tip revision from master?
[02:05] <axw> wallyworld: no. I branched off v5 and added my commit, then updated juju to use that. someone else had previously pulled a commit from v6-unstable
[02:05] <axw> so when I made my change, the fixes on v6-unstable were lost
[02:07] <wallyworld> ah right
[02:15] <menn0> wallyworld: ok, definitely another upgrade problem
[02:15] <wallyworld> axw: looks like it was a cloud sigma change - they use a stale version of v5
[02:16] <axw> wallyworld: don't understand
[02:17] <wallyworld> they commited to dependencies.tsv and updated it with a stale charm.v5 revision ie one that was about a week old
[02:17] <axw> wallyworld: I was trying to avoid laying blame, but the problem occurred on this commit https://github.com/juju/juju/commit/73b4f331085fabd5bef5188e5af193118ec573fb
[02:18] <axw> wallyworld: note the change in dependencies.tsv
[02:18] <axw> the commit changed to does not exist in v5
[02:18] <wallyworld> not so much laying blame but understanding the cause so we can ensure it doesnt happen again
[02:19] <axw> wallyworld: I replied to mgz's email, with a note on what to avoid doing
[02:19] <axw> and explained what happened
[02:19] <wallyworld> rightio, need to check mail
[02:31] <axw> waigani: can you please hold off on retrying that build for a minute
[02:31] <axw> your change doesn't look quite right
[02:31] <waigani> axw: yeah for sure
[02:31] <waigani> axw: what's your concern?
[02:32] <axw> waigani: I'm just reading over the code and thinking, pretty sure what was there was doing the right thing... need to go over it some more
[02:33] <waigani> axw: servers that were not available and wanted vote where getting demoted
[02:33] <waigani> axw: that's the crux of the bug
[02:34] <axw> waigani: that's exactly the specified behaviour of ensure-availability
[02:34] <waigani> axw: with the fix servers that not available, have vote and want vote now get demoted
[02:35] <waigani> axw: "adding vote" == want vote + don't have vote
[02:35] <waigani> axw: we shouldn't be demoting servers that are getting a vote added to them
[02:36] <axw> waigani: why not?
[02:36] <axw> the machine is unavailable, so why would we not demote it?
[02:36] <axw> that prevents another machine from becoming a state server
[02:40] <axw> waigani: I suspect what's happened in the bug scenario is that the machine's pinger hasn't started yet
[02:40] <axw> so it looks like the machine isn't available, but it kinda-sorta really is
[02:40] <waigani> axw: I've manually tested this on aws, I ran ensure-availability, turned off an instance, reran ensure-availability, dead server got demoted an new one got added
[02:41] <axw> waigani: yes... that is what is meant to happen
[02:41] <waigani> axw: and it still does
[02:44] <axw> waigani: how long did you wait? if you wait long enough, that state server should lose its "has vote" status
[02:45] <axw> waigani: i.e. because the peer grouper has noticed that the mongo has stopped responding
[02:45] <axw> waigani: and once that happens, the machine will be "maintained" forever
[02:47] <waigani> axw: I've just destroyed the environment, but let me do the same again and we can poke around
[02:48] <axw> waigani: thanks. I think you should be able to see from the peergrouper logs which machines have/haven't got a vote
[02:48] <axw> oh you already said it's in the status, never mind
[02:49] <waigani> axw: yep. this is from last run: http://pastebin.ubuntu.com/10987944/
[02:50] <waigani> axw: I'm just bootstrapping again now, I'll run HA, turn off an instance, run HA again. Then I'll ping you.
[02:50] <axw> waigani: thanks
[03:00] <axw> wallyworld: can you please close https://github.com/juju/juju/pull/2209?
[03:00] <wallyworld> axw: yep, sure
[03:01] <axw> wallyworld: ehm, hmm. on 1.24, I get "2015-05-05 03:01:02 WARNING juju.cmd.envcmd environmentcommand.go:253 invalid JUJU_CLI_VERSION value:"
[03:01] <axw> I guess something is not checking for ""
[03:01] <axw> I guess something is not checking for ""
[03:01] <axw> oops
[03:02] <wallyworld> axw: hmmm, was supposed to check
[03:02] <wallyworld> i'll double check and fix
[03:03] <wallyworld> axw: ah, the warning is printed unnecessarily
[03:03] <wallyworld> ffs
[03:07] <waigani> axw: here is where the env is up to: http://pastebin.ubuntu.com/10987993/
[03:07] <waigani> axw: stopped machine is being demoted, new machine is getting the vote
[03:07] <axw> waigani: it'll do that as long as the machine "has-vote"
[03:07] <axw> waigani: I think it should lose the vote after a long enough time of mongo not being running
[03:10] <waigani> axw: are you saying, if I stop an instance and don't run HA straight away, but wait, that instance will loose "has-vote" and then the logic is screwed?
[03:10] <axw> waigani: right (assuming has-vote does get lost)
[03:11] <waigani> axw: how about l stop another instance and let it sit for the afternoon - checking in now and then
[03:12] <axw> waigani: I'd appreciate it. I'm looking at the code now, but do need to get back to other things soon
[03:12] <waigani> axw: I'll also read through the code and see if I can spot anywhere that removes it
[03:12] <axw> waigani: worker/peergrouper is the place to look
[03:12] <waigani> axw: ha, I was just about to ask - thanks
[03:13] <waigani> axw: on an interesting aside, I used destroy-environment --force on last env, but it didn't terminate the stopped instance - it's still there in aws
[03:14] <axw> waigani: hrm :/
[03:15] <waigani> axw: yeah. Other instances are marked as terminated. But that one is still there as "stopped".
[03:15] <axw> I'm guessing that we're filtering for only running instances
[03:18] <waigani> axw: a reasonable guess.
[03:22] <axw> waigani: okay, here's a scenario where it would (I think) occur: ensure-availability, then immediately stop jujud and mongod on the new state server
[03:22] <axw> i.e. regardless of has-vote being lost
[03:22] <axw> (I think has-vote will never be lost unless another eligible state server comes along)
[03:24] <waigani> axw: I currently have two stopped state server machines: one with vote, one without. How about I spin up the one without?
[03:24] <waigani> axw: then we'd have another eligible state server right?
[03:24] <axw> waigani: sure. I'll try this in my own env too
[03:36] <waigani> axw: is this the scenario we need: http://pastebin.ubuntu.com/10988070/
[03:37] <waigani> axw: machine 2 is started and has no vote. machine 3 is stopped and has vote
[03:38] <axw> waigani: that's *a* scenario, yeah. what happens with ensure-availability once all of the agents are running/available?
[03:40] <waigani> axw: yikes, I just noticed that 0 and 1 are down - they shouldn't be?
[03:41] <waigani> axw: okay they are up now, I don't understand why they went down?
[03:41] <axw> waigani: so anyway, I'm going to have to NOT LGTM this. with your change, if a state server machine never gets provisioned, then it'll never get replaced
[03:42] <wallyworld> waigani: there's a small window as machines and units start where they are marked as down - the default presence value is 0
[03:43] <wallyworld> this is all messed up but that's how it was written
[03:43] <waigani> axw: yeah, but 0 and 1  were started, I just stopped and started 2. Anyway, focusing on the bug at hand
[03:44] <axw> waigani: sorry, I don't know why those agents are going up and down
[03:44] <waigani> axw: so they are up now. the stopped server still has no vote.
[03:44] <axw> waigani: I don't think there is a simple fix here. We need to improve agent presence, as wallyworld mentions
[03:46] <wallyworld> s/improve/burn in hell
[03:46] <wallyworld> it is horrid
[03:46] <waigani> hehe
[03:46] <thumper> wallyworld: not landing stuff on a blocked branch are you?
[03:46] <waigani> axw: how could a state server get the vote without being provisioned?
[03:47] <wallyworld> thumper: yes i am
[03:47] <wallyworld> thumper: i talked to curtis a few days ago and we agreed onecould be prmatic
[03:47] <wallyworld> and this ia a small fix to a bug in a previous landing
[03:48] <axw> waigani: it doesn't ever need to have gotten a vote.
[03:48] <waigani> axw: sorry, got my logic backwards - it needs the vote to get demoted
[03:48] <axw> right
[03:48] <wallyworld> axw: once you finish your conversation, do you have time for a quick chat about storage with me and anastasia?
[03:49] <waigani> axw: so we can't tell the difference between a state server that is un-provisioned because of an error and one that is in the process of being provisioned
[03:49] <axw> waigani: or one where provisioning failed but can be retried, or a provisioned one where the agents's dead, or mongo's dead, or ...
[03:50] <axw> wallyworld: sure, tanzanite hangout?
[03:50] <wallyworld> yep
[03:50] <wallyworld> anastasiamac: ^^^
[03:50] <anastasiamac> wallyworld: m here :D
[03:52] <anastasiamac> wallyworld: can u hear us?
[03:52] <wallyworld> no, hangout is hanging
[03:52] <axw> ironic
[03:53] <wallyworld> ffs
[03:59] <waigani> axw: what if we use a wait and retry policy?
[03:59] <waigani> axw: after n retries and x time, it gets demoted?
[04:05] <axw> waigani: IMO that should be part of the "availability"
[04:05] <axw> which is more general than HA
[04:05] <axw> although HA availability *should* consider more than just agent availability
[04:09] <waigani> axw: but it's not (or the window is not big enough). because we are getting servers that are not available because they are in the process of being provisioned
[04:09] <axw> I know it's not, I'm saying it should be
[04:09] <axw> it's known that agent availability/presence is not great
[05:11] <menn0> wallyworld or axw: I am seeing a case of incorrect field ordering of docs in mongodb which is causing an assert to fail
[05:12] <wallyworld> oh?
[05:12] <menn0> wallyworld or axw: could be upgrade related. i see it when upgrading to 1.23
[05:12] <wallyworld> example?
[05:12] <menn0> wallyworld or axw: it breaks address setting on machines
[05:13] <menn0> wallyworld: i see it happen fairly often on either the addresses and or machineaddresses fields
[05:13] <menn0> 	"machineaddresses" : [
[05:13] <menn0> 		{
[05:13] <menn0> 			"addresstype" : "ipv4",
[05:13] <menn0> 			"networkscope" : "local-machine",
[05:13] <menn0> 			"value" : "127.0.0.1"
[05:13] <menn0> 		},
[05:13] <menn0> 		{
[05:13] <menn0> 			"value" : "192.168.122.107",
[05:13] <menn0> 			"addresstype" : "ipv4",
[05:13] <menn0> 			"networkscope" : "local-cloud"
[05:14] <menn0> 		},
[05:14] <menn0> 		{
[05:14] <menn0> 			"addresstype" : "ipv6",
[05:14] <menn0> 			"value" : "fe80::5054:ff:fe92:1645"
[05:14] <menn0> 		},
[05:14] <menn0> 		{
[05:14] <menn0> 			"value" : "fe80::c8a9:ceff:fef4:18fb",
[05:14] <menn0> 			"addresstype" : "ipv6"
[05:14] <menn0> 		}
[05:14] <menn0> 	],
[05:14] <menn0> note how the field ordering isn't consistent
[05:14] <menn0> mongo cares about field ordering when doing comparisons
[05:14] <menn0> so the assertion fails
[05:14] <axw> there have been issues in the past where something would sort addresses when it shouldn't; order of addresses in state is meant to be maintained
[05:14] <menn0> axw: it's not the order of the docs that's the issue. that seems to be correct.
[05:14] <menn0> axw: it's the order of the fields inside the docs
[05:15] <wallyworld> hmmm
[05:15] <wallyworld> menn0: perhaps someone changed the struct definition
[05:15] <menn0> axw: in this case the 2nd and 4th docs matches the struct definition. the others don't
[05:15] <axw> hrm :/
[05:15] <menn0> wallyworld: i've checked that and it doesn't look like it
[05:16] <wallyworld> wh should mongo care about field ordering ffs?
[05:16] <wallyworld> what sort of retarted "db" it that
[05:16] <menn0> i'm wondering if it's related to the dict randomisation in Go 1.3
[05:16] <menn0> i'm on vivid today
[05:16] <menn0> but mgo/bson seems to do everything right
[05:16] <menn0> i might stick some debugging stuff into the transaction layer and see what I can find
[05:17] <wallyworld> even with map odering, surely a map comparison doesn't care
[05:17] <menn0> wallyworld: mgo/txn actually makes a query to MongoDB to do the comparison
[05:17] <menn0> wallyworld: so it's being done at the MongoDB side, so field ordering does matter.
[05:17] <wallyworld> so are you saying if mgo gets a map{1:2, 3:4} that will be considered diferent to {3:4, 1:2}
[05:18] <wallyworld> if so, wtf
[05:18] <menn0> wallyworld: if you replace the word "map" with "document" then yes
[05:18] <wallyworld> why?
[05:18] <wallyworld> that just makes no sense to me
[05:18] <menn0> wallyworld: beats me but that's what it does
[05:18] <wallyworld> ffs
[05:18] <wallyworld> i hate mongo even more now
[05:19] <menn0> http://devblog.me/wtf-mongo
[05:19] <menn0> wallyworld: ^^^ first item
[05:19] <wallyworld> great link title
[05:21] <wallyworld> menn0: so the link offers a work around
[05:21] <wallyworld> alter the find syntax slightly
[05:21] <menn0> wallyworld: that doesn't really work for this code.
[05:21] <davecheney> mwhudson: https://go-review.googlesource.com/#/c/9526
[05:21] <menn0> wallyworld: it's asserting that a long list of network address structs matches
[05:21] <davecheney> if you have time
[05:21] <davecheney> a bit messy
[05:22] <davecheney> but it gets the job done
[05:22] <menn0> wallyworld: I guess you *could* translate it
[05:22] <wallyworld> menn0: we *may* have to :-(
[05:23] <menn0> wallyworld: but first I'd like to understand what's going on because as far as I can see we only write using our structs which should preserve order (i've been trying to break mgo/bson and so far it's doing the right thing)
[05:23] <wallyworld> menn0: are you sure the struct field order is that same for 1.18 vs 1.23 etc?
[05:24] <menn0> wallyworld: fairly sure... I've been comparing both codes bases
[05:24] <menn0> wallyworld: i'll do some more digging
[05:24] <menn0> wallyworld: this is pretty frustrating
[05:24] <wallyworld> indeed :-(
[05:41] <menn0> wallyworld: well fuck, I see the problem and this is big
[05:41] <wallyworld> oh no
[05:41] <menn0> wallyworld: just about all the env UUID migrations use one helper function
[05:42] <menn0> wallyworld: and that helper loads the doc to be migration into a bson.M, modifies the _id, adds an env-uuid field and writes it back out
[05:42] <menn0> wallyworld: b/c it's loading it into a bson.M which is just a map (and in this case a map of maps)
[05:42] <menn0> wallyworld: and b/c it's Go 1.3
[05:43] <menn0> wallyworld: the field orderings get messed up
[05:43] <wallyworld> oh dear
[05:43] <wallyworld> needs to be into a slice
[05:43] <menn0> wallyworld: this will be a problem where-ever we have a Go 1.3+ compiled Juju and nested docs or arrays of docs
[05:44] <wallyworld> luckily our releases are all 1.2 at present
[05:44] <menn0> wallyworld: I'm pretty sure that the releases for vivid are being compiled on 1.3
[05:44] <menn0> wallyworld: at least I thought I saw sinzui say that
[05:44] <wallyworld> hmmm, could be
[05:44] <wallyworld> shit
[05:45] <menn0> I can fix the migration helper
[05:45] <wallyworld> luckily i feel most vivid usages of juju are to play with openstack kilo
[05:45] <wallyworld> new environments
[05:46] <wallyworld> but still
[05:46] <menn0> i'll create a new ticket
[05:46] <wallyworld> i think we are fine to ship aplha with this unfixed
[05:47] <wallyworld> but needs fixing for 1.24.0
[05:47] <menn0> wallyworld: yeah, it's still alpha. if you upgrade an important environment to an alpha release then you deserve it.
[05:47] <wallyworld> yup
[05:48] <wallyworld> and we will add in big red letters to release notes
[06:15] <mup> Bug #1451674 was opened: Broken DB field ordering when upgrading to Juju compiled with Go 1.3+ <blocker> <juju-core:New> <juju-core 1.23:New> <juju-core 1.24:New> <https://launchpad.net/bugs/1451674>
[07:42] <mattyw> fwereade, ping?
[08:09] <TheMue> morning o/
[08:14] <axw> wallyworld: when you're free, please take another look at the storage hook order review. it's changed a bit, to fix unit termination
[09:00] <TheMue> hangout, omw
[09:01] <dooferlad> voidspace: hangout!
[09:01] <voidspace> dooferlad: omw
[09:28] <rogpeppe> mgz: hiya
[09:29] <mgz> rogpeppe: hey
[09:29] <rogpeppe> mgz: i've just landed a change to godeps that should make our life easier: https://codereview.appspot.com/230460044/
[09:29] <rogpeppe> mgz: (i wanted your review but you weren't in sight :-) )
[09:30] <mgz> rogpeppe: I saw that one, thanks (I also added a hack in CI to delete stuff, your change is much nicer)
[09:30] <rogpeppe> mgz: this should put paid to all those transitory dependency issues
[09:30] <mgz> rogpeppe: yeah, sorry, but saw it was getting looked at
[09:31] <rogpeppe> mgz: if your change just deleted extraneous dependencies, then that was kinda flawed... :)
[09:31] <mgz> rogpeppe: indeed. relies on the build/test actually catching stuff it's missing but cares about.
[09:32] <rogpeppe> mgz: well, the whole point of the test (and why the build was failing) was to catch places where we have a dependency that's not mentioned in dependencies.tsv, i think
[09:32] <mgz> yup
[09:32] <rogpeppe> mgz: you'd've been better off just removing that check...
[09:32] <rogpeppe> mgz: but anyway, now there is a Better Way :)
[09:32] <mgz> anyway, not using godeps is just much better
[09:33] <rogpeppe> what's the alternative?
[09:33] <rogpeppe> or did you mean "not using go get" ?
[09:33] <mgz> rogpeppe: we also needed an actually correct tarball for releasing, which means not including code we don't declare (and have done debian copyright checking on and such like)
[09:34] <mgz> rogpeppe: I did
[09:34] <mgz> using godeps, not using go get
[09:36] <rogpeppe> mgz: actually deleting the extras was probably fine in fact, 'cos it would cause the build to fail if the dep was actually being used
[09:36] <mgz> that was the idea.
[09:36] <rogpeppe> mgz: BTW I was using godeps -P 20 to fetch dependencies and it seemed to work reliably. i wonder if you might want to experiment with that to speed up turnaround time.
[09:36] <axw> mgz: re https://github.com/juju/juju/pull/2209#issuecomment-99005800 -- the last windows unit test job passed, so I don't think 1.24 is blocked by the issue anymore?
[09:37] <mgz> axw, right, in process of opening both branches, one more forward port and some joyent junk to look at
[09:38] <axw> ok then
[09:41] <mgz> oh, and functional-restricted-network is borked on 1.24 still... did that get a bug filed...
[09:54] <mgz> can I have a re-stamp on eric's fix, http://reviews.vapour.ws/r/1576
[09:56] <jam> mgz: stamped
[09:56] <mgz> ta
[09:59] <natefinch> morning all
[09:59] <mgz> hey nate
[10:01] <mgz> sorry if I came off as pissy in the email, should not be tracking down bugs late at night
[10:02] <natefinch> I thought it was fine.  You're absolutely right that we shouldn't be landing branches that don't fix the blockers.  To be fair, I did land a bugfix on 1.24 that was marked critical... but that one on master I had intended to let just sit there until master was unblocked.
[10:15] <mwhudson> davecheney: looks good to me
[10:19] <mwhudson> davecheney: thanks, btw
[10:48] <perrito666> morning
[11:11] <perrito666> lazyPower: around?
[11:12] <wallyworld> axw: back from soccer, will have dinner and then look
[11:13] <perrito666> wallyworld: your network seems to have settled, what did you do?
[11:13] <wallyworld> perrito666: replaced the @$*@&!^ modem
[11:13] <perrito666> that happens
[11:16]  * perrito666 destroy one cheap AP per year and one modem every two
[11:19]  * perrito666 frowns at bugs that asume one knows things
[11:31] <lazyPower> perrito666: i am
[11:35] <perrito666> lazyPower: https://bugs.launchpad.net/juju-core/+bug/1444861
[11:35] <mup> Bug #1444861: Juju 1.23-beta4 introduces ssh key bug when used w/ DHX <dhx> <ssh> <juju-core:Triaged> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1444861>
[11:36] <perrito666> I would appreciate a bit more detail on reproducing this, I have never used the plugin
[11:36] <perrito666> so what provider where you using, version of the plugin, steps?
[11:38] <mgz> perrito666: I have a plan for that fix, I'll add to the bug
[11:38] <mgz> it's basically an api versioning/breakage issue
[11:40] <perrito666> mgz: tx
[11:41] <perrito666> mgz: do you know what time sinzui starts his day? I am waiting for his cut to merge a minor bug in 1.24
[11:41] <mgz> perrito666: shortly
[11:41] <mgz> perrito666: I'm trying to open both master and 1.24 currently, master should be clear shortly
[11:42] <lazyPower> perrito666: launchpad is rejecting edits to the bug - is adding it as a comment fine?
[11:43] <perrito666> lazyPower: it is
[11:43] <lazyPower> Updated as comment #2
[11:45] <perrito666> tx
[11:58] <wallyworld> axw: reviewed with a test suggestion
[12:06] <perrito666> mgz: should I wait for your input on the bug or go on and try to fix it?
[12:06] <mgz> perrito666: enarly there
[12:20] <mgz> perrito666: commented, yell if anyhting doesn't make sense
[12:21]  * perrito666 yells because life is senseless
[12:21] <mgz> I think you can do a 1.23 fix that unbreaks the api, but refactors enough to be useful for a new api call with a better result struct
[12:22]  * perrito666 just read: make a dirty hack just don t make it look as a dirty hack
[12:22] <mgz> ... I think it's pretty elegant really... :)
[12:23] <mgz> you make runSSHKeyImport do what we want, the just adapt how it's mapped to ErrorResults in ImportKey different
[12:23] <perrito666> I havent read the comment, I was talking about what you justsaid
[12:24] <mgz> :P
[12:39] <axw> mgz: are there any unverified bug fixes on 1.24? can I JFDI a fix for storage?
[12:40] <mgz> axw: just gone through, functional-restricted-network is still unhappy but all blockers done
[12:40] <mgz> one sec and I'll unblock, no forcing needed
[12:40] <axw> mgz: sweet, thanks
[12:41] <mgz> axw: go ahead and $$merge$$
[12:42] <axw> mgz: thanks
[12:42] <mgz> I'll file something about the network test, may not be juju breakage but something needs fixing
[12:50] <mgz> meh, I think 1.24 bootstrapping may actually be doing something wrong on limited networks
[12:54] <axw> mgz: if I cancel the current github-merge-juju job on Jenkins, will it clean up the instance and so on?
[12:54] <axw> missed a test fix while changing a signature
[12:55] <mgz> axw: possibnly not, but we'll catch and clean it up later anyway
[12:55] <perrito666> axw: beware, that might be mine
[12:55] <axw> mgz: ok, thanks
[12:55] <perrito666> so perhaps you still have some room to fix :p
[12:56] <axw> perrito666: it's not, promise
[12:56] <mgz> anh, axw won the race
[12:56] <perrito666> axw: bummer
[12:58] <axw> mgz: :(  where'd that blocker come from
[12:58] <mgz> axw: is terminated
[12:59] <mgz> so yeah, our script does handle manual aborts fine
[12:59] <axw> mgz: cool
[13:00] <mgz> menno filed it, probably doesn't need to block 1.24 or trunk at this point
[13:01] <mgz> gimme a sec
[13:01] <axw> thanks
[13:01] <axw> 1.23 would be fair
[13:02] <sinzui> perrito666, mgz; 'sup
[13:02] <dooferlad> voidspace, TheMue: https://plus.google.com/hangouts/_/canonical.com/maas-juju-net
[13:03] <mgz> sinzui: so, but 1451674 should absolutely block a release on any of those branches, but doesn't actually need to block development on 1.24/trunk right?
[13:03] <mgz> we have bug fixes that won't interfer with that, it's not a new regression
[13:04] <mgz> *bug 1451674
[13:04] <mup> Bug #1451674: Broken DB field ordering when upgrading to Juju compiled with Go 1.3+ <blocker> <golang> <upgrade-juju> <vivid> <juju-core:Triaged> <juju-core 1.23:Triaged> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1451674>
[13:04] <sinzui> mgz: it won't block the alpha release today
[13:04] <mgz> sinzui: so, we do not want to mark it as a blocker, right?
[13:04] <wallyworld> we shouldn't necessarilyt block an alpha release on thsi bug
[13:04] <perrito666> sinzui: just wanted to know what time where you going to cut 1.24.xxxx so I can merge something into 1.24 (a fix)
[13:04] <wallyworld> it's an alpha - so we can tell people not up upgrade using vivide
[13:04] <perrito666> wallyworld:  Y U NO SLEEPING
[13:04] <mgz> 1.24 should be open atm
[13:04] <wallyworld> axw: ^^^^
[13:05] <axw> woop
[13:05] <sinzui> mgz: it is a blocker, because we cannot do a real release with it, it is not critical for 1.24-alpha at this moment
[13:05] <sinzui> sleep gives you cancer
[13:06] <wallyworld> sinzui: so we need to land storage work so we can get 1.24 alpha out
[13:06] <mgz> okay, we need to sort out what we're doing here then, currently blocker prevents landing, which I want for "we have new regressions that need handling before ci can do useful work"
[13:06] <perrito666> sinzui: so does everything else
[13:06] <sinzui> wallyworld, oh, I thought we agreed to release today.
[13:06] <wallyworld> this issue should be resolved before 1.24.0 but not alpha1
[13:06] <mgz> that's not the case for this bug, it's a real upgrade issue, but not a new one from recent landings that needs resolving before we get meaningful ci results
[13:06] <wallyworld> sinzui: we did, hence trying to land thi storage fix which we cant release witout
[13:07] <mgz> I'd prefer we don't jfdi things through, but label bugs to reflect the state of a branch vs release/devel clearly
[13:07] <wallyworld> mgz: CI hasn't failed so far with this bug, so it's not fully testing gor it
[13:07] <wallyworld> mgz: this issue should not block landings for 1.24 alpha 1
[13:07] <mgz> yup, we are not testing vivid state servers or --upload-tools from vivid
[13:08] <sinzui> wallyworld, and you know why, there wasn't a version to test with
[13:08] <wallyworld> we need to be prgatic about blocking landings
[13:08] <mgz> wallyworld: that is what I am saying
[13:08] <sinzui> We can add upgrade testing today for vivid and we can add gce
[13:08] <wallyworld> yes, all good reasons why this lsipped through
[13:09] <mgz> I want you guys to not land trivial stuff when which means I have to read 2700 line diffs to work out which change broke something, but blocking currently doesn't cause that pain
[13:09] <wallyworld> you shouldn't have to do that
[13:09] <sinzui> mgz, to be clear, --upload-tools is just a gateway for developers doing what we officially tell users not to do
[13:09] <wallyworld> that's devel's job to fix our stuff
[13:10] <sinzui> mgz, vivid's compiler is 1.3 and until we released last week, it wasn't possible to test this case
[13:10] <sinzui> wallyworld, all devel release come with an advisory that it doesn't support upgrade, so I remain unconcerned
[13:11] <mgz> sinzui: so, how do we want to mark bugs that we cannot release with, but have already been tracked down and have limited disruption for ci
[13:11] <mgz> I'd have thought "critical" but no "blocker" tag
[13:11] <wallyworld> sinzui: +1, me too, that is my position also
[13:11] <sinzui> mgz, the opposite. it is a blocker that that release team is tracking. it is just high on the alpha
[13:12] <wallyworld> hence the push back for blocking landings
[13:12]  * sinzui already fixes it
[13:12] <mgz> okay.
[13:12] <mgz> so, 1.24 currently unblocked
[13:13] <mgz> there *is* an issue with cloud-utils but I'm trying to get details on that still
[13:13] <sinzui> mgz: I am more concerned about what we don't know. Why does the restricted network test fail. I fear Juju changes something that breaks private clouds
[13:14] <mgz> sinzui: canonistack also fails the same way - which supports that idea
[13:14] <sinzui> mgz, expand the list at http://juju-ci.vapour.ws:8080/view/Juju%20Revisions/job/functional-restricted-network/ and you will see 99% passes become 90^ fails
[13:14] <mgz> am trying to find out more
[13:26] <wallyworld> sinzui: that last storage branch just landed
[13:27] <sinzui> wallyworld, okay. I will watch the next run
[13:27] <wallyworld> i don't think there's anything else blocking a release of alpha 1 as soon as CI passes
[13:36] <rogpeppe1> mgz: i'm just trying to get my head around git cherry-pick to forward-port the changes that have landed in charm.v5. I was a bit confused by 87cf11b8cb2ab830c4ed9c2eab4d633004bb4689 which *looks* like a merge (from the message) but isn't actually. did you by any chance use cherry-pick to create that branch?
[13:36] <rogpeppe1> s/that branch/that commit/
[13:39] <mgz> yeah, charm got super-confused
[13:39] <mgz> because the head of v5 moved
[13:44] <rogpeppe1> mgz: hmm, i hope that wasn't my fault
[13:45] <mgz> rogpeppe1: may have been, but we should be able to fix up somehow
[13:46] <rogpeppe1> mgz: ah! i think i might know what happened
[13:47] <rogpeppe1> mgz: i accidentally pushed the new v6-unstable branch to v5 and quickly reverted it, but i guess i must have force-pushed an earlier version
[13:47] <mgz> rogpeppe1: my theory was v5 got renamed to v6-unstable
[13:47] <mgz> aha, that also sounds possible
[13:48] <rogpeppe1> mgz: probably because i hadn't done a git fetch recently enough
[13:48] <mgz> anyway, we probably want to check nothing else got dropped, axw merged the two I highlighted back into v5
[13:49] <rogpeppe1> mgz: ... and i thought it was unproblematic because i realised my mistake after only 5 seconds...
[13:49] <mgz> git is dangerous:)
[13:55] <rogpeppe1> axw: ping
[13:58] <axw> rogpeppe1: pong
[13:58] <rogpeppe1> axw: i just noticed https://github.com/juju/charm/pull/125/files
[13:59] <rogpeppe1> axw: and thought i should mention that, for future reference, that kind of change is technically API-breaking
[13:59] <axw> rogpeppe1: it was not in any release yet
[13:59] <rogpeppe1> axw: currently we don't have the 'bot guarding API-breaking changes, but we should have...
[13:59] <rogpeppe1> axw: any juju-core release?
[14:00] <axw> rogpeppe1: correct
[14:00] <rogpeppe1> axw: there are quite a few users of the charm package
[14:00] <rogpeppe1> axw: not just juju-core
[14:00] <axw> rogpeppe1: ok. does anything care about storage hooks yet?
[14:00] <rogpeppe1> axw: and the major versioning *should* apply regardless of release status
[14:01] <rogpeppe1> axw: dunno
[14:01] <rogpeppe1> axw: we try to keep to the letter of the rules about API versioning, i guess
[14:01] <perrito666> rogpeppe1: is there a merge bot there?
[14:01] <rogpeppe1> axw: because it's so easy to break things
[14:01] <rogpeppe1> perrito666: not yet
[14:01] <axw> rogpeppe1: ok, will remember that for next time
[14:02] <rogpeppe1> axw: thanks.
[14:02] <mgz> I will happily mergebot charm now I have a godeps that can actually work#
[14:02] <perrito666> that explains why the $$merge$$ in https://github.com/juju/charm/pull/122 never did anything
[14:02] <rogpeppe1> axw: tbh the gopkg.in thing is still in trial - it's good to try to be honest with ourselves
[14:03] <katco> natefinch: stand up
[14:03] <mgz> issue before was it didn't have dependencies.tsv but had branches that din't work against tip of all deps
[14:03] <rogpeppe1> mgz: right. it needs a dependencies.tsv
[14:03] <mgz> now I can at least manufacture one that will work
[14:03] <rogpeppe1> mgz: cool
[14:25] <natefinch> mgz, rogpeppe1: do we have more than one dependencies.tsv in the tree of dependencies that juju uses?
[14:25] <katco> natefinch: not sure if this is related; axw has a pr for the charm.v5 stuff: http://reviews.vapour.ws/r/1575/diff/#
[14:28] <natefinch> mgz_: what should I be doing with https://bugs.launchpad.net/juju-core/+bug/1450919 ?  The windows patch that got backed out, I can fix easily.  But I have no idea how the charm.v5 code has anything to do with windows breakages
[14:28] <mup> Bug #1450919: many window unit tests failures <blocker> <ci> <regression> <windows> <juju-core:Fix Committed by gz> <juju-core 1.24:Fix Released by axwalk> <https://launchpad.net/bugs/1450919>
[14:29] <rogpeppe1> natefinch: in a call
[14:31] <voidspace> dooferlad: sorry I missed the call. a) I forgot it was Tuesday - feels like Monday to me.
[14:31] <voidspace> dooferlad: and b) one my neighbours got burgled (by another of my neighbours - in a normally sleepy village!)
[14:31] <voidspace> dooferlad: and I got inveigled
[14:31] <voidspace> dooferlad: anything I should know about?
[14:31] <dooferlad> voidspace: no, all quiet
[14:32] <perrito666> voidspace: nice negbours :)
[14:32] <voidspace> perrito666: yeah :-/
[14:35] <katco> voidspace: dooferlad: hey just a heads-up. we are cutting 1.24-alpha1 today if you can wrap up any of the bugs you guys are working on
[14:35] <voidspace> katco: thanks
[14:35] <katco> voidspace: np, and wb
[14:35] <voidspace> katco: I don't think I've got anything I can get in 1.24 today
[14:35] <voidspace> katco: o/
[14:36] <dooferlad> katco: thanks. I am in a similar situation to voidspace :-|
[14:36] <natefinch> voidspace: wow, I was sure inveigled must have been a typo.  It's not often people use words in common conversation that I've never even heard of before
[14:36] <katco> dooferlad: understood, and wb as well
[14:36] <perrito666> katco: is it already unblocked?
[14:37] <katco> perrito666: no, wwitzel3 is working on the blocking bug
[14:37] <perrito666> natefinch: really? voidspace does that to me all the time
[14:37] <natefinch> perrito666: I guess he *is* british
[14:37] <katco> perrito666: well, and natefinch is working on the windows blocker
[14:37] <perrito666> but then again, my list of english words must be way shorter than yours
[14:38] <voidspace> natefinch: heh
[14:38] <voidspace> perrito666: I'm sorry...
[14:39] <perrito666> voidspace: dont be, its enriching, most of my english is otherwise based on techincal words only
[14:39] <natefinch> yeah, I always like learning new words
[14:39] <voidspace> me too
[14:39] <voidspace> or three actually I guess...
[14:40] <perrito666> voidspace: although none of the definitions of inveigled I found make sense in your sentence
[14:40] <voidspace> perrito666: get involved by trickery and subterfuge
[14:41] <natefinch> perrito666: he got snookered
[14:41]  * natefinch is probably not helping ;)
[14:41] <voidspace> perrito666: there wasn't really much subterfuge, but it was rather against my will that I got dragged into it
[14:41] <voidspace> hehe
[14:41] <perrito666> natefinch: ... I am not even gonna try....
[14:41] <perrito666> voidspace: ahh I see
[14:41] <voidspace> perrito666: I was inveigled into the situation
[14:45] <voidspace> TheMue: ping
[14:45] <TheMue> voidspace: pong
[14:45] <voidspace> TheMue: bug https://bugs.launchpad.net/juju-core/+bug/1442801
[14:45] <mup> Bug #1442801: aws containers are broken in 1.23 <blocker> <ci> <deployer> <ec2-provider> <lxc> <regression> <juju-core:Fix Released by themue> <juju-core 1.23:Fix Released by dooferlad> <juju-core 1.24:Fix Released by dimitern> <https://launchpad.net/bugs/1442801>
[14:46] <voidspace> TheMue: for juju-core this is marked as assigned to you and fix released
[14:46] <voidspace> TheMue: as far as I *know*, part of the fix is putting addressable containers behind a feature flag
[14:46] <voidspace> TheMue: which hasn't yet landed
[14:46] <voidspace> TheMue: am I incorrect?
[14:46] <voidspace> TheMue: (I have a PR for putting addressable containers behind a feature flag on trunk and I'm wondering if this is the right issue)
[14:47] <voidspace> TheMue: if I am correct, I'll JFDI it as it's a critical bug
[14:47] <TheMue> voidspace: it has been a small fix by dooferlad, ported by dimiter to 1.24 and now by me
[14:47] <voidspace> TheMue: ah
[14:47] <voidspace> TheMue: so it's not the issue I'm thinking of
[14:47] <voidspace> TheMue: thanks
[14:48] <voidspace> TheMue: this was the DNS search domains one then I guess
[14:48] <TheMue> voidspace: take a look at http://reviews.vapour.ws/r/1564/diff/#, there you see the few changes
[14:48] <voidspace> ah ok
[14:48] <voidspace> not that one either :-)
[14:48] <voidspace> TheMue: thanks
[14:50] <TheMue> voidspace: yw
[14:51] <jamespage> fwereade, do peer relations remain accessible during upgrade-charm execution?
[14:52] <fwereade> jamespage, yes, they should do
[14:52] <fwereade> jamespage, are you seeing otherwise?
[14:52] <jamespage> fwereade, no sure - still triaging
[14:52] <jamespage> fwereade, maybe bad charm code
[14:55] <bac> evilnickveitch: typo fix branch for review: https://github.com/juju/docs/pull/410
[14:56] <evilnickveitch> bac, thanks
[15:40] <katco> fwereade: leadership log spam review if you have a moment (small review): http://reviews.vapour.ws/r/1577/
[16:01] <rogpeppe1> natefinch: sorry for not answering - i was in a call, then forgot you'd asked a question...
[16:01] <rogpeppe1> natefinch: the answer is yes
[16:01] <rogpeppe1> natefinch: (but i don't think it matters)
[16:01] <rogpeppe1> pwd
[16:02] <perrito666> rogpeppe1: /home/hduran/gocode/src/github.com/juju/juju
[16:02]  * perrito666 hates to leave wrong window command unanswered
[16:02] <rogpeppe1> omg i've changed identity!
[16:03] <perrito666> rogpeppe1: I am pretty sure you would be waaay more freaked if I had produced your pwd
[16:04] <rogpeppe1> perrito666: i don't think it would be hard to guess...
[16:04] <rogpeppe1> perrito666: well, one of them anyway
[16:23] <katco> all blockers cleared!
[16:29] <mup> Bug #1450919 changed: many window unit tests failures <blocker> <ci> <regression> <windows> <juju-core:Fix Released by gz> <juju-core 1.24:Fix Released by axwalk> <https://launchpad.net/bugs/1450919>
[16:29] <mup> Bug #1451100 changed: TestCheckProviderProvisional fails on ppc64 <blocker> <ci> <ppc64el> <regression> <test-failure> <juju-core:Fix Released by ericsnowcurrently> <juju-core 1.24:Fix Committed by ericsnowcurrently> <https://launchpad.net/bugs/1451100>
[16:31] <katco> wwitzel3: test ping
[16:31] <wwitzel3> katco: pong, yes!
[16:31] <wwitzel3> great success
[16:31] <katco> wwitzel3: :)
[16:37] <sinzui> abentley, mgz, katco, wallyworld: CI blessed master, and it closed th remaining blocking bugs by itself
[16:37] <katco> sinzui: yay :)
[16:38] <sinzui> Ci will now test 1.24 with the new storage feature
[16:38] <mup> Bug #1450919 was opened: many window unit tests failures <blocker> <ci> <regression> <windows> <juju-core:Fix Released by gz> <juju-core 1.24:Fix Released by axwalk> <https://launchpad.net/bugs/1450919>
[16:38] <mup> Bug #1451100 was opened: TestCheckProviderProvisional fails on ppc64 <blocker> <ci> <ppc64el> <regression> <test-failure> <juju-core:Fix Released by ericsnowcurrently> <juju-core 1.24:Fix Committed by ericsnowcurrently> <https://launchpad.net/bugs/1451100>
[16:47] <mup> Bug #1450919 changed: many window unit tests failures <blocker> <ci> <regression> <windows> <juju-core:Fix Released by gz> <juju-core 1.24:Fix Released by axwalk> <https://launchpad.net/bugs/1450919>
[16:47] <mup> Bug #1451100 changed: TestCheckProviderProvisional fails on ppc64 <blocker> <ci> <ppc64el> <regression> <test-failure> <juju-core:Fix Released by ericsnowcurrently> <juju-core 1.24:Fix Committed by ericsnowcurrently> <https://launchpad.net/bugs/1451100>
[17:12] <fwereade> katco, shit it
[17:12] <fwereade> er
[17:12] <fwereade> ship it
[17:12] <mgz> ...
[17:12]  * fwereade crawls off to hide in a corner somewhere
[17:12] <mgz> that's not how we do releases around here fwereade :P
[17:15] <rogpeppe1> lol
[17:25] <gsamfira> LOL
[17:39]  * fwereade is off to catch some crabs with laura, will be back later
[17:40] <fwereade> maybe even an eel
[17:50] <katco> wwitzel3: looks like you don't have to look at that critical bug... blessing of master closed it
[18:00] <wwitzel3> katco: awesome, that's good timing too
[18:00] <wwitzel3> katco: since it was on my list for after lunch :)
[18:01] <katco> :)
[18:22] <katco> is anyone looking at bug 1451674? it's a blocker for master
[18:22] <mup> Bug #1451674: Broken DB field ordering when upgrading to Juju compiled with Go 1.3+ <blocker> <golang> <upgrade-juju> <vivid> <juju-core:Triaged> <juju-core 1.23:Triaged> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1451674>
[18:28] <mattyw> fwereade, that sounds like awesome fun
[18:29] <mattyw> katco, I'm just EODing, if no one picks it up between now and me starting tomorrow I'll take a look
[18:29] <katco> mattyw: looks like menn0 might be involved, but i'm sure ian will coordinate
[18:29] <mattyw> katco, but if someone is able to look at it in the meantime that would be nice - I have stuff waiting to land :)
[18:29] <katco> mattyw: ty though
[18:29] <katco> mattyw: yeah :p
[18:32] <katco> wwitzel3: perrito666: natefinch: cherylj: i need a volunteer ^^
[18:38] <katco> (let ((volunteers '(wwitzel3 perrito666 cherylj)))
[18:38] <katco>   (elt volunteers (random (length volunteers)))) => cherylj
[18:38] <katco> cherylj: what are you up to?
[18:38] <katco> :)
[18:39] <cherylj> katco: sorry, was on the phone with the vet.  Had to take my cat in for surgery today.  But other than that, I can help
[18:40] <katco> eek.. everything ok?
[18:40] <cherylj> Yeah...  it wasn't an emergency surgery
[18:40] <katco> oh good
[18:40] <cherylj> she had to get some teeth extracted and it turned out to be a lot worse than they thought initially
[18:40] <cherylj> fun times
[18:41] <katco> oh =/ yeah we almost had to do that to one of our cats
[18:41] <katco> hope they're ok in the end
[18:41] <katco> can you take a look at bug 1451674? it doesn't seem like it should be too bad... looks like a map ordering issue
[18:41] <mup> Bug #1451674: Broken DB field ordering when upgrading to Juju compiled with Go 1.3+ <blocker> <golang> <upgrade-juju> <vivid> <juju-core:Triaged> <juju-core 1.23:Triaged> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1451674>
[18:42] <cherylj> yeah, I'll take a look
[18:42] <katco> cherylj: ty! fyi, it's a blocker (as indicated by the blocker tag)
[18:42] <cherylj> yay!
[18:42] <cherylj> hehe
[18:43] <katco> lol
[18:43] <katco> cherylj: you will be a hero to all, as TheMue was in the yesterday
[18:44] <katco> cherylj: your name will ring out in CI logs across the build farm
[18:44] <cherylj> well that all depends on the timeliness of my fix
[18:44] <cherylj> heh
[18:44] <katco> we have faith! :)
[18:44] <katco> start with 1.23 and work your way up, as is custom
[18:45] <cherylj> ah, menn0 was talking about this in our standup yesterday
[18:46] <katco> oh awesome, so you have some context then?
[18:47] <cherylj> not much more than what's in the report
[18:59] <perrito666> hey I am back, cherylj if you get tired of it you can toss it this way
[19:02] <cherylj> Thanks, perrito666.  I can take it, but wouldn't mind some pointers as this part of the code is all new to me.
[19:03] <cherylj> perrito666: Do I just need to change the doc to be bson.D rather than bson.M?
[19:05] <perrito666> cherylj: yes, you might need to do some cascade fixing since I have seen some queries use.M just to make the envuuid thinguie and will blow if you change it
[19:06] <cherylj> ok, cool, thanks
[19:06] <perrito666> if it werent for point 2 on that report this would not be critical
[19:26] <cherylj> are we supposed to be using go 1.3 for our local development?
[19:27] <perrito666> cherylj: not really, but juju in vivid is supposed to be compiled with it iirc
[19:27] <cherylj> ok, I'm just trying to compile with 1.3 to try and recreate and I'm not having a lot of luck
[19:28] <perrito666> cherylj: mm, test not failing  right?
[19:28] <cherylj> perrito666: it seems to be complaining that the dependencies are not also built with 1.3
[19:29] <perrito666> cherylj: rm $GOPATH/pkg
[19:30] <cherylj> perrito666: thanks!!!
[19:30] <perrito666> np that is a nassty one, I have it in my buildjuju alias
[19:58] <natefinch> katco: huge thank you for fixing the leadership log spam
[19:58] <katco> natefinch: lol
[21:08] <sinzui>  wallyworld katco xwwt: I wont be at the meeting in 25 minutes. we got CI passes for master and 1.24.
[21:08] <wallyworld> yay
[21:08] <katco> sinzui: :) thanks for the heads-up
[21:08] <wallyworld> sinzui: so 1.24 release status?
[21:09] <wallyworld> katco: btw, i think those log levels should be trace
[21:09] <sinzui> wallyworld, katco xwwt bug 1451674 is the last blocker. CI cannot test it yet.
[21:09] <wallyworld> vry noisy even for debug
[21:09] <mup> Bug #1451674: Broken DB field ordering when upgrading to Juju compiled with Go 1.3+ <blocker> <golang> <upgrade-juju> <vivid> <juju-core:Triaged> <juju-core 1.23:Triaged by cherylj> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1451674>
[21:09] <wallyworld> sinzui: menn0 is onto that today i believe
[21:09] <sinzui> wallyworld, We just got the bless. I will queue the release when I get back from my son's school
[21:09] <wallyworld> sinzui: yay tyvm
[21:10] <katco> wallyworld: i think you're right. some of them should remain debug, but i should have made the majority trace
[21:10] <wallyworld> yeah :-)
[21:10] <wallyworld> can be fixed for 1.24 final
[21:10] <menn0> wallyworld, sinzui: I am. will get on to it straight after the standup
[21:10] <wallyworld> \o/
[21:11] <wallyworld> sinzui: we still having issues with restricted network tests last time i looked?
[21:11] <sinzui> wallyworld, I just sussed it. I will let you read the MP when I explain it
[21:11] <sinzui> wallyworld, not a juju issue
[21:11] <wallyworld> ok
[21:11] <katco> menn0: wallyworld: sinzui: i believe cherylj is looking into that as well
[21:12] <wallyworld> cool :-)
[21:12] <menn0> katco: yep saw that
[21:12] <cherylj> yeah, my next step was to bring in menn0.  I'm having problems reproducing
[21:31] <mwhudson> davecheney: looks like all the patches to go 1.3 in vivid are backports so hopefully they are all fixed in 1.4.2
[21:31] <mwhudson> (which is what debian sid has)
[21:31] <mwhudson> one is a patch to go-mode.el so that's irrelevant now
[21:31] <mwhudson> two are archive/tar things
[21:31] <mwhudson> no, three
[21:32] <mwhudson> and one is an armhf linker thing
[21:32] <mwhudson> hm, linker thing appears not to be present in 1.5
[22:02] <menn0> davecheney, mwhudson: possible Go 1.4 release notes error: "much of the runtime was translated from Go to C". I thought it was the other way around. https://golang.org/doc/go1.4#performance
[22:02] <mwhudson> haha really?
[22:03] <mwhudson> lolz
[22:04] <mwhudson> er
[22:04] <menn0> mwhudson: I've just seen elsewhere in the notes that it's the other way around
[22:04] <mwhudson> menn0: that's not what i see on the page
[22:04] <mwhudson> oh
[22:04] <menn0> mwhudson: the performance section
[22:04] <menn0> mwhudson: sorry, ignore me
[22:04] <mwhudson> that says "much of the runtime was translated to Go from C"
[22:04] <menn0> mwhudson: I read it the wrong way
[22:04] <mwhudson> :)
[22:04] <menn0> sorry
[22:05]  * menn0 had a bad night with both children being awake a lot
[22:05]  * menn0 goes to make another coffee
[22:05] <menn0> seems like a great day to fix some bugs
[22:12] <mwhudson> menn0: find . -name *.go | xargs rm
[22:12] <mwhudson> no more bugs!
[22:17] <mup> Bug #1452050 was opened: Add log when firing hook <landscape> <juju-core:New> <https://launchpad.net/bugs/1452050>
[22:20] <menn0> mwhudson: good plan! i wonder how hard it'll be to get that PR past review...
[22:20]  * menn0 seeks out a Juju devel who's equally tired...
[22:20] <mwhudson> menn0: it'll be huge, the reviewer won't read it properly
[22:20] <menn0> ha :)
[23:02] <mup> Bug #1452050 changed: Add log when firing hook <landscape> <juju-core:Invalid> <https://launchpad.net/bugs/1452050>
[23:08] <mup> Bug #1452050 was opened: Add log when firing hook <landscape> <juju-core:New> <https://launchpad.net/bugs/1452050>
[23:17] <wallyworld> perrito666: standup?
[23:17] <mup> Bug #1452050 changed: Add log when firing hook <landscape> <juju-core:Invalid> <https://launchpad.net/bugs/1452050>
[23:19]  * thumper heading out for early lunch