[00:01]  * thumper sighs
[00:01] <thumper> why oh why is the bot fubared
[00:31] <davecheney> wallyworld_: thumper sinzui axw https://docs.google.com/a/canonical.com/document/d/1m9R2n6LPLNLGjdopcNkQYVG8D5V4FTyvc1vvn-9ZifM/edit#
[00:32] <wallyworld_> davecheney: thanks, will look
[00:32] <thumper> davecheney: ta
[00:32] <wallyworld_> thumper: bot is farked because or a tarmac issue
[00:32] <wallyworld_> i think
[00:32] <thumper> :(
[00:33] <wallyworld_> it has merged 4 branches to it's local tree but not pushed to lp
[00:33] <wallyworld_> i'm trying to see if those branches actually passed the tests
[00:33] <thumper> haha
[00:33] <wallyworld_> if so, i can push manually
[00:36] <wallyworld_> thumper: ffs.
[00:36] <wallyworld_> 2014-03-06 18:39:30 WARNING  ConnectionReset calling 'Branch.last_revision_info', retrying
[00:36] <wallyworld_> 2014-03-06 18:39:32 WARNING  ConnectionReset calling 'Branch.last_revision_info', retrying
[00:36] <wallyworld_> 2014-03-06 18:39:35 INFO     Committing to: /home/tarmac/trees/src/launchpad.net/juju-core/
[00:37] <wallyworld_> so if it can't push to lp, it gives up and just merges locally
[00:37] <wallyworld_> which results in complaining ever after
[00:37] <thumper> can the machine see launchpad?
[00:37] <thumper> hang on
[00:37] <thumper> it must be able to
[00:37] <wallyworld_> it can and has - i think those are transient
[00:37] <thumper> otherwise it wouldn't merge...
[00:37] <wallyworld_> yep
[00:38] <wallyworld_> so transient errors = it's all broken
[00:38] <thumper> gah
[00:38] <wallyworld_> i'll look for the branches that have passed testing in the log file and push those manually
[00:48] <thumper> wallyworld_: while you're in there
[00:48] <wallyworld_> that's what she said
[00:48] <thumper> wallyworld_: want to 'go get github.com/juju/loggo' ?
[00:48] <wallyworld_> sure
[00:48] <thumper> ta
[00:49] <thumper> wallyworld_: so since it failed once, it no longer tried to push to lp?
[00:49] <wallyworld_> yep :-(
[00:49] <wallyworld_> i had to push manually
[00:49] <wallyworld_> there were 5 i think
[00:49] <bodie_> trying to take a crack at fixing a few small bugs to get my head in the game with Go -- one problem, i'm not sure which part of the codebase this refers to on Github
[00:50] <bodie_> https://bugs.launchpad.net/juju-core/+bug/1217868
[00:50] <_mup_> Bug #1217868: move from state.D to bson.D <hours> <tech-debt> <juju-core:Triaged> <https://launchpad.net/bugs/1217868>
[00:50] <bodie_> it says juju-core, but i'm not seeing a juju-core repo
[00:50] <thumper> bodie_: juju-core is still on launchpad
[00:50] <bodie_> is there a doc that explains how things are put together?
[00:50] <bodie_> ah
[00:51] <thumper> go get launchpad.net/juju-core/...
[00:51] <bodie_> alrighty
[00:51] <thumper> wallyworld_: I have a physio appt during our standup
[00:51] <thumper> wallyworld_: when do you need to finish today?
[00:52] <wallyworld_> around 3pm
[00:53] <wallyworld_> thumper: when does your appt finish? we can just delay the standup
[00:53] <thumper> I should be back just after 3pm my time
[00:53] <thumper> noon your time
[00:53] <waigani> that suits me, I've got a school thing at 2pm
[00:53] <wallyworld_> ok, so let's delay by 30 mins then
[00:53] <waigani> wallyworld_: sent you an email.
[00:54] <wallyworld_> waigani: changing writeSampleConfig by itself is not sufficient - i was just point out where the config is used
[00:54] <wallyworld_> you need to add a method to optionally add to the sample config
[00:55] <wallyworld_> do this during set up
[00:55] <waigani> wallyworld_: right, thought so
[00:55] <wallyworld_> and have writeSampleConfig use that
[00:55] <thumper> wallyworld_: perhaps by 45 minutes
[00:55] <thumper> as I need to get home
[00:55] <wallyworld_> sure
[00:55] <waigani> axw: man of the moment! Sorry for spamming your inbox
[00:56]  * axw opens inbox
[00:56] <wallyworld_> thumper: btw, i can see the bot landing your latest branch
[00:56] <wallyworld_> running tests i mean
[00:57] <wallyworld_> axw: waigani: standup will be delayed while thumper gets repaired. approx 45 minutes
[00:57] <axw> hehe. no worries
[00:57] <thumper> wallyworld_: ok, cool
[00:57] <thumper> wallyworld_: we'll see if it pushes when done
[00:57] <wallyworld_> yeah
[00:58] <axw> waigani: re "do you think it is worth while making the validation optional but on by default in UpdateEnvironConfig" -- yes, it is already optional, by opening state with a nil policy
[00:59] <waigani> axw: haha, point taken
[01:00] <waigani> wallyworld_: okay 3:15. My girl asked me to watch her at assembly which is on now. So that works well.
[01:00] <wallyworld_> cool :-)
[01:04] <axw> waigani: first, when reopening don't use Initialize, use "Open". second, if you follow through to JujuConnSuite you can see how it opens a state connection. It uses juju.NewConn, passing in the environ
[01:05] <axw> waigani: take a look in juju/conn.go to see what NewConn does, and how it opens a state conn
[01:05] <axw> waigani: you can then do something similar in your test, but pass nil in as the policy
[01:09] <thumper> wallyworld_: is the bot finished?
[01:09]  * thumper doesn't see a push
[01:09]  * wallyworld_ looks
[01:10] <wallyworld_> thumper: looks like push to lp has failed again
[01:10] <thumper> :(
[01:10] <wallyworld_> thumper: i'll mention to martin
[01:10] <thumper> does the bot have a ssh key that lp knows about?
[01:11] <wallyworld_> don't know, seems not i guess
[01:11] <wallyworld_> actually
[01:11] <thumper> wallyworld_: how are you pushing to lp from there?
[01:11] <wallyworld_> it must have
[01:11] <wallyworld_> cause i'm pushing from the bot
[01:11] <wallyworld_> so it just may be a small setup issue
[02:15]  * thumper joined standup hangout
[02:16] <waigani_> axw: http://pastebin.ubuntu.com/7047615
[02:19] <wallyworld_> axw: standup time :-)
[09:54] <voidspace> https://code.launchpad.net/~rogpeppe/juju-core/511-instancepoller-aggregate/+merge/209505
[09:54] <voidspace> wwitzel3: https://code.launchpad.net/~rogpeppe/juju-core/511-instancepoller-aggregate/+merge/209505
[09:54] <voidspace> one time special offer
[09:54] <voidspace> review our branch, again!
[09:54] <voidspace> https://codereview.appspot.com/71490045/
[09:56] <mgz> wwitzel3: https://code.google.com/p/irssi-libnotify/
[10:12] <mgz> wwitzel3: it's a trap!
[10:13] <natefinch> morning all
[10:13] <wwitzel3> Morning natefinch
[10:13] <wwitzel3> should we get the video online?
[10:13] <natefinch> wwitzel3: yes please
[10:16] <jam> natefinch: https://plus.google.com/hangouts/_/7acpj3k2j7gc114cifi39l6sf0
[10:36] <rogpeppe> wwitzel3:  lp:~rogpeppe/juju-core/512-maas-bootstrap-bridge-utils
[10:37] <rogpeppe> wwitzel3:  lp:~rogpeppe/juju-core/512-maas-bootstrap-bridge-utils
[11:01] <voidspace> A long and difficult CL for someone
[11:01] <voidspace> https://codereview.appspot.com/72450043
[11:52] <jamespage> mgz, does this look familiar @ 1.17.4
[11:52] <jamespage> unit-percona-cluster-6: 2014-03-07 11:51:52 ERROR juju runner.go:220 worker: exited "upgrader": cannot read tools metadata in tools directory: open /var/lib/juju/tools/1.17.4-trusty-amd64/downloaded-tools.txt: no such file or directory
[11:52] <jamespage> I get that continually from all of my service units
[12:09] <mgz> jamespage: bug 1285901
[12:09] <_mup_> Bug #1285901: error starting unit upgrader on local provider <local-provider> <lxc> <regression> <juju-core:Fix Committed by wallyworld> <https://launchpad.net/bugs/1285901>
[12:10] <jamespage> mgz,good oh
[12:10] <jamespage> I'll stop making noise
[12:12] <rogpeppe> voidspace: https://github.com/golang/lint
[12:16] <fwereade> adeuring, basically LGTM, ping me if any of my suggestions are unclear
[12:16] <fwereade> adeuring, oh! did we want handling for JUJU_TEST_MODE as well?
[12:17] <adeuring> fwereade: what do you mean with "handling for JUJU_TEST_MODE"?
[12:18] <fwereade> adeuring, actually forget I said anything
[12:18] <adeuring> ;)
[12:18] <fwereade> adeuring, it'll cause more problems than it solves
[14:25] <cjohnston> dimitern: what version of jujuclient were you using yesterday when you deployed trying to reproduce our issue? I'm wondering if you had the newest version which I guess has a workaround for the issue
[14:27] <dimitern> cjohnston, i'm using juju-core from trunk
[14:28] <cjohnston> dimitern: client... jujuclient
[14:28] <dimitern> cjohnston, i don't quite know what's jujuclient
[14:28] <cjohnston> python-jujuclient ?
[14:28] <dimitern> cjohnston, didn't have to install that
[14:28] <cjohnston> we also started pinning a specific revno from trunk a couple days ago as a work around for this
[14:28] <mgz> cjohnston: wat?
[14:29] <mgz> why not just use one of the released 1.17 ones
[14:29] <mgz> or are you talking about kapil's thing?
[14:30] <cjohnston> not sure what kapil's thing is, but he has been helping us with this issue
[14:30] <mgz> cjohnston: specifically, what do you mean by "jujuclient"?
[14:30] <cjohnston> python-jujuclient
[14:30] <cjohnston> AIUI it has a workaround for the api watcher issue we are hitting..
[14:30] <cjohnston> ev, hazmat ^
[14:31] <mgz> okay, so you are talking about kapil's thing
[14:31] <hazmat> mgz, i had a workaround for the api server going awol and disconnecting all clients
[14:31] <hazmat> in python jujuclient  so that it recovered for extant watches and reconnected and restablished the watch
[14:32] <hazmat> mgz, i think the question is if the underlying issue of the api server going awol is addressed
[14:32] <cjohnston> mgz: so I am trying to figure out how dimitern tested it, and it appears as though it would need to be retested without hazmat's workaround in order to try to reproduce..
[14:32] <mgz> they've only given us steps to repo that include spinning up 15 manchines
[14:32] <cjohnston> however we have a number of people on our team who have been hitting it
[14:32] <mgz> so, dimitern has been struggling to actually get traction on it
[14:32] <cjohnston> mgz: that's how we produce the error
[14:33] <hazmat> mgz, specifically there's a log entry in syslog showing mongodb with no clients
[14:33] <cjohnston> we don't have time right now to go look for other ways to produce it.. we have a deadline that we have to meet
[14:33] <hazmat> cause of a client error from the api server, which then restarts the api server worker
[14:34] <hazmat> mgz, cjohnston i also have no idea how to reproduce this.. it does seem more prevalent on canonistack..
[14:34] <mgz> cjohnston: sure, I know you guys are pretty limited in what you can put in here
[14:34] <jam> sinzui: bug #1289376   conversation should be here when you're around
[14:34] <_mup_> Bug #1289376: Cannot build r2379 on amd64+precise <ci> <regression> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1289376>
[14:34] <jam> sinzui: so certainly you should have been using newer golang
[14:35] <jam> but the failures in that bug look exactly what I would expect if you tried to compile juju-core with go 1.0
[14:35] <jam> (the default in precise)
[14:35] <jam> sinzui: I'd be willing to help debug it with you
[14:35] <sinzui> jam, I will look further...oh the makefile didn't exec and add the deps...
[14:36] <sinzui> jam, I switched the rules to run tests since I saw failures everywhere...but I didn't switch back the to the makefile tests.
[14:36] <sinzui> sorry
[14:36] <jam> sinzui: no problem, I'm happy it was easy to fix :)
[14:36] <jam> we'd love to get 1.17.5 today (as a potential 1.18 candidate) though things are looking pretty slim at this ponit
[14:36] <jam> point
[14:36] <jam> bug #1289316 is serious
[14:36] <_mup_> Bug #1289316: lxc not installed from ubuntu-cloud.archive on precise <lxc> <maas> <precise> <regression> <juju-core:Triaged by wwitzel3> <https://launchpad.net/bugs/1289316>
[14:42] <sinzui> jam, I will release 1.17.5 when CI blesses a recent rev. It is 7 revs behind as we did rapid landings.
[14:42] <sinzui> jam, I will release over the weekend if I need to
[14:44] <jam> sinzui: sure, just that if we don't have fixes like bug #1289316 landed, we can't make it 1.18 as it is regressed from 1.16, and arguably we shoudln't make it 1.17.5 either
[14:44] <_mup_> Bug #1289316: lxc not installed from ubuntu-cloud.archive on precise <lxc> <maas> <precise> <regression> <juju-core:Triaged by wwitzel3> <https://launchpad.net/bugs/1289316>
[14:44] <sinzui> jam, noted
[14:45] <jam> sinzui: I'll hopefully have a short list for you as I uncover things, and target them to 1.17.5 as Critical, will that help you ?
[14:47] <jam> sinzui: or would you prefer a different way of indicating blockers
[14:55] <jam> ?
[15:04] <sinzui> jam, sorry, distracted by meeting. I would love a list of blockers. we can target the bugs to 1.17.5 to be clear they need to be fixed in it
[15:05] <sinzui> jam, https://launchpad.net/juju-core/+milestone/1.17.5 has a few bugs that stakeholders care about. We can push the off to the next release though since I can release every week
[15:49] <mgz> jam: https://bugs.launchpad.net/juju-core/+bug/1288141
[15:52] <jam> sinzui: did you start duping the bugs against https://bugs.launchpad.net/juju-core/+bug/1227450 ?
[15:52] <jam> we have probably ~3 bugs I think that are related enough to be a dupe
[16:07] <_mup_> Bug #1288141: Errors when a 1.17.4 unit is rebooted <landscape> <reboot> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1288141>
[16:07] <_mup_> Bug #1227450: juju does not retry provisioning against transient provider errors <charmers> <landscape> <juju-core:Triaged> <https://launchpad.net/bugs/1227450>
[16:07] <sinzui> jam, hazmat did with my blessing.
[16:07] <sinzui> jam, looks like CI is catching up I see r2390 has started testing
[16:19] <mgz> fwereade_, natefinch, jam: https://codereview.appspot.com/72310044
[16:26] <natefinch> mgz LGTM'd.  Thanks.
[17:45] <wwitzel3> mgz: https://codereview.appspot.com/72270044
[17:45] <wwitzel3> also dimitern ^