/srv/irclogs.ubuntu.com/2014/06/19/#juju-dev.txt

menn0davecheney: so did I do something wrong to cause that panic on power? I've gone back and looked at that code and nothing's jumping out at me.00:18
davecheneymenn0: no idea00:20
davecheneyhaven't looked00:20
davecheneyyet00:20
davecheneyfighting with the bot for my previous commit00:20
davecheneymenn0: if three was a mistake, it would relate to tools selection00:20
davecheneyand an unchecked error path00:20
davecheneythis is just suposition and does not indicate assignment of blame00:20
menn0davecheney: hmmm, I'll wait to see what you figure out. Let me know if I can help.00:27
davecheneythis bug is all fucked up00:28
davecheneyhttps://bugs.launchpad.net/bugs/1331744/+attachment/4134402/+files/run-unit-tests-ppc64el.log00:28
_mup_Bug #1331744: panic: reflect: Call of unexported method ppc64el/gccgo unit tests <ci> <gccgo> <ppc64el> <regression> <juju-core:Triaged by jameinel> <https://launchpad.net/bugs/1331744>00:28
davecheneyis just one long line00:28
davecheneyno \r's00:28
thumperdavecheney: view source00:36
thumperdavecheney: it is being rendered as html00:36
thumperdavecheney: it should have the content type of "text/plain"00:36
thumperbut it doesn't00:36
davecheney:sadtrombone:00:38
davecheneygrr, mongo keeps shitting itself in the tests00:52
davecheneypanic: Session already closed00:53
davecheneygoroutine 3633 [running]:00:53
davecheneyruntime.panic(0xa66ca0, 0xc2103c9720) /usr/lib/go/src/pkg/runtime/panic.c:266 +0xb600:53
davecheneylabix.org/v2/mgo.(*Session).cluster(0xc210a96c60, 0xc21035a820) /home/ubuntu/juju-core_1.19.4/src/labix.org/v2/mgo/session.go:1153 +0x6600:53
wallyworldthumper: reschedule 1:1 after core meeting?01:03
davecheneywallyworld: axw is there an issue for these mgo panics ?01:24
davecheneyhttps://github.com/juju/juju/pull/12001:24
davecheney3 in a row trying to land this PR01:24
axwyes01:25
axwthere's a couple01:25
axwapiserver seems to fail more than others01:25
thumperwallyworld: yeah...01:27
davecheneyis it possible to get a build plan that runs with -race ?01:29
davecheneyit'd be a very small amount of SABDFL's massivel pile of SSL money that there is a race in there01:29
davecheneys/be/bet01:30
axwI've run the race detector and already fixed the races - sadly they were not related to the failures01:31
davecheney:(01:32
axwdo I get SSL money now?01:32
davecheneyaxw: sure, it's in the Isle of Man, i'll wait01:33
perrito666sinzui: it is very naive for me to expect for you to be still here right?01:46
sinzuiperrito666, i am here. I have no social life01:47
perrito666sinzui: neither do I, that is why I am also here01:48
perrito666sinzui: how flexible is CI (can you replay old rev tests?)01:48
sinzuiI can play old revs, though only a few tests can play a specific rev. I often need to setup ci to retest a revision, but that is all the tests01:50
sinzuiperrito666, which rev do you care about?01:50
sinzuiI assume you want the backup-restore tests run again...they use the published streams so the early part of the test suite needs to be run to put those streams in place01:51
perrito666sinzui: r0cbd5b87 its the last to have succesfully run01:51
perrito666sinzui: I am trying to reduce the possible culprits of this odd breakage01:51
sinzuiI see it01:51
perrito666sinzui: if r0cbd5b87 doesnt run I think I will say its something outside juju that changed if not I can look inside juju, in any case I can produce a fix somewhere tomorrow but I would really like to go less blind about it01:53
sinzuiperrito666, I ran the tests on Hp and aws01:53
sinzuiand switched between trusty and precise01:53
sinzuiperrito666, We can wait for CI to run that revision, I can could publish an alternate stream from these debs: http://juju-ci.vapour.ws:8080/job/publish-revision/496/01:57
perrito666sinzui: well in the beginning apt-get upgrade is run, so that could produce a version change although it should not01:58
sinzuiperrito666, the tools/streams we tested where made from those debs. The function tests use streams, so we need to republish to test again02:00
perrito666sinzui: as you wish, let me know if you can replay r0cbd5b87, if you cant I think I can do a fairly decent work reproducing the test conditions but will be tomorrow02:00
perrito666ok I think its better that I do it then02:00
perrito666its a good thing I neved deleted that script02:03
* perrito666 creates cheap stream02:05
perrito666sinzui: running testsuite with the old revision, will see the result tomorrow, I am going to bed now02:08
wallyworldthumper: want to get it over and done with now?02:35
thumperwallyworld: yeah02:35
thumperwallyworld: just use the hangout on our normal meeting02:35
wallyworldalready there02:35
=== thumper is now known as thumper-cooking
* thumper-cooking back in an hour or so after kids have eaten04:26
davecheneymenn0: the bug is herer04:34
davecheney        status, err := apiclient.Status(c.patterns)04:34
davecheney        // Display any error, but continue to print status if some was returned04:34
davecheney        if err != nil {04:34
davecheney                fmt.Fprintf(ctx.Stderr, "%v\n", err)04:34
davecheney        }04:34
davecheney        result := newStatusFormatter(status).format()04:34
menn0aha04:35
davecheneyi'll fix04:35
menn0funny because that's mostly pre-existing code04:35
menn0I guess the way that status is used has changed04:35
davecheneythere may be several errors04:36
davecheneyi don't understand why comment 2 is unrelated to the original error04:37
davecheneyhttps://bugs.launchpad.net/juju-core/+bug/133174404:37
waiganimenn0, thumper-cooking, davecheney: user info cmd PR ready for review: https://github.com/juju/juju/pull/12604:37
_mup_Bug #1331744: panic: reflect: Call of unexported method ppc64el/gccgo unit tests <ci> <gccgo> <ppc64el> <regression> <juju-core:Triaged by jameinel> <https://launchpad.net/bugs/1331744>04:37
davecheneys/error/bug04:37
waiganiI had to rewrite the branch because rebasing made a mess of it04:37
davecheneyrun_test.go:293: c.Check(testing.Stdout(context), gc.Equals, string(jsonFormatted)+"\n")04:40
davecheney... obtained string = "[{\"Error\":\"command timed out\",\"MachineId\":\"1\",\"Stdout\":\"\"},{\"MachineId\":\"0\",\"Stdout\":\"megatron\\n\"}]\n"04:40
davecheney... expected string = "[{\"MachineId\":\"0\",\"Stdout\":\"megatron\\n\"},{\"Error\":\"command timed out\",\"MachineId\":\"1\",\"Stdout\":\"\"}]\n"04:40
davecheneyyay, map ordering errors04:40
davecheneyhmm, that is odd, these should always be sorted04:41
waiganidavecheney: jc.SameContents ?04:45
davecheneyi think both are strings04:46
davecheneynot slices04:46
waiganioh right04:46
davecheney % juju destroy-environment -y $(juju switch)04:58
davecheneyuh oh05:04
davecheneylucky(~/src/github.com/juju/juju) % juju bootstrap --upload-tools05:04
davecheneyuploading tools for series [precise trusty]05:04
davecheneyLaunching instance - i-4376dc1105:04
davecheneyWaiting for address05:04
davecheneyAttempting to connect to ec2-54-91-67-54.compute-1.amazonaws.com:2205:04
davecheneyAttempting to connect to ip-10-183-142-21.ec2.internal:2205:04
davecheneyAttempting to connect to 54.91.67.54:2205:04
davecheneyAttempting to connect to 10.183.142.21:2205:04
davecheneyLogging to /var/log/cloud-init-output.log on remote host05:04
davecheneyInstalling add-apt-repository05:04
davecheneyAdding apt repository: deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/cloud-tools main05:04
davecheneyoh phew05:07
davecheneyit bootstrapped a precise machine05:07
davecheneymenn0: https://github.com/juju/juju/pull/12705:17
wallyworld_davecheney: are you serios? that printf caused the bug?05:19
davecheneywallyworld_: nope05:19
davecheneywallyworld_: the printf didn't cause the bug05:20
davecheneycontinuing to use the value of status even though it's value could be nil caused the bug05:20
wallyworld_ok, ta. i just skimmed the pr05:21
wallyworld_but yes, i see now05:21
davecheneyi'm looking at the original report05:22
davecheneyi think the first report was unrelated to the bug in #205:22
wallyworld_when i looked at it briefly, it seemed like another of those call stack differences05:22
davecheneynot sure, i'm trying on ppc now05:23
davecheneyif i can reproduce the issue i'll have to propose a branch with some extra debugging information05:23
menn0davecheney: just responded. I'm wondering if we should solve this in a different way (described on the PR).05:24
davecheneycrap05:24
menn0wallyworld_: ^^05:24
davecheneyi already $$Merge$$05:24
davecheneyi'll propose another branch05:25
wallyworld_i can kill the jib05:25
davecheneyok05:25
davecheneywallyworld_: do that05:25
menn0davecheney, wallyworld_: cheers05:25
davecheneythen i'll push another change05:25
wallyworld_sorry menn005:25
davecheneymenn0: wallyworld_ PTAL05:27
wallyworld_davecheney: can't we stop the double print05:29
wallyworld_s/can't/shouldn't ?05:30
davecheneywe can return nil05:30
davecheneywhen the command will exit with 005:30
menn0that's what I just suggested05:30
wallyworld_or not print it in the command?05:30
wallyworld_and let the top level command print it?05:30
davecheneywallyworld_: then we have two errors to return05:30
davecheneythe one from the call to get the status05:30
davecheneyand one from possibly printing the status05:30
menn0solution: move the if status == nil above the if err != nil05:31
davecheneyok05:31
menn0(and keep returning err if status == nil)05:31
jcw4fwereade, cmars happy Thursday: https://github.com/juju/juju/pull/12805:31
jcw4easy reading: diffstat +92,-005:32
wallyworld_davecheney: menn0 : oh, can status be not nil even with an err?05:32
davecheneywallyworld_: traditionally you can make no assertion about the contents of status if err != nil05:33
menn0wallyworld_: that's what the comment appears to suggest05:33
wallyworld_well that sucks05:33
wallyworld_i assumed if err != nil status would be unusable as you just said05:34
davecheneywallyworld_: i agree05:34
menn0I guess the scenario will be that the status output was partially generated when an error was encountered05:34
davecheneyi don't know who put in the original logic05:34
davecheneythat was my original fix05:34
menn0it's still better to show something05:34
davecheneymenn0: you could argue that the contents of status aren't incomplete05:34
davecheneythey are unknown05:34
menn0original logic has been there for a while I suspect (before my time)05:34
wallyworld_yeah05:34
menn0davecheney: you could argue that but the original author seemed to think it was better to show partial results than nothing at all05:35
wallyworld_sure confusing if you don't look in detail05:35
menn0which I can see05:35
wallyworld_menn0: so, i will be away at the sprint when you are in brisbane :-(05:35
menn0wallyworld_: I gathered that from the dates I heard during today's meeting :(05:36
menn0wallyworld_: no matter, I'll work from my parent's place.05:37
wallyworld_you only here a week right?05:37
menn0A little over a week - 5th to the 15th05:37
* menn0 will be in davecheney's timezone05:37
davecheneymenn0: you can stay at jools' place, right ?05:37
menn0davecheney: my parents live in Brisneyland ... that's the main reason I'll be there05:37
davecheneynot to visit jools?05:38
davecheneyhe'll be hurt05:38
menn0working with Ian was going to be a side benefit05:38
davecheneythat's an odd turn of phrase05:38
wallyworld_menn0: i'll be back morning of the 14th, i won't work that day but maybe we can catch up05:38
menn0I don't think I've met jools in person before although we are indirectly connected outside of canonical05:39
menn0wallyworld_: that'd be good if we can make it work05:39
menn0I'm flying on the 15th so it'd have to be the 14th05:39
wallyworld_yup05:39
wallyworld_leave it with me05:39
menn0davecheney: responded again to the PR05:42
davecheneywallyworld_: which version of gccgo is installed on the build machine ?05:42
davecheneyactaully, maybe I answer that05:42
wallyworld_davecheney: sec, otp05:42
davecheneynope i can't05:42
davecheneyneed help05:43
wallyworld_davecheney: it's an ephemeral image so it pulls down whatever is in the archive05:45
wallyworld_the console logs should show it05:45
davecheneyhttp://juju-ci.vapour.ws:8080/job/local-deploy-trusty-ppc64/99/console05:45
davecheneydoesn't show me05:46
wallyworld_http://juju-ci.vapour.ws:8080/job/github-merge-juju/203/consoleFull05:47
wallyworld_ah05:47
wallyworld_you mean the ppc machine05:47
davecheneyyup05:47
wallyworld_sorry05:47
davecheneygotta check if it isn't using the broken compiler we shipped with trusty05:47
wallyworld_still looking, but i have to duck out to get my son, i'll be back soon05:51
dimiternmorning05:54
davecheneywallyworld_: ping06:41
wallyworld_davecheney: hi, just got back06:41
davecheneyexcellent06:41
wallyworld_still figuring out how to tell what version06:41
davecheneygccgo -v06:41
davecheneywill do06:41
wallyworld_davecheney: gotta figure out the machine ip address, it's not obvious from the job that i can see06:43
TheMueMorning06:46
TheMueWill start a bit later today, I'm in the garage with my car.06:47
TheMueAt least some net access via phone.06:47
=== vladk|offline is now known as vladk
davecheneywallyworld_: i've reproduce the panic on ppc06:51
davecheneystill need to know the compiler version your running06:51
davecheneyalthought I suspect it is the unfixed versino06:51
davecheneybecause the fixed version has not landed in trusty-updates06:52
wallyworld_davecheney: yeah, i still can't find the ip address of the machine running the tests06:52
davecheneyfuk it06:54
davecheneyi'm 90% sure that the fix hasn't made it to trusty yet06:54
davecheneyso we're probably boned06:54
wallyworld_davecheney:06:58
wallyworld_ubuntu@stilson-07:~$ gccgo -v06:58
wallyworld_Using built-in specs.06:58
wallyworld_COLLECT_GCC=gccgo06:58
wallyworld_COLLECT_LTO_WRAPPER=/usr/lib/gcc/powerpc64le-linux-gnu/4.9/lto-wrapper06:58
wallyworld_Target: powerpc64le-linux-gnu06:58
wallyworld_Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.9-20140406-0ubuntu1' --with-bugurl=file:///usr/share/doc/gccgo-4.9/README.Bugs --enable-languages=c,c++,go --prefix=/usr --program-suffix=-4.9 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.9 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --06:58
wallyworld_enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-libmudflap --disable-libsanitizer --disable-libquadmath --enable-plugin --with-system-zlib --enable-secureplt --with-cpu=power7 --with-tune=power8 --disable-multilib --enable-multiarch --disable-werror --with-long-double-128 --enable-checking=release --build=powerpc64le-linux-gnu --host=powerpc64le-linux-gnu --target=powerpc64le-linux-gnu06:58
wallyworld_Thread model: posix06:58
wallyworld_gcc version 4.9.0 20140405 (experimental) [trunk revision 209157] (Ubuntu 4.9-20140406-0ubuntu1)06:58
davecheneyyup06:58
davecheneyhmm06:58
wallyworld_finally found the ip address06:58
davecheneyi've definitely got the dud version on winton-0906:59
davecheneyhttps://bugs.launchpad.net/ubuntu/+source/gccgo-4.9/+bug/130475407:00
_mup_Bug #1304754: gccgo has issues when page size is not 4kB <ppc64el> <trusty> <gcc:Fix Released> <gcc-4.9 (Ubuntu):Fix Released> <gccgo-4.9 (Ubuntu):Invalid> <gcc-4.9 (Ubuntu Trusty):Invalid> <gccgo-4.9 (Ubuntu Trusty):Fix Committed by doko> <gcc-4.9 (Ubuntu Utopic):Fix Released> <gccgo-4.9 (Ubuntu Utopic):Invalid> <https://launchpad.net/bugs/1304754>07:00
davecheneynot fix released07:00
wallyworld_:-(07:00
* davecheney tries throwing ppa's at the problem07:00
davecheneywallyworld_: good news and bad news07:18
davecheney1. bad news, compiler bug is not fixed07:18
davecheneygood news, this isn't the compiler bug07:18
davecheneywallyworld_: [LOG] 0:00.458 DEBUG juju.state.apiserver -> [95] machine-0 171us {"RequestId":2,"Error":"objId is empty, dipshit","Response":{}} Machiner[""].Life07:20
davecheneyserver_test.go:77: c.Assert(err, gc.IsNil)07:20
davecheney... value *params.Error = &params.Error{"", "objId is empty, dipshit"} ("objId is empty, dipshit"07:20
davecheneyso, there is the cause07:21
sparkiegeekif I have two units of the same subordinate service "hulk-smashed" onto a single machine, will I get hooks being run concurrently? (e.g. unit/0 hookX and unit/1 hookX being run at the same time)?07:34
sparkiegeekI tried asking last night but didn't get a definitive answer :)07:35
davecheneysparkiegeek: no07:45
jamfwereade: rogpeppe1: ping about interfaces and reflect, I'm trying to fix bug https://bugs.launchpad.net/juju-core/+bug/1331744  I'm pretty sure I understand *why* it is failing, what I can't understand is why it ever worked.07:45
_mup_Bug #1331744: panic: reflect: Call of unexported method ppc64el/gccgo unit tests <ci> <gccgo> <ppc64el> <regression> <juju-core:Triaged by jameinel> <https://launchpad.net/bugs/1331744>07:45
davecheneywallyworld_: [LOG] 0:00.376 DEBUG juju.state.apiserver <- [95] machine-0 {"RequestId":2,"Type":"Machiner","Request":"Life","Params":{"Entities":[{"Tag":"machine-0"}]}}07:45
davecheney[LOG] 0:00.377 DEBUG juju.state.apiserver -> [95] machine-0 184.387us {"RequestId":2,"Error":"objId is empty, dipshit","Response":{}} Machiner[""].Life07:45
sparkiegeekdavecheney: great, thank you07:45
rogpeppe1jam: looking07:45
jamfwereade: rogpeppe1: I'm in https://plus.google.com/hangouts/_/canonical.com/juju-sapphire07:45
jamrogpeppe1: I can give the context on the hangout07:45
jamthe logs there like to be formatted as "html" which makes them wrap in horrible horrible ways :)07:46
davecheneyjam: look at the debug I sent wallyworld_07:47
davecheneyobjId is empty07:47
jamdavecheney: doesn't matter, I know what the bug is, I'm trying to figure out how to fix it07:48
voidspacemorning all08:02
jamrogpeppe1: a thought about why it might have worked in golang go08:22
jamif it puts its methods in sorted order08:22
jamand all of our types that were being exposed via interfaces08:22
rogpeppe1jam: yeah, if gccgo doesn't, then that's the reason08:22
jamdidn't actually implement more public methods than just the interface methods08:22
jamrogpeppe1: well gccgo could, but it looks like it puts private methods first08:22
rogpeppe1jam: right08:22
jamand Ping comes before ping depending on how you sort :)08:22
rogpeppe1jam: that makes sense08:22
rogpeppe1jam: i suspected something like that08:23
jamso I might have a good test case for golang go then08:23
rogpeppe1sounds like it's not a bug08:23
jamrogpeppe1: well its a bug in our code, because of assumptions that only happened to match most of the time08:23
jam(well, in all our current use cases :)08:23
rogpeppe1jam: yeah, worth adding a test case, sure08:24
davecheneyjam: if you have a test case08:24
davecheneyi can submit it to iant08:24
davecheneyhe's pretty receptive to stuff like this08:24
davecheneyeven if it's 'in spec'08:24
jamdavecheney: so again, it doesn't appear to be a gccgo bug08:24
davecheneyhe knows weird arse shit that makes gccgo act differently drives adoption away08:24
jamdavecheney: it is a case where we were relying on accidental behavior which isn't even always true in golanggo but happened to be for how we were structuring our code.08:24
jamrogpeppe1: yeah, I have a test which shows that it fails if your concrete type has extra methods.08:40
rogpeppe1jam: cool08:40
jambugfix for bug #1331744 https://github.com/juju/juju/pull/12908:48
_mup_Bug #1331744: panic: reflect: Call of unexported method ppc64el/gccgo unit tests <ci> <gccgo> <ppc64el> <regression> <juju-core:Triaged by jameinel> <https://launchpad.net/bugs/1331744>08:48
jamrogpeppe1: ^^ if you want to review it.08:50
jamWithout the code fix, the test returned "A" instead of "second"08:50
jambbiab08:53
=== Ursinha is now known as Ursinha-afk
=== vds` is now known as vds
wallyworld_mgz: i just got back and need to eat, can i ping you after the team lead meeting?10:28
mgzsure thing10:29
dimiternfwereade, thanks for reviewing the networking model doc10:34
voidspacejam: by the way, rfc 3230 digest headers don't support sha256 :-/ (only sha1 and md5)10:35
dimiternfwereade, can you join us on our standup to discuss and agree upon what's not clear?10:35
voidspacejam: we could use the digest header and an "officially non-standard" algorithm10:36
voidspacejam: Digest: sha256=<base64 encoded hash>10:37
fwereadedimitern, yeah, I'm not 100% sure I've finished orthat I'm very coherent today either10:37
voidspacejam: that would be slightly more standard than our current X-Content header10:37
voidspacejam: which I forgot to fix yesterday10:37
fwereadedimitern, ...and hmm we have the leads meeting after the standup, and I have wallyworld_ after that10:38
fwereadedimitern, after that maybe?10:38
dimiternfwereade, whenever works for you10:39
jamvoidspace: are we using sha256? we could still stick with sha110:40
wallyworld_mgz: i finished eating, let's do it quickly now if you're free10:41
jamdimitern: TheMue: standup?10:45
TheMuejam: yep10:45
=== thumper-cooking is now known as thumper
jamvladk: I like that it shows you are thinking about it, and we can discuss how we've solved some of the problems you're bringing up, I just ran out of time.11:05
* perrito666 is seconds away to having to use git bissect11:06
voidspaceperrito666: I've changed your backup functions to return a base64 encoded sha1 hash instead of a hex-encoded sha 256 hash11:06
voidspaceperrito666: I hope you have no objections to that11:07
perrito666voidspace: nope, some reason in particular?11:07
mfoordperrito666: the only standard http response header (defined by rfc) for sending a file hash11:08
mfoordperrito666: uses base64 encoded sha1 hash11:08
mfoordperrito666: and doesn't support sha 25611:08
perrito666mfoord: I see, I really hope you added a comment there :p11:09
wwitzel3perrito666: git biset is really useful .. most people use git in a way that makes it less useful, but I've used it plenty of times to locate bugs.11:09
mfoordperrito666: http://www.ietf.org/rfc/rfc3230.txt11:09
wwitzel3s/biset/bisect11:09
mfoordperrito666: I added as many comments as there were already...11:09
mfoordperrito666: j/k - I'll add one11:09
perrito666mfoord: "the code was self explanatory"®11:10
mfoordperrito666: hehe11:10
mfoordperrito666: the code is just as easy to read11:10
mfoordperrito666: I've added a comment anyway11:10
perrito666wwitzel3: yes, git bissect rocks, sadly I have no time to build a run command so it does all the test automatically11:13
* perrito666 notices gmail header when the email has a flight booking ... creepy11:20
natefinchperrito666: have we just cranked up the timeouts and see if that fixes things?  Like making the timeouts 60 minutes or something?11:27
perrito666natefinch: I have identified the what is not working, not yet the why11:28
perrito666I have narrowed to a breaking commit between 240db2a2 0cbd5b8711:29
perrito666so now I am bisecting with hope to obtain the culprit and be able to work a solution from there11:29
perrito666luckily I have a setup in place that allows me to re-create what jenkins does, stream included11:31
natefinchperrito666: awesome11:31
mfoorda lovely simple PR for someone to review11:33
mfoordhttps://github.com/juju/juju/pull/130/files11:33
perrito666natefinch: have you decided to stop sleeping? you where here when I left lastnight and are still here when I come back11:33
wwitzel3perrito666: he has kids ;)11:33
natefinchheh11:35
natefinchperrito666: I would say the same to you11:35
perrito666natefinch: I gave up sleeping11:36
perrito666I stayed late debugging this and woke up to have breakfast with my wife11:37
mfoordwwitzel3: thanks11:38
wwitzel3that reminds me, I am really hungry .. thanks perrito66611:38
perrito666wwitzel3: np11:38
wwitzel3mfoord: np11:38
mfoordI like the way github diffs highlight the changed *part* of a line that was only modified11:39
wwitzel3mfoord: you can get that behavior on the command line too using the diff-highlight script11:42
mfoordcool11:42
wwitzel3mfoord: you just need to enable it, it should be in your git/contrib11:42
mfoordwill look, thanks11:42
wwitzel3natefinch: should we do our meeting befores standup day since we missed it yesterday?11:55
natefinchwwitzel3: sounds good11:58
jamcmars: I'd like to send this email, shall we meet up in 5-10 ?12:09
cmarsjam, sounds good12:10
bodie_morning all12:15
bodie_jcw4, saw your Actions PR, woot!12:15
bodie_this might be useful to others (I know at least jcw4 was asking about it)13:12
bodie_https://gist.github.com/piscisaureus/334224713:12
=== vladk is now known as vladk|offline
perrito666ok, says git bisect that 4a2f56528641c12f2e80cb17cc84e58d425d73bd is the culprit13:16
wwitzel3natefinch: I'm in moonstone13:30
natefinchwwitzel3: coming13:30
TheMuefwereade, cmars: a PR for review: https://github.com/juju/juju/pull/131 thanks13:36
natefinchwwitzel3: you see frozen?13:37
sinzuiHi juju devs. Joyent had a storage outage 10 hours ago, that was the cause of the joyent deploy failure. I am stopping CI to reenable publication to joyent. I will restart the current revision.13:54
perrito666sinzui: I think I found the offending commit, sadly its a very long one :s13:56
natefinchperrito666: link me, maybe I can help13:58
sinzuiperrito666, I am SO SO sorry for not getting your rev tested. I had a power outage and joyent failed all about the same time.13:58
=== Ursinha-afk is now known as Ursinha
perrito666sinzui: no hurries, I have a pretty good setup for that already13:59
natefinchsinzui: I wonder if we could set up some kind of provider baseline test that proves the provider is currently working in an acceptable fashion before running our tests... that way we'd better understand what's failing, and our CI tests would look less spotty.13:59
sinzuinatefinch, we set that up last november14:00
sinzuinatefinch, http://juju-ci.vapour.ws:8080/view/Cloud%20Health/14:00
natefinchsinzui: oh awesome, I didn't realize14:00
sinzui^ the joyent job has a sad recent history14:00
natefinchericsnow: standup?14:02
bodie_blocking on https://github.com/juju/juju/pull/128 if anyone can spare a sec to lgtm14:28
jcastroHey guys do we have Juju API docs on the web anywhere?14:40
ericsnowjcastro: I was looking yesterday and couldn't find any14:41
ericsnowjcastro: the best thing I could find was bindings for Python: https://launchpad.net/python-jujuclient14:42
TheMuejcastro: we’ve so far only got a rough text about the general design14:48
TheMuejcastro: a developer documentation is currently wip14:48
natefinchTheMue: it would make me really happy if this could be a reasonable place to look for docs: http://godoc.org/github.com/juju/juju/state/api14:50
ericsnowTheMue: that would be great14:50
natefinchTheMue: obviously having something on juju.ubuntu.com would also be good... but the closer you are to the code, the more likely the docs will stay up to date14:51
TheMuenatefinch: they will be written in markdown, so that they can be rendered directly in GH. additionally they will be rendered for juju.ubuntu.com14:51
TheMuenatefinch: the docs are no documentation of the individual funcs, that may be indeed better there14:52
TheMuenatefinch: they are a design documentation about RPC and versioning for maintenance and extension14:52
TheMuenatefinch: so docs != docs14:52
natefinchTheMue: I think when people ask for docs about the API, they mean "what methods does it contain and what do they do?"14:53
ericsnownatefinch, TheMue: right14:53
TheMuenatefinch: that’s one perspective, yes. another one is design, to get new team members and contributers faster into our system14:54
TheMuenatefinch: that’s where I’m working on14:54
natefinchTheMue: cool14:54
TheMuenatefinch: finding a way to generate also a good overview of the API functions automatically too would be nice, yes. will think about it14:55
jcastroTheMue, what's the timeline for API docs? someone is making a production decision based on when we have that14:55
ericsnowTheMue: another perspective is people writing bindings--they need both14:55
natefinchjcastro: are they looking for docs about what endpoints are exposed and what they do, etc?14:55
jcastroyeah14:55
natefinchjcastro: this is a good start: http://godoc.org/github.com/juju/juju/state/api14:55
TheMuejcastro: which perspective is the interesting one here? I think nates view of the available functions?14:55
jcastrothey want to integrate with jenkins etc.14:55
jcastrohe's the guy talking in #juju14:56
jcastroI have to run the cloud cross team call, if one of you guys can take over in #juju TheMue or natefinch14:57
natefinchjcastro: I'm on it14:58
natefinchwwitzel3: can you try helping perrito666 figure out his mongo issue?  It's not fun, but it's quite important14:59
perrito666natefinch: well I know I am no fun, but its not like you have to say it on public :p15:00
natefinchhaha15:00
natefinchit's mongo that's no fun15:00
* fwereade goes to lie down a bit more, back later15:00
wwitzel3natefinch: yep15:19
wwitzel3perrito666: I'm in moonstone15:19
perrito666wwitzel3: hold a sec15:20
perrito666wwitzel3: your room is on moonstone15:21
=== vladk|offline is now known as vladk
* perrito666 plays silent cinema with wwitzel315:41
wwitzel3perrito666: lol15:41
wwitzel3perrito666: that CI link you sent me 404, can you link it here15:41
perrito666http://4.bp.blogspot.com/-5jfw91ZJefw/UvxN4qJNfoI/AAAAAAAATNc/9Jh7whIzWmE/s1600/Cinelists+-+Silent+Movie-+Mel+Brooks+(20).jpg15:42
perrito666wwitzel3: http://juju-ci.vapour.ws:8080/job/functional-ha-backup-restore/15:42
perrito666sinzui:15:54
perrito666this happens only with ha right?15:54
sinzuiperrito666, I have seen just backup-restore fail, but that passed yesterday15:55
=== seelaman` is now known as seelaman
ericsnowvoidspace:  with the new backup client I see that you are following the lead of AddLocalCharm and UploadTools16:03
voidspaceericsnow: in terms of?16:03
voidspaceericsnow: what's the link to your PR by the way?16:03
voidspaceericsnow: I'm about ready for it16:03
ericsnowvoidspace: the direct HTTP request stuff rather than using the RPC call machinery16:04
voidspaceericsnow: right, indeed16:04
rogpeppe1dimitern, natefinch, voidspace, mgz, perrito666, fwereade, jam: mechanical changes here, preparing for adding bundle support to the charm package, review appreciated: https://github.com/juju/charm/pull/716:04
ericsnowvoidspace: https://github.com/ericsnowcurrently/juju/pull/1/files16:05
voidspaceericsnow: it needs to be, I don't want to read the whole (potentially gigabyte sized) file into memory before writing it to disk16:05
ericsnowvoidspace: can't you use a different codec with the RPC/websocket code that does it while still using the existing RPC machinery?16:06
ericsnowvoidspace: WatchDebugLog already does something along these lines16:06
voidspaceericsnow: I don't know :-)16:06
voidspaceericsnow: ah right, I looked for examples and the first I found was the charm stuff16:06
voidspaceericsnow: it isn't much work though16:06
voidspaceericsnow: five lines of code to prepare and do the call16:07
voidspaceericsnow: or so16:07
ericsnowvoidspace: I'm guess that those 3 oddball API methods do their own thing because it was the easiest way to get it done :)16:07
voidspaceright16:07
voidspaceI'm not sure how far out of our way we should go to find a harder way... ;-)16:07
ericsnowvoidspace: considering the comments in AddLocalCharm() and UploadTools(), it may be worth factoring at least some of the code into a common binary blob RPC client16:08
dimiternrogpeppe1, lgtm16:09
voidspaceericsnow: maybe16:09
ericsnowvoidspace: at which point it may be worth just going the binary-blob-instead-of-JSON codec route16:09
voidspaceericsnow: grabbing coffee and then will take a proper look at the code16:09
ericsnowvoidspace: if I read all that code right (it gets a little twisty in there)16:09
ericsnowvoidspace: sounds good :)16:10
ericsnowBTW, I just put up a doc patch for backup/restore (once the CLI changes land): https://github.com/juju/docs/pull/12416:11
=== vladk is now known as vladk|offline
=== vladk|offline is now known as vladk
voidspacenatefinch: ping16:20
voidspaceactually16:20
voidspacenatefinch: unping16:20
voidspacealexisb: ping16:20
alexisbvoidspace, pong16:20
=== vladk is now known as vladk|offline
=== vladk|offline is now known as vladk
=== vladk is now known as vladk|offline
=== vladk|offline is now known as vladk
=== vladk is now known as vladk|offline
=== vladk|offline is now known as vladk
=== vladk is now known as vladk|offline
=== vladk|offline is now known as vladk
=== vladk is now known as vladk|offline
voidspaceericsnow: as far as I know we can't extend the Client type from another file16:41
voidspaceericsnow: which is why we don't break the client up into multiple files16:42
voidspaceericsnow: we could move common functionality into a base class and then have several client types16:42
voidspaceericsnow: but that seems like extra complexity16:42
ericsnowvoidspace: oh, I figured intra-package was okay16:42
voidspaceericsnow: I *may* be wrong16:43
voidspacenatefinch: ping? ^^^16:43
voidspaceericsnow: there isn't much common code I can see to factor out between the binary blob methods16:43
=== vladk|offline is now known as vladk
ericsnowvoidspace: at least the part around  "resp, err := utils.GetNonValidatingHTTPClient().Do(req)"16:44
voidspaceericsnow: right - I intend to do that already, for when we switch to a validating client16:45
voidspaceericsnow: (have getting the client be a single method used by all of them)16:45
ericsnowvoidspace: exactly16:45
voidspaceericsnow: WatchDebugLog shows how to do that correctly I think, so I'll just copy that code :-)16:46
voidspaceericsnow: except the error handling looks janky - it reads the first line to see if there's an error16:48
voidspaceericsnow: I mean, we *could* do that with Backup, assuming a newline appears *somewhere*16:48
voidspaceericsnow: but it's a bit yucky16:48
ericsnowvoidspace: yeah :)16:49
voidspaceericsnow: hmmm16:50
voidspaceericsnow: that looks a little hard to get round16:50
natefinchvoidspace: you can put functions for a type in any file in the same package as that type.  files are just an organizational tool, they don't really interact with the language itself.16:51
voidspaceericsnow: ah no, I think that API endpoint always sends an error response in the first line16:51
voidspaceericsnow: and you tell if there was an error by checking the Error of the unmarshalled json of the first line16:52
voidspaceericsnow: so it's not a problem - it's a janky api endpoint, not a feature of websockets16:52
voidspacenatefinch: ok, cool16:52
ericsnowvoidspace: good16:52
voidspacenatefinch: sooo... for the client Backup method, ericsnow is suggesting we put this in a new file16:52
voidspacenatefinch: especially as Backup will have related methods for restore soon16:53
natefinchsure16:53
voidspaceokey-dokey16:53
voidspaceericsnow: I'm going to punt on factoring it out into a different type though16:53
natefinchfiles don't really matter.  It can be helpful to break stuff up.  it's your call16:53
voidspaceericsnow: I want to use some of the client methods for obtaining the websocket16:53
ericsnowvoidspace: no worries (limited resources, etc.)16:53
voidspaceaye16:53
voidspaceericsnow: as I'm away from Saturday I'll get done what I get done (probably most of it bar complete tests)16:54
voidspaceericsnow: and hand the rest over if not complete16:54
ericsnowvoidspace: sounds good16:54
jcw4fwereade, cmars : happy Thursday :) https://github.com/juju/juju/pull/13217:20
jcw4PTAL17:20
voidspaceericsnow: hmmm... it seems like the janky api in WatchDebugLog maybe because websockets don't have a statuscode17:28
voidspaceericsnow: and they may not have headers easily accessible either17:28
voidspaceericsnow: so this may be a dead end17:28
voidspaceericsnow: or we duplicate the janky api to send any error response and the sha17:29
voidspaceericsnow: (i.e. send a json response as the first line and *then* the file)17:29
ericsnowvoidspace: that's what I was thinking :)17:29
voidspaceericsnow: I'll look tomorrow17:29
voidspaceI'm EOD now17:29
ericsnowvoidspace: alrighty17:30
voidspaceericsnow: I pushed the changes that split out the Backup method into its own file at least17:30
voidspaceericsnow: I haven't pushed the websocket changes as I'm not sure whether or not to go down that road17:30
voidspacesee you tomorrow everyone17:30
voidspaceg'night17:30
bodie_blocking on a LGTM for https://github.com/juju/juju/pull/13218:16
cmarsbodie_, jcw4 not familiar enough with actions or watchers to approve, but I don't see anything wrong with it either18:19
jcw4cmars: thanks; this part has nothing new that is inherent to actions... just a new filtered watcher on the actions collection18:20
jcw4cmars appreciate your comments!  I don't mind waiting for someone who knows watchers better to LGTM18:21
jcw4(isn't that a weird nounification ^^ ?)18:22
jcw4or wait... verbification?18:22
bodie_I think it's a verbed noun, yes18:22
bodie_verbing is the best18:23
jcw4haha18:23
jcw4I see what you did there18:23
bodie_and LGTM itself is a nouned exclamation...18:23
bodie_english...18:24
bodie_not a well-typed language18:24
jcw4... is why they invented Go18:24
jcw4har har18:24
bodie_nyuk nyuk18:24
perrito666sinzui: will you stop growing that bug?18:25
sinzuiperrito666, sorry, I am just looking for evidence. since branches are linked to bugs any more. I had to read every commit since the last release. I just happened to see things that looked relevant to the tests failing18:27
perrito666just kidding I identified the commit that breaks at least in aws18:27
sinzuiperrito666, fab, I moved both jobs back to aws this morning18:28
bacsinzui: are there any known issues with juju and local providers on utopic?18:30
sinzuibac no18:31
bacthanks18:31
sinzuibac: it has a perfect record in fact *after* the template is created. If you abort the first bootstrap, the template is borked18:32
=== vladk is now known as vladk|offline
perrito666mm, I think something is really making our tree ugly18:50
wwitzel3perrito666: ?18:51
hazmatrogpeppe1, one item of concern with machines in the bundle spec is the issues of concurrency around usage of those machines18:52
perrito666wwitzel3: something Was not making sense to me so I went into a graphical git thinguie and our tree looks a bit messy but perhaps its what I am using18:52
hazmatrogpeppe1, ie. juju will blithely assign a different workload to an unused machine even though it was created for the bundle18:52
bodie_in case anyone missed the pr, actions work is blocking on https://github.com/juju/juju/pull/13218:57
bodie_pretty simple one18:57
jcw4someone with watcher knowledge could review it in just a few minutes ^^18:57
perrito666natefinch: lol, what a rant19:41
perrito666natefinch: I think the preferred g-apps way is share this as a gdrive attachment19:41
natefinchI have a pet peeve about email size limits19:41
natefinchsome people like pretending we're all still on dialup19:42
perrito666heh, I recall some old dev on my team, some time ago telling me "but what if the user has js dissabled" ... "me: who does that" "him: blind people" .."me: for gods sake, we are making a movies guide site, really? blind people?"19:43
perrito666and just to make it worse, the discussion was around some js for a button that had the caption "I have seen this"19:45
natefinchlol19:45
TheMuenatefinch: great pic *smile*19:50
natefinch:)19:50
natefinchthrilled to have a boy.  I know a lot of people with three girls.  Of course, now I have to figure out how to raise a boy.19:50
perrito666natefinch: you should have gone througt that process at least once19:52
natefinchperrito666: and look how *that* turned out! :)19:52
perrito666well, you have your kids go to standups, the apple cant fall so far from the tree19:53
hazmatsinzui, which azure regions are being tested?19:58
sinzuihazmat, US West...the only one that ever worked20:10
perrito666wwitzel3: got any luck?20:17
perrito666wallyworld_: lemme know when you come to life please20:17
wallyworld_perrito666: hey, having my morning coffee. only 6am here so i have a few things to do to get the kid ready for school so i can't give you my full attention, but i may be able to help20:19
perrito666well, I dont want to ruin your morning coffee but https://github.com/juju/juju/commit/4a2f56528641c12f2e80cb17cc84e58d425d73bd#diff-dec2c5aafeb31667d3e3abc2232ec107R50 is most likely the commit that broke restore, I could use some help figuring what exactly it is20:20
perrito666so yea, I can wait until your full attention is available :p20:20
wallyworld_perrito666: well that sucks :-( do you know what the failure is? a permissions issue?20:22
wallyworld_perrito666: note that there was a follow up to that branch that did fix a permissions issue20:23
perrito666wallyworld_: I wish, the failure is, there is no failure, there is a part of the restore process that is undone :| Its a bit of a hassle to explain over chat, we can have a quick call whenever you finish your morning setup20:23
wallyworld_so if you run with that branch and not the follow up one, it will fail20:23
wallyworld_the follow up branch adds a ClusterAdmin permission to user creation20:24
perrito666wallyworld_: even if I am running as admin?20:24
wallyworld_i think so20:24
perrito666ok, what is the branch?20:24
perrito666d1e0262efcf878bad24c4235873b0d740f21f32020:25
sinzuibodie_, is there anything to announce regarding Actions in 1.19.4?20:25
wallyworld_perrito666: https://github.com/juju/juju/commit/94e7f692cc2c145d67de9c1f5d3e6cda53097c4120:26
jcw4sinzui: not for 1.19.420:26
sinzuigoody20:26
jcw4:) okay...20:26
wallyworld_perrito666: i have to be afk for a bit, will chek back with you later20:27
perrito666tx20:27
hazmatsinzui, ack, thanks.. apparently there have been some issues in the azure api in eu region.. was talking to cpc folks20:53
wallyworld_perrito666: raining here so i have to drive the kid to school. so i'll bbiab if you need anything21:03
perrito666wallyworld_: np, drive safely21:04
thumperwaigani: I have a guy here quoting on windows, may not be done by 11, can you tell dave?22:48
waiganiwow, confusing - I just had the same! yep, shall we go ahead without you?22:49
jcw4I don't suppose there are any folks down under that want to do a quick review on a new Watcher?22:58
jcw4The first part already got merged and this is part 222:58
jcw4https://github.com/juju/juju/pull/13222:58
thumperwaigani, davecheney: window guy still here, perhaps another 10-15 min23:06
thumperok, here now23:16
thumperwaigani, davecheney: go now?23:16
waiganithumper: we are in hangout now :023:16
waigani:)23:17
sinzuiWTF, ec2 trusty cannot bootstrap with a recent rev23:20
* sinzui looks for bad CI23:20
perrito666wallyworld_: returned?23:25
wallyworld_perrito666: yup, a while ago, sorry23:25
perrito666wallyworld_: thats ok, I took a nap, its the night here, wanna do a quick hangout before I go have dinner?23:25
wallyworld_thumper: has the window guy taken the window out and put in back in again?23:26
wallyworld_sure23:26
perrito666wallyworld_: come to https://plus.google.com/hangouts/_/canonical.com/moonstone?authuser=323:26
thumperwallyworld_: nah, just quoting at this stage23:31
perrito666thumper: so, how much for the licence :p?23:34
thumperhaha23:34
* sinzui slaps termination protect on the bootstrap node so juju cannot destroy the evidence23:35
sinzuiwallyworld_, thumper , if you have bzr-git and qbzr installed, I have found that qlog makes more sense out of what jujubot merged than git itself23:36
thumperha23:36
sinzuigit log  shows me  many possible revs that can break juju, but qlog shows the real merges23:37
sinzuibugger, I wonder if trusty changed its mongo, or the new trusty images used by aws don't work with juju23:39
sinzuithumper, wallyworld_ More suck, https://bugs.launchpad.net/juju-core/+bug/133236523:49
_mup_Bug #1332365: trusty EC2: cannot initiate replica set <bootstrap> <ec2> <mongodb> <trusty> <juju-core:Triaged> <https://launchpad.net/bugs/1332365>23:49
wallyworld_:-(23:50

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!