/srv/irclogs.ubuntu.com/2016/03/24/#juju-dev.txt

thumperhttp://reports.vapour.ws/releases/3811/job/run-unit-tests-trusty-ppc64el/attempt/491300:17
thumperso need to get off gccgo00:17
rick_h_thumper: around now00:18
mgzthumper: master merge, dat bug fixed I see bug 156102300:19
mupBug #1561023: charmstore v5.WillIncludeMetadata gccgo build failure <ci> <gccgo> <ppc64el> <juju-core:Fix Committed by gz> <https://launchpad.net/bugs/1561023>00:19
thumperrick_h_: hey, wondering if you wanted to chat about maas2 or not00:20
thumperrick_h_: I forwarded you my sentiments00:20
rick_h_thumper: yea, saw that while at the boy's school. Haven't given it a careful read yet.00:22
thumperrick_h_: so... chat or not?00:22
rick_h_thumper: sure00:22
thumperhttps://plus.google.com/hangouts/_/canonical.com/tim-rick?authuser=100:22
wallyworldaxw_: we should probably land that rename Server->ResolvedAPIEndpoints so it is locked in00:38
anastasiamacwallyworld: axw_: so I want to have 2 controllers, each with a hosted model00:43
anastasiamacwallyworld: axw_: so that models in both have the same name00:43
wallyworldanastasiamac: juju now creates a hosted model when bootstrapping00:43
anastasiamacwallyworld: axw_: (to see if I can kill one and still connect to the other)...00:43
wallyworldit defaults to the name "default"00:43
wallyworldimaginative right00:43
wallyworldbut you can also use the --default-model <name> arg00:44
anastasiamacwallyworld: exactly what I need!00:44
wallyworldthe controller model is always called "admin"00:44
anastasiamacwallyworld: but I cannot really add more models right as we r updating create-model?00:44
wallyworldbut you need latest master00:44
wallyworldno, you can also use create model00:44
anastasiamacyep, the tip :D00:44
wallyworldcreate model has been enhance00:44
wallyworldit now no longer *requires* credentials if you are admin00:45
wallyworldif you are admin the new model inherits the credentials from the admin model00:45
anastasiamacwallyworld: reall?! even better :D00:45
wallyworldbut you can also specify different ones00:45
anastasiamacwallyworld: i'll give it a spin right now! tyvm00:46
wallyworldyou used to *have* to always specify credentials each time00:46
wallyworldand non admons still need to00:46
menn0thumper: this is something that came out of a discussion I had with fwereade this morning: https://github.com/nylas/stress-tester01:05
menn0not that :)01:05
menn0this: http://reviews.vapour.ws/r/4332/01:05
menn0thumper: ^01:06
thumperoh hai01:06
menn0wow, that means I hadn't copied anything into the clipboard since last night :)01:06
thumpermenn0: a question01:09
menn0thumper: yes?01:10
thumperon the pr01:10
thumperwell, review01:10
menn0ok01:10
menn0thumper: responded01:12
thumpershipit01:12
menn0cheers01:12
wallyworldanastasiamac: what's the status of bug 153679201:15
mupBug #1536792: Some providers release wrong resources when destroying hosted models <juju-core:In Progress by anastasia-macmood> <https://launchpad.net/bugs/1536792>01:15
anastasiamacwallyworld: this is what m testing now... I think it's all good but need to confirm01:15
anastasiamacI suspect most of it would have been fixed with tag fixes in joyent :D01:16
wallyworldanastasiamac: awesome, i'll mark as resolved in the release notes01:16
anastasiamacwallyworld: not yet01:16
wallyworldi'm optimistic01:16
anastasiamacwallyworld: i'll mark as resolved when m 100% sure01:16
anastasiamacgimme an hr or two? :P01:16
anastasiamacwallyworld: i have 2 controllers, both with default models01:30
anastasiamaci have added a machine to both, so each default model has a machine 001:30
anastasiamacnow i try to01:30
anastasiamacjuju remove-machine 0 -m=default01:30
anastasiamacERROR model local.tags:admin@local:=default not found01:30
anastasiamacif i do not set -m, the command runs fine01:30
anastasiamacbut I can still see the machine in status :(01:30
wallyworldok, we'll need to test and see if there's an issue01:31
wallyworldi wonder if it is getting confused because of two model names the same on different controllers01:31
mupBug #1558333 changed: juju's logging the literal "$cmd" instead of value of $cmd <juju-log> <logging> <juju-core:Invalid> <https://launchpad.net/bugs/1558333>01:31
anastasiamacoh and m on joyent.. so it could be that joyent resources, the bug u quoted aboce is not quiet there yet..01:31
wallyworldaxw_: have you seen the above issue recently? ^^^^^^01:32
anastasiamacmachine in nether controller is removed...01:32
anastasiamaclet me kill one controller...01:32
anastasiamachowever, mayb is non-issue as both machines are shown as "pending" still..01:34
anastasiamacso i think that this a joyent issue...01:39
anastasiamaci've killed one controller01:39
anastasiamacand now, i can still list-controllers01:39
anastasiamacbut both status and list-models hangs..01:39
wallyworldit may be that the current controller is still set to the one killed?01:39
anastasiamacwallyworld: so please do not mark this bug as resolved - it clearly isnt01:39
anastasiamacno, i've switched controllers once one was successfully killed!01:39
wallyworlddepends - may just be joyent01:39
wallyworldwe can put the caveart on there01:40
wallyworldwe'll need to test with other providers to be sure01:40
anastasiamacwallyworld: i will assume it's joyen and will work on this bug 153679201:40
mupBug #1536792: Some providers release wrong resources when destroying hosted models <juju-core:In Progress by anastasia-macmood> <https://launchpad.net/bugs/1536792>01:40
wallyworldok, tyvm01:40
wallyworldmaybe worth a test on aws to be sure01:40
wallyworldin case we need to fix modelcmd or something01:40
anastasiamack. i'll run the same on aws just to make sure01:42
wallyworldaxw_: ping?01:53
thumperwallyworld: well... in a relatively short time I have a vmaas setup with one machine comissioned01:55
thumperwith xenial and maas201:55
wallyworldawesome sauce01:55
wallyworldthumper: uses those notes?01:55
thumperI annotated the doc with my findings01:55
thumperyeah01:55
thumperpretty trivial01:56
wallyworldthumper: well that's another thing - you can't annotate bloody videos01:56
thumper:)01:56
wallyworldi'll have to do the same01:56
wallyworldanastasiamac: probably already on your todo list - i've add a heading in What's New, can you remember to add a section further down with details? New Openstack machines can be provisioned based on virtualisation type02:04
anastasiamacwallyworld: sure, so r release notes ready from ur perspective?02:10
wallyworldmore or less02:10
wallyworldstill need to do a little more02:10
anastasiamacwallyworld: excellent. tyvm02:10
mupBug #1561293 opened: remove machine model flag <juju-core:New> <https://launchpad.net/bugs/1561293>02:43
wallyworldaxw: hey02:57
axwwallyworld: hey03:00
wallyworldaxw: there's a section already added to release notes on CLI login (for SSO). can you add the local macaroon stuff there or in a subsection as appropriate?03:00
axwwallyworld: got the link to notes handy?03:00
wallyworldhttps://docs.google.com/document/d/1ID-r22-UIjl00UY_URXQo_vJNdRPqmSNv7vP8HI_E5U/edit#03:00
wallyworldaxw: also see if i've forgotten something. we've added a lot03:01
axwwallyworld: sure03:01
wallyworldaxw: lgtm also, much nicer, ty03:04
axwwallyworld: thanks03:04
wallyworldaxw: did you want to do that rename as a drive by?03:04
wallyworldso it lands together in one go03:04
axwwallyworld: can do03:04
wallyworldwould be good i think03:05
wallyworldone less thing to worry about later03:05
wallyworldaxw: and if you land soon, it will get included in the next CI run, current one looking ok so far except that log rotation still broken03:07
axwwallyworld: ok. can you please take a look at the paragraph I added to "command-line login"?03:08
wallyworldsure03:08
wallyworldaxw: lgtm, ty03:09
axwwallyworld: I forget what the suggestion was. UnresolvedAPIAddresses and APIAddresses?03:10
wallyworldResolvedAPIEndpoints (instead of Servers) I think03:10
wallyworldresolved-api-endpoints for yaml03:10
axwwallyworld: Servers is the unresolved set03:10
wallyworldand then we also still have api-endpoints for the ip addresses03:10
wallyworldoh, did i get that wrong way around03:11
axwwallyworld: I think I'd rather say Unresolved for that, since it's the one that should typically not be used03:11
wallyworldsgtm03:11
wallyworldyep, just re-read my notes, i got it backwards03:12
mupBug #1561293 changed: remove machine model flag <juju-core:Invalid> <https://launchpad.net/bugs/1561293>03:14
mupBug #1561300 opened: juju debug-log does honor current model aside from the admin model <conjure> <juju-core:New> <https://launchpad.net/bugs/1561300>03:14
axwwallyworld: we currently render both servers and api-endpoints in show-controller. do you think we should remove the unresolved ones?03:25
axwI don't think they have any value to the user03:25
wallyworldaxw: we can do and put them back if there's complaints, now is the time03:25
axwwallyworld: ok, doing that now03:26
mupBug #1561315 opened: Ensure availability uses wrong constraints <juju-core:Triaged> <https://launchpad.net/bugs/1561315>03:47
axwwallyworld: can you take a quick look at the PR again?03:53
axwplease03:53
wallyworldyup03:53
wallyworldaxw: lgtm, let's hope we beat the next CI run :-)03:58
=== Spads_ is now known as Spads
wallyworldaxw: did you see the test failure?04:30
axwwallyworld: went to make lunch, not yet04:56
axwdoh04:56
wallyworldnp, list a small one04:56
wallyworldjust04:56
axwwallyworld: there's new simplestreams in the works for azure already? does it include windows and centos?05:02
wallyworldaxw: yup05:02
wallyworldnot far off being ready05:02
axwwallyworld: ubuntu too?05:02
wallyworlddidn't want to distract :)05:02
wallyworldaxw: so yeah, believe so. the windows and centos streams will be maintained by us on streams.canonical.com, the ununtu streams on cloud-images05:03
wallyworldsimplestreams search will look in 2 search paths05:03
axwwallyworld: ok05:03
wallyworldaxw: i added the extra search path and signing key back in dec at oakland05:04
wallyworldhas taken till now to get the streams done05:04
* axw nods05:04
wallyworldall needs to be tested etc05:04
wallyworldmight be issues, who knows05:04
davecheneyallwatcher_internal_test.go:3066: tw.c.Assert(tw.NumDeltas(), jc.GreaterThan, 0)05:06
davecheney... obtained int = 005:06
davecheney... expected int = 005:06
davecheneygolf clap05:06
mupBug #1561339 opened: environs/sync: test failure <juju-core:New> <https://launchpad.net/bugs/1561339>05:14
davecheneymenn0: https://github.com/juju/juju/pull/488705:32
davecheney^ urgent05:32
davecheneyaxw: https://github.com/juju/juju/pull/488705:39
axwdavecheney: LGTM05:46
davecheneydanka, I'll hulk smash that in05:47
davecheneyaxw: as mwhudson noted, Juju's compiler breaking bug came early this development season05:48
axwheh :)05:48
davecheneyfwiw, I don't think working around that bug made the code any worse05:49
davecheneyin fact, I think it made it better05:49
axwdavecheney: agreed05:49
davecheneybut I make no comment on the name 'processedStatus'05:49
davecheneyi was going to change to to just status05:49
davecheneybut then we'd have shit like05:49
davecheneystatus.Status.Status =05:49
axwheh05:49
davecheneyand I couldn't see through the tears to keep typing05:50
axwjuju/juju/juju/Status.Status.Status05:50
mwhudsongo 1.6 is in trusty-proposed05:50
mwhudsonhttps://launchpad.net/ubuntu/+source/golang-1.6/1.6-0ubuntu1~14.0405:51
davecheney16:50 < axw> juju/juju/juju/Status.Status.Status05:51
axwmwhudson: woohoo, thank you :)05:51
davecheneymwhudson: fantastic05:52
davecheneyI think that deserves a toot of your horn on juju-dev@05:52
mwhudsonone thing to note is that this package doesn't install /usr/bin/go05:55
mwhudson(rather it's /usr/lib/go-1.6/bin/go)05:55
mwhudsonah yeah05:56
wallyworldaxw: doh, next CI run started without your change05:58
axwindeed05:59
wallyworldanastasiamac: my theory is correct06:00
wallyworldhttps://apidocs.joyent.com/cloudapi/06:00
anastasiamac\o/06:00
anastasiamacm fixing now :D06:00
wallyworldaxw: the reason joyent is farked up - we need to prefix tags with "tag."06:00
wallyworldwe pass in tags from ResourceTags()06:00
wallyworldeg model uuid etc06:00
axwah, so it is a special prefix06:00
wallyworldappears so06:00
axwokey dokey06:00
wallyworldi just had an educated guess, i think it's corretc06:01
wallyworldwe'll find out soon enogu, but the api doc appears to confirm it06:01
axwwallyworld: yeah, `An arbitrary set of tags can be set at provision time, but they must be prefixed with "tag."`06:01
wallyworldaxw: but what i did see was an extraordinary long time between instance start up and the log entry "spaces discovery complete, client connections now allowed"06:02
wallyworld10 minutes or thereabouts06:02
axwyikes06:02
wallyworldand also worker machines not able to talk back to start server06:03
wallyworldbut one thing at a time - we'll get the tags fixed first06:03
wallyworldanastasiamac: looks like it worked :-)06:15
anastasiamacyes.. m just running couple of other tests a nd then that's it!06:15
davecheney16:55 < mwhudson> one thing to note is that this package doesn't install /usr/bin/go06:22
davecheney16:55 < mwhudson> (rather it's /usr/lib/go-1.6/bin/go)06:22
davecheney^ this is a good thing, it's why we have update-alternatives06:22
mwhudsoni guess you can use that too06:23
mwhudsonthe package could even include an alternative, but it doesn't06:23
mwhudsonalternatives are a bit iffy for things that end up as build-depends06:23
mwhudson(alternatives for go went away in xenial yesterday)06:24
mupBug #1561375 opened: state: TestRegisterNoSecretKey unreliable <juju-core:New> <https://launchpad.net/bugs/1561375>08:20
dooferladvoidspace: sorry, just noticed the time. On my way to the hangout.10:04
voidspacedooferlad: me too10:04
voidspaceno babbageclunk yet10:04
fwereadevoidspace, he's in there already10:04
voidspacefwereade: ah, just not on irc10:05
voidspacebabbageclunk: hey, hii10:23
voidspacebabbageclunk: so you're on maas 2.0 for the morning then10:23
babbageclunkvoidspace: yup yup10:23
babbageclunkvoidspace: might drop off the standup then10:24
voidspacebabbageclunk: yeah10:24
voidspacebabbageclunk: my merge of master onto the "drop-1.8" branch landed and I'm now landing that onto our maas2 branch10:25
voidspacebabbageclunk: which we still haven't done anything with yet...10:25
babbageclunkvoidspace: should be an easy merge then!10:30
voidspacebabbageclunk: :-)10:35
mgzso, we nearly got a blessed master, one test failure off. then admin-controller-model merged.10:39
voidspacemgz: I had a load of "unknown" failures on my drop-1.8-support branch, all seemingly joyent related10:56
voidspacemgz: http://reports.vapour.ws/releases/381110:56
voidspacemgz: plus a couple of test timeouts and a couple with known bug numbers10:57
voidspacemgz: is the joyent problem just a spurious thing, known, or something else? (do you think)10:57
rogpeppe1does anyone here understand how the new model/controller/account hierarchy works?10:57
mgzvoidspace: what version of master did that branch last merge... ah, my rev, so includes admin-controller10:59
voidspacemgz: ah, they're all failing on master as well11:02
voidspacemgz: so my branch has consistently been "no worse than master"... :-)11:03
mgzvoidspace: three things11:03
mgzvoidspace: one, you have the new regressions from master (timeout etc across a lot of tests)11:03
mgzvoidspace: two, you mis-resolved conflicts in dependencies.tsv and dropped a bug fix11:03
voidspacemgz: misresolved dependencies.tsv? I didn't resolve a conflict there11:04
voidspacemgz: and dropped a bug fix - you mean calling the test setup twice was a bug fix?11:04
mgzvoidspace: three, the joyent config is borked for reasons unclear11:04
voidspaceunless that was a previous conflict resolution11:05
mgzvoidspace: well, it's not clear how exactly, and talking in shas makes life complicated11:05
voidspacemgz: can you point me to the dependencies.tsv issue11:05
mgzbut the rev you merged includes the fix for bug 156102311:05
mupBug #1561023: charmstore v5.WillIncludeMetadata gccgo build failure <ci> <gccgo> <ppc64el> <juju-core:Fix Committed by gz> <https://launchpad.net/bugs/1561023>11:05
voidspacemgz: also the bugfix I dropped - I'll fix those11:05
mgzbut the dependencies.tsv in your branch at that rev doesn't have the charmstore dep bump11:05
mgzso... something happened11:06
mgzit's gone on master11:06
mgzI wonder if the joyent cred issue is similar?11:06
voidspaceI'm tempted to blame git, until someone can prove me wrong...11:07
mgzvoidspace: well, it's certainly not wrong to blame git11:07
voidspace:-)11:07
voidspacemgz: I do have that old version in dependencies.tsv - but I don't have anything that would have conflicted with it11:08
voidspacemgz: that's an easy fix though11:08
voidspacemgz: you said screwed up dependencies.tsv *and* dropped a bug fix11:08
voidspacemgz: was that two things or one thing?11:08
mgzvoidspace: merging that branch into master produces a sane diff at least11:09
mgzso, nothing to borked happened git-wise11:09
mgzvoidspace: one thing, and I'm not actually sure what's up11:10
voidspacemgz: yeah, I just wonder how that dependencies.tsv line got reverted11:10
voidspacemgz: I'll update it manually in my branch - it's just concerning if things are missing11:10
voidspacemgz: the diff doesn't show it as a change against master, so it looks like it just hasn't been pulled into that branch11:11
mgzvoidspace: the rev tested doesn't include the change from 1abf825dcb71f198c3895e24fecc99537454c64e ... which predates rev 9e2a02b17a92c90b524b33938b5b32bb74451538 which... aha11:11
mgzvoidspace: is *not* the rev you merged11:11
mgzvoidspace: so actually, just merging master again would resolve that, and likely the joyent creds issue11:12
voidspacemgz: right, and pull in new problems instead :-)11:12
voidspacemgz: maybe I'll wait until there's a bless and pull that in11:12
mgzvoidspace: that would be reasonable11:12
voidspacethe timeouts are worrying but I don't *think* they're consistent against test runs on my branch11:12
voidspaceI'll check11:12
voidspaceif they are I'll need to look at them11:13
mgzI certainly don't see any failures that are obviously from changes in that branch11:13
voidspacethe changes are all in the maas provider so it would be "interesting" if it caused failures elsewhere11:14
wallyworldvoidspace: what joyent creds issue are you having?11:21
mgzwallyworld: ERROR cmd supercommand.go:448 validating "credentials" credential for cloud "joyent": manta-user: expected string, got nothing11:23
wallyworldand btw joyent has been quite sick - they were upgrading their data centre, and now we have issue which are still being diagnosed, maybe network. one instance took 10 or more minutes from agent start up to get the "maas spaces all discovered" message11:23
wallyworldnetwork spaces i mean11:23
wallyworldmgz: you need to remove the manta items11:23
wallyworldthey are no longer needed11:23
voidspacewallyworld: thanks11:23
wallyworldwe added the instructions to the release notes, but have not advertised yet11:24
wallyworlddidn't think anyne besides Qa was using joyent :-)11:24
mgzwallyworld: this is just a sync issue between different versions of the code and those changes11:25
wallyworldah ffs {\\\"code\\\":\\\"NotAuthorized\\\",\\\"message\\\":\\\"QuotaExceeded:11:25
wallyworldthat's the latest reason for joyent failures11:25
wallyworldsigh, we're having no luck11:25
mgzwallyworld: yeah, that's some of them11:25
wallyworldmgz: the joyent provider was also misusing tags11:26
wallyworldthat was done in master a while back i think11:26
wallyworldfixed in latest CI run11:26
wallyworldthe machines were not being tagged properly with uuid11:27
wallyworldso ControllerInstances() was all messed up11:27
wallyworldas was bootstrap finalisation11:27
babbageclunkvoidspace: ok, I've got a maas2 cluster going, I think. I might try doing the setup to get power addresses working so that I can try dooferlad's script11:30
babbageclunkvoidspace: unless there's something else I should do?11:30
wallyworldmgz: voidspace: i sent an email - we have a routing issue with joyent machines, nfi what's wrong11:48
mgzwallyworld: thank you11:50
wallyworldmgz: any ideas welcome :-)11:50
mgzwallyworld: did you also look at the windows test run?11:51
wallyworldthe unit test failure?11:51
mgzwith admin-controller it's now hitting timeouts11:51
wallyworldi have only seen one test failure in the builds i have looked at, and it says there's an existing file handles bug11:52
wallyworldi'll look at the latest run11:52
mgzwallyworld: that does sometime happen anyway, but last three runs have all hit11:53
mgz*** Test killed: ran too long (10m0s).11:53
mgzFAILgithub.com/juju/juju/api609.327s11:53
wallyworldmgz: nothing has changed inthe api package at all lately, and the last time in the admin comtroller branch it was the file hadnles bug11:54
wallyworldi'll look at the logs though, maybe something changed i don't know about11:55
wallyworldbut i would have expected the same pass or fail now in master as in the latest admin controller runs11:55
mgzwallyworld: also http://reports.vapour.ws/releases/3813/job/run-unit-tests-race/attempt/1216#highlight11:55
wallyworldas what was run before the merge into master11:55
wallyworldi have not looked at the races at all11:56
wallyworldthey have been happening for a while in master, have not been ob the radwr for us11:56
mgzwallyworld: http://reports.vapour.ws/releases/3810 <- this is my baseline11:57
mgzwallyworld: that's just before admin-controller-model merged, followed by a trivial win fix11:57
wallyworldmgz: right, but in the latest admin controller model runs we are not seeing the same issues11:57
wallyworldthe only windows test failure i recall was the file handle one'11:58
mgzcompare 3812 3813 3814 with the merge in11:58
wallyworldmgz: here's the latest admin controller run http://reports.vapour.ws/releases/380911:58
wallyworldall known issues from master11:58
mgzwhich all timeout on the windows tests, and have a new data race11:59
wallyworld?11:59
wallyworldbug 1521699 is not a tineout11:59
mupBug #1521699: windows unit tests fail because handles are not available <ci> <intermittent-failure> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1521699>11:59
wallyworldthat's the only windows test failure11:59
wallyworldin that last admin controller run12:00
wallyworldwhat are you referring to when you say timeout?12:00
wallyworldso just before we merged admin controller into master - we got a clean run apart from a few known inherited issues, so we got agreement from qa team to nerge12:01
wallyworldone of the issues is a know CI script issue - the log rotation one12:01
mgzwallyworld: http://juju-ci.vapour.ws/job/run-unit-tests-win2012-amd64/buildTimeTrend12:01
wallyworldthe azure arm deploy is also known - an lxc issue on xenial12:02
mgzmaster was failing (with one small issue) in 30 mins12:02
mgzit's now 2 hrs12:02
mgzwallyworld: anyway, lunch...12:02
wallyworldright so we need to root cause which it is wrong to straight away blame admin controller model when that branch was clean12:03
wallyworldbefore merging12:03
wallyworldmgz: is the machine under load perhaps, there's things like this Panic: cannot create index for logs collection: WSARecv tcp 127.0.0.1:55607: i/o timeout (PC=0x41597B)12:21
wallyworldthat's quite low level, can't immediately see the issue is caused by juju12:21
voidspacebabbageclunk: ah sorry, missed your tweet12:39
voidspacebabbageclunk: where are you at now?12:39
babbageclunkvoidspace: no worries, unless you're going to say "Nooooo, you should have been working on something else!"12:39
babbageclunkvoidspace: juuuuust about got the last piece of the power management setup done - will try using the kvm_maas script to add nodes.12:40
voidspacebabbageclunk: cool12:41
voidspacebabbageclunk: I'm not sure how much you can help on the API spelunking stuff I'm doing12:41
voidspacebabbageclunk: getting some stuff deployed with juju on maas 1.9  might be useful12:42
voidspacebabbageclunk: get familiar with the juju command line and concepts12:42
babbageclunkOk, I'll start on that once I do this and get it written up.12:43
babbageclunkvoidspace: (Also, just about to pop out and pick up some monitors.)12:43
voidspacebabbageclunk: ok, cool12:49
mupBug #1561526 opened: api/usermanager: no way to find if a user doesn't exist <juju-core:New> <https://launchpad.net/bugs/1561526>13:27
cheryljmorning, everyone.13:46
* cherylj reads backscroll13:46
mgzcherylj: wotcha13:47
cheryljhey mgz, how are things?  :)13:48
cheryljso ,looks like we need a bug for the github.com/juju/juju/worker/environ data race failure13:48
cheryljand we need to get someone on the stringforwarder failure for ppc13:49
cheryljand someone on the windows test timeouts13:50
cheryljand the joyent networking stuff13:50
cheryljI'll open a bug for the environ data race, unless you already did mgz ?13:51
mgzcherylj: not yet13:51
rogpeppe1anyone know of a handy mongodb key sanitizer function? nice if it was reversible.13:51
cheryljk, I'll do it now13:51
cheryljkatco: do you have someone who can look at bug 1560203?13:51
mupBug #1560203: stringForwarderSuite.TestRace sometimes fails <ci> <intermittent-failure> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1560203>13:52
cheryljkatco: right now it's blocking our ability to release13:52
cheryljwe seem to hit it every time13:52
mgzah, 3814 is done now, ace13:52
cheryljblergh, katco is out13:53
cheryljnatefinch, ericsnow, can either of you take bug 1560203?13:53
mupBug #1560203: stringForwarderSuite.TestRace sometimes fails <ci> <intermittent-failure> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1560203>13:53
mgzgood to bug jam on that too as it's his code :)13:53
cheryljcmars: can you spare people to help us get the release out?13:54
mgzI couldn't get the test to fail on its own, but seems as part of a full run on ppc64 it comes up a lot13:54
natefinchcherylj: katco is out today and tomorrow btw13:54
mgzso I tuink t13:54
mgz*think the 'race' assumptions are just not conservative enough13:54
natefinchcherylj: oops, sorry, I see you realized that already13:56
cherylj:)13:56
mgzcherylj: the dumb option is just skip the test13:56
cheryljhey dooferlad, how are things going with the proxy bug?  (I'm sure it will come up in the cross team call this morning)13:57
natefinchcherylj: that's some gnarly code13:58
dooferladcherylj: moving along. Have had some useful discussions with fwereade on how to fix it properly and have some code that I am reasonably happy with.13:58
cheryljdooferlad: excellent.  Is early next week still a sane target?13:58
cheryljlike Monday / Tuesday?13:58
mgzcherylj: 3814/joyent-deployer-bundle failed due to routing issues when trying to fetch tools from the state server13:59
dooferladcherylj: yes13:59
natefinchgah, why is this code under juju/juju?13:59
mgzcould be our manual firewall cleanup rules not working any more?13:59
dooferladcherylj: though remember that Friday and Monday are public holidays (at least in the UK)13:59
cheryljdooferlad: d'oh, that's right13:59
cheryljthanks :)14:00
dooferladcherylj: no problem :)14:00
cheryljdooferlad: you'll want to touch base with mgz to discuss CI testing with proxies for 1.25.514:00
cheryljdooferlad: make sure that we can get a test set up for it14:00
mgzyeah, I've looked at that a little this week14:00
dooferladcherylj: sure.14:00
cheryljmuchas gracias14:01
cheryljnatefinch: are you looking at that stringforwarder bug?14:01
natefinchcherylj: sort of.14:01
cheryljheh14:01
natefinchcherylj: both my managers are out, so I'm a little unclear on whether or not I should take this bug over what I was otherwise working on14:02
natefinchcherylj: on the upside (for you), it looks interesting and the code seems to need some cleanup, so it makes me want to fix it.14:03
cheryljsweet!14:03
rick_h_natefinch: what are you working on today?14:04
natefinchrick_h_: getting the resources juju code to actually support channels14:05
natefinchrick_h_: it's a "bug" that we don't :D14:06
alexisbnatefinch, today we need help with bugs14:07
alexisbeveryone14:07
rick_h_natefinch: ok, +1 ^14:07
natefinchalexisb: ok, thanks for the clarification14:07
rick_h_:)14:07
cheryljI'm making the bugs that are currently blocking us actual blockers, so they'll show up on juju.fail14:08
natefinchjuju.fail is probably my favoritest thing that marcoceppi has ever done :)14:12
cheryljyeah, it's nice :)14:12
marcoceppinatefinch: I also have juju.qa - still waiting for a good idea to come to me14:12
mgzcherylj: I'm unsure what to do over the win2012 unit test timeout,14:13
mgzian is right that the last run of the feature branch didn't have issues14:13
mgzbut nothing else in master particularly affects that14:13
cheryljmgz: the last run of the feature branch may have been masking it14:13
mgzcherylj: my best bet may be restart the windows slave and rerun?14:13
cheryljbecause of the setuptest14:13
mgzthat's one small test later in the process than where the timeout happens14:14
cheryljI'm suspicious of this because of the change could have brought in, namely around utils.OutgoingAccessAllowed14:14
cheryljchange it could have brought in14:15
natefinchOh man, that is so confusing. StringForwarder has a Receive() method that actually sends a message. :/14:17
cheryljlol14:17
natefinchI think I'll rename it "Forward"... since that's what it's doing14:18
cheryljbogdanteleaga: would you be able to help with bug 1561566?14:21
mupBug #1561566: Many windows tests fail with "WSARecv tcp 127.0.0.1:53731: i/o timeout (PC=0x41593B)" <blocker> <ci> <juju-core:Triaged> <https://launchpad.net/bugs/1561566>14:21
alexisbericsnow, ping14:26
bogdanteleagacherylj, looks really weird, but I have a feeling I've seen it before, I think it was transient though14:28
bogdanteleagaI'll ask gabriel too14:28
cheryljthanks, bogdanteleaga!14:29
mgzcherylj: I may have some good news on that14:29
cheryljoh?14:29
cheryljthat would be awesome14:29
mgzcherylj, bogdanteleaga: http://paste.ubuntu.com/15487526/14:29
bogdanteleagamgz, if those were still around you probably wanna check the tempdir folder as well14:31
mgzyeah, rm -rf $TMP/* running14:31
mgzbogdanteleaga: what do I run to restart the (cloud) machine cleanly?14:32
mupBug #1561555 opened: Data race in github.com/juju/juju/worker/environ <blocker> <ci> <juju-core:Triaged> <https://launchpad.net/bugs/1561555>14:33
mupBug #1561566 opened: Many windows tests fail with "WSARecv tcp 127.0.0.1:53731: i/o timeout (PC=0x41593B)" <blocker> <ci> <juju-core:Triaged> <https://launchpad.net/bugs/1561566>14:33
bogdanteleagamgz, shutdown -r -t 0 should do14:35
wallyworldcherylj: a small one https://github.com/juju/juju/pull/488914:37
ericsnowalexisb: pong14:39
alexisbericsnow, we have a push on bugs to get a very important beta3 out14:40
alexisbericsnow, can you please work with cherylj on any needs she may have of you today14:40
wallyworldericsnow: a +1 of this small date race fix would be great https://github.com/juju/juju/pull/488914:41
ericsnowalexisb: k14:41
natefinchwallyworld: you should go to bed.  it's after midnight on the first day of your vacation14:41
wallyworldnatefinch: soon, just one more fix :-)14:42
ericsnowalexisb: keep in mind that we still have to land the implementation of resources in the charm store in short term14:42
alexisbericsnow, yes I know14:42
ericsnowcherylj: keep me posted on how I can help14:42
natefinchalexisb: you should make wallyworld take an extra day of vacation for every hour he works past midnight, that would teach him ;)14:42
alexisbnatefinch, agreed ;)14:43
ericsnownatefinch, alexisb: lol14:43
alexisbericsnow, natefinch this is just to support beta3 over the next 2 days14:44
ericsnowwallyworld: ship-it14:44
wallyworldtyvm14:44
alexisbericsnow, natefinch I will work with katco and wallyworld on adjusting schedules with the 2 day impact14:44
alexisbbut we need to get beta3 out14:44
ericsnowalexisb: the nice thing is that the charm store patches don't have quite the same deadlines :)14:45
alexisbericsnow, yep :)14:45
alexisband sorry for the shift in priority but this one is important14:45
ericsnowalexisb: not that we can affort to be complacent about them!14:45
ericsnowalexisb: np14:45
wallyworldericsnow: ok, that should be landing, can you monitor for me? i've asked abentley to pause CI pending any more short term work to address any remaining issues; we'll need to keep him inthe loop14:46
ericsnowwallyworld: k14:46
ericsnowwallyworld: should I ping him once that's merged?14:47
abentleyericsnow: yes, please.14:47
ericsnowabentley: will do14:47
wallyworldericsnow: just need to check what else there is - mgz did we think a windows slave restart would help the windoes tests?14:47
wallyworldthere's also a potential log rotation fix14:47
mgzwallyworld: I belive so, I'm doing that now14:48
wallyworldhaven't looked at that14:48
wallyworldi think the only other issue is joyent?14:48
wallyworldand its networking issue14:48
wallyworldericsnow: mgz: it's possible TestAddUserAndRegister has an issue on windows cleaning up the logsink.log file at the end of the test. nfi why14:49
mgzwallyworld: the other big thing is joyent routing14:49
wallyworldthere's nothing special in that test14:49
wallyworldso maybe we c.Skip() on windows with a todo for now14:49
wallyworldmgz: yeah, i don't know enough to fix joyent routing issues14:50
mgzhttp://data.vapour.ws/juju-ci/products/version-3814/joyent-deploy-trusty-amd64/build-1965/machine-0.log.gz14:50
mgzthat's not actually the most helpful14:52
mgzbasically... the provisioned machines can't talk back to admin:machine-0 to get tools14:52
mgzthis is similar to what we added hack firewall cleanup rules to work around14:53
wallyworldmgz: yep, but only if they use the cloud address14:53
mgzit's possible those hacks need updating to the new world of admin controllers14:53
wallyworldthe public address is fine14:53
wallyworldmgz: admin controller should make 0 difference14:53
wallyworldit's just a controller14:53
mgzthe hack being:14:54
mgz$RELEASE_TOOLS/joyent-curl.bash /cpcjoyentsupport/fwrules | sed -e 's/[\[\{]/\n\0/g;' | grep $JOB_NAME | sed -e 's/.*"id":"\([^"]*\)".*/\1/' | xargs -I{} $RELEASE_TOOLS/joyent-curl.bash /cpcjoyentsupport/fwrules/{} -X DELETE || true14:54
wallyworldi'll take your word for it :-)14:54
mgzbut given all the joyent downtime and other misc fallout from the last few days it could also be something else14:54
wallyworldericsnow: mgz: one difference in that testAdduserAndRegister test is that it does a c.Assert(api.Close(), jc.ErrorIsNil) at the end whereas other tests just do a api.Close(). we could try that or just skip the test for now14:55
wallyworldbut that's just clutching at straws14:55
wallyworldtrying to find something wrong14:55
wallyworldthe test itself doesn't do anything with logsink.log14:55
wallyworldit's all setup by JujuConnSuite14:55
wallyworldunder the covers14:55
mgzwindows machine restarted. rerunning 3814 win2012 unit tests.15:06
abentleywallyworld, mgz: cherylj and I think the joyent issues are caused by the admin model and the default model being in different regions.15:14
wallyworldhmmm, that may explain it15:15
wallyworldwould need to check the code to be sure15:16
ericsnowabentley: in case you didn't notice, this merged: https://github.com/juju/juju/pull/488915:16
mgzabentley: actual different regions? not just different network zones (or whatever joyent calls that)15:16
cheryljmgz: yeah15:16
cheryljmgz: I see that my controller is in us-east-315:16
ericsnowabentley: first try too <wink>15:16
cheryljmgz: but when I deploy to the default model15:17
wallyworldAFAIR, there's no attempt to be in different regions, but also no attempt to be in the same region15:17
abentleywallyworld: Should we test now or hold for more fixes?15:17
cheryljmgz: it gets put in us-east-115:17
abentleywallyworld: We specify region in the bootstrap config, so we'd expect the same one to be used for both.15:17
cheryljmgz, abentley, deploying a machine in the admin model properly puts the machine in the same region as the controller15:18
wallyworldabentley: depends on whether that is passed through, i'd need to check. but have we looked at the joyent instances to confirm?15:18
cheryljwallyworld: yes15:18
wallyworldso seems like we need to explicitly constrain the region of any hsted models15:19
cheryljwallyworld: yeah, seems to be15:19
wallyworldi'd need to check the code15:19
wallyworldcan't recall exactly the setup behaviour ottomh15:19
cheryljwallyworld: I can take this, go to bed!15:19
wallyworldlet me take a quick look15:20
wallyworldabentley: maybe just hold off on a new run for  bit longer15:20
abentleywallyworld: Okay, let me know.15:20
abentleywallyworld: The functional-log-rotation-unit issue also seems to be the same joyent issue.  I re-ran it on AWS and it passed.15:21
wallyworldabentley: oh good :-)15:21
abentleycherylj: ^^15:21
wallyworldso really one more issue15:21
cheryljyay15:21
abentleywallyworld: Does this cross-region apply to other providers?  We saw a bunch of admin machines all by themselves in aws earlier this week.15:27
abentleyMaybe their default model machines were in a different region.15:27
cheryljahhhh, I can check on AWS15:27
cheryljthat reminds me I need to check my dashboard for other regions!!15:28
wallyworldabentley: not sure, from what i can see, the region for joyent comes from the sdc url which is the same for both admin and hosted15:28
cheryljwhat's the default amazon region?15:28
wallyworldus-east-115:28
abentleywallyworld: On bootstrap, we're not allowed to pass in sdc-url.  Only region.15:29
wallyworldabentley: SDC-URL COMES FROM CREDENRIALS15:29
wallyworldcaps lock fail15:29
wallyworldsorry15:29
ericsnownatefinch: if you're working on bug #1560203, would you mind assigning it to yourself?15:29
cheryljI was like woah!15:29
mupBug #1560203: stringForwarderSuite.TestRace sometimes fails <blocker> <ci> <intermittent-failure> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1560203>15:29
cheryljdidn't know you felt so strongly about SDC-URL15:29
wallyworldlol15:29
wallyworld'A' key too close to caps lock15:29
natefinchericsnow: oops, yep, thanks15:30
cheryljlol15:30
natefinchwallyworld: you should remap caps lock to something useful... I made it the cmopose key so I can easily make things like ™ :)15:32
wallyworld:-)15:32
wallyworldcherylj: https://github.com/juju/juju/blob/master/provider/joyent/environ_instance.go#L9915:33
wallyworldthe region for start instance appears to be the same for all models based on https://github.com/juju/juju/blob/master/provider/joyent/config.go#L17015:34
wallyworldand sdc-url comes from credentials15:35
wallyworldwhich is the same for both admin and default model15:35
wallyworldwith create-model  the user can pass in different credentials so could shoot themselves in the foot15:35
natefinchahhh.... stupid gocheck15:36
natefinchalways put your func TestPackage(t *testing.T) { gc.TestingT(t) } in the internal package, not _test package, otherwise it doesn't actually run any internal tests :/15:37
natefinchsome might call that a feature ;)15:37
wallyworldcherylj: so it seems confusing based on the above how machines in hosted models can be in different regions for joyent15:38
cheryljwallyworld: I'm not specifying sdc-url and I see this ssue15:38
wallyworldcherylj: let me see where sdc-url comes from15:39
cheryljwallyworld:  I'm using bootstrap joyent joyent/us-east-315:39
wallyworldcherylj: right, so it comes from the clouds.yaml file; will be the same for all models15:40
wallyworldso i don't see off hand how machines get created in different regions15:41
cheryljThere's nothing in my clouds.yaml for joyent15:41
wallyworldcherylj: fallback-public-clouds.yaml15:41
wallyworldin the code base15:41
cheryljah15:42
cheryljwallyworld: I'll keep looking into this15:42
wallyworldok, it annoying me that the regions are different and they shouldn't be15:43
wallyworldaccording to the code as i see it, but i need sleep15:43
wallyworldi'll find out i guess at the release standup :-)15:43
wallyworldcherylj: the only other thing is to consider skipping that 1 failing test on windows as per backscroll a bit ago15:44
wallyworldcherylj: func (joyentProvider) RestrictedConfigAttributes() []string {15:47
wallyworldadd sdc-utl15:47
wallyworldsdc-url15:47
wallyworldto the result15:47
wallyworldthat should fix it15:47
wallyworldthat will force histed model config to share the sdc-utl value15:48
wallyworldwith the admin model15:48
cheryljnice, thanks wallyworld15:48
wallyworldyou may need to add additional ones, i think that's all that's needed15:49
cheryljok, will test15:49
wallyworldtl;dr; - add any attrs there that must be duplicated between admind and host model15:49
wallyworldhosted15:49
wallyworldcherylj: eg for cloudsigma, the result is return []string{"region"}15:50
wallyworldso for joyent return []string{"sdc-url"}15:50
wallyworldhopefully a one line fix :-D15:50
cheryljawesome, thanks :)15:50
wallyworldyep, also "region" for ec2 :-)15:51
wallyworldjoyent provider is the red headed step child15:51
wallyworldthat no one loves15:51
rick_h_wallyworld: fyi, cancelled our call. Obviously you're here late and off so want to make sure you don't expect to have it :)15:52
wallyworldrick_h_: i don't mind, i need to be up 30 miuntes before hand for the relese standup15:52
wallyworldhappy to still have it15:52
rick_h_wallyworld: nope, I'm not :P15:52
wallyworldrick_h_: ok, hopefuly my status email filled you in15:53
rick_h_wallyworld: rgr15:53
rick_h_wallyworld: email me if you need anything15:53
wallyworldwill do15:53
wallyworldi am happy i think we got joyent sorted out15:53
wallyworldbeta3 will rock15:53
wallyworldttyl15:54
wallyworldabentley: one last thing before i go - we think there's a one line fix for joyent - cherylj will ping you when  landed15:56
redirmorning15:59
natefinchsomeday reviewboard will pick this up: https://github.com/juju/juju/pull/489016:05
natefinchif anyone wants to review it16:05
abentleycherylj: Here is a list of attrs that may occur in environments.yaml that we have blacklisted from the bootstrap config in 2.0: https://pastebin.canonical.com/152723/16:10
cheryljericsnow: can you review natefinch's PR?  https://github.com/juju/juju/pull/489016:15
ericsnowcherylj: will do16:15
jammgz: bug me about what?16:15
mupBug #1561611 opened: Joyent machines deployed to hosted models use wrong region <blocker> <ci> <juju-core:Triaged by cherylj> <https://launchpad.net/bugs/1561611>16:16
mgzjam: look at nate's pr ^16:16
mgzbug you about that16:16
cheryljmgz: I see the windows tests are still failing :(16:16
mgzcherylj: well, good news bad news...16:16
mgzright, that16:16
mgzgo tests to run again, still super unreliable compared to before on master16:17
mgztwo things under apiserver hitting 600s timeout, strongly suggests actual deadlock16:17
mgzbogdanteleaga: can you normally get a clean test run locally? if so, can you try with current master?16:18
jamnatefinch: so the first thing that jumps out at me is that if you call 'Stop()' twice you'll get a panic. Its often hostile to not make cleanup actions reentrant16:19
natefinchjam: ahh you're here, great16:21
bogdanteleagamgz, I stopped trying to run the whole thing on windows a while ago, they seem to be way slower on windows16:21
natefinchjam: I wanted to talk to you about my change, but figured you were out16:21
bogdanteleagaI remember hitting something like 299.x seconds with a 300s timeout on a suite16:22
bogdanteleagabut I'll give it a shot16:22
mgzbogdanteleaga: yeah, can do it on 30 mins on big cloud machine but it's not much use for narrowing down problems16:22
mgzbogdanteleaga: apiserver seems the interestng package16:22
natefinchjam: the reason I did it is to make it more obvious that it's not thread-safe to call stop multiple times. When I first looked at the code for Stop, I figured the nil check was an attempt to make it threadsafe (which obviously it's not)16:22
natefincher not thread safe to call stop from multiple threads16:22
mgzmaster as-of before admin controller and HEAD, if they behave differently16:23
bogdanteleagamgz, I'll try head first, you're saying apiserver times out?16:23
natefinchjam: and given that we don't actually need to call stop multiple times in production, that seems ok16:23
bogdanteleagaalso what go version are you using?16:24
mgzbogdanteleaga: yeah16:24
jamnatefinch: So i think it isn't all that uncommon to want to do something like defer (Stop()), and then end up with logic that might also Stop early.16:24
jamgenerally re-entrant cleanups are going to play nicer16:24
jamI'd rather just put a Mutex in there16:24
natefinchjam: yeah, the defer and then also call stop is true16:24
mgzbogdanteleaga: urk, that thar is a reasonable question, not sure if we've updated the window build to 1.6 yet16:25
mgz# C:/go/bin/go version16:25
mgzgo version go1.2.2 windows/amd6416:25
mgzsoo... that's also on our list to get done16:26
bogdanteleaga2507 files changed, 160673 insertions(+), 117711 deletions(-)16:26
bogdanteleaga:)16:26
natefinchjam: I just kind of hate writing code for conditions that we don't actually need or care about16:26
mgzbogdanteleaga: :)16:26
bogdanteleagamgz, are the binaries used for deployment built using 1.6 though?16:27
natefinchjam: anyway, I can add the mutex if you think that's the right fix.16:28
jamnatefinch: done vs stopch is fine, New vs NewFoo is good, and I'm happy with therest16:29
jamnatefinch: yeah, most is just spelling that I'm agnostic about, I'd just like Stop() to be reentrant16:29
natefinchjam: thanks... sorry for stomping all over your code16:29
jamnatefinch: if its clearer for someone that isn't me, then its all good16:29
mgzbogdanteleaga: not yet I think, cross builds also on go 1.216:30
mgzwe're about to move all the remaining bits to go 1.6 though16:30
natefinchjam: do you agree with the changes to TestRace?  That's the one that was failing.  It seemed like checking that the goroutine stops before running out of runway was not actually the point of the test.16:35
mgzbogdanteleaga: basically, I'm fine signing off windows test failures for the release until we've done some of that upgrade work,16:35
mgzif you can confirm that master as of now is not vastly more borked than it was a few days ago16:36
cheryljjam, do you think that this PR could have anything to do with these windows failures?   https://github.com/juju/juju/pull/479816:39
voidspacealexisb: you back?16:42
jamcherylj: yes, with the caveat that it is exposing brokenness in the test suite.16:42
jamcherylj: we were doing stuff like not calling SetUpSuite16:42
jamor calling PatchValue before calling SetUpTest16:43
jamwhich would cause the patched value to never be cleaned up.16:43
jamcherylj: IIRC mgz had a patch that changed at least one of them to correctly call SetUpSuite16:43
alexisbvoidspace, I am and will be free shortly16:44
alexisbwill ping16:44
voidspacealexisb: cool16:44
cheryljnatefinch, ericsnow, can I get a review? http://reviews.vapour.ws/r/4340/16:54
* ericsnow reviews16:54
ericsnowcherylj: LGTM16:55
natefinchjam, ericsnow: the reason I made loop into a stanadalone function is that it's a goroutine... it doesn't have any state its storing, and it shouldn't be implied that it is.  There's two separate concerns, a goroutine looping over channel, and a type that can send messages to that channel.  linking them to the same object is conceptually incorrect, and unnecessary.  Plus it means the goroutine has receive-only halves of done and messages, making it16:57
natefinchclear that it should not (and cannot) be the one closing them.16:57
ericsnownatefinch: my point was that it is confusing that way16:59
natefinchericsnow: hmm, ok.17:04
natefinchericsnow: my default is to make a goroutine a standalone function, since that's what it is.  I actually found it confusing that it was a method, since it didn't really need to be :)17:06
natefinchericsnow: I can hit a middle ground I think... just share the done and messages channels in the struct17:06
ericsnownatefinch: even putting that loop function inside New() as a closure would help, though jam's point about other loop methods is still correct17:07
natefinchericsnow: what was your oops comment about?17:31
ericsnownatefinch: you had pushed up some typos17:31
natefinchericsnow: oh, did I fix it?17:31
ericsnownatefinch: apparently you caught them anyway :)17:31
natefinchheh17:31
natefinchericsnow: also, the fix was actually just removing the limited for loop inside TestRace, and removing the error that happened if the goroutine went through the for loop before the test called stop17:33
ericsnownatefinch: k17:34
bogdanteleagamgz, it was successful, but it almost timed out with several packages over 550s17:46
mgzbogdanteleaga: thanks17:47
mgzcherylj: ^I'm fine signing off windows test for a beta317:47
bogdanteleagamgz, trying 1.6 now17:47
redircherylj: review please http://reviews.vapour.ws/r/4341/17:48
bogdanteleagamgz, oh and it was apiserver only :)17:48
mgzbogdanteleaga: that should do, other things timed out in CI as well but seems like we just made juju slower on windows tests17:51
natefinchW: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty-updates/main/source/Sources  Hash Sum mismatch18:02
natefinchcute18:02
natefinchteam meeting anyone?18:03
cheryljalexisb: you going to make the is call?18:03
alexisbcherylj, sorry looks like I missed it18:08
cheryljyeah, there wasn't anything to talk about anyway.  Unless you had something18:08
alexisbnope18:15
mgznatefinch: we failed...18:25
natefinchmgz: indeed18:27
=== redir is now known as redir-lunch
=== redir-lunch is now known as redir
redirericsnow, natefinch review please: http://reviews.vapour.ws/r/4341/19:37
ericsnowredir: will do19:38
ericsnowredir: done19:55
redirericsnow: tx, I'll have a couple questions -- after a reboot.20:03
ericsnowredir: k20:03
natefinchredir: I reviewed as well.20:15
natefinchericsnow: it's just you and me for the standup today, mind if we meet early?20:16
ericsnownatefinch: sounds good20:16
ericsnownatefinch: now?20:16
natefinchericsnow: yep20:18
redirholler when you're off ericsnow20:21
ericsnowredir: k20:21
alexisbcherylj, I am finding my first issue testing out beta320:49
alexisbI had a controller I created with beta220:50
alexisbtrying to kill it with beta3 fails20:50
alexisbbut when I kill it with the old beta2 it works20:50
cheryljalexisb: that doesn't surprise me20:50
cheryljthere have been problems like that between the other betas20:51
cheryljthe expectation is that you create and destroy with the same versionj20:51
alexisbwe should note it in known issues though, dont you think?20:51
alexisbor somewhere in the release notes20:51
cheryljalexisb: yeah, could say something about the different betas not guaranteed to be compatible with eachother20:55
alexisbcherylj, if you are good with saying something I can add20:56
=== natefinch is now known as natefinch-afk
cheryljalexisb: yeah, probably worth noting.  Thanks :)21:00
wallyworldalexisb: beta3 is not compatible with beta221:15
wallyworldwe make no guarantees of compatability21:16
wallyworldthe releases notes will say as much when i update them21:16
alexisbwallyworld, you are not allowed to correct me on your vacation day21:17
wallyworldalexisb: will disappear after release standup21:17
wallyworldalexisb: we changed the format ofr controllers.yaml21:18
wallyworldbetter to do to now than after releases21:18
alexisbwallyworld, agreed,  I am just trying to be a 'new' user21:19
wallyworldsure np, we just need to be clear that folks need to "start again" between betas21:19
redirAm I supposed to 'resolve' issues in review board when I fix them or leave them for the original reviewer to resolve?21:25
redirsomebody say new user?21:25
ericsnowredir: feel free to mark them as resolved21:34
redirericsnow: one outstanding21:34
ericsnowredir: if you aren't sure if you've satisfied the reviewer though, it sometimes pays to leave it alone until you get more feedback21:34
redirericsnow: your first one, I am not sure...21:35
ericsnowredir: I've dropped that one21:41
redirOK.21:41
ericsnowredir: so you should be ready to go :)21:41
redirSo then does it automatically merge?21:41
redirsorry this is my first bug21:41
redirso first time through.21:41
redirOr do I merge it myself?21:41
ericsnowredir: you have to add a comment in the PR with $$merge$$ in it21:41
redirk. tx.21:42
redirAnd the buildbots don't run the tests before the merge?21:42
* redir wonders where the build dashboard is.21:43
ericsnowredir: there's a merge bot that adds a link to the PR for the merge request and then runs the tests and does the merge for you21:49
redirawesome. so if the tests fail the merge should too. thanks ericsnow21:50
ericsnowredir: congrats on your first patch merged :)21:50
ericsnowredir: yep21:50
ericsnowredir: np21:50
redir...insert fancy ascii art in here...21:50
alexisbi would like to point out that wallyworld did not ask about mongo22:08
alexisbin the release call22:08
wallyworldalexisb: no point :-( i saw the email22:08

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!