/srv/irclogs.ubuntu.com/2015/03/25/#juju-dev.txt

katcoericsnow: if you're around, looks like review board isn't picking this up? https://github.com/juju/juju/pull/193100:12
katcoericsnow: oh shoot and i know why.00:12
katcoericsnow: i recently changed my github username: katco- to kat-co00:12
ericsnowkatco: that would do it :)00:13
katcoericsnow: any way to move my account and get that PR up?00:13
ericsnowkatco: I should be able to do that real quick00:13
katcoericsnow: you are a gentleman and a programmer.00:14
ericsnowkatco: done; hopefully it works ;)00:16
katcoericsnow: account looks good, but PR still isn't there?00:17
katcorbt post is asking for a pw00:17
katcoericsnow: and the instructions you sent out last year aren't working for me =/00:26
katcoaxw: wallyworld_: https://github.com/juju/juju/pull/1931 until it gets onto RB. i have to go take care of some things.00:28
wallyworld_ty00:28
davecheneyboom, gccgo bug is fixed upstream00:34
davecheneynow ... PAPERWORK!00:34
thumperdavecheney: \o/00:39
davecheneyyeah, lynn's patch was submitted this morning00:40
davecheneyok, that joy was short lived, our other fix that went into gccgo 4.9 needs to be forward ported to 5.000:53
davecheneyi'll raise another bug00:53
davecheneywin1201:06
thumperwallyworld_: got a minute?01:22
wallyworld_sure01:22
thumper1:1 hangout?01:22
=== kadams54 is now known as kadams54-away
fwereadeholy crap, that mostly works02:08
fwereaderight, going to bed02:09
ericsnowkatco: if you're still around, try killing that PR, go to RB, log out, log back in, and then re-submit the PR over on github02:22
ericsnowthumper: ping04:29
axwwallyworld: FYI, I've proposed the environ-storageprovisioner and dynamic EBS branches -- http://reviews.vapour.ws/r/1258/ and http://reviews.vapour.ws/r/1259/04:38
axwwallyworld: I guess I'll work on volume-backed filesystems now04:38
axwwallyworld: FYI, I've proposed the environ-storageprovisioner and dynamic EBS branches -- http://reviews.vapour.ws/r/1258/ and http://reviews.vapour.ws/r/1259/04:41
axwwallyworld: I guess I'll work on volume-backed filesystems now04:41
wallyworldaxw: great ty, will look soon. volume backed fs sounds good04:42
ericsnowwallyworld: does this sounds familiar (relative to state log pruning): "failed to retrieve log counts: no such cmd: scale"05:19
bradmanyone know what the error "juju.rpc server.go:554 error writing response: EOF" in machine-0.log means?  the juju environment isn't very healthy, juju sets aren't triggering the appropriate config-changed hooks05:47
urulamawallyworld: the sessions added, alexisb notified.06:45
wallyworldaxw: off to soccer for a couple of hours, reviews done06:55
axwwallyworld: thanks06:56
axwlater06:56
mupBug #1436191 was opened: gce: bootstrap instance has no network rule for API <firewall> <gce> <juju-core:New> <https://launchpad.net/bugs/1436191>07:17
mattywmorning all o/07:24
dimiternaxw, ping08:30
dimiternaxw, I'm +100 on moving replicaset out of the main repo, but will this reduce the tests run time for core?08:31
axwdimitern: hey sorry, was afk. this change will not. this change is just preparing to move it out of core08:59
axwand to generate discussion09:00
dimiternaxw, right, so a step in the right direction :)09:00
axwyup09:00
axwdimitern: in case I misunderstood - the change on RB atm won't reduce test time, but moving replicaset out of core will09:03
axwunless of course CI starts running tests for all the github.com/juju packages09:03
dimiternaxw, yep, got it09:04
dimiternaxw, well, that happening at some point certainly won't hurt09:04
axwit'll hurt the run time ;)  I think we should do it, not sure about as part of the merge gating, maybe just as a voting CI job09:05
dimiternaxw, no no - definitely not part of the merge gating :)09:05
natefinchwallyworld, dimitern, axw, anyone else: I'm getting a "no reachable servers" trying to connect to state, which then causes all the workers to shut down... but jujud hangs around, and doesn't restart or retry or anything.  Why is that?  Shouldn't it retry?  This is on a machine that I've converted from a regular server to a state server in a replicaset.09:22
natefinchIt's a bummer, because I think it's just that mongo is slow, and if I kill jujud, it restarts and is able to connect.09:22
axwnatefinch: not sure off hand, I'll have a look at the code and see if I can think of a reason...09:22
dimiternnatefinch, I'm not quite sure why, but it should retry09:22
dimiternnatefinch, maybe some worker / runner is hanging instead of exiting09:23
natefinchdimitern: ahh, hmm, yeah, that's a good point09:23
axwnatefinch: only reason I can see that would explain that is if the state-serving info in the agent config changed09:31
axwi.e. from having it to not having it09:31
axwnot sure if that's even possible09:32
dimiternwow that's a first - http://paste.ubuntu.com/10676289/ - build failure in the net-cli branch - anyone seen something like that with the gc compiler?09:33
axwdimitern: did you update your go compiler?09:34
axwI've seen that sort of thing when there's version mismatch09:34
dimiternaxw, no - still go 1.2.1 amd64 from trusty09:34
natefinchtry deleteing your gopath/pkg directory and rebuild09:36
dimiternweird.. so I've wiped out my /tmp and $GOPATH/pkg/* dirs, rebuilt and now it's fine09:36
natefinchyep09:36
dimiternnatefinch, yep, cheers09:36
natefinchsome weird mismatch.. maybe gccgo or something?09:37
dimiternI did have both gccgo and gc binaries/object files in pkg09:37
dimiternin separate subdirs, but still09:38
natefinchoh right forgot they're in separate dirs...  well, I dunno, I've had it happen once in a blue moon09:38
dimiternyeah09:39
dimitern(as they say quite often lately around here - "putin is most likely behind this" :D)09:40
axwlol09:40
natefinchhaha09:40
dooferladAnyone around who could help with a test failure due to an unexpected receive? http://paste.ubuntu.com/10676354/09:42
dooferladThis seems to happen if I am running the entire suite sometimes, but never if I just run that one test.09:42
dimiterndooferlad, oh, I know that one09:42
dimiterndooferlad, it's known to be flaky, esp. under load09:42
dooferladdimitern: *sigh*09:43
dimiterndooferlad, I did try fixing it, but gave up because it wasn't easy without refactoring the way the filter uses watchers09:43
TheMueI love our tiny logs. ;)09:43
dimiterndooferlad, our SpaceTag caused quite a stir apparently09:47
dooferladdimitern: do you remember what caused the problem? (filter tests)09:47
dooferladdimitern: yea, I expect that will be renamed at some point!09:48
dooferladnetwork-group maybe09:48
natefinchis spacetag anything like freeze tag?  sounds like fun ;)09:48
dimiterndooferlad, the problem is synchronization between the apiserver, api, and state watchers09:49
dimiternnatefinch, it's more like a space cake I guess :D09:49
dimiterndooferlad, TheMue, standup?10:00
TheMuedimitern: yes. *lol*10:01
voidspacedimitern: so I fixed the first test problem (only a new worker sees the broken provider)10:39
voidspacedimitern: for the second test issue I've pushed a failing test10:40
voidspacedimitern: when we start the worker we have two dead addresses 0.1.2.4 and 0.1.2.610:40
voidspacedimitern: I'm expecting to see two ReleaseAddress calls, one for each10:40
voidspacedimitern: what I *actually* see is *three* calls10:40
voidspacedimitern: two for the first address and one for the second10:40
voidspacedimitern: I'm looking into it10:40
dimiternvoidspace, hmm weird10:40
voidspacedimitern: I have a test that reliably passes asserting those three calls...10:41
dimiternvoidspace, do the first 2 calls have the same args?10:42
voidspacedimitern: well, the OpReleaseAddress we get is the same10:42
voidspaceso yes10:42
voidspacesame subnetid etc10:42
voidspacethey're the same address10:42
voidspacedimitern: I'll add some logging to see why10:43
dimiternvoidspace, yeah10:43
dimiternvoidspace, it might be the last call comes from Handle()10:43
dimiternvoidspace, oh, no actually.. if the first 2 are the same maybe the first one does10:43
dimiternvoidspace, make sure you check the addresses you get in Handle are both dead and not gone10:44
voidspacedimitern: I check that10:46
voidspacedimitern: I think I found a bug, not sure how it can be the cause10:46
voidspacedimitern: or how the code worked at all10:46
voidspacedimitern: I have a select that removes the dead addresses - but it is a single select it doesn't loop10:47
voidspacedimitern: so it should only do the first address10:47
voidspaceso how I'm seeing three releases is a real mystery...10:49
voidspacerunning with logging10:49
dimiternvoidspace, good catch then10:49
voidspacedimitern: right, I see two attempts for each address10:49
voidspacefrom the logging10:49
dimiternvoidspace, I guess one from SetUp and one from Handle ?10:50
voidspacelet me add to Handle to check that10:50
voidspacedimitern: it's possible that the removal triggers the watcher too - but inside Handle I'm specifically checking if the address exists10:50
wallyworldfwereade: you around?10:50
voidspacedimitern: if it's *already* been removed we should fail to fetch it from State10:50
fwereadewallyworld, heyhey10:51
voidspacedimitern: it looks like the removal succeeds, triggering the watcher and then we *successfully* pull the address back out of state10:51
voidspacedimitern: race condition due to transactions?10:51
voidspaceis that possible10:51
dimiternvoidspace, hmm10:51
dimiternvoidspace, it is possible10:51
wallyworldfwereade: if you had time, i'd love a quick pre-impl chat10:51
voidspacedimitern: I'm going to confirm that this is the case (one attempt from Handle and one from SetUp)10:51
dimiternvoidspace, that's the thing with JujuConnSuite10:51
fwereadewallyworld, will be in our hangout in 5 mins10:51
dimiternvoidspace, we have 2 states there - State and BackingState - the latter is the real one used by the apiserver, the former is the state used by the api client10:52
wallyworldsure10:52
dimiternvoidspace, and they're not synced wrt watchers10:52
dimiternvoidspace, try calling s.BackingState.StartSync() before each assert on the watcher10:53
voidspacedimitern: what i'm seeing on start is Handle called with every address10:56
dimiternthose were exactly the same issues that lead to flaky tests10:56
voidspacedimitern: so the initial SetUp is trying to remove them at the same time as Handle10:56
dimiternvoidspace, yeah, that's what I suspected10:56
voidspaceI thought State and BackingState were the same these days, we fixed it by just having one State?10:57
voidspaceI'll try the extra sync anyway10:57
dimiternvoidspace, so you might not need to do the out-of-bound goroutine in setup perhaps?10:57
voidspacedimitern: maybe not10:57
voidspacedimitern: I read a docstring saying "Initial" was only the alive entities for a lifecyclewatcher10:58
voidspacethat maybe out of date10:58
voidspacedimitern: I'll read the lifecycle watcher code10:58
voidspaceI maybe able to just delete the SetUp10:58
voidspacewell, most of it anyway10:58
fwereadevoidspace, I think it has to include dying ones as well and frequently the dead ones10:59
fwereadevoidspace, consider something like a provisioner10:59
fwereadevoidspace, it needs to know about its dead machines so it can remove them10:59
voidspacefwereade: "frequently"... I'd quite like a deterministic result :-)10:59
voidspacefwereade: right, I was just going off some docstring that said "initial()" was called with only alive entities11:00
fwereadevoidspace, ok, lifecycle watchers should return everything in their initial results, including dying and dead11:00
dimiternvoidspace, fwereade, so the lifecycle watcher is smarter than I thought :)11:00
voidspacefwereade: I'll dig it out11:00
voidspacethat's great11:00
voidspaceremoves some code11:00
fwereadevoidspace, cool11:00
voidspace The first event emitted will contain the ids of all non-Dead  entities11:01
voidspacestate/watcher.go line 13611:01
voidspacefwereade: so in lifecycleWatcher.initial it fetches the whole collection, *stores* the ids of "non-dead" entities, but returns *all* ids11:03
voidspaceI'll update the docstring11:03
voidspacedammit, I should no better never to trust a docstring11:03
fwereadevoidspace, yeah, that sounds right11:03
fwereadevoidspace, thanks11:03
voidspacefwereade: we need to go back to calling them lies...11:03
voidspace*know better*11:04
voidspacedimitern: so if I just delete that code in SetUp the test passes...11:06
voidspacedimitern: which is good11:06
dimiternvoidspace, only one comment on your review11:07
dimiternvoidspace, great!11:07
dimiternvoidspace, I'll wait for the changes around SetUp and will re-review it11:07
voidspacedimitern: that select is gone11:08
voidspacedimitern: ah, no it's not11:08
voidspacedimitern: that's iin the test, fine I'll fix that11:08
voidspacedimitern: SetUp change pushed11:08
dimiternvoidspace, looks nice and red :)11:09
voidspace:-)11:09
* dimitern steps out for ~2h11:12
wallyworld_ericsnow: here's a trivial fix for an intermittent failure on ppc64 http://reviews.vapour.ws/r/1261/12:17
frankbanocr" could anyone please take a look at http://reviews.vapour.ws/r/1263/ ? thanks!12:43
* dimitern is back12:51
* fwereade omw uk for a bit, will be around again later but was up until 3 last night so might be a bit relaxed about it13:09
dimiternvoidspace, hey13:39
dimiternvoidspace, the releaser looks much better now, thanks!13:40
dimiternvoidspace, you have a review with a provisional "ship it!" and mostly minor changes suggested.13:40
ericsnowjw4: ping13:45
jw4ericsnow: ola13:45
ericsnowjw4: could you take a look at this log from CI: http://data.vapour.ws/juju-ci/products/version-2478/run-unit-tests-vivid-amd64/build-289/consoleText13:46
jw4ericsnow: that should be fixed?13:47
ericsnowjw4: it shows a failure that you fixed the other day with the log pruning stuff13:47
jw4right13:47
jw4did that error occur recently?13:47
ericsnowjw4: I'm trying to figure out why13:47
ericsnowjw4: Tuesday13:47
ericsnow(yesterday)13:48
jw4ericsnow: ah, look at the documentation for the mgo http://godoc.org/labix.org/v2/mgo#Database.Run13:48
ericsnowjw4: right, but I thought you fixed this already13:49
jw4ericsnow: yeah, I'm looking to see if there's another call to Database.Run13:50
ericsnowjw4: good point13:50
dimiternericsnow, jw4, I did see that patch replacing bson.M with bson.D land, but maybe the dependencies.tsv was not updated after?13:50
jw4dimitern: there should be no dependencies.tsv update13:51
dimiternjw4, oh, right - that's in state only13:51
jw4ericsnow: weird I don't see any other call13:52
jw4ericsnow: I see that run was testing a version before my fix landed: Testing gitbranch:master:github.com/juju/juju 0752a4bc13:53
ericsnowjw4: ah, never mind then :)13:54
jw4:)13:54
ericsnow(I was just checking that)13:54
* ericsnow promises himself not to try to reason about problems late in the evening13:54
jw4hehe13:54
jw4(but aren't you in Mountain Time? 8 AM now?)13:55
natefinchwell, ha --to 1,2 works now, though I have to use a hack to get jujud to restart after the StateWorker errors out from not being able to connect to state.  I gotta figure out why the damn workers aren't shutting down.13:58
voidspacedimitern: cool, thanks14:02
voidspacedimitern: you want me to use the  envtesting package in the worker itself?14:02
voidspacedimitern: in fact, I don't think envtesting.ShortAttempt exists, which is why I was using the other one in the tests14:03
dimiternvoidspace, ah14:03
voidspacedimitern: I'm pretty sure you've hallucinated its existence14:03
dimiternvoidspace, let me double check14:03
voidspacedimitern: and I didn't want a dependency on a test package in the production code14:03
dimiternvoidspace, oh, you're right14:04
dimiternvoidspace,  so then use common.ShortAttempt in both the worker and the tests?14:05
voidspacedimitern: it's in the provider package, but ok14:06
voidspacewell, provider/common14:06
dimiternvoidspace, yeah14:06
voidspaceseems like a weird dependency14:06
voidspacedimitern: and I dropped one of your issues - you slightly misunderstood the test14:06
voidspacedimitern: the rest are good and I'll fix them, thanks14:06
voidspacedimitern: after a break...14:07
dimiternvoidspace, ok, cheers14:08
TheMueYeeeeeeeeeeeeeeeeeeeeeeeeeeeeeehaw! Local vMAAS acquiring works. *phew*14:12
dimiternTheMue, awesome!14:13
TheMueNeeded some patches/changes in their hack, but now the first node started as wanted.14:13
dimiternTheMue, does it stop as well? :)14:14
TheMuedimitern: Yeah, will pass this information to the MAAS team. Maybe they are interested in adding it too.14:14
TheMuedimitern: Oh, good question. one moment.14:14
TheMuedimitern: Hehe, yeah, it does. *dancingThroughTheRoom*14:16
dimiternTheMue, great! then you can have a look at that bug 1427814 :)14:17
mupBug #1427814: juju bootstrap fails on maas with when the node has empty lshw output from commissioning <bootstrap> <maas> <maas-provider> <network> <juju-core:Triaged> <https://launchpad.net/bugs/1427814>14:17
dimiternTheMue, in order to reproduce it, you'll need to remove the lshw output for a node (or all nodes) from maas db14:18
TheMuedimitern: shit, I should have been quiet *lol*14:22
dimiternTheMue, :)14:22
TheMuedimitern: so, assigned to me and will add a card. but let me provision my fresh nodes first14:23
dimiternTheMue, sure, np14:23
dimiternTheMue, there's already a card btw14:23
TheMuedimitern: ah, ok, then I'll grab it14:24
dimiternTheMue, cheers, ping me if anything is unclear14:24
TheMuedimitern: yep, will do14:24
sinzuiericsnow, I have bug 1435974 into something we can do. In short, the license files don't have copyrights...the owner of the licence is not clear the owner of the code14:43
mupBug #1435974: Copyright information is not available for some files <juju-core:Triaged> <juju-core 1.22:Triaged> <juju-core 1.23:Triaged> <https://launchpad.net/bugs/1435974>14:43
ericsnowsinzui: ah, that makes sense14:45
ericsnowsinzui: so does that mean we should have a separate top-level COPYRIGHT file?14:45
sinzuiericsnow, no it means stop cargo-culting licenses. just put the copyright and meaningful paragraph in the LICENCE file. see my comment int the bug14:46
ericsnowsinzui: got it; I'll take a look14:48
ericsnowsinzui: should I get someone to write up patches or were you planning on it?14:52
sinzuiericsnow, I think core staff are best suited to it since they have to change the project, then the juju 1.22.1 dependencies.tsv14:53
ericsnowsinzui: sounds good14:53
dimiternericsnow, hey14:58
ericsnowdimitern: hi14:59
dimiternericsnow, any idea why I'm getting this with GCE ? Bootstrap failed, cleaning up the environment.15:00
dimitern2015-03-25 14:58:45 ERROR juju.cmd supercommand.go:430 there was an issue examining the environment: retrieving auth token for <my client email from the json key file>: Invalid Key15:00
ericsnowdimitern: I'm guessing your key is invalid <wink>15:00
dimiternericsnow, I've followed the instructions, registered for a gce free trail, generated a client id as instructed15:00
dimiternericsnow, and the gce provider is not liking it for some reason - triple checked I pasted it ok15:01
ericsnowdimitern: it may be, as axw said, that it's simply not clear what to put into environments.yaml15:01
natefinchericsnow, sinzui: yeah, just slapping a copyright line on the license seems to be the easiest way to do it15:01
natefinchdimitern: you probably have to put it in quotes15:02
dimiternericsnow, should the private-key setting be a patch to the json file I downloaded?15:02
ericsnownatefinch: as sinzui indicates in the bug, we should also simplify the license file15:02
dimiternericsnow, s/patch/path/15:02
ericsnowdimitern: nope, though that is basically what axw suggested (good idea too)15:02
dimiternericsnow, ah, so it needs to be the verbatim contents of the file then?15:03
ericsnowdimitern: you have to copy-and-paste stuff out of that JSON file15:03
sinzuiI think QA can change the tarball scripts to check for copyrights in LICENSE/LICENCE file so we don't get these surprises from Ubuntu15:03
dimiternericsnow, oh, well, I'll try this15:03
ericsnowdimitern: also, I've put better instructions in the 1.23 release notes15:04
dimiternericsnow, I'll check there as well then, thanks15:05
natefinchdimitern: something like this: https://pastebin.canonical.com/128327/15:05
natefinch(replace <tons of garbage> with your actual key)15:05
ericsnowdimitern: BTW, where did you get that JSON file?15:05
natefinchericsnow: google's site creates it for you15:06
ericsnownatefinch: right, but at what point?  during the project creation process?  I don't remember15:06
dimiternericsnow, from the GCE web ui - API & auth15:07
ericsnowdimitern: k15:07
ericsnowdimitern: I remember now, thanks15:07
Muntanerhi devs15:13
MuntanerI have a question about charm distros15:13
MuntanerI need to be free in what distro I launch with my charms: CentOS, Kali, etc... can I actually do that with juju?15:13
ericsnowdimitern: thanks for looking at the bug, BTW15:13
dimiternericsnow, np, looked like a relatively easy fix and a chance for me to try the gce awesomeness :)15:14
ericsnowdimitern: haha15:14
alexisbMuntaner, currently juju supports deploying Ubuntu and Windows workloads15:19
alexisbMuntaner, we will soon have support of Centos15:19
alexisbbut it is not currently available15:19
Muntaneralexisb, so I'm not able to choose the distro in my charm?15:20
Muntaneralexisb, maybe by writting my own metadatas?15:21
natefinchMuntaner: you definitely can create your charm for a specific distro... it just has to be a version of Ubuntu or Windows right now15:22
marcoceppi_Muntaner: charm distro isn't specified by metadata.yaml, it's defined by where you push the charm to lp or to a local repo. So if you create ~/charms/win2012r2/charm-name then juju deploy --repository ~/charms local:win2012r2/charm-name Juju will attempt to create a VM with Windows Server 2012r215:26
marcoceppi_Muntaner: but that series, which defines distro (trusty, precise, win2012r2, etc)15:26
mupBug #1436390 was opened: GCE provider config should support extracting auth info from JSON file <gce-provider> <juju-core:Triaged> <https://launchpad.net/bugs/1436390>15:27
mupBug #1436397 was opened: map-order sensitive test in md/juju/storage needs to be fixed <map-order> <juju-core:Triaged> <juju-core 1.23:New> <https://launchpad.net/bugs/1436397>15:27
Muntanermarcoceppi_, there is no way that allows me to use other distros atm?15:27
marcoceppi_Muntaner: no, because Juju has to know how to handle that distro and that distro needs to have an image spun up with cloudinit running. Juju is doing the machine setup so if it can't speak to that distro it won't work.15:28
dimitern\o/ gce is bootstrapping now15:29
marcoceppi_Muntaner: We're adding support for other distros all the time though, the more distros we add support for the easier juju becomes to port to other platforms15:29
Muntanermarcoceppi_, where can I find a list of the usable distros?15:29
marcoceppi_Muntaner: right now it's Ubuntu and Windows, with support for CentOS underway15:30
ericsnowsinzui: FYI, this last run of 1.23 (2482) had only 1 unit test fail :)15:30
natefinchMuntaner: CentOS support will be submitted pretty soon, in a matter of weeks.    As for current distros, it's just Ubuntu (precise or later) and Windows... I would have ot look to see which versions of windows we support. Just the server versions for the most part, and I think 2012 or later.15:30
sinzuiericsnow, \0/15:32
aisraelDoes anyone know the status of this critical bug? https://bugs.launchpad.net/juju-deployer/+bug/142131515:34
mupBug #1421315: subordinate service not appearing in watch output <api> <juju-gui> <oil> <regression> <juju-core:Triaged> <juju-core 1.22:Incomplete> <juju-deployer:Triaged> <https://launchpad.net/bugs/1421315>15:34
dimiternericsnow, bootstrap fails now -I did  2 attempts, the first one waited for a while, then I stopped it, the second failed right away with RESOURCE_NOT_READY when trying to start the instance15:39
ericsnowdimitern: do you see an instances or other resources in the GCE [web] console for the project?15:41
dimiternprobably because I'm using region europe-west115:41
mattywfolks - is there a way I can open an api connection acting as the state server in JujuConnSuite?15:41
dimiternericsnow, I did see a deprecation warning for zone europe-west1-a15:41
mattyw^^ seems like there must be a way - but I can't find it15:41
dimiternericsnow, now I've switched to us-central1, and so far no errors, it's trying to connect with ssh15:42
dimiternmattyw, you need to log in as a machine with JobManageEnviron15:42
alexisbMuntaner, did you get your question answered?15:43
dimiternericsnow, it worked this time - bootstrapping is mostly done15:43
mattywdimitern, yeah, I'm trying to access an api that is looked to machines with JobManageEnviron15:43
mattywdimitern, but I can't create a new machine with OpenAPIasNewMachine because I'm not allowed to start machines with that job15:44
ericsnowdimitern: oh good :)15:44
dimiternmattyw, what do you mean not allowed? you're getting an error from state.AddMachine?15:44
mattywdimitern, correct15:45
mattywdimitern, I can do a quick screen share if you like15:45
Muntaneralexisb, yes, I don't know what to do now ;(15:45
MuntanerI should instantiate automatically subnets with different OS15:45
ericsnowcould I get a volunteer to work on lp:1436407?15:45
dimiternmattyw, hmm, let me look in a few places first15:46
ericsnowit's a blocker for 1.2315:46
Muntanerconnected between them... and I wanted to use juju bundles15:46
dimitern#143640715:46
mupBug #1436407: certSuite.TestNewDefaultServer failed on vivid <ci> <juju-core:Invalid> <juju-core 1.23:Triaged> <https://launchpad.net/bugs/1436407>15:46
alexisbcmars, if you sill have someone available ^^15:46
dimiternericsnow, I do believe one of the patches you've reviewed should fix that - I've left a comment15:46
ericsnowdimitern: ah, cool15:47
dimiternmattyw, so is the machine you're adding the first one?15:47
mattywdimitern, I tried s.OpenAPIAsNewMachine(c, state.JobManageEnviron)15:48
ericsnowdimitern: yep, http://reviews.vapour.ws/r/1261/15:48
mattywdimitern, but that fails because I'm not allowed15:48
dimiternmattyw, ok, let me have a look at your code?15:48
dimiternericsnow, that's the one15:48
voidspacedimitern: ah, can't use worker.AssertStop as worker.Worker doesn't implement Stop15:49
ericsnowdimitern: thanks for remembering :)15:49
dimiternericsnow, lucky catch I guess :)15:49
voidspacestatetesting.AssertStop I mean15:49
dimiternvoidspace, well, how about defer worker.Stop(w) then ?15:49
voidspacedimitern: don't need the func call around it as it's a single liner I guess15:51
ericsnowall, we also need someone to work on #1435974, which should be pretty simple but will involve patches to most of our repos15:51
mupBug #1435974: Copyright information is not available for some files <juju-core:Triaged> <juju-core 1.22:Triaged> <juju-core 1.23:Triaged> <https://launchpad.net/bugs/1435974>15:51
alexisbcherylj, are you working anything critical??15:51
ericsnowthat one is needed before Ubuntu will take 1.22, so it is pretty important15:51
alexisbif not can you take a look at 143597415:52
frankbandimitern: what's the goal of the revision updater? is it just for juju status?15:57
voidspacedimitern: nope, needs wrapping15:59
voidspacedimitern: http://play.golang.org/p/4N_ydeMKEB15:59
voidspacedimitern: the *first* panic is triggered inside the defer - never get to the second15:59
voidspacedimitern: which is why we wrap in a function call!15:59
dimiternvoidspace, sorry, will get back to you in a bit16:02
voidspacedimitern: no problem, just an interesting observation16:03
voidspacedimitern: I can't change the defers as you suggested16:03
mupBug #1436415 was opened: vivid local template container "juju-vivid-lxc-template" did not stop' <ci> <lxc> <vivid> <juju-core:In Progress by ericsnowcurrently> <juju-core 1.23:In Progress by ericsnowcurrently> <https://launchpad.net/bugs/1436415>16:03
voidspacedimitern: I made all the other changes and now one test is failing for no discernable reason [yet]16:03
mupBug #1436421 was opened: juju panics after upgrade from 1.18.4 to 1.20.11 <juju-core:New> <https://launchpad.net/bugs/1436421>16:03
voidspacedimitern: will fix and ship16:03
dimiternvoidspace, I guess you're not looking your PMs :)16:15
ericsnowdimitern: do you know if wallyworld's cert expiry patch needed to be forward-ported to master?16:32
dimiternericsnow, no, I don't off hand, but check if the bug is16:35
dimitern..affecting both master and 1.2316:35
ericsnowdimitern: I'm pretty sure it either doesn't apply to master or was already fixed :)16:37
ericsnowdimitern: I'll ask wallyworld when he's around16:37
ericsnowdimitern: I do have a question about networking, if you have a minute16:37
dimiternericsnow, shoot16:38
ericsnowdimitern: for the vmware provider (from alteros) it looks like the low-level API does not support managing much networking details for instances (in this case virtual machines)16:39
dimiternericsnow, really? that's a bit surprising16:40
dimiternericsnow, but shouldn't matter much anyway - networking features are optional16:40
ericsnowdimitern: so opening/closing ports and assigning internal/external IP addresses is not supported and must be handle manually16:40
ericsnowdimitern: I'm talking about for the core firewall functionality of a provider16:40
ericsnowdimitern: for supporting juju expose/unexpose16:41
ericsnowdimitern: though that is certainly related16:41
dimiternericsnow, right, ok16:41
ericsnowdimitern: I'm going off the information I have about the capability of the "vSphere" API that we're using16:42
dimiternericsnow, so you're saying the provider will need to do some manual steps to open/close ports on each instance16:42
ericsnowdimitern: so I wanted to get your thoughts on what might be the best approach to take in this situation16:42
ericsnowdimitern: right16:42
ericsnowdimitern: and to manage internal/external IP of an instance16:43
dimiternericsnow, what sort of management?16:43
ericsnowdimitern: (just for the sake of juju's networking needs)16:43
ericsnowdimitern: not talking about the juju networking API16:43
TheMueaaargh, shitty yaml again, disliking a tab :(16:43
dimiternericsnow, well, not having support for open/close port is not uncommon - the provider can just return an error16:45
ericsnowdimitern: so what might be the most sensible approach for the provider to manually set these things16:45
dimiternericsnow, but I need to understand a bit better the issue around internal/external IPs you mention16:45
ericsnowdimitern: really?  how does juju handle that when handling an expose call?16:46
dimiternericsnow, expose triggers calling OpenPort on the environ via the firewaller16:46
ericsnowdimitern: right16:46
ericsnowdimitern: so what happens when OpenPort returns an error saying it isn't supported?16:47
dimiternericsnow, the firewaller logs an error/warning and that's it16:48
ericsnowdimitern: but then the unit doesn't get exposed, right?16:49
ericsnowdimitern: that seems like a show-stopper for any provider16:49
dimiternericsnow, no16:50
dimiternericsnow, so first, the service is exposed or not, not its units16:51
ericsnowdimitern: right16:51
dimiternericsnow, then, "exposed" is just a flag, it doesn't guarantee access, it declares intent16:51
ericsnowdimitern: got it16:52
dimiternericsnow, whether a given unit is accessible after exposing the service - that's where the firewaller comes into play16:52
dimiternericsnow, so if it fails to open the port - too bad; however this is not an issue for *some* providers16:52
dimiternericsnow, e.g. local doesn't have (or assume) restrictive firewall rules so even without exposing a service, once it's running you can access it16:53
dimiternericsnow, now, if the provider does not allow you to control access (egress/ingress), I'm not sure you can do much about it16:54
mbruzekwwitzel3: Can I get a review of this go code? https://github.com/mbruzek/virtualservices/blob/master/parse_endpoint.go16:57
mupBug #1436407 was opened: certSuite.TestNewDefaultServer failed on vivid <ci> <juju-core:Incomplete by wallyworld> <juju-core 1.23:Fix Committed by ericsnowcurrently> <https://launchpad.net/bugs/1436407>16:57
mbruzekwwitzel3: https://github.com/mbruzek/virtualservices16:57
ericsnowdimitern: so to summarize: OpenPort/ClosePort isn't critical as long as there is still external access to the port on the instance17:05
ericsnowdimitern: what specifically does juju require regarding internal IP addresses?17:08
dimiternericsnow, that's correct17:24
TheMuedimitern: any idea why bootstrapping fails with "You may want to use the 'agent-metadata-url' configuration setting to specify the tools location." even if the configuration setting is done?17:30
=== anthonyf is now known as Guest6628
dimiternericsnow, sorry, got in another call17:49
ericsnowdimitern: no worries :)17:49
dimiternericsnow, re internal IP addresses17:49
dimiternericsnow, there's no special requirement - just that an instance can talk to the apiserver17:50
dimiternericsnow, on its published cloud-local address17:50
ericsnowdimitern: k17:50
dimiternTheMue, have you imported the boot images?17:50
ericsnowdimitern: it seems like communication between charms via the cloud-local address is pretty common, so that should be a factor, right?17:51
TheMuedimitern: you mean "--sync-tools" into a local directory?17:52
dimiternericsnow, yes, all charms inside the environment use the private address of the unit (which in turn comes from the unit's assigned machine)17:52
dimiternTheMue, no, I mean does your maas have any images you can see in the web ui?17:52
ericsnowdimitern: got it; thanks for all the clarification :)17:53
TheMuedimitern: yes, 14.04 and 14.1017:53
dimiternericsnow, np, I hope I was helpful :)17:53
ericsnowdimitern: totally17:53
TheMuedimitern: but I get an error when doing juju bootstrap17:53
dimiternTheMue, and you're bootstrapping with --upload-tools --debug?17:54
TheMuedimitern: so far w/o debug, will try again17:54
dimiternTheMue, might be helpful, yeah17:55
TheMuedimitern: hmm, currently not directly. but will ping you tomorrow morning17:56
cheryljalexisb, ericsnow, I can take bug 143597417:56
mupBug #1435974: Copyright information is not available for some files <juju-core:Triaged> <juju-core 1.22:Triaged> <juju-core 1.23:Triaged> <https://launchpad.net/bugs/1435974>17:56
dimiternTheMue, ok, paste a dump of the bootstrap log along with it17:57
alexisbcherylj, thank you!17:57
ericsnowcherylj: awesome, thanks!17:57
* dimitern steps out17:57
* TheMue too18:00
=== kadams54 is now known as kadams54-away
ericsnowcherylj: make sure to assign the bug to yourself and mark it as in progress18:24
cheryljericsnow: done18:25
ericsnowcherylj: thanks :)18:25
mupBug #1434437 changed: juju restore failed with "error: cannot update machines: machine update failed: ssh command failed: " <backup-restore> <maas-provider> <juju-core:Invalid> <juju-core 1.22:Incomplete by ericsnowcurrently> <https://launchpad.net/bugs/1434437>18:27
natefinchI think turning up logging to show DEBUG logs fixed my problem...18:31
cheryljericsnow, sinzui:  Just to sanity check:  http://reviews.vapour.ws/r/1264/18:41
cheryljI'll modify that text to reflect which license is used in the other packages.18:42
sinzuicherylj, perfect, and the diff shows the comedy that happened18:44
cheryljah, yeah18:44
cheryljsinzui: should I go back in and add that one line package description too?18:45
ericsnowcherylj: a friendly reminder that the patch will also need to be applied to 1.22 and 1.23 :)18:46
cheryljericsnow: I haven't had to apply changes to a non-master branch yet.  Is there an easy way to do it?18:50
ericsnowcherylj: you should probably be able to just create a new pull request with a different target branch (I expect it will apply cleanly on all branches)18:51
ericsnowcherylj: but you *may* need to rebase your branch to the target and force-push it before the new PR will apply cleanly18:52
cheryljericsnow: yeah, I think I do18:52
ericsnowcherylj: not the funnest dance18:52
cheryljhehe18:52
ericsnowcherylj: be sure to land it in each branch before moving on to the next branch18:53
ericsnowcherylj: to be honest, I think we could make this easier by null-merging older branches up the chain and then strictly forward-porting from there18:54
ericsnowcherylj: if I understand it right that would eliminate the need to rebase for each PR18:55
ericsnowcherylj: it would also mean the forward port would be accomplished by merging the older branch into the newer one (e.g. 1.22 -> 1.23 -> master)18:55
ericsnowcherylj: however, that isn't the state of things at the moment so you'll have to go the rebase-then-PR route :)18:56
cheryljericsnow: now that you've gotten me really confused, I should just go back and do the first thing?  ;)18:57
ericsnowcherylj: sorry, I was afraid of that :)18:57
mupBug #1436495 was opened: sporadic test failure: storeManagerStateSuite.TestStateWatcher <ci> <juju-core:New> <https://launchpad.net/bugs/1436495>18:57
ericsnowcherylj: do what you were doing 1. land your PR against master, 2. rebase your branch against 1.23  3. force-push it 4. create a new pull requrest from your branch to 1.23 5. repeat 2-4 for 1.2218:59
cheryljericsnow: that I can do :)  Thanks18:59
ericsnowcherylj: (and land the PR from 4 before moving on to 5)19:00
ericsnowcherylj: there are other ways to do this too, perhaps even simpler, but I've already muddied the waters a bit :)19:01
cheryljericsnow: for a package description (from your RB comment), should I add something like "This package contains the core functionality for juju"?19:03
ericsnowcherylj: that sounds fine to me19:04
ericsnowsinzui: ^^^19:04
ericsnowcmars: intermittent(?) test failure possibly related to your team's work:  #143650719:06
mupBug #1436507: sporadic test failure: FilterSuite.TestMeterStatusEvents <ci> <juju-core:New> <https://launchpad.net/bugs/1436507>19:06
cmarsericsnow, will take a look19:07
ericsnowcmars: thanks19:07
ericsnowcmars: I'm trying to get the vivid unit tests passing in CI :)19:07
cmarsericsnow, it just so happens I have a vivid instance ready for action19:07
mupBug #1436507 was opened: sporadic test failure: FilterSuite.TestMeterStatusEvents <ci> <juju-core:New> <https://launchpad.net/bugs/1436507>19:10
ericsnowsinzui: looks like run-unit-tests-vivid-amd64 just passed on vivid :)19:12
ericsnowsinzui: on 1.2319:12
sinzuiericsnow, yeah19:14
ericsnowsinzui: master is close (maybe just intermittent failures)19:15
sinzuiericsnow, I am whipping the precise aws upgrade job. I think aws + precise is experiencing unreliability this week and it is hurting the test results19:15
ericsnowsinzui: I've opened bugs for recent failures and noted all related bugs in #143357719:16
mupBug #1433577: Vivid unit tests need to pass <ci> <test-failure> <vivid> <juju-core:In Progress by ericsnowcurrently> <juju-core 1.23:In Progress by ericsnowcurrently> <https://launchpad.net/bugs/1433577>19:16
sinzuiericsnow, I saw :)19:17
ericsnowsinzui: now if only I could get the vivid deploy test to pass :(19:18
mupBug #1436507 changed: sporadic test failure: FilterSuite.TestMeterStatusEvents <ci> <juju-core:New> <https://launchpad.net/bugs/1436507>19:19
=== kadams54-away is now known as kadams54
ericsnowwhat changed, mup?  are you just making this up?19:21
ericsnownatefinch: were you going to be able to give me a review on http://reviews.vapour.ws/r/1234/?19:22
natefinchlol:   return newStorageWorker("", api, api, api, api)19:22
ericsnownatefinch: obviously :)19:23
natefinchericsnow: yeah, sorry, got distracted by getting my HA stuff working19:23
ericsnownatefinch: hey, I'm okay with that19:23
natefinchapi/converter/converter.go: needs merge19:27
natefinchapiserver/converter/converter.go: needs merge19:27
natefinchworker/converter/converter.go: needs merge19:27
natefinchYou must edit all merge conflicts and then19:27
natefinchmark them as resolved using git add19:27
natefinch$ git mergetool19:27
natefinchNo files need merging19:27
natefinchffffff.....19:27
mupBug #1436507 was opened: sporadic test failure: FilterSuite.TestMeterStatusEvents <ci> <juju-core:New> <https://launchpad.net/bugs/1436507>19:28
sinzuiericsnow, I paused CI. I am going to restart vivid-slave-b and give it one more try with 1.2319:38
ericsnowcherylj: looks like godeps (really a flaky code hosting service) is making your merge fail :(19:56
cheryljericsnow:  I was wondering what was causing that.  Is there anything I can do other than keep retrying?19:57
ericsnowcherylj: I just ran godeps on my local host and it took a looooong time (and CI is less patient)19:57
natefinchericsnow: weird, what host is being slow?19:58
ericsnowcherylj: wait for launchpad (or whoever) to speed back up19:58
ericsnownatefinch: not sure19:58
ericsnownatefinch: it wasn't clear which, though it *seemed* to correspond to gopkg.in19:59
ericsnownatefinch: which doesn't tell me much19:59
ericsnowcherylj: also, keep in mind that I enabled the reviewboard webhook on a number of our github repos the other day but not all of them20:10
ericsnowcherylj: so if no review request shows up it probably means there's no hook and it will have to be reviewed on github20:11
ericsnowcherylj: if you bump into that please let me know so I can double-check the webhook is set up right20:12
cheryljericsnow: will do20:14
cheryljdavecheney: If you could take a look today :)  http://reviews.vapour.ws/r/1239/20:33
davecheneycherylj: sure thing20:39
=== kadams54 is now known as kadams54-away
dpb1rick_h_: how do I download a bundle from the store from the command line?20:48
rick_h_dpb1: curl https://api.jujucharms.com/v4/mongodb-cluster/archive/bundle.yaml ?20:48
dpb1ok20:48
rick_h_dpb1: though that's the new format, so you'll need a not-yet-released deployer to use it20:49
rick_h_dpb1: or quickstart I think will do that20:49
dpb1rick_h_: https://jujucharms.com/u/landscape/landscape-dense-maas/20:49
dpb1rick_h_: why is that preview blank?20:49
dpb1rick_h_: (sorry for the rapid fire questions). :)20:49
rick_h_dpb1: looking20:49
rick_h_dpb1: no gui annotations for positions20:50
rick_h_dpb1: we've got a todo to add a default layout algo to the svg generator20:50
dpb1rick_h_: OK, I'll export something from the GUI and see if I can figure it out.  Thanks, that is probably enough to go on20:51
dpb1rick_h_, Makyo, I'm giving the branch a final test now, things looking much better test coverage wise.  Thanks.20:53
rick_h_dpb1: awesome, ty for bearing through us with it.20:53
Makyodpb1, seconded, thanks a ton20:53
rick_h_dpb1: big chunk of work to move forward the new bundle story and such20:53
dpb1np, sorry it's taken a while, other prios came up.20:54
rick_h_and hopefully we can iterate on it to smooth it out as a nice useful feature20:54
dpb1rick_h_: agreed20:54
rick_h_dpb1: understand, unfortunately it's not really anyone's day job :)20:54
ericsnownatefinch: so about that review :)20:54
=== kadams54-away is now known as kadams54
ericsnowcmars: did that bug end up being related to your stuff?21:34
ericsnowcherylj: would you mind targeting 1.22 first?  It's the key one we need to get fixed first.21:34
alexisbcherylj, thumper, the bug cherylj is working is very time sensitive, so we will want to pass that off to another team member at eod for cherylj21:37
cheryljericsnow: yes, I can do that21:37
ericsnowcherylj: thanks21:38
thumperalexisb: ack21:40
alexisbthumper, cherylj thank you!21:40
cmarsericsnow, i seriously doubt it. nothing is stable on vivid afaik on my instance21:42
cmarsericsnow, mongo sockets left open, all kinds of timeouts, etc.21:44
ericsnowcmars: lovely21:56
cmarsericsnow, still poking at it21:56
ericsnowcmars: please keep a list of everything you notice needs fixing on vivid :)21:57
cmarsericsnow, will do21:57
ericsnowcmars: thanks!21:57
=== kadams54 is now known as kadams54-away
cmarswallyworld, could this be related to storage? what might cause this timeout? https://bugs.launchpad.net/juju-core/+bug/1436507/comments/122:09
mupBug #1436507: sporadic test failure: FilterSuite.TestMeterStatusEvents <ci> <intermittent-failure> <juju-core:In Progress by cmars> <https://launchpad.net/bugs/1436507>22:09
cheryljericsnow, sinzui:  For the juju/cmd case, there is an exception to the license noted, so I just added the copyright information and left the rest of the LICENSE file intact:  http://reviews.vapour.ws/r/1269/22:12
ericsnowcherylj: sounds good22:13
cheryljericsnow: can you merge the juju/cmd PR?22:13
ericsnowcherylj: sure22:13
cheryljthanks22:13
wallyworldcmars: not the new storage feature - the storage in the error is the environment blob store where charms and tools are saved (instead of cloud storage)22:14
cmarswallyworld, ok, got it22:14
wallyworldcmars: for whatever reason, the mongo db used for that storage became disconnected - i/o timeout22:14
wallyworldit attempts to rollbacl failed operations, but the rollback failed too22:15
cmarswallyworld, ah, that's *very* consistent with other errors i'm seeing22:15
cmarswallyworld, i'm running mgo tests now22:15
ericsnowcherylj: did you try adding a merge comment?  the landing bot was added to a number of repos recently and I believe cmd is one of them22:15
wallyworldcmars: maybe on a busy system things can't keep up? not sure22:16
wallyworldbut for whatever reason, juju->mongo is not happy22:16
cmarsso i decided to run mgo.v2's tests. http://paste.ubuntu.com/10680888/22:18
cmarsthis is on canonistack. i'll try a kvm locally for a second opinion22:19
ericsnowcmars: yikes22:36
cmarsericsnow, that might be a red herring.. i get the same error on trusty22:41
cmarsericsnow, which is a different sort of yikes22:42
ericsnowcmars: :)22:42
ericsnowcmars: at least that package isn't important to juju <wink>22:42
cmarsyeah, i mean, close enough, right? http://paste.ubuntu.com/10680988/22:45
cmarsthat's an mgo.v2/txn test, which I just ran on trusty22:46
ericsnowcmars: bank error *not* in your favor :)22:47
cmarsericsnow, just curious, does the vivid torrent work for you? i'm getting a tracker error22:52
cmarsericsnow, http://cdimage.ubuntu.com/ubuntu-gnome/releases/vivid/alpha-1/22:53
cmarsoh, i should probably get beta-122:53
cmarsnevermind22:53
ericsnowcmars: don't know22:53
cmarsbeta-1 works22:53
ericsnowcmars: lesson learned :)23:00
cheryljericsnow: no, I hadn't, the last time I made changes to cmd it was still a manual merge repo23:05
ericsnowcherylj: well, I tried it and nothing happened :)23:05
cheryljericsnow: I also see that charm doesn't trigger RB23:05
ericsnowcherylj: charm is one for which I did not add the hook23:05
ericsnowcherylj: I'll push the merge button on cmd23:06
cheryljericsnow:   thanks :)23:06
cheryljCan you also review the 1.22 change?  http://reviews.vapour.ws/r/1268/23:06
ericsnowcherylj: be sure to target 1.22 with your PRs (you have to manually change the target brach from master)23:07
ericsnowcherylj: you bet23:07
ericsnowcherylj: I see you're a step ahead of me :)23:07
cheryljhehe :)  It took me a little bit of time to figure out how to actually do the changes / PR for 1.2223:08
cheryljI haven't contributed to most of these repos before, so I have to so all the initial steps to get things set up first23:10
ericsnowcherylj: sorry about that; I really appreciate that you are working on this :)23:25
cheryljericsnow:  It's no problem at all.23:26
davecheneyalexisb: oh shit, sorry i'm late23:35
alexisbdavecheney, nws23:35
alexisbI am on the hangout when you are ready23:35
davecheneyalexisb: jumping in the hangout now23:35
axwwallyworld: can you please create github.com/juju/replicaset when you have a moment. I can't do that, but I'll do the rest23:47
wallyworldsure23:48
wallyworldaxw: mongo-replicaset perhaps?23:48
wallyworldor is that too wordy23:48
wallyworldor tautological23:49
axwI'd rather just go with replicaset as that's what it's called now, we can rename it later if we need to23:49
axwwallyworld: ^23:50
wallyworldok, i was thinking replicaset on its own was ambiguous23:50
wallyworldbut we can rename if needed23:50
wallyworldaxw: that's done, i've also added the bots as collaborators so we can automate landings if needed23:54
wallyworlds/automate/gate23:54
axwwallyworld: thanks23:55

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!