[00:01] katco: how optional am I on the second standup? [00:03] perrito666: it's almost philosophical - what variations of optional there are? hard optional? soft optional? :D [00:03] perrito666: disclaimer: this question ^^ does not imply ur optionality [00:05] anastasiamac: there is a whole range between "we cant live if living is without you" and "I dont ever want so see you again" [00:05] (I had to google for the second song, there aren't as many hate you songs as one would expect) [00:06] perrito666: really, i was told swift covers a lot of hate ranges [00:07] anastasiamac: the thing is, that is very close to my bike time and I might not make it everyday [00:08] perrito666: ack. i have school drop-off at that time too, so cannot do either :/ [00:11] menn0: thanks for review, i've updated pr [00:11] wallyworld: for some reason RB isn't showing any changes between diffs 1 and 2 [00:11] wallyworld: but I trust you, so ship it [00:11] hmm, ok, ty, i double check [00:12] wallyworld: it says there's files changed but when I what to see the diffs there's nothing [00:12] looks like a RB problem [00:13] * perrito666 sends the kind of mail that makes someone hate you [00:13] menn0: i just checked github and the commit it there [00:13] so wtf rb [00:13] * menn0 looks quickly at the changes on GH [00:14] * wallyworld waits in anticipation [00:14] wallyworld: looks good! my ship it stands :) [00:14] yay, tyvm [00:23] anastasiamac: hey, i commented on the pr, sorry it needs rework [00:23] wallyworld: i disagree with ur stmt [00:24] wallyworld: quick hangout? [00:24] sure [00:24] 1:1 [00:25] Bug #1546348 opened: cannot use or destroy controller after bootstrap [00:25] [00:25] Bug #1546350 opened: All juju models tagged with "local" [00:36] state tests take a bit too long [00:40] someone thought this was a useful test failure error: [00:40] Error: entity contents mismatch; same length 1 [00:41] perrito666: juju deals with paradoxes :D [00:44] wallyworld: I think the CI issue is due to the change from -> local. [00:44] ah [00:45] axw: that sounds like it's Ci scripts not juju [00:45] wallyworld: model "aws-deploy-trusty-amd64" not found <-- that'd be because it's called local.aws-deploy-trusty-amd64 [00:45] axw: that's the model though not the controller [00:46] wallyworld: yeah, the admin model is currently called the same as the controller [00:46] axw: ah, ok, so when we create the admin model, we need to strip local. [00:46] wallyworld: that's going to cause problems, due to code expecting controller==model [00:46] wallyworld: (without the branch I'm working on) [00:46] fark [00:47] axw: so we need to get your current branch landed i guess [00:47] wallyworld: yeah, I think so. [00:47] wallyworld: in the mean time, CI could prefix with local. [00:47] not sure how much effort that'd take [00:47] ok, i'll let the qa folks know. i don't think they'd be up for changing scripts now [00:56] wallyworld: they'll need to change them to expect the model name to be "admin" anywya, in case that's not obvious [00:56] unless we change it to be the non-prefixed controller name as an intermediate step [00:57] axw: i'm discussing with sinzui in #juju now === ses is now known as Guest63483 [04:00] Anyone able to explain to me why juju does what it does to /root/.ssh/authorized_keys, and what I can do to fit in with what it does whilst still enabling what I want, which is that a charm needs to be able to write working entries in there. [04:00] There should have been a ? somewhere in that line. [04:04] np [04:18] does anyone know how to debug the uniter tests? The failures basically read like "Here's a big struct vaguely describing what the test is... oh something failed somewhere" [04:19] brb [04:20] anyone here know about the multi user stuff? say we were to run it on top of an openstack cloud, we would only need the top env to be deployed with juju 2.0, right? the underlying cloud could be a stable juju, say 1.25.3? [04:36] back [04:37] bradm: yeah, the juju that deploys openstack wouldn't need to be multi-user unless you cared about having multiuser for controlling openstack itself [04:37] bradm: you could definitely have whatever juju deploys on top of openstack be 2.0+ to get multi-user for those services [04:38] bradm: the layers don't talk to each other at all. As far as the top level juju is concerned, it doesn't know or care how openstack is deployed. [04:39] natefinch: excellent, I thought that was the case but wanted to confirm before deploying with 1.25.3 for the underlying stack [05:08] wallyworld: RB bot seems to be asleep, here's the first fix: https://github.com/juju/juju/pull/4442 [05:09] ta, will look soon [05:20] axw: damn, you and i are going to conflict :-) [05:20] wallyworld: where? register? [05:21] SetBootstrapEndpointAddress [05:21] maybe elsewhere [05:21] wallyworld: ah. my changes there are pretty trivial [05:21] yeah, i have 2 tests to fix before proposing, we can land yours first [05:28] axw: so just to confirm, "-m model" works, and tags are without local- [05:28] wallyworld: I'll bootstrap aws now to verify, but tested with lxd and -m model works [05:28] great [05:36] axw: wallyworld: when/if u get a chance, PTAL "simple"streams PR - surface difference increased somewot :P [05:42] wallyworld: confirmed that tags don't have local in them, and I can do "juju status -m " [05:43] awesome [05:44] anastasiamac: if you must have them at all: please move all of the bug references out of the production code, and into the tests that verify they're fixed [05:45] if we break the code, the tests will fail and the broken tests will point at the bugs [05:45] then we don't have bug references littered all over the code [05:48] axw: sure [05:58] wallyworld: based on your comment "we don't want one or the other", should we be changing the data sources to take a set of public keys? [05:59] axw: we could do that, or create separate datasources. it's really a corner case for this image-metadata-url datasource. in practice, we don't have the private key used on cloud-images.u.c so maybe we just use the custom key if defined [06:00] and leave off the key if not defined [06:01] wallyworld: so custom ones just use the "user signing key"? sounds sane [06:01] axw: except that we also control the official key for tools [06:01] and so the qa guys may want to test that with a custom source [06:01] simple :) [06:02] streams :-) [06:02] but we do know what's an image vs tools data source so we can make a call there [06:02] yup [06:02] except for the common data source at bootstrap [06:03] which points to a top level dir containing tools and images from memory [06:03] maybe we construct a couple of data sources as needed and allow a signing verification error to be non fatal [06:03] i think that's what we do now [06:04] wallyworld: different topic: would you be ok with temporarily reverting adding the "local." prefix altogether? then when we're through CI we can add it back, and change the initial mode to admin at the same time [06:05] mode=model [06:05] axw: sure, the thought had crossed my mind, but the pr landing now makes things good right? [06:06] wallyworld: still can't destroy-controller with the controller name minus prefix [06:06] wallyworld: dropping the prefix would fix that [06:06] we were going to match local if there were no non-local? [06:06] that might be a smaller change [06:08] wallyworld: so I'm a bit reluctant to add that logic into jujuclient, which currently is agnostic of the names. but adding it in anywhere else would be error prone, given the number of places we're going to write them. and the spec doesn't call for it... [06:09] it is agnotic, it was to be a short term fix. the issue is that if we ship with the UX deviating from the spec in such a visible way, there may be issues [06:09] ie fix for beta1, remove as soon as beta1 ships and Ci gets updated [06:11] wallyworld: it's definitely not not going to be a smaller fix, btw. we'd need to canonicalise the controller names, which we currently expect are canonical [06:11] ok, i guess we can release note it loudly [06:11] wallyworld: and also already deviating from spec, since model is not admin [06:12] yes, but that's not as in your face :-) [06:12] it should be trivial to fix both at the same time [06:12] ok [06:12] let's revert for beta1 [06:12] and we'll ensure Ci drops the -m etc at the same time [06:14] wallyworld: another question I have is whether it should be local. or local: -- we use local: for clouds [06:14] yeah, and the spec uses local.model [06:14] we can ask rick [06:15] wallyworld: hrm actually it can't be local:, because then there'd be ambiguity with "local" as a controller name [06:15] ah yes [06:17] axw: right, time to see how bad the conflicts are [06:17] * wallyworld takes a deep breath [06:17] wallyworld: sorry :) [06:17] shouldn't be too bad [06:17] tis all good [06:19] 4 files [06:24] anastasiamac: the answer to your last question on the PR is ^^^^ [06:24] wallyworld: (above "different topic") [06:24] erer [06:24] anastasiamac: [06:29] axw: wallyworld: plz clarify in PR for posterity (and my sanity) [06:30] axw: m not seeing a definite answer above, just tossing ideas :D [06:49] wallyworld: trivial one: http://reviews.vapour.ws/r/3881/ [06:49] looking [07:25] axw, hey, could this https://github.com/juju/juju/pull/4444 also fix bug 1546043? [07:25] Bug #1546043: unit tests leave apiserver/logsink.log behind [07:25] dimitern: that's the one [07:25] axw, awesome! it was bugging me for a while now :) [07:29] wallyworld: I'm going to rename the "create admin model..." card and create another one, since bulk of the work is done. just need to flip a switch to set it to admin [07:29] +1 [07:29] actually I'll just create a new card and transfer points [07:31] wallyworld: when you're done, can we have a chat about what's next? [07:31] axw, you've got a review btw, and I hope you don't mind - I assigned yourself to that bug [07:31] dimitern: thanks, not at all [07:31] axw: sure, more tests to fix, won't be long [07:33] dimitern: master's blocked though, I'll land it once the beta's out I guess [07:36] dimitern: why do we care about sorting the addresses w.r.t. v4/v6 on the client? [07:36] axw, that's ok - btw I've just confirmed your patch prevents apiserver/logsink.log creation [07:36] dimitern: thanks [07:37] axw, ah, that due to the somewhat well-intended-but-not-well-implemented prefer-ipv6 setting [07:38] dimitern: where does that preference actually matter though? [07:38] dimitern: I'm wondering if we should care on the client [07:38] axw, it will change soon (first for maas, later for all providers gradually), as we're making controller endpoint selection based on spaces [07:38] because we try all addresses [07:39] axw, we do try all of them, but we still put the last we successfully connected to on top [07:39] axw, and yeah, it shouldn't matter at the client side [07:40] dimitern: ok, thanks [07:41] wallyworld: ^^ maybe we don't need to worry about it anymore [07:41] that would be good [07:42] axw: quick chat now in standup channel? [07:42] wallyworld: be there in a sec [08:20] dooferlad_: running 15 mins late for our sync. sorry! [08:23] axw: st.ModelUUID is different to st.Model().ComtrollerUUID() since the latter returns model.serverUUID [08:23] wallyworld: the UUID of the admin model is the same as the controller UUID [08:24] axw: i guess that's true but it seems then we're making an assumption that may change and the code as it is is immune to that [08:24] wallyworld: there's also a controllerTag field on state, we could just expose that [08:24] ok, maybe the latter is better [08:25] wallyworld: avoids a db read [08:25] indeed, but mongo is web scale === ashipika is now known as ashipika__ [08:35] wallyworld: finished reviewing === urulama__ is now known as urulama [09:10] anastasiamac: I suggest, for now, you change the image datasources just to use the user-supplied public key [09:10] anastasiamac: nobody is likely to be using the official one with a custom location [09:11] anastasiamac: and that's a separate issue as you noted in the bug [09:11] anastasiamac: so, drop the ImagePublicKey function you added, and just use simplestreams.UserPublicSigningKey in the *image* data sources only [09:11] anastasiamac: the tools ones should continue to use SimplestreamsJujuPublicKey [09:14] dooferlad_: ping [09:16] anastasiamac: you have a shipit once that's addressed [09:17] dimitern: ping [09:17] frobware, pong [09:17] dimitern: can we talk about L2 in the standup HO? If not convenient just say. [09:19] frobware, sure, sounds good [09:19] frobware, omw === ashipika__ is now known as ashipika [09:22] wallyworld: still buggered: http://juju-ci.vapour.ws:8080/view/Juju%20Revisions/job/canonistack-deploy-trusty-amd64/4041/console [09:23] oh [09:24] axw: so it's just the kill. the cli *appears* ok [09:25] oh, also the connect [09:25] after bootstrap, damn [09:25] wallyworld: actually the kill worked, bootstrap returned an error code [09:25] oh, maybe both failed actually [09:26] wallyworld: cannot read controller info: model "canonistack-deploy-trusty-amd64:canonistack-deploy-trusty-amd64" not found [09:26] for bootstrap and kill [09:26] axw: for legacy, we record foo:foo, in new client store, we want just foo, right [09:26] so maybe there's a mix up somewhere [09:27] wallyworld: oh, my bad. bootstrap is failing due to ssh timeout [09:28] ... probably because canonistack [09:28] and destroy is failing because it didn't bootstrap [09:28] * axw checks a different substrate [09:30] wallyworld: sorry, canonistack is non-voting. I'll raise the alarm if another voting substrate fails [09:30] whew [09:30] so far so good on azure.. [09:33] wallyworld: it's a little bit unclear, but I think that test you removed is to ensure we can connect with the addresses/creds on disk, without requiring the bootstrap config [09:34] ok, i'll take a look [09:46] wallyworld: winner: http://juju-ci.vapour.ws:8080/view/Juju%20Revisions/job/azure-arm-deploy/217/console [09:47] goo :-) [09:47] good [09:55] wallyworld: sorry, I didn't see that you had just changed the existing blank address check... leave it if you think there's potential for errors there [09:56] axw: a bunch of tests fail which i am fixing. maybe we can hope juju itself is now not broekn anymore [09:57] wallyworld: I'll have to go soon, but will check back later to see your changes [09:57] ok, np, have a few to go [09:57] wallyworld: or I can review tomorrow if you want to finish it tomorrow [09:58] would be nice to land tonight, take a peek if you want later and hopefully will be +1 [09:58] sure [11:16] dimitern: frobware: http://reviews.vapour.ws/r/3884/ [11:17] voidspace, looking [11:18] dimitern, am bumping up against cmd/jujud/agent.MachineSuite.testAddresserWorkerNewResult [11:18] dimitern, is it still relevant, or should we actually just never run it? [11:18] fwereade, let me have a look [11:18] dimitern, (or always? which would be bad, because it currently seems like "never" ;p) [11:18] cheers [11:19] fwereade, ah, it's disabled as flaky [11:20] fwereade, but we're dropping the address-allocation feature flag soon [11:20] dimitern, indeed, but I want to know what the status is of the stuff it's dealing with [11:21] fwereade, you mean whether to migrate it to use the dep engine? [11:22] dimitern, I already have, I'm just not 100% sure what the expected behaviour is [11:22] dimitern, and the agent tests are not making it any easier to figure out ;p [11:23] fwereade, as it currently is, it should start and stop immediately when address-allocation feature flag is off (as it calls CanDeallocateAddresses first thing before it starts, which in turn uses the flag) [11:24] fwereade, was that helpful? :) [11:24] dimitern, it confirms that it's currently doing the right thing [11:24] dimitern, and I think that *that* specific behaviour doesn't need testing at agent level [11:25] dimitern, well, ok, it's arguable [11:25] fwereade, agreed - those couple of tests effectively test the finished worker behavior [11:26] dimitern, ehh, it's mainly just a bit annoying that my check-all-workers-are-running thing fails when one of the standard workers is sanely and blamelessly not running [11:27] fwereade, ah :) I see [11:27] dimitern, and while I *could* just ignore that skipped test I would *know* it'd fail against my current changes when reactivated, and that would be evil [11:28] dimitern, so I need to take some action and am rambling at you in the hope that it will become clearer :) [11:28] fwereade, I wouldn't worry too much about that particular test - perhaps an expected failure when not skipped + TODO will make it obvious? [11:29] dimitern, it's still a timebomb for whoever hits it -- do you recall the nature of the flakiness? [11:30] fwereade, yeah, it failed to pass under heavy load IIRC [11:30] dimitern, ok, then I'm entirely unsurprised, bloody singularRunner patch :) [11:31] * fwereade hopes what he has is a bit less awful [11:31] fwereade, my point about not worrying too much about is that we'll drop the addresser soon anyway [11:31] dimitern, that's even better then [11:32] frobware: thanks for the review [11:33] voidspace, I have a concern re updated dependencies.tsv btw [11:33] frobware: the uint comes from the gomaasapi Space object - so unrelated [11:33] voidspace: np, the only thing that really stuck out was int/uint for ID [11:33] voidspace, changing just the sha hash makes it difficult to compare the dates [11:34] dimitern: how do I get the date? Running the update steps for dependencies.tsv shown in CONTRIBUTING.md omitted the date. [11:34] dimitern: I can just put todays date in [11:34] voidspace, godeps github.com/juju/juju/... [11:34] voidspace, dumps the result at the end [11:35] dimitern: with no dates [11:35] dimitern: on my machine [11:36] voidspace, make sure you have the latest godeps [11:37] voidspace, go get -u -v launchpad.net/godeps/... [11:38] frobware: the "int" is purely to convert from the float that comes out of json [11:38] frobware: I can use uint I suppose [11:38] voidspace, that's what I got on my in-progress branch when running godeps github.com/juju/juju/...: http://paste.ubuntu.com/15099666/ [11:38] dimitern: are you sure *you* have the latest version? [11:39] heh [11:39] voidspace, yes, I did run go get -u -v launchpad.net/godeps/... before suggesting it :) [11:39] dimitern: I've updated and I get dates [11:39] dimitern: I'll add it [11:40] dimitern: I've updated that [11:42] Bug #1546492 opened: It's difficult to debug-hooks storage hooks [11:42] voidspace, tyvm [11:43] voidspace, almost done with your review btw [11:45] Bug #1546492 changed: It's difficult to debug-hooks storage hooks [11:45] frobware: fmt.Sprintf("%.0f", float) works... [11:45] and is less round the houses [11:46] network.Id(fmt.Sprintf("%v", x)) will also work for v interface{} :) [11:46] voidspace: yep [11:46] dimitern: but if x is a float %v will do the wrong thing [11:47] dimitern: i.e. 2.0 will become network.Id("2.0") and we need network.Id("2") [11:47] dimitern: and what comes out of json is a float even if the source is an int [11:47] because json [11:47] because javascript [11:48] axw: if you're around, pr updated === jamespag` is now known as jamespage [11:48] Bug #1546492 opened: It's difficult to debug-hooks storage hooks [11:48] voidspace, uint(2.0) then? [11:50] or fmt.Sprintf("%.0f", v) [11:50] rather than strconv.Itoa(uint(...)) [11:53] dimitern: frobware: did you get disconnected too or was it just me? [11:57] dimitern: I don't think newAddressOnSpaceWithId should move to network [11:57] dimitern: in future we'll be setting provider id *or* name [11:57] dimitern: at which point we'll probably need a "NewAddressOnSpaceId [11:57] dimitern: this PR is a little bit of an intermediate one where we're setting both, but it won't stay like this [11:57] dimitern: probably next PR... [11:58] voidspace:just you [11:58] frobware: weird [11:58] voidspace, I think freenode is experiencing DDoS attacks today (yet again) [12:00] dimitern: ah [12:00] voidspace, I'm not sure about "or" [12:00] voidspace: I take it back. [12:00] frobware: hehe [12:00] dimitern: that's the current plan [12:00] dimitern: we shouldn't use space names directly from the provider [12:00] dimitern: so if the space comes from the provider we should set the id not the name [12:01] dimitern: so if we have a name set we *know* it's a juju name [12:01] voidspace, what's wrong with having either name and no ID or both? [12:01] dimitern: rather than having a mix of juju names and provider names floating through our code [12:01] dimitern, voidspace: why can't we indirect through the ID for the name? [12:01] dimitern: because if you see a name on an address you need to know if it's a juju name or provider name [12:02] frobware: we should do, that's what I'm saying [12:02] voidspace, frobware, I think a space name should always be a valid juju name [12:02] frobware: we won't have an id for ec2 addresses of course because there's no provider id [12:02] dimitern: but that isn't currently the case with maas [12:02] dimitern: and our current definition of "valid juju name" [12:02] dimitern: if they change to be in sync that's a different matter [12:03] but even then there's no *problem* with being strict on using id *or* name - there's just less need [12:03] but at the moment there's a need [12:04] voidspace, yeah - they should change in names so that [A-Za-z0-9-_]+ is a valid juju space name [12:04] voidspace, but as it will be the case in maas [12:05] voidspace, s/but// [12:05] dimitern: yep [12:05] voidspace, that's why I suggested to keep the name even if we have an ID [12:05] dimitern: my next PR will address this anyway, so I'm going to drop that issue from this PR [12:06] dimitern: the trouble is that until they do it (which they say they will but may never do) it's dangerous to mix up juju names and maas names [12:06] voidspace, I'm ok with that [12:06] dimitern: cool [12:08] voidspace, dimitern: totally agree with "it's dangerous to mix up juju names and maas names" +1 [12:11] frobware, voidspace, the only danger I can see is due to bugs in maas atm [12:12] dimitern: we should stop working around bugs, IMO. fix the root cause. [12:13] wallyworld: are you still working on changing UpdateControllerAddresses to not take UUID? [12:14] frobware, I'm not saying we should work around the bugs in maas that prevent us in some cases to use a valid maas space name as a juju space name [12:14] dimitern: well, that maas space name definition is broader than juju is not a "bug", it's due to a lack of communication between us [12:14] dimitern: it sounds like they will fix it though [12:14] dimitern: duplicate names are a bug :-) [12:15] axw: it takes a controllerUUID in one case - after api login when controller name is not known, only uuid. i guess i could look up controller name at that point [12:15] voidspace, yeah - spaces allowed in names and duplicate names are 2 bugs they need to fix [12:15] wallyworld: when would we ever not know the controller name? [12:16] voidspace, they said they'll fix those two and we should consider them "fixed" - i.e. build our code onto that assumption [12:16] damn, in that one place, i didn't see we had controlle rname [12:16] i'll rework a bit [12:17] voidspace, which means if it's not so - i.e. CI maas setups or other (older) 1.9 maas deployments won't work [12:17] until those bugs are fixed [12:21] Repeating earlier question: Why does juju mangle /root/.ssh/authorized_keys, and what I can do to fit in with that, whilst still enabling my charm to write a working entry in there? [12:26] axw: updated, feature test for register runs ok [12:26] wallyworld: ok, looking in a sec. I noticed in CI that manual bootstrap fails because we're now expecting "juju bootstrap manual/", and CI is passing bootstrap-config in config.ayml [12:27] wallyworld: should I try and fix it, or just ask CI to retest with that small change? [12:27] axw: yeah, saw that too, i beliebe aaron knows about that [12:27] wallyworld: we could allow bootstrap-host in config as well, would be better not to deviate from other providers tho [12:28] i will ask them to re-test tomorrow, i prefer consistency [12:29] wallyworld: LGTM, thanks for changes [12:30] axw: np, thanks for review, Ci looking good too [12:30] except for manual of course [12:31] wallyworld: and a few things say they're still pending, but yeah, lots of green :) [12:32] release notes tomorrow, will be quite an essay === perrito667 is now known as perrito666 [13:15] dimitern: frobware: my branch has an unrelated failure in featuretests [13:15] dimitern: frobware: it's actually a space discovery related failure (can't login) [13:15] dimitern: frobware: I'll land my branch on maas-spaces2 and fix this failure with a follow-up on master [13:18] voidspace, sounds good +1 [13:40] is there a quick getting started guide on building juju from source? [13:40] I think I found a bug in the alpha but I wanted to confirm with trunk [13:44] jcastro, maybe there's a ppa with daily builds from trunk? [13:44] that would be a good idea regardless :) [13:46] (I realize this is not what you're asking for but figuring out such ppa would make me happy as well solve your problem to test trunk :)) [13:50] yeah I am wondering why there's not just a daily PPA [13:59] natefinch: wasn't there a thread recently that talked about a doc on building juju core? I can't seem to find it and thought you replied in it. [14:00] Ursinha: jcastro we're looking at ways to get builds out faster. Because of the use of streams and such, we don't do daily builds. We need a less invasive way to get that out. [14:06] rick_h_: it's not really very well written for first timers: https://github.com/juju/juju/blob/master/CONTRIBUTING.md [14:06] natefinch: ah ok thanks [14:07] rick_h_: we really should write a step by step "this is how you build juju"... which I think that document wants to be, but tries to do too much, and so obscures the actual steps you need to do to build [14:07] natefinch: agreed, good to find this and we'll see what we can do. [14:10] natefinch, I used that doc for my first build, wasn't too bad [14:12] fwereade: [14:12] fwereade: heh, oops. Wanted to ask about uniter tests [14:15] Bug #1546561 opened: juju sets wrong default gateway for lxc containers when using JUJU_DEV_FEATURE_FLAGS=address-allocation [14:18] Bug #1546561 changed: juju sets wrong default gateway for lxc containers when using JUJU_DEV_FEATURE_FLAGS=address-allocation [14:24] Bug #1546561 opened: juju sets wrong default gateway for lxc containers when using JUJU_DEV_FEATURE_FLAGS=address-allocation [14:31] redacting the API params during tests is dumb. === Ursinha_ is now known as Ursinha [14:34] natefinch, oops sorry; uniter tests? [14:36] fwereade: there's a bunch of "match hooks" where we expect a specific list of hooks to be fired. I presume that if that list changes, that's a bad thing? I'm still working on this resources code, and what I'm getting is an extra upgrade-charm hook firing [14:36] uh, that part still exists? [14:37] unless something changed in the last 5 or 6 days? [14:38] I hoped that to go in the uniter refactor sprint [14:38] also, it seems like there's a bug, or at least an inefficiency in the matchHooks code: https://github.com/juju/juju/blob/master/worker/uniter/util_test.go#L250 [14:39] This match hooks call gets retried until a LongWait expires... but really, after the first time this check fails, we know it'll never ever pass [14:40] natefinch: it is not a bug, its a feature :p [14:40] we don't ever remove things from the hooksCompleted list, so if one of them doesn't match... that's it, we're done, it'll never match. [14:41] no, really its a design flaw [14:42] that whole thing is a big bad idea, it is depending a lot on an implementation accident [14:42] but fwereade might not agree with me :p [14:42] heh [14:43] yeah, that was my question... I'm changing the implementation slightly... in theory it should fire the exact same events that it did before, but it's possible some niggling bits in watcher/worker code cause a hook to fire more than once... so I can't tell if I've introduced a bug, or if the tests are just too specific about their expectations. [14:45] ok, fwereade might correct me here, tests rely on a hook order that we dont really guarantee, much of the fact that the expected and obtained hook match depends on delicate time balance [14:46] som, for example, here's what my change does to the "SteadyStateUpgrade" hooks: http://pastebin.ubuntu.com/15100335/ (I changed the line prefixes so they make more sense without looking at the code, and so the lists match up for easier visual comparison) [14:47] (that used to be ctx.hooks vs. ctx.hooksCompleted) [14:50] natefinch: you are triggering more hook calls, which is not bad per se [14:57] fwereade: maybe related to your login issue ^^ "voidspace> dimitern: frobware: it's actually a space discovery related failure (can't login)" [15:03] dimitern, voidspace: release note sync [15:06] frobware: oops, soory - lost track of time [15:06] frobware: 5 mins, grabbing coffee [15:11] oops sorry, omw [15:19] natefinch: ericsnow_: just want to o/ since no standup in the morning [15:20] natefinch: ericsnow_: how's everything going? [15:20] dimitern: frobware: http://reviews.vapour.ws/r/3886/ [15:21] katco: good [15:21] katco: talked with Ian about that worker [15:21] frobware: fwereade: what login issue? === ericsnow_ is now known as ericsnow [15:21] ericsnow_: yeah? what's the consensus [15:22] katco: we don't need a worker for that [15:22] katco: the precedent (downloading lxc images) indicates that a worker should not be used [15:23] ericsnow: it occurred to me this morning that maybe i didn't represent it accurately. this will be running 100% of the time polling the charm store for all the services, correct? [15:23] katco: no [15:23] katco: that is a different worker [15:23] ericsnow: oh, i see you picked up a diff. card [15:23] ericsnow: ok, so the download worker, is not a worker. gotcha. what is it? [15:24] katco: the one we were discussing related to using a worker to download from the charm store into state in response to resource-get [15:24] Bug #1546272 changed: doc: docs/devel/developer-layers-interfaces typo: get_remove === Ursinha_ is now known as Ursinha [15:25] katco: in the API code we stream directly from the charm store into the blob store [15:25] katco: (API server side) [15:25] ericsnow: and if the stream errors out? [15:25] katco: since resource-get blocks a worker doesn't add anything [15:25] katco: then we retry (on the server side) [15:26] ericsnow: what's the mechanism for doing so? [15:26] katco: the usual suspect: utils.AttemptStrategy [15:26] * natefinch is here but doesn't want to interrupt. [15:26] natefinch: it's irc ;) [15:27] katco: :) [15:27] ericsnow: ok, well glad you worked it out. thanks for bringing it up [15:28] ericsnow: natefinch: i'm working on release notes for resources; i'll want you to review here in a bit [15:28] katco: also, per wallyworld regarding workers: "workers are more for reacting to changes in state" [15:28] katco: no, thanks for the discussion [15:28] katco: the uniter tests are a beast. There are currently about 5 failures that I'm trying to determine if they represent bugs I have introduced, or tests that need to be updated. [15:28] ericsnow: good to know [15:28] katco: it helped paint an even clearer picture about workers in general :) [15:29] natefinch: :( i understand those are difficult to deal with. thanks for reaching out to perrito666. we need that done today, so do whatever is needed to make that happen (e.g. pair if necessary) [15:29] ericsnow: definitely [15:30] katco: FYI, wallyworld also pointed me to the "charmrevisionupdater" worker as something similar to the charmstore poller I'm writing for resource info [15:30] natefinch: ericsnow: btw we have a new card we need to point: validating file extensions [15:30] ericsnow: oh, awesome! [15:31] katco: it's not particularly re-usable for this, but does provide an example; and validation that I'm on the right track :) [15:31] ericsnow: just as useful imo :) [15:31] katco, natefinch: I'm free to point that whenever y'all are ready [15:32] fwereade: let me know if you come available [15:32] ericsnow: natefinch: gimme 5 and we'll hop in moonstone? [15:32] ericsnow katco sounds good [15:33] katco: +1 [15:38] ericsnow: natefinch: ok i'm in there [15:54] frobware, voidspace: nah, I was being dumb, had changed a client-side code path without full awareness [15:55] natefinch, perrito666: sorry! short version: I am inclined to consider unexpected hook firings to be a problem [15:56] natefinch, I am available, just quite deep into code, sorry [15:56] fwereade: understandable. that was my assumption as well. [15:56] natefinch, cool [15:58] natefinch, it *may* be the case that the layers of indirection around the tests are such that we can't reliably induce specific uniter behaviour, but that's still a problem [15:59] natefinch, by the way, I think I'm misunderstanding something -- did I see a recent mail from jam saying that new resources are only delivered on resource-get? [16:00] natefinch, is it the case that we do an upgrade-charm to inform of new *available* resource versions or something, but nothing has actually changed in the charm at that point? === kwmonroe_ is now known as kwmonroe [16:03] fwereade: we fire upgrade-charm when you push up a local resource to the controller. At that point, the unit does not have the bytes from the new resource, and must call resource-get [16:04] natefinch, upgrade-charm seems a bit surprising compared to resources-changed or something? [16:06] natefinch, the only connection I can see is that they're both sort of to do with the charm, and you can say that about almost anything ;p [16:06] fwereade: the idea is that the resource bytes are integral to the functionality of the charm. So changing a resource is just as disruptive as changing the charm version [16:07] fwereade: firing upgrade-charm is what is declared in the spec, per rick_h_, mark, jam, etc [16:07] natefinch, oh joy [16:08] * natefinch is Not In Chargeā„¢ [16:08] natefinch, we start by saying "charm logic and payloads can vary independently, let's add tooling to make those use cases easier" and we end up with "upgrade-charm now means two completely distinct things, have fun charmers!" [16:09] * marcoceppi watches as this develops [16:10] * fwereade turns to marcoceppi and gestures vaguely -- if this *is* a happy outcome for you and those you represent, that's awesome -- is it? [16:11] marcoceppi, fwereade: http://media0.giphy.com/media/4pMX5rJ4PYAEM/giphy.gif [16:11] natefinch, haha [16:11] fwereade: I had concerns of not being a hook available, but was convinced that this was okay. If there's still a way to get a hook I'd be happier [16:12] fwereade: if charms have a way to query the version of the resource, without fetching it, then it's not that bad. Just some additional logic in upgrade-charm to determine if it's is new charm code or new resources [16:12] marcoceppi, yeah, absolutely, any state we show you as a charmer *should* be guarded by some hook that's guaranteed to fire if it changes [16:13] marcoceppi, or both ;p [16:13] but I'd rather have this feature with upgrade-chram than delay to 2.1 [16:13] we can build libraries in the new reactive framework to work around this [16:13] or rather, not work around, but enhance this [16:15] marcoceppi, sure, understood, I'm just digging to check that "when we introduce new state to charmers, we should introduce new hooks to inform of changes to same" fits with your baseline desires [16:16] fwereade: I'd prefer new hooks, the more granular the event dispatcher the better [16:16] marcoceppi, because, y'know, if we don't tell you about the changes that's obviously stupid, and if we repurpose old hooks we potentially expose latent bugs all over the charm store by changing what hooks will get run in what situations [16:17] marcoceppi, cool [16:18] marcoceppi, to put it another way: you *can* work around a mish-mash of events, many of which mean "some combination of X, Y and Z distinct things changed", but it kinda sucks to have to [16:18] in so many words [16:19] with the exception of the hook thing, th eresources stuff is brilliant [16:22] fwereade: marcoceppi: sorry, just read through the backscroll [16:23] fwereade: marcoceppi: i think the reason that decision was made is because resources never stand alone, it's always the tuple of the charm and resources that is the concept to consider [16:24] right, but you can also upgrade resources seperately from the charm [16:24] fwereade: marcoceppi: so having being notified of a resource being updated is meaningless if you accept that premise [16:24] marcoceppi: my understanding is that you cannot [16:24] I was informed otherwise [16:24] marcoceppi: resource updated = new rev of the charm [16:24] but since you've done the work I'm happy to accept that [16:24] marcoceppi: who am i contradicting? [16:24] so if I release a new version of my applicaiton, that's a charm rev? [16:25] marcoceppi: yep [16:25] but it's just an updated resource [16:25] marcoceppi: the 'revision' of a charm becomes a 'revision set' [16:25] marcoceppi: that set is the published set of the charm rev, and the rev of each resource [16:26] marcoceppi: updating any part of that, is an new revision set that much be processed by the upgrade-charm hook. [16:26] that's the way it's currently set out. [16:26] marcoceppi: check out the 3rd bullet of the 1.0 mvp [16:26] okay, so is there a way for me to query, without first accepting the payload, that I've got a new version? [16:26] in the charm [16:26] marcoceppi: as a charmer, not currently. as an admin, yes [16:26] marcoceppi: we need that yes. It's not there now, but agree. Honestly it was an optimization thing that came up as we get into using it [16:27] okay [16:27] marcoceppi: and good feedback that we need to go back with [16:27] then I'm cool [16:27] marcoceppi: you are cool, marcoceppi. [16:27] as long as I can say "resource update" [16:27] very cool :) [16:27] at the end of the day, with reactive, we're not going to care about upgrade-charm hook [16:27] well some will, most wont [16:27] marcoceppi: katco I think we need something like resource-version or resource-changed to go with resource-get. My concern with resource-changed is that if something fails and you retry, will we have the right answer. [16:27] we'll only really care about @when('resource..updated') [16:28] marcoceppi: katco so I think there's some thought we need to get that right [16:28] rick_h_: well, isn't resource-changed redundant? i.e. anytime it would fire, upgrade-charm would also have fired [16:28] katco: not as a hook, but as a hook command [16:28] katco: e.g. resource-changed mysourcecode [16:28] returns true/false [16:29] rick_h_ katco is resource-list a hook-tool? [16:29] katco: so if one new resource is uploaded out of 3, I can tell which one I need to untar, process, etc [16:29] marcoceppi: no, juju list-resources [16:29] katco, rick_h_: the issue is surely that upgrade-charm means "your charm logic has changed" and resource-changed means "some of your tweakable blobs have changed" [16:30] rick_h_: well, that is an optimization. i believe resource-get on a resource that hasn't changed is a nop [16:30] so, as a charmer, I want to be able to run "resource-list" in a hook with --format json and get something like "resource_name": "version" for each resource [16:30] katco: I'm not so sure [16:30] katco, rick_h_: that we have a tuple of things that represent "exactly what's running on this unit" is... coincidental, if you like? we don't even officially expose charm urls to charmers right now [16:31] ericsnow: natefinch: is resource-get on an unchanged resource a no-op right now? [16:31] FWIW, we certainly could add a command that tells the unit whether or not it has the latest version of a resource. We store the version each unit has and the version that is current for the service. [16:31] fwereade: well it was designed/built to not be coincidental. I see what you mean on "charm logic" vs "blob of data" though. [16:32] fwereade: the goal is that when you publish to the charmstore you're declaring "this revision of the charm logic with this revision of this blob and that blob pass tests and work together" [16:32] natefinch: exposing that I think will be crucial to making the join charmversion + resourceversion + upgrade-charm work for a charm authors perspective [16:32] fwereade: it's the local case that's more muddy [16:32] joint* [16:32] rick_h_: fwereade: yeah it all begins at 1st principles. either the tuple defines the charm, or it doesn't. if it does, then there's no such thing as updating just the blob [16:32] marcoceppi: +1 the thing is catching the failure cases and making sure it behaves predictably [16:33] katco: we don't do that optimization... resource-get will always just download the bytes [16:33] natefinch: fair enough for v1 [16:33] natefinch: right, I think we can't noop right now because it means the hooks aren't idempotent [16:33] rick_h_: mm not true [16:33] natefinch, katco: so the upshot is that every time you get an upgrade-charm, you *have* to resource-get *all* your resources before you can safely move on? [16:33] rick_h_: sure they are... the bytes on disk are the same before and after [16:33] rick_h_: idempotent = end state is always the same [16:33] katco: ah, I see. the bytes are in the folder [16:34] rick_h_: correct [16:34] fwereade: yes :/ [16:34] fwereade: yeah, as designed currently\ [16:34] how does this even happen with a supposed "user focus" in design? [16:34] fwereade: no, it just means you have to process your logic. [16:34] fwereade: happy to hop on a call if you've got questions [16:34] rick_h_: would join to be a fly on the wall [16:34] fwereade: but you're starting to cross into unproductive territory here [16:35] rick_h_, let's, I fear that I misunderstand something horribly [16:35] https://plus.google.com/hangouts/_/canonical.com/rick?authuser=1 for all interested parties [17:08] I think I broke RB [17:10] k people bbl [17:10] cheers [17:28] Bug #1544796 changed: Backup restore fails: upgrade in progress [17:37] Bug #1544796 opened: Backup restore fails: upgrade in progress [17:40] Bug #1544796 changed: Backup restore fails: upgrade in progress [17:43] Bug #1544796 opened: Backup restore fails: upgrade in progress [17:49] Bug #1544796 changed: Backup restore fails: upgrade in progress [17:58] Bug #1546662 opened: juju does not respect --agent-version [18:01] Bug #1546662 changed: juju does not respect --agent-version [18:12] anyone ever heard of StateServerInstances() ? [18:13] Bug #1546662 opened: juju does not respect --agent-version [18:16] Bug #1546662 changed: juju does not respect --agent-version [18:16] fwereade: still here? [18:17] fwereade: nevermind [18:19] Bug #1546662 opened: juju does not respect --agent-version [18:19] mup: you are a very annoying thing [18:19] perrito666: Strictly speaking, I'm not intelligent enough to understand that. [18:20] it shows [18:20] :p [18:33] bbl [18:40] katco: so in talking with william and rick, it sounds like we really need a hook command that can tell the charm what resources are out of date for the unit. That should be really easy, since we have all the backend code, we'd just replicate a bunch of juju resources in a hook command. [18:43] natefinch: i agree that sounds like a good idea. let's revisit after we deliver what's on our plate already [18:46] katco: yep... and FWIW, rick was able to convince william that we should stick with the charm-upgrade hook, mostly because it's sorta too late to do anything else. [18:46] natefinch: yeah [18:47] natefinch: i wonder if we looked at how we support things, we couldn't support releasing changes like that more easily [18:47] katco: I think making a new hook wouldn't been the end of the owrld, but it woudl definitely be a few more days work than finishing up what we have... and we don't really have a few days to spare. [18:47] natefinch: then the conversation becomes, "that's a good idea. we'll do that next month" instead of "omg change course! change course! we can't have a breaking change!" [18:48] katco: rick was positing that it could be delivered later... I'm not 100% convinced that's true, though, since it would break charms that rely on charm-upgrade getting called. [18:48] (for resource changes) [18:48] natefinch: we should make more liberal use of our versioned concepts outside of the macro version of juju [18:49] natefinch: and deprecate old versions of things more regularly (apis, hook commands, cli) [18:49] katco: well, yeah, we should deliver min-juju-version, which would help with that [18:49] natefinch: true [18:51] katco: (in our spare time) [18:51] perrito666: you back yet? [19:02] super short review for someone? re-enables showing full API params during tests (if you use the verbose flag during the test). http://reviews.vapour.ws/r/3887/diff/# [19:12] EOW, bye all [19:23] gah, what I wouldn't give for focused unit tests in the uniter code, so I could narrow down where the bug is in my code :/ [19:39] natefinch: i think you and ericsnow better pair up to get through this [19:40] katco: yep [19:40] katco: sorry, missed when he came back online. [19:41] ericsnow: help me ericsnowcurrently, you're my only hope! [19:41] natefinch: let's get it done! [19:42] ericsnow: natefinch: i'm done with admin stuff. reviewing your code now [19:56] cherylj: I've got to get the boy from school and will be late to our meeting with folks. Can you bug thumper please? [19:56] cherylj: I'll have to work with him to move that back since it's sitting in this bad spot [19:57] rick_h_: sure, will do === urulama is now known as urulama__ [20:42] * thumper heading to dentist with daughter two [20:43] thumper: doh, looks like I missed you [20:55] 0 for 2 rick_h_ [20:56] davecheney: yea, he's avoiding me. Scheduling things during school time and then running off to dentists. [20:56] :P [21:23] Katco I won't make it to your standup, stuck in traffic sorry [21:23] perrito666: np, i just marked you and anastasiamac as optional [21:23] perrito666 do it from the car :P [21:24] I can IRC from traffic jams, how cool is that? [21:24] Honestly if the lte dongle wasn't in the trunk I could [21:25] ahh, the person grabbed it from you as you pushed them in there? [21:25] ;) [21:26] There is no respect anymore [21:28] * perrito666 shouts at the trunk [21:35] Traffic is moving bbl [21:53] * perrito666 is back [22:02] wallyworld: around? [22:05] in a meeting [22:06] ericsnow: sorry... I'll give that a test and see if it needs tweaking or anything [22:06] ericsnow: thanks for the help [22:06] natefinch: cool === natefinch is now known as natefinch-afk [22:23] yay go 1.6 [22:28] perrito666: time to pack it up and move juju to 1.6 ;) [22:33] I'm actually thinking of going back to an older version to save those insane compilation times [22:38] bogdanteleaga: the only way to advance is forward [22:40] Bug #1546790 opened: availability zone not set [22:41] perrito666, at least by golang 2.0 the test suite run time won't be the bottleneck [22:58] Bug #1546794 opened: lxd provider's namespace property seems to be ignored [22:58] Bug #1546795 opened: bootstrap manual broken: failed to check provisioned status: subprocess encountered error code 1 [23:01] Bug #1546794 changed: lxd provider's namespace property seems to be ignored [23:01] Bug #1546795 changed: bootstrap manual broken: failed to check provisioned status: subprocess encountered error code 1 [23:11] Bug #1546794 opened: lxd provider's namespace property seems to be ignored [23:11] Bug #1546795 opened: bootstrap manual broken: failed to check provisioned status: subprocess encountered error code 1 [23:13] * perrito666 prepares for standup, suddenly remembers there is no standup today [23:27] perrito666: anastasiamac: axw: meeting finished early, let's have standup [23:29] Bug #1546805 opened: storageprovider seems to be stuck in an infinite loop [23:35] Bug #1546805 changed: storageprovider seems to be stuck in an infinite loop [23:38] Bug #1546805 opened: storageprovider seems to be stuck in an infinite loop