[00:20] <menn0> thumper: looking now
[00:33] <menn0> thumper: ship it with some small fixes
[00:45] <mup> Bug #1538241 changed: 2.0-beta1 stabilization <blocker> <juju-core:Invalid> <https://launchpad.net/bugs/1538241>
[00:54] <mup> Bug #1538241 opened: 2.0-beta1 stabilization <blocker> <juju-core:Invalid> <https://launchpad.net/bugs/1538241>
[01:01] <mup> Bug #1538241 changed: 2.0-beta1 stabilization <blocker> <juju-core:Invalid> <https://launchpad.net/bugs/1538241>
[01:17] <thumper> menn0: ta
[01:25] <axw> rick_h_: I'll test azure and see what's up
[01:25] <rick_h_> axw: ok, there's some general "stuff left behind" i get every once in a while with kill-controller. I'm not sure if this is part of that or just how much is meant to get cleaned up.
[01:25] <rick_h_> axw: ty for following up
[01:26] <axw> rick_h_: you should expect there to be nothing at all left behind
[01:26] <axw> rick_h_: everything is scoped by the resource group, each model gets a resource group
[01:27] <rick_h_> axw: going to see if we can get a GSoC student to help automate that process of pulling the info for the credentials. Wow that's painful.
[01:27] <axw> rick_h_: the azure creds?
[01:27] <rick_h_> axw: ok, yea that's what I assumed, but I need to look more at what you create with the azure cli process vs resource group vs model
[01:27] <rick_h_> axw: working through the whole nodejs cli to create the AD entries and such
[01:28] <axw> rick_h_: sounds good. yes, very painful. amazingly, it's actually much better than when I started working on the provider :)
[01:28] <rick_h_> ugh
[01:28]  * rick_h_ can't imagine
[01:28] <rick_h_> the fact that I have to install nodejs makes me cry much less the multi-step "run this command with this UUID, and this one with this UUID...and..."
[01:28] <axw> rick_h_: I had envisaged a CLI tool to automate all the nodejs scripty bits
[01:29]  * axw nods vigorously
[01:29] <rick_h_> axw: right, it must be something that we can even build into juju/Go based on using the same api in the same way
[01:29] <rick_h_> start out as a plugin and see how far we can get
[01:29] <axw> sounds good
[01:30] <axw> rick_h_: FWIW, MS have acknowledged that it sucks. I'm not sure what they're doing about it though.
[01:30] <rick_h_> axw: yea, well I got pulled into a "juju sucks on AWS" meeting with our folks that work with them
[01:30] <rick_h_> and I promised it'd get all better with the new ARM...and now I'm not so sure heh
[01:31] <axw> the creds bit is worse, I think the rest is much better
[01:31] <rick_h_> definitely
[01:31] <rick_h_> hopefully just a scripting headache to hide from users
[01:47] <thumper> hmm...
[01:47] <thumper> we have this thing called annotations
[01:47] <thumper> and it can be set on "anything"
[01:47] <thumper> but in practice, it seems like it is only ever stored on services
[01:47] <thumper> anyone know of other places it is used?
[01:48] <anastasiamac> thumper: not on anything...
[01:48] <anastasiamac> thum
[01:49] <anastasiamac> thumper: on "gloab entities"... essentially, anything that has a global key and  a tag
[01:50] <anastasiamac> thumper: machines, units, services, charms, models
[01:50] <thumper> well... sure
[01:50] <thumper> but in practice, I think it is just services
[01:50] <anastasiamac> thumper: juju itself may not use all of these but clients might.. i believe GUI is a prime example
[01:50] <thumper> yeah
[01:50]  * thumper goes for the JFDI everything card
[01:51] <anastasiamac> thumper: it would b lovely to know how customers are using Juju (as in feature-focused customer feedback)
[01:51] <axw> rick_h_: I can't reproduce leftovers after kill-controller
[01:51]  * anastasiamac dreaming 
[01:52] <axw> rick_h_: sure they weren't there before you bootstrapped? can you delete manually and see if it happens again?
[01:53] <rick_h_> axw: will see. I gave my creds to mark to try out so have to check if he's using it first
[01:53] <rick_h_> axw: did you deploy anything into it?
[01:54] <axw> rick_h_: no rush I guess. I think CI will notice if junk gets left behind
[01:54] <axw> rick_h_: nope. did you?
[01:54] <rick_h_> axw: yes, jenkins/haproxy, related them, exposed haproxy to get the public IP addr
[01:55] <rick_h_> axw: hmm, maybe I was impatient, I don't see the resource group in the portal at all now
[01:55] <axw> rick_h_: deleting resource groups is super duper slow
[01:55] <rick_h_> axw: I might have caught it with no machines/etc but with the IP/availability set left or something
[01:55] <rick_h_> axw: hmm, ok. I'll try again and see and report back
[01:55] <rick_h_> axw: ty for attempting to reproduce
[01:55] <axw> thanks, no worries
[02:22] <menn0> thumper: here's the State reference counter: http://reviews.vapour.ws/r/3921/
[02:22] <menn0> axw: ping
[02:22] <axw> menn0: pong
[02:22] <menn0> axw: I noticed something strange in master
[02:22]  * menn0 digs out some links
[02:24] <menn0> axw: https://github.com/juju/juju/blob/master/cmd/jujud/agent/machine.go#L1062-L1074
[02:24] <menn0> axw: in master the stateworker has some logic to call unregisterSimplestreamsDataSource when the *state.State is closed (when the worker terminates)
[02:25] <menn0> axw: but the related code to NewStorage and registerSimplestreamsDataSource is no longer present
[02:25] <menn0> axw: openStateForUpgrade still has it though
[02:25] <menn0> axw: and it's in the 1.25 branch
[02:26] <menn0> axw: do you think this is an unintended deletion? or are the calls to NewStorage and registerSimplestreamsDataSource no longer required any more?
[02:27] <axw> menn0: not actually sure. anastasiamac, ideas on above? ^^  I think this is related to the structured metadata changes
[02:27]  * axw pokes around code
[02:27] <anastasiamac> axw: m not sure why it was removed? an overzealous merge maybe?
[02:28] <rick_h_> axw: wow, 6min at "All hosted models reclaimed, cleaning up controller machines
[02:28] <rick_h_> and counting
[02:28] <axw> rick_h_: heh, yeah :/
[02:28] <axw> rick_h_: that's the cost of being 100% sure we're not leaking resources on azure, sadly
[02:28]  * menn0 digs into the history a bit
[02:28] <rick_h_> axw: but yes, all things look gone this time
[02:29] <rick_h_> axw: so I'll chalk it up to impatience last time ty
[02:29] <axw> rick_h_: cool, thanks for confirming
[02:29] <anastasiamac> menn0: eyes of the eagle \o/
[02:29] <anastasiamac> *an
[02:30] <menn0> anastasiamac: well I'm completely changing how the state worker starts so I need to understand how this works :)
[02:37] <axw> menn0: I *think* we don't need it anymore, because we asynchronously populate the state db with metadata, and then return those results to StartInstanceParams. anastasiamac, you did that work... does that sound right?
[02:37] <axw> anastasiamac: that wasn't done for 1.25 was it? which would explain why it's different in the 1.25 branch?
[02:38] <anastasiamac> axw: m not remembering it being asynch.... but i *hope* that it was in 1.25 rathern than not...
[02:38] <menn0> anastasiamac, axw : except what's in master looks wrong... like someone half completed a change or messed up a merge
[02:38] <axw> anastasiamac: asynchronous, as in we have an image metadata worker.
[02:39] <menn0> 1.25 looks sensible
[02:39] <axw> menn0: right, the unregister certainly shouldn't be there if we don't need to register it anymore. still looking..
[02:42] <axw> menn0:  removed by wallyworld in https://github.com/juju/juju/commit/66a6121c6598b2de7d8ac3acec7afbeccdcd958a
[02:43] <axw> wallyworld: do you recall why this change was made? https://github.com/juju/juju/commit/66a6121c6598b2de7d8ac3acec7afbeccdcd958a#diff-f5ec9ed405cc8f3a833355afdc629bd3L1110
[02:43] <wallyworld> axw: otp, give me a few minutes
[02:56] <menn0> axw, anastasiamac: thanks for the thinking and digging so far.
[03:00] <anastasiamac> menn0: axw:wallyworld so the review for it http://reviews.vapour.ws/r/3459/
[03:00] <anastasiamac> has comment
[03:01] <wallyworld> still in meeting sorry
[03:01] <anastasiamac> to my question of "why is this removed/no longer needed"
[03:01] <anastasiamac> the response is
[03:01] <anastasiamac> We only use simplestreams datasources are first searching in state database. Why add state database as a simplestreams datasource? What ends up happening is we search twice in the same location. And the logs show the search the second time errors anyway, so it's doublely evil.
[03:02] <anastasiamac> .... as a consequence, the registration was removed (registerSimplestreamsDataSource)
[03:02] <axw> so we just need to remove the unregister call
[03:02] <anastasiamac> looks like \o/
[03:02] <menn0> anastasiamac: ok, so that means that the openStateForUpgrade should be similarly updated too
[03:03] <menn0> I might JFDI that into master now
[03:03] <menn0> actually no, i'll do it in my feature branch
[03:03] <menn0> it's relevant to the changes there and is almost done
[03:04] <anastasiamac> menn0: i think so :D...
[03:04] <anastasiamac> axw: the call to unregsiter is not doing anything, i think... removing it would be clean \o/
[03:18] <wallyworld> axw: menn0: i think that change was because we were using the stream setting for tools to filter image metadata
[03:39] <menn0> wallyworld, anastasiamac: http://reviews.vapour.ws/r/3922/
[03:39] <wallyworld> looking
[03:40] <wallyworld> menn0: awesome, ty
[08:50] <axw> wallyworld: a handful of juju/api.go-related changes, when you have a moment: https://github.com/juju/juju/pull/4490
[08:50]  * axw bbl
[10:08] <marcoceppi> halp
[10:31] <voidspace> frobware: you want to do 1:1?
[10:32] <frobware> voidspace: yep
[10:32] <voidspace> frobware: I'd like coffee first ~5mins?
[10:32] <frobware> voidspace: ok
[10:41] <voidspace> frobware: omw
[10:42] <voidspace> frobware: uhm... I joined the wrong room. Really on my way now...
[12:53] <perrito666> morning all
[13:29] <frobware> dooferlad, dimitern, voidspace: http://reviews.vapour.ws/r/3914/ the very small master->maas-spaces2 merge we talked about this morning
[13:31] <dimitern> frobware, why not bring in the latest master instead?
[13:34] <frobware> dimitern: can do. I thought we said this morning we would just do this and then the next...
[13:34] <frobware> dimitern: happy to drop and go for the bigger change.
[13:34] <frobware> dimitern: btw, do you have a link to your PR from this morning?
[13:36] <frobware> dimitern: if it doesn't turn up in RB my experience is that RB does not like much markdown formatting. Remove all that and it will probably turn up. This is why I have a bunch of discarded reviews - It took me a while to realise that it was the commit message formatting that was "objectionable".
[13:37] <dimitern> frobware, I think it's better to keep tracking the latest master - i'll deal with merging stuff once my PRs are approved
[13:37] <dimitern> frobware, here it is: https://github.com/juju/juju/pull/4481
[13:38] <frobware> dimitern: <wild-speculation> looking at that I'm sure the bullet lists are "objectionable" for RB
[13:42] <dimitern> frobware, I added those later; I suspect the issue is with PRs that have only first line but no description, so the RB bot fails to add the link to the description
[13:42] <frobware> dimitern: I had problems where I had 4 spaces instead of 2 at the beginning of a line
[13:43] <dimitern> weird..
[13:43]  * dimitern will be back in about 1h
[14:21] <mup> Bug #1548333 opened: 2.0 bootstrap error messages and help are not helpful <juju-core:New> <https://launchpad.net/bugs/1548333>
[14:24] <mup> Bug #1548333 changed: 2.0 bootstrap error messages and help are not helpful <juju-core:New> <https://launchpad.net/bugs/1548333>
[14:27] <mup> Bug #1548333 opened: 2.0 bootstrap error messages and help are not helpful <juju-core:New> <https://launchpad.net/bugs/1548333>
[14:30] <mup> Bug #1548333 changed: 2.0 bootstrap error messages and help are not helpful <juju-core:New> <https://launchpad.net/bugs/1548333>
[14:42] <mup> Bug #1548333 opened: 2.0 bootstrap error messages and help are not helpful <juju-core:New> <https://launchpad.net/bugs/1548333>
[15:35] <natefinch> morning ericsnow
[15:35] <ericsnow> natefinch: hey
[15:36] <natefinch> ericsnow: almost got the uniter stuff in over the weekend, but adding a resource to the wordpress testcharm caused a lot of failures in tests that use that charm but didn't register resources :/
[15:36] <ericsnow> natefinch: nice (and not so nice)
[15:37] <natefinch> ericsnow: yeah, and trying to register resources caused import cycles.... so now I'm just changing all of the uniter tests to hardcode starsay rather than wordpress (after making starsay a duplicate of wordpress, with some resources added).
[15:38] <ericsnow> natefinch: yeah, I ran into those same import cycles (remember the last time I was complaining about internal tests)
[15:38] <natefinch> ericsnow: heh
[15:38] <natefinch> ericsnow: yeah
[15:39] <natefinch> ericsnow: (lack of) import cycles *are* a benefit of external tests :)
[15:39] <ericsnow> natefinch: our future selves always thank us :)
[15:40] <ericsnow> natefinch: anyway, I too ended up going a different route to avoid the circular imports, rather than fixing them (a too-large project)
[15:42] <natefinch> ericsnow: yeah, kinda sucks.
[15:46] <ericsnow> natefinch: regardless, let me know if there's anything I can do to help
[15:48] <natefinch> ericsnow: I'm making a copy of the wordpress charm to use only in the uniter tests where we need wordpress+resources, and hopefully that'll isolate it so it doesn't break everything else that wanted the old wordpress charm.  I think that's the last thing to do for this.
[15:48] <ericsnow> natefinch: sounds good
[15:50] <natefinch> ericsnow: now to wait 5 minutes for the uniter tests to run
[16:22] <ericsnow> natefinch: see my reply comment about adding the IncCharmModifiedVersionOps in Staged.Activate()
[16:23] <natefinch> ericsnow: ok, thanks for pinging me about it.
[16:23] <ericsnow> natefinch: yep
[16:24] <natefinch> ericsnow: ahh, thanks for the note on what you were thinking.  I was overcomplicating it in my head.
[16:25] <ericsnow> natefinch: yeah, I could tell what you are thinking
[16:25] <ericsnow> natefinch: dealing with that *in* the transaction is indeed a pain, though not quite a nightmare :)
[16:28] <natefinch> ericsnow: also, I just stumbled on a better way to test resources, I think: https://github.com/juju/juju/blob/master/worker/uniter/uniter_test.go#L1976
[16:29] <natefinch> ericsnow: we can append resources to the existing metadata.yaml for just the tests we care about
[16:29] <ericsnow> natefinch: smart!
[16:29] <natefinch> ericsnow: figured it out because this test was failing with invalid yaml
[16:30] <natefinch> ericsnow: sort of surprised this test works at all... it seems to assume there's already a newline at the end of the yaml
[16:48] <voidspace> dimitern: heh, so I have to reimplement environ.Spaces() ...
[16:49] <voidspace> dimitern: the assumption that SpaceProviderId is a name is built into the way we fetch subnets and spaces
[16:49] <voidspace> dimitern: and so switching requires changing the way we fetch subnets, which means rewriting Spaces because it builds the spaces from the subnets...
[16:49] <voidspace> dimitern: ah well
[16:54] <dimitern> voidspace, in a call, sorry - will get back to you shortly
[16:54] <voidspace> dimitern: no need, was just commenting
[16:55] <dimitern> voidspace, ah, ok then
[17:06] <natefinch> ericsnow: keeping error values and call names in sync in the stub tests is killing me
[17:07] <ericsnow> natefinch: sorry
[17:08] <natefinch> ericsnow: we might want to build in a test that makes sure the number of errors prepopulated is the same as the number of calls... I think I see a place where I had more errors than calls
[17:09] <ericsnow> natefinch: maybe so
[17:27] <cmars> dooferlad, can I trouble you with a couple of small reviews? just deps updates to fix LP:#1534643
[17:27] <mup> Bug #1534643: cookies file locked for too long <ci> <intermittent-failure> <juju-core:In Progress by cmars> <https://launchpad.net/bugs/1534643>
[17:50] <cmars> anyone up for some short & sweet reviews? http://reviews.vapour.ws/r/3931/ and http://reviews.vapour.ws/r/3932/
[18:29] <natefinch-lunch> I think the unit tests just popped up a browser window :/
[18:29] <natefinch-lunch> Juju Charms API -> log in with Ubuntu One
[18:32] <perrito666> dang
[18:32] <perrito666> that is new?
[18:34] <natefinch> I dunno.. was just running a full juju-core test
[18:36] <alexisb> cmars, if you havent gotten your reviews, tim should be on soon
[18:48]  * perrito666 cuts storage bucket out of providers
[18:48] <perrito666> I have been removing stuff for a month from juju :p
[18:57] <natefinch> perrito666: you are doing god's work
[18:57] <perrito666> lol
[18:57] <perrito666> I wouldn't put ian so high :p
[19:03] <natefinch> heh
[19:16] <perrito666> natefinch: there is a mail talking about the browser issue
[19:17] <natefinch> perrito666: yeah, cmars let me know it's fixed in latest master
[19:47] <natefinch> neat, a new sporadic failure: /home/ubuntu/juju-core_2.0-beta1/src/github.com/juju/testing/mgo.go:489: &errors.errorString{s:"no reachable servers"} ("no reachable servers")
[19:48] <alexisb> natefinch, we dont need more of those
[19:48] <natefinch> alexisb: indeed
[19:48] <alexisb> and with that I go to lunch :)
[20:20] <lazyPower> is it common that i see this with a longer-running beta-2 model-controller? consistently when i creat-models and destroy them after about the third one i start seeing this plague my units http://paste.ubuntu.com/15173762/
[20:20] <lazyPower> if i upgrade-charm it typically goes away and triggers the install
[20:22] <natefinch> if user==lazypower && rand.Under(0.33) { return failwhale() }
[20:22] <lazyPower> natefinch - git-blame schenanigans.go
[20:23] <natefinch> lazyPower: it's weird that it's a sha hash mismatch... I wonder if it's a network problem screwing up the download
[20:32] <natefinch> I'm hitting them all today: github.com/juju/juju/state/cloudimagemetadata/image.go:86: cannot save cloud image metadata
[20:46] <natefinch> rick_h_, ericsnow: are charmers allowed to update a charm and add/remove resources from the metadata.yaml?  I presume yes.  I don't think that's something we've thought about
[20:47] <ericsnow> natefinch: adding new ones should work fine as-is, removing old ones should be covered once we handle removal
[20:50] <natefinch> ericsnow: your charmstore poller looks like it's just marking what resources are out of date, but not storing the data on what's in the charmstore anywhere... don't we need both?
[20:50] <ericsnow> natefinch: yeah, I'm working on that right now
[20:51] <natefinch> ericsnow: ok, no problem... wanted to make sure I wasn't misunderstanding either your code or the spec :)
[20:52] <natefinch> ericsnow: heh, we're going to have like 4 or 5 representations of a resource in the DB, depending on its state... staged, pending, active, waiting to be updated from the store...  yeesh.
[20:53] <ericsnow> yep
[20:53] <natefinch> we should scrap it all and turn it into a state machine ;)
[20:54]  * natefinch waits for eric to say he already did that ;)
[20:55] <natefinch> OMG... third sporadic test failure... good old "bad record MAC"
[21:14] <wallyworld> thumper: with the new manifold stuff, do you know how to conditionally start a worker? previously, you'd just use an if statement in jujud/machine.go
[21:18] <fwereade> wallyworld, heyhey
[21:18]  * thumper lets fwereade answer
[21:18] <wallyworld> fwereade: hey, maybe you can answer :-)
[21:18] <fwereade> wallyworld, an if statement in th start func should do the trick -- depending on how changeable the value you're depending on is
[21:19] <wallyworld> fwereade: what do i return from the start func? can't be nil
[21:19] <wallyworld> is there a specific error?
[21:19] <fwereade> wallyworld, what people always used to do was return FinishedWorker IIRC
[21:19] <fwereade> wallyworld, that squicks me out
[21:19] <fwereade> wallyworld, dependency.ErrMissing would probably do the trick though
[21:20] <fwereade> wallyworld, that means "I'm not going to do anything with my current set of dependencies"
[21:20] <wallyworld> sounds good, i'll try that
[21:20] <fwereade> wallyworld, but will try again if any dependency does come up or go doown
[21:20] <wallyworld> yeah, that was sort of my issue
[21:20] <wallyworld> i didn't want it to retry
[21:48] <katco> cherylj: where can i see the last bless of master?
[21:49] <cherylj> katco: http://reports.vapour.ws/releases#master
[21:50] <katco> cherylj: duh ty
[21:50] <cherylj> np :)
[22:01] <rick_h_> natefinch: yes, they can add/remove. I'm not sure what removing will do. Since you're always upgrading ahead and can downgrade, it'd have to keep it around for a while just like the charms themselves?
[22:02] <natefinch> rick_h_: yeah, I don't think removing will be a problem if we keep them around (which we do).
[22:03] <rick_h_> natefinch: yea, I think we'll need some admin way to garden those out at some point, as the infinite charms has shown to be a problem in large production deploys
[22:04] <thumper> alexisb: you have frozen for me
[22:04] <alexisb> thumper, lost you on the hangout
[22:47] <lazyPower> natefinch - looks like a repeat offender. i just hit again on my second model :(
[22:48] <lazyPower> is there anything i can tail/append here to help other than shasum mismatch?
[23:36] <cherylj> can I get a quick / easy review?  http://reviews.vapour.ws/r/3936/
[23:36] <anastasiamac> cherylj: looking :D
[23:37] <anastasiamac> cherylj: LGTM :D
[23:55] <cherylj> thanks, anastasiamac!