menn0thumper: looking now00:20
menn0thumper: ship it with some small fixes00:33
mupBug #1538241 changed: 2.0-beta1 stabilization <blocker> <juju-core:Invalid> <https://launchpad.net/bugs/1538241>00:45
mupBug #1538241 opened: 2.0-beta1 stabilization <blocker> <juju-core:Invalid> <https://launchpad.net/bugs/1538241>00:54
mupBug #1538241 changed: 2.0-beta1 stabilization <blocker> <juju-core:Invalid> <https://launchpad.net/bugs/1538241>01:01
thumpermenn0: ta01:17
axwrick_h_: I'll test azure and see what's up01:25
rick_h_axw: ok, there's some general "stuff left behind" i get every once in a while with kill-controller. I'm not sure if this is part of that or just how much is meant to get cleaned up.01:25
rick_h_axw: ty for following up01:25
axwrick_h_: you should expect there to be nothing at all left behind01:26
axwrick_h_: everything is scoped by the resource group, each model gets a resource group01:26
rick_h_axw: going to see if we can get a GSoC student to help automate that process of pulling the info for the credentials. Wow that's painful.01:27
axwrick_h_: the azure creds?01:27
rick_h_axw: ok, yea that's what I assumed, but I need to look more at what you create with the azure cli process vs resource group vs model01:27
rick_h_axw: working through the whole nodejs cli to create the AD entries and such01:27
axwrick_h_: sounds good. yes, very painful. amazingly, it's actually much better than when I started working on the provider :)01:28
* rick_h_ can't imagine01:28
rick_h_the fact that I have to install nodejs makes me cry much less the multi-step "run this command with this UUID, and this one with this UUID...and..."01:28
axwrick_h_: I had envisaged a CLI tool to automate all the nodejs scripty bits01:28
* axw nods vigorously01:29
rick_h_axw: right, it must be something that we can even build into juju/Go based on using the same api in the same way01:29
rick_h_start out as a plugin and see how far we can get01:29
axwsounds good01:29
axwrick_h_: FWIW, MS have acknowledged that it sucks. I'm not sure what they're doing about it though.01:30
rick_h_axw: yea, well I got pulled into a "juju sucks on AWS" meeting with our folks that work with them01:30
rick_h_and I promised it'd get all better with the new ARM...and now I'm not so sure heh01:30
axwthe creds bit is worse, I think the rest is much better01:31
rick_h_hopefully just a scripting headache to hide from users01:31
thumperwe have this thing called annotations01:47
thumperand it can be set on "anything"01:47
thumperbut in practice, it seems like it is only ever stored on services01:47
thumperanyone know of other places it is used?01:47
anastasiamacthumper: not on anything...01:48
anastasiamacthumper: on "gloab entities"... essentially, anything that has a global key and  a tag01:49
anastasiamacthumper: machines, units, services, charms, models01:50
thumperwell... sure01:50
thumperbut in practice, I think it is just services01:50
anastasiamacthumper: juju itself may not use all of these but clients might.. i believe GUI is a prime example01:50
* thumper goes for the JFDI everything card01:50
anastasiamacthumper: it would b lovely to know how customers are using Juju (as in feature-focused customer feedback)01:51
axwrick_h_: I can't reproduce leftovers after kill-controller01:51
* anastasiamac dreaming 01:51
axwrick_h_: sure they weren't there before you bootstrapped? can you delete manually and see if it happens again?01:52
rick_h_axw: will see. I gave my creds to mark to try out so have to check if he's using it first01:53
rick_h_axw: did you deploy anything into it?01:53
axwrick_h_: no rush I guess. I think CI will notice if junk gets left behind01:54
axwrick_h_: nope. did you?01:54
rick_h_axw: yes, jenkins/haproxy, related them, exposed haproxy to get the public IP addr01:54
rick_h_axw: hmm, maybe I was impatient, I don't see the resource group in the portal at all now01:55
axwrick_h_: deleting resource groups is super duper slow01:55
rick_h_axw: I might have caught it with no machines/etc but with the IP/availability set left or something01:55
rick_h_axw: hmm, ok. I'll try again and see and report back01:55
rick_h_axw: ty for attempting to reproduce01:55
axwthanks, no worries01:55
menn0thumper: here's the State reference counter: http://reviews.vapour.ws/r/3921/02:22
menn0axw: ping02:22
axwmenn0: pong02:22
menn0axw: I noticed something strange in master02:22
* menn0 digs out some links02:22
menn0axw: https://github.com/juju/juju/blob/master/cmd/jujud/agent/machine.go#L1062-L107402:24
menn0axw: in master the stateworker has some logic to call unregisterSimplestreamsDataSource when the *state.State is closed (when the worker terminates)02:24
menn0axw: but the related code to NewStorage and registerSimplestreamsDataSource is no longer present02:25
menn0axw: openStateForUpgrade still has it though02:25
menn0axw: and it's in the 1.25 branch02:25
menn0axw: do you think this is an unintended deletion? or are the calls to NewStorage and registerSimplestreamsDataSource no longer required any more?02:26
axwmenn0: not actually sure. anastasiamac, ideas on above? ^^  I think this is related to the structured metadata changes02:27
* axw pokes around code02:27
anastasiamacaxw: m not sure why it was removed? an overzealous merge maybe?02:27
rick_h_axw: wow, 6min at "All hosted models reclaimed, cleaning up controller machines02:28
rick_h_and counting02:28
axwrick_h_: heh, yeah :/02:28
axwrick_h_: that's the cost of being 100% sure we're not leaking resources on azure, sadly02:28
* menn0 digs into the history a bit02:28
rick_h_axw: but yes, all things look gone this time02:28
rick_h_axw: so I'll chalk it up to impatience last time ty02:29
axwrick_h_: cool, thanks for confirming02:29
anastasiamacmenn0: eyes of the eagle \o/02:29
menn0anastasiamac: well I'm completely changing how the state worker starts so I need to understand how this works :)02:30
axwmenn0: I *think* we don't need it anymore, because we asynchronously populate the state db with metadata, and then return those results to StartInstanceParams. anastasiamac, you did that work... does that sound right?02:37
axwanastasiamac: that wasn't done for 1.25 was it? which would explain why it's different in the 1.25 branch?02:37
anastasiamacaxw: m not remembering it being asynch.... but i *hope* that it was in 1.25 rathern than not...02:38
menn0anastasiamac, axw : except what's in master looks wrong... like someone half completed a change or messed up a merge02:38
axwanastasiamac: asynchronous, as in we have an image metadata worker.02:38
menn01.25 looks sensible02:39
axwmenn0: right, the unregister certainly shouldn't be there if we don't need to register it anymore. still looking..02:39
axwmenn0:  removed by wallyworld in https://github.com/juju/juju/commit/66a6121c6598b2de7d8ac3acec7afbeccdcd958a02:42
axwwallyworld: do you recall why this change was made? https://github.com/juju/juju/commit/66a6121c6598b2de7d8ac3acec7afbeccdcd958a#diff-f5ec9ed405cc8f3a833355afdc629bd3L111002:43
wallyworldaxw: otp, give me a few minutes02:43
menn0axw, anastasiamac: thanks for the thinking and digging so far.02:56
anastasiamacmenn0: axw:wallyworld so the review for it http://reviews.vapour.ws/r/3459/03:00
anastasiamachas comment03:00
wallyworldstill in meeting sorry03:01
anastasiamacto my question of "why is this removed/no longer needed"03:01
anastasiamacthe response is03:01
anastasiamacWe only use simplestreams datasources are first searching in state database. Why add state database as a simplestreams datasource? What ends up happening is we search twice in the same location. And the logs show the search the second time errors anyway, so it's doublely evil.03:01
anastasiamac.... as a consequence, the registration was removed (registerSimplestreamsDataSource)03:02
axwso we just need to remove the unregister call03:02
anastasiamaclooks like \o/03:02
menn0anastasiamac: ok, so that means that the openStateForUpgrade should be similarly updated too03:02
menn0I might JFDI that into master now03:03
menn0actually no, i'll do it in my feature branch03:03
menn0it's relevant to the changes there and is almost done03:03
anastasiamacmenn0: i think so :D...03:04
anastasiamacaxw: the call to unregsiter is not doing anything, i think... removing it would be clean \o/03:04
wallyworldaxw: menn0: i think that change was because we were using the stream setting for tools to filter image metadata03:18
menn0wallyworld, anastasiamac: http://reviews.vapour.ws/r/3922/03:39
wallyworldmenn0: awesome, ty03:40
=== ses is now known as Guest3272
=== gberginc_ is now known as gberginc
=== blahdeblah_ is now known as blahdeblah
axwwallyworld: a handful of juju/api.go-related changes, when you have a moment: https://github.com/juju/juju/pull/449008:50
* axw bbl08:50
voidspacefrobware: you want to do 1:1?10:31
frobwarevoidspace: yep10:32
voidspacefrobware: I'd like coffee first ~5mins?10:32
frobwarevoidspace: ok10:32
voidspacefrobware: omw10:41
voidspacefrobware: uhm... I joined the wrong room. Really on my way now...10:42
=== kadams54_ is now known as kadams54
perrito666morning all12:53
frobwaredooferlad, dimitern, voidspace: http://reviews.vapour.ws/r/3914/ the very small master->maas-spaces2 merge we talked about this morning13:29
dimiternfrobware, why not bring in the latest master instead?13:31
frobwaredimitern: can do. I thought we said this morning we would just do this and then the next...13:34
frobwaredimitern: happy to drop and go for the bigger change.13:34
frobwaredimitern: btw, do you have a link to your PR from this morning?13:34
frobwaredimitern: if it doesn't turn up in RB my experience is that RB does not like much markdown formatting. Remove all that and it will probably turn up. This is why I have a bunch of discarded reviews - It took me a while to realise that it was the commit message formatting that was "objectionable".13:36
dimiternfrobware, I think it's better to keep tracking the latest master - i'll deal with merging stuff once my PRs are approved13:37
dimiternfrobware, here it is: https://github.com/juju/juju/pull/448113:37
frobwaredimitern: <wild-speculation> looking at that I'm sure the bullet lists are "objectionable" for RB13:38
dimiternfrobware, I added those later; I suspect the issue is with PRs that have only first line but no description, so the RB bot fails to add the link to the description13:42
frobwaredimitern: I had problems where I had 4 spaces instead of 2 at the beginning of a line13:42
* dimitern will be back in about 1h13:43
mupBug #1548333 opened: 2.0 bootstrap error messages and help are not helpful <juju-core:New> <https://launchpad.net/bugs/1548333>14:21
mupBug #1548333 changed: 2.0 bootstrap error messages and help are not helpful <juju-core:New> <https://launchpad.net/bugs/1548333>14:24
mupBug #1548333 opened: 2.0 bootstrap error messages and help are not helpful <juju-core:New> <https://launchpad.net/bugs/1548333>14:27
mupBug #1548333 changed: 2.0 bootstrap error messages and help are not helpful <juju-core:New> <https://launchpad.net/bugs/1548333>14:30
mupBug #1548333 opened: 2.0 bootstrap error messages and help are not helpful <juju-core:New> <https://launchpad.net/bugs/1548333>14:42
natefinchmorning ericsnow15:35
ericsnownatefinch: hey15:35
natefinchericsnow: almost got the uniter stuff in over the weekend, but adding a resource to the wordpress testcharm caused a lot of failures in tests that use that charm but didn't register resources :/15:36
ericsnownatefinch: nice (and not so nice)15:36
natefinchericsnow: yeah, and trying to register resources caused import cycles.... so now I'm just changing all of the uniter tests to hardcode starsay rather than wordpress (after making starsay a duplicate of wordpress, with some resources added).15:37
ericsnownatefinch: yeah, I ran into those same import cycles (remember the last time I was complaining about internal tests)15:38
natefinchericsnow: heh15:38
natefinchericsnow: yeah15:38
natefinchericsnow: (lack of) import cycles *are* a benefit of external tests :)15:39
ericsnownatefinch: our future selves always thank us :)15:39
ericsnownatefinch: anyway, I too ended up going a different route to avoid the circular imports, rather than fixing them (a too-large project)15:40
natefinchericsnow: yeah, kinda sucks.15:42
ericsnownatefinch: regardless, let me know if there's anything I can do to help15:46
natefinchericsnow: I'm making a copy of the wordpress charm to use only in the uniter tests where we need wordpress+resources, and hopefully that'll isolate it so it doesn't break everything else that wanted the old wordpress charm.  I think that's the last thing to do for this.15:48
ericsnownatefinch: sounds good15:48
natefinchericsnow: now to wait 5 minutes for the uniter tests to run15:50
ericsnownatefinch: see my reply comment about adding the IncCharmModifiedVersionOps in Staged.Activate()16:22
natefinchericsnow: ok, thanks for pinging me about it.16:23
ericsnownatefinch: yep16:23
natefinchericsnow: ahh, thanks for the note on what you were thinking.  I was overcomplicating it in my head.16:24
ericsnownatefinch: yeah, I could tell what you are thinking16:25
ericsnownatefinch: dealing with that *in* the transaction is indeed a pain, though not quite a nightmare :)16:25
natefinchericsnow: also, I just stumbled on a better way to test resources, I think: https://github.com/juju/juju/blob/master/worker/uniter/uniter_test.go#L197616:28
natefinchericsnow: we can append resources to the existing metadata.yaml for just the tests we care about16:29
ericsnownatefinch: smart!16:29
natefinchericsnow: figured it out because this test was failing with invalid yaml16:29
natefinchericsnow: sort of surprised this test works at all... it seems to assume there's already a newline at the end of the yaml16:30
voidspacedimitern: heh, so I have to reimplement environ.Spaces() ...16:48
voidspacedimitern: the assumption that SpaceProviderId is a name is built into the way we fetch subnets and spaces16:49
voidspacedimitern: and so switching requires changing the way we fetch subnets, which means rewriting Spaces because it builds the spaces from the subnets...16:49
voidspacedimitern: ah well16:49
dimiternvoidspace, in a call, sorry - will get back to you shortly16:54
voidspacedimitern: no need, was just commenting16:54
dimiternvoidspace, ah, ok then16:55
natefinchericsnow: keeping error values and call names in sync in the stub tests is killing me17:06
ericsnownatefinch: sorry17:07
natefinchericsnow: we might want to build in a test that makes sure the number of errors prepopulated is the same as the number of calls... I think I see a place where I had more errors than calls17:08
ericsnownatefinch: maybe so17:09
cmarsdooferlad, can I trouble you with a couple of small reviews? just deps updates to fix LP:#153464317:27
mupBug #1534643: cookies file locked for too long <ci> <intermittent-failure> <juju-core:In Progress by cmars> <https://launchpad.net/bugs/1534643>17:27
=== natefinch is now known as natefinch-lunch
cmarsanyone up for some short & sweet reviews? http://reviews.vapour.ws/r/3931/ and http://reviews.vapour.ws/r/3932/17:50
natefinch-lunchI think the unit tests just popped up a browser window :/18:29
natefinch-lunchJuju Charms API -> log in with Ubuntu One18:29
=== natefinch-lunch is now known as natefinch
perrito666that is new?18:32
natefinchI dunno.. was just running a full juju-core test18:34
alexisbcmars, if you havent gotten your reviews, tim should be on soon18:36
* perrito666 cuts storage bucket out of providers18:48
perrito666I have been removing stuff for a month from juju :p18:48
natefinchperrito666: you are doing god's work18:57
perrito666I wouldn't put ian so high :p18:57
perrito666natefinch: there is a mail talking about the browser issue19:16
natefinchperrito666: yeah, cmars let me know it's fixed in latest master19:17
natefinchneat, a new sporadic failure: /home/ubuntu/juju-core_2.0-beta1/src/github.com/juju/testing/mgo.go:489: &errors.errorString{s:"no reachable servers"} ("no reachable servers")19:47
alexisbnatefinch, we dont need more of those19:48
natefinchalexisb: indeed19:48
alexisband with that I go to lunch :)19:48
=== alexisb is now known as alexisb-afk
lazyPoweris it common that i see this with a longer-running beta-2 model-controller? consistently when i creat-models and destroy them after about the third one i start seeing this plague my units http://paste.ubuntu.com/15173762/20:20
lazyPowerif i upgrade-charm it typically goes away and triggers the install20:20
natefinchif user==lazypower && rand.Under(0.33) { return failwhale() }20:22
lazyPowernatefinch - git-blame schenanigans.go20:22
natefinchlazyPower: it's weird that it's a sha hash mismatch... I wonder if it's a network problem screwing up the download20:23
natefinchI'm hitting them all today: github.com/juju/juju/state/cloudimagemetadata/image.go:86: cannot save cloud image metadata20:32
natefinchrick_h_, ericsnow: are charmers allowed to update a charm and add/remove resources from the metadata.yaml?  I presume yes.  I don't think that's something we've thought about20:46
ericsnownatefinch: adding new ones should work fine as-is, removing old ones should be covered once we handle removal20:47
natefinchericsnow: your charmstore poller looks like it's just marking what resources are out of date, but not storing the data on what's in the charmstore anywhere... don't we need both?20:50
ericsnownatefinch: yeah, I'm working on that right now20:50
natefinchericsnow: ok, no problem... wanted to make sure I wasn't misunderstanding either your code or the spec :)20:51
natefinchericsnow: heh, we're going to have like 4 or 5 representations of a resource in the DB, depending on its state... staged, pending, active, waiting to be updated from the store...  yeesh.20:52
natefinchwe should scrap it all and turn it into a state machine ;)20:53
* natefinch waits for eric to say he already did that ;)20:54
natefinchOMG... third sporadic test failure... good old "bad record MAC"20:55
wallyworldthumper: with the new manifold stuff, do you know how to conditionally start a worker? previously, you'd just use an if statement in jujud/machine.go21:14
=== alexisb-afk is now known as alexisb
fwereadewallyworld, heyhey21:18
* thumper lets fwereade answer21:18
wallyworldfwereade: hey, maybe you can answer :-)21:18
fwereadewallyworld, an if statement in th start func should do the trick -- depending on how changeable the value you're depending on is21:18
wallyworldfwereade: what do i return from the start func? can't be nil21:19
wallyworldis there a specific error?21:19
fwereadewallyworld, what people always used to do was return FinishedWorker IIRC21:19
fwereadewallyworld, that squicks me out21:19
fwereadewallyworld, dependency.ErrMissing would probably do the trick though21:19
fwereadewallyworld, that means "I'm not going to do anything with my current set of dependencies"21:20
wallyworldsounds good, i'll try that21:20
fwereadewallyworld, but will try again if any dependency does come up or go doown21:20
wallyworldyeah, that was sort of my issue21:20
wallyworldi didn't want it to retry21:20
katcocherylj: where can i see the last bless of master?21:48
cheryljkatco: http://reports.vapour.ws/releases#master21:49
katcocherylj: duh ty21:50
cheryljnp :)21:50
rick_h_natefinch: yes, they can add/remove. I'm not sure what removing will do. Since you're always upgrading ahead and can downgrade, it'd have to keep it around for a while just like the charms themselves?22:01
natefinchrick_h_: yeah, I don't think removing will be a problem if we keep them around (which we do).22:02
rick_h_natefinch: yea, I think we'll need some admin way to garden those out at some point, as the infinite charms has shown to be a problem in large production deploys22:03
thumperalexisb: you have frozen for me22:04
alexisbthumper, lost you on the hangout22:04
=== blahdeblah_ is now known as blahdeblah
lazyPowernatefinch - looks like a repeat offender. i just hit again on my second model :(22:47
lazyPoweris there anything i can tail/append here to help other than shasum mismatch?22:48
cheryljcan I get a quick / easy review?  http://reviews.vapour.ws/r/3936/23:36
anastasiamaccherylj: looking :D23:36
anastasiamaccherylj: LGTM :D23:37
cheryljthanks, anastasiamac!23:55

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!