#juju-dev 2012-10-08
<fwereade_> davecheney, are you seeing http://paste.ubuntu.com/1266985/ ?
<davecheney> nope, but i haven't updated recently
<rogpeppe> davecheney, fwereade_: mornin'
<fwereade_> rogpeppe, heyhey
<davecheney> ... obtained map[string]interface {} = map[string]interface {}{"title":"My Title", "username":"admin001", "skill-level":9000}
<davecheney> ... expected map[string]interface {} = map[string]interface {}{"title":"My Title", "username":"admin001", "skill-level":9000}
<davecheney> o_0!
<fwereade_> davecheney, huh
<TheMue> morning
<davecheney> morning
<fwereade_> TheMue, heyhey
<TheMue> davecheney and fwereade_: hi
<Aram> morning.
<fwereade_> Aram, heyhey
<TheMue> hi amithkk
<TheMue> oops
<TheMue> hi Aram
<wrtp> fwereade_:  i was thinking about your Context interface, and wondering if the MapChanger type was necessary. i was thinking something like this might work a tad more neatly: http://paste.ubuntu.com/1267165/
<wrtp> fwereade_: probably this is just bikeshedding tho'
<fwereade_> wrtp, I think I'd prefer to keep being able to just return a ConfigNode there
<fwereade_> wrtp, Context.RelationId(); Context.RemoteUnitName()
<fwereade_> wrtp, niemeyer has suggested adding error returns
<fwereade_> wrtp, ISTM that, in practice, what this does is bloat the crap out of all the code that uses it
<wrtp> fwereade_: i was trying to see the interface from the context of a user of it. ISTM that it would be nice if it mapped fairly closely to the interface provided to the hooks. but i see yr point about just returning ConfigNode
<fwereade_> wrtp, can you offer a perspective from which this change is a good thing?
<wrtp> fwereade_: as a consumer of the interface, it's not really clear why the MapChanger type is there.
<fwereade_> wrtp, feels to me like it just means more code, with error paths that can't happen
<wrtp> fwereade_: oh, sorry, you mean adding error returns?
<fwereade_> wrtp, yeah
<wrtp> fwereade_: error returns from what?
<fwereade_>  wrtp, Context.RelationId(); Context.RemoteUnitName()
<wrtp> fwereade_: so niemeyer would like you to return an error rather than relation-id == -1?
<fwereade_> wrtp, yeah
<wrtp> fwereade_: i'm with you tbh. there's only one kind of error that can happen, and it's not really an error.
<fwereade_> wrtp, and I was ok with this, it seemed reasonable, until it came to actually switching the commands over to use the interface
<wrtp> fwereade_: returning an error makes it look like all kinds of bad things could happen
<fwereade_> wrtp, yeah exactly
<wrtp> fwereade_: there is another possibility, i suppose
<fwereade_> wrtp, trying to *use* that interface has put me in a totally foul mood
<wrtp> fwereade_: which is to have an IsRelationContext (or something) method, and make Context.RelationId panic if called when IsRelationContext is false.
<wrtp> fwereade_: that way you make the precondition clear, and there's no way you can inadvertently get it wrong.
<fwereade_> wrtp, tbh I would like to take it further, and use HasRelation to guard Relation
<fwereade_> wrtp, hell, HasRemoteUnit for RemoteUnitName
<fwereade_> wrtp, that might lead to code that doesn't make me want to slit my eyes
<fwereade_> wrtp, might actually even be more readable
<wrtp> fwereade_: that sounds reasonable to me
<wrtp> fwereade_: i don't know that it'll make niemeyer happy though. you can still run RelationId and forget it might be invalid. you'll just get a panic if you do that.
 * fwereade_ goes off for a ciggie and a grumble to himself while he tries to figure out whether it's even worth bothering to change anything until niemeyer comes online
<fwereade_> actually, going to get some food, bbiab
<wrtp> fwereade_: one other possibility: http://paste.ubuntu.com/1267207/
<fwereade_> morning niemeyer
<fwereade_> niemeyer, I'm finding the error returns from RelationId and RemoteUnitName to be extremely unwieldy... I think the biggest problem is that the errors imply that more could go wrong than just ErrNoRelation etc
<fwereade_> niemeyer, I'm wondering how you'd feel about pairs like HasRemoteUnit() bool and RemoteUnitName() string, and a requirement that clients look before they leap?
<niemeyer> fwereade_: Good morning
<niemeyer> Morning all
<fwereade_> niemeyer, heyhey
<niemeyer> fwereade_: It wouldn't really help much in the sense I was considering
<niemeyer> fwereade_: The point was really to prevent using context.RelationId() as if the relation existed, when in fact it doesn't
<niemeyer> fwereade_: We have the same logic in state itself, for example
<niemeyer> fwereade_: E.g. unit.InstanceId()
<fwereade_> niemeyer, I just don't see how it's anything but obfuscatory to have to go through identical error checking on RelationId and Relation
<niemeyer> fwereade_: Do you have the paste at hand?
<niemeyer> fwereade_: Actually, it was already a review wasn't it
 * niemeyer looks
<fwereade_> niemeyer, yeah
<niemeyer> fwereade_: Alternatively, what about having RemoteRelation (analogous to RemoteUnitName) that returns (ContextRelation, error), possibly returning ErrNoRemote, and moving Id to within ContextRelation
<fwereade_> niemeyer, not sure about the name; where does Remote come in? and actually I don't think we even need the ID once we've got the relation...
<niemeyer> fwereade_: The name doesn't sound great indeed
<fwereade_> niemeyer, quick access to the current relation would be a good thing though
<fwereade_> niemeyer, but still, RemoteUnitName... in that case I really don't think we want any sorts of errors: I really don't see that checking for 2 different possible error kinds (one of which can't happen, but the interface says it might) is any better than `if ctx.RemoteUnitName != ""`
<fwereade_> niemeyer, sorry, `if ctx.RemoteUnitName() != ""`
<fwereade_> niemeyer, does the switch from struct to interface really mandate that we codify every zero value as a specific error?
<niemeyer> fwereade_: Which error is it saying it can't happen but the interface says it might?
<fwereade_> niemeyer, any error other than ErrNoRemote
<niemeyer> fwereade_: Sorry, I think I'm still a bit slow today
<fwereade_> niemeyer, if I want to get a remote unit name, I now have to worry about 3 code paths
<niemeyer> fwereade_: Even if there's a single error, that's still a reason to report it.. (?)
<fwereade_> niemeyer, doesn't exist; does exist; something else went wrong
<fwereade_> niemeyer, the something else is not possible
<niemeyer> fwereade_: What something else?
<niemeyer> fwereade_: Sorry
<fwereade_> niemeyer, anything
<niemeyer> fwereade_: Erm.. great
<niemeyer> fwereade_: Foo
<fwereade_> niemeyer, unless we're saying that RemoteUnitName can *only* return ErrNoRemote?
<niemeyer> fwereade_: I'm completely out of context
<niemeyer> fwereade_: RemoteUnitName may be called when there's no remote unit name.. there are two ways to go about this: 1) We panic; 2) We error
<niemeyer> fwereade_: There's no in-between
<niemeyer> fwereade_: If you want to panic, and use IsRelationHook or similar, that sounds fine too
<fwereade_> niemeyer, wooo!
<fwereade_> niemeyer, cool, i thought I'd suggested that clearly at the beginning :0
<niemeyer> fwereade_: I don't recall a panic suggestion, but perhaps I just missed it
<fwereade_> niemeyer, "require that clients look before they leap"
<niemeyer> fwereade_: Cool, sorry, I didn't think that was it
<niemeyer> fwereade_: +1 either way
<fwereade_> niemeyer, that was I admit a sl obscure way to put it :)
<fwereade_> niemeyer, ok, I'll try those
<fwereade_> niemeyer, thanks
<niemeyer> fwereade_: np, and sorry for being slow to understand
<fwereade_> niemeyer, no worries :) am just pondering names... ActiveRelation? CurrentRelation?
<niemeyer> fwereade_: HookRelation?
<fwereade_> niemeyer, +1, thanks
<niemeyer> fwereade_: I find the interface slightly non-conventional, in that rather than doing "if err != nil", we'll be doing the same thing with HasFoo in several places
<niemeyer> fwereade_: We can go ahead and see, though, if you're happy with this
<fwereade_> niemeyer, I'm not sure I have the best solution, but the error-juggling was getting painful... it will probably evolve a touch if we run with it for a bit and listen to what it's saying :)
<niemeyer> fwereade_: Sure.. error handling is painful, but part of the deal
<niemeyer> fwereade_: In some cases, it looks like it went too far
<niemeyer> fwereade_: E.g. Relation(id int) needs an error return, I think
<fwereade_> niemeyer, well, I'm not usually bothered by error handling :)
<niemeyer> fwereade_: Otherwise you'll simply be doing if HasRelation(id); Relation(id) all the time
<rogpeppe> huh, maybe i wasn't attached
<rogpeppe> fwereade_: did you see my alternative suggestion above?
<fwereade_> rogpeppe, I did... wasn't quite sure what you were suggesting, because it seemed to drop a lot of important bits from the interface
<rogpeppe> fwereade_: oh, i hadn't intended to drop anything.
<rogpeppe> fwereade_: i omitted ContextRelation because it was the same as before
<fwereade_> niemeyer, well, not really... I'll be making use of *knowing* that a given relation exists, I think
<niemeyer> fwereade_: How do you know it?
<niemeyer> fwereade_: The only way is calling HasRelation, I believe
<fwereade_> niemeyer, well, at the moment I know it because in all the commands that need relations Init will fail if the relation isn't valid
<fwereade_> niemeyer, I'm not sure it's really worth checking that the relation is valid *every* time we're about to use it
<fwereade_> niemeyer, against the same, frozen, context...
<niemeyer> fwereade_: That's exactly the danger
<niemeyer> fwereade_: You don't want to check that it is valid, but you have to anyway, or you have to memorize the code paths
<niemeyer> fwereade_: so that it doesn't blow up
<niemeyer> fwereade_: Writing down err != nil is cheap.. memorizing code paths is not
<niemeyer> fwereade_: Either way, LGTM.. I'm happy to see the code and bother you later :)
<fwereade_> niemeyer, we'll see how much it makes your eyes bleed when I propose then :)
<rogpeppe> fwereade_: suggestion in full: http://paste.ubuntu.com/1267395/
<niemeyer> fwereade_: My eyes won't bleed.. I'll simply laugh the first time we get a panic
<fwereade_> niemeyer, heh :)
<fwereade_> rogpeppe, yeah: how do I get an arbitrary relation out of that?
<rogpeppe> fwereade_: ctxt.RelationContext().Relation(id)
<rogpeppe> fwereade_: one final alternative: http://paste.ubuntu.com/1267396/
<rogpeppe> fwereade_: then it would be: ctxt.(RelationContext).Relation(id)
<fwereade_> rogpeppe, and in a non-relation hook?
<niemeyer> rogpeppe: Are you really suggesting we have both ContextRelation and RelationContext?
<rogpeppe> fwereade_: ah, yeah
<rogpeppe> niemeyer: i thought it seemed reasonable, in the erm... context.
<fwereade_> niemeyer, well, we already do, but I hope it will become nicer :/
<niemeyer> fwereade_: We do!?
<rogpeppe> niemeyer: they mean quite different things, so i think it's not unreasonable to have those names
<fwereade_> niemeyer, we have a concrete RelationContext type and your suggested ContextRelation
<niemeyer> rogpeppe: I think it's completely crackful
<rogpeppe> niemeyer: ContextRelation represents a particular relation; RelationContext represents a particular context.
<rogpeppe> niemeyer: but obviously mileage varies...
<niemeyer> rogpeppe: This is insane, soryr
<fwereade_> niemeyer, that's why I liked the idea of a  jujuc.Relation interface -- IMO that is perfectly distinct from state.Relation
<rogpeppe> fwereade_: i like that too.
<niemeyer> fwereade_: Why do we have both?
<fwereade_> niemeyer, bear in mind that RelationContext will almost certyainly not keep that name, though
<niemeyer> fwereade_: Ahhh, okay
<fwereade_> niemeyer, well, we have a concrete type we're trying to hide behind an interface
<niemeyer> fwereade_: Yes, and that concrete type doesn't live in the same package
<fwereade_> niemeyer, not yet
<niemeyer> fwereade_: Or won't (thinking end goal)
<niemeyer> fwereade_: We can call it Mama for now
<fwereade_> niemeyer, haha
<niemeyer> fwereade_: :)
<niemeyer> fwereade_: What I'm really interested on is where we're headed
<niemeyer> fwereade_: The concrete type can have exactly the same name
<niemeyer> fwereade_: Under the uniter or whoever will have the concrete implementation
<fwereade_> niemeyer, I *think* I know, roughly, but the final shape is contingent on what I do with Relationer/RelationContext... and I don't have the foresight to know exactly how that will look until I'm free to hack them around a bit behind an interface, so I can experiment without spending 99% of my time fixing the build ;p
<niemeyer> fwereade_: I thought RelationContext was quite clearly being defined by the ContextRelation interface, which will of course need a concrete implementation
<fwereade_> niemeyer, sure, yes; so, you're right, it would make most sense to call it uniter.ContextRelation (unless it ends up in hook, which I think it will)
<niemeyer> fwereade_: Yeah, in hook or wherever the concrete implementation ends up being.. it just feels like at least that portion of its role in the problem we're defined now..
<niemeyer> defining
<fwereade_> niemeyer, but I believe it will come to include some of Relationer's responsibilities, and that the rest of Relationer will migrate into a new type
<niemeyer> fwereade_: Understood, that stuff I'm less sure/aware about
<fwereade_> niemeyer, in particular, I'm not sure whether that new type is going to be in uniter or hook
<fwereade_> niemeyer, (well, *I* am as sure as I can be at this stage that it'll be in hook... but that's too many reviews in the future for that to hold much verifiable water)
<niemeyer> fwereade_: Same here. Happy to see that stuff growing organically so we get a feeling of it while we go and don't spend too much time bikesheding on stuff that we can't really get right without further insight.
<niemeyer> rogpeppe: This is ready on the pipeline, btw: https://code.launchpad.net/~rogpeppe/juju-core/105-cloudinit-initial-password/+merge/128222
<rogpeppe> niemeyer: i realise. unfortunately it doesn't pass live tests, and i'm working on fixing that.
<niemeyer> rogpeppe: Super, thanks
<fwereade_> niemeyer, rogpeppe: btw, I misspoke earlier, I *do* need an Id() on ContextRelation, but that is not a big deal
<niemeyer> fwereade_: Agreed, it makes sense anyway
<niemeyer> fwereade_: Reviewing the follow up now
<niemeyer> fwereade_, rogpeppe, Aram: Btw, for synchronization, I intend to change state so it doesn't create the config nodes on demand, and instead handles them upfront at proper control points
<niemeyer> I'll do that after I clean up the reviews
<fwereade_> niemeyer, cool, thanks
<rogpeppe> niemeyer: that seems like a good thing
<niemeyer> fwereade_: This is part of multi-config blah, btw
<Aram> niemeyer: finally we change that. thank you.
<niemeyer> Aram: np
<fwereade_> niemeyer, <3
<niemeyer> fwereade_: https://codereview.appspot.com/6624062/diff/3001/worker/uniter/jujuc/config-get.go
<niemeyer> fwereade_: Are these removals intended?
<niemeyer> Hmm
<fwereade_> niemeyer, yeah, I moved the logic behind Config() in HookContext, in the interest of implementing the interface "correctly" without duplicating the logic
<fwereade_> niemeyer, I am aware it's not actually correct
<fwereade_> niemeyer, but it's pre-existing and I'm resisting a lot of tempting rabbit holes in this pipeline
<niemeyer> fwereade_: Cool
<niemeyer> fwereade_: That's okay.. I just don't want us to mistakingly screw up Dave's efforts of having working charms
<fwereade_> niemeyer, trust me, that thought was high up in my mind :)
<niemeyer> fwereade_: Thanks
<fss> niemeyer: I've sent two new CL's this morning
<fss> niemeyer: https://codereview.appspot.com/6621062/ and https://codereview.appspot.com/6618064/
<niemeyer> fss: Thanks
<fss> niemeyer: after 6621062 get merged, I will start to send iamtest package (it uses RequestIdResp)
<niemeyer> fwereade_: Review sent
<fwereade_> niemeyer, tyvm
<niemeyer> fwereade_: np
<fwereade_> niemeyer, the public fields remain because the next step (after switching the commands to use the interface) will be to move the concrete HookContext into uniter... which will break the jujuc tests if they can no longer construct a HookContext neatly
<fwereade_> niemeyer, they won't be there for long...
<fwereade_> niemeyer, because the followup will be to write the command tests against a local implementation of the interface
<niemeyer> fss: First looks nice.. just a couple of trivials
<fwereade_> niemeyer, and re stupid doc comments... is it better IYO to leave them blank? or to duplicate the interface comments?
<niemeyer> fwereade_: Okay, that's fine then, but let's show the path forward rather than documenting the transition as permanent then
<niemeyer> fwereade_: Just leave blank IMO
<niemeyer> fwereade_: They're documented in the interface
<fwereade_> niemeyer, cool, I was going to do that and then I freaked out at the sight of a whole screen without comments ;p
<niemeyer> fwereade_: Regarding _, can we please have a TODO comment inside the type explaining what we're doing, and tweaking the actual type docs so it doens
<niemeyer> doesn't feel like the status quo is intended as permanent
<fwereade_> niemeyer, absolutely so
<niemeyer> fwereade_: Thanks a lot
<niemeyer> fss: Second reviewed too.. good stuff as well with a trivial
<fss> niemeyer: cool :) I'm addressing your comments right now
<fwereade_> niemeyer, rogpeppe: btw, is anyone else seeing http://paste.ubuntu.com/1267489/ on trunk? is there something I need to update?
<fss> niemeyer: thanks for you reviews
<Aram> niemeyer: fwereade_: what do you want the new service relations watcher to return, []int?
<niemeyer> Aram: +1
<Aram> because we don't have a function that takes an integer relation id and returns a relation ATM.
<Aram> ok, so I'll add that function then.
<niemeyer> fwereade_: I'm not sure.. I can't see the failure here
<niemeyer> I'll step out for an early lunch.. biab for more fun
<fwereade_> Aram, sgtm
<fwereade_> Aram, I guess you're not seeing the failure I linked above?
<Aram> no, I do not.
<Aram> everything is fine here.
<fwereade_> Aram, cool, thanks
<mramm> US Holiday today so I will only be around sporadically
<Aram> have fun
<rogpeppe> niemeyer: a trivial lbox fix for a panic i just had: https://codereview.appspot.com/6632043/
<rogpeppe> niemeyer: PTAL at this - i had to make a few more changes to get all the tests to work correctly. (i changed the dummy provider to check EntityName too to catch the errors i was seeing in the live tests) https://codereview.appspot.com/6612054
<rogpeppe> fwereade_: yes, i see that failure too
<fwereade_> rogpeppe, ah, phew, I'm not insane :)
<rogpeppe> fwereade_: i'll have a look at fixing it, one mo
<fwereade_> rogpeppe, you rock :)
<rogpeppe> fwereade_: i haven't fixed it yet :-)
<fwereade_> rogpeppe, FWIW I think the deferred StopInstances just below that block is a bit too low down
<fwereade_> rogpeppe, er just above
<rogpeppe> fwereade_: agreed
<rogpeppe> TheMue: ping
<TheMue> rogpeppe: pong
<rogpeppe> TheMue: i see a test failure in trunk that seems to be connected with port handling
<rogpeppe> TheMue: do you see the same failure?
<TheMue> rogpeppe: for the dummy provider? just submitted the fix
<rogpeppe> TheMue: ah, ok, cool
<rogpeppe> TheMue: although... i did just pull and i still see the failure, i think
<TheMue> rogpeppe: the test for the global firewall mode, which works fine for ec2 local and live, fails, my fault
<rogpeppe> TheMue: yeah, i still see the failure.
<TheMue> rogpeppe: eh, ok, wrong wording, i made a proposal, not submitted it
<rogpeppe> TheMue: ah!
<rogpeppe> TheMue: link?
<rogpeppe> TheMue: (i don't see many proposal emails sadly)
<TheMue> rogpeppe: https://codereview.appspot.com/6635043/
<TheMue> rogpeppe: i wanted to extend the firewaller for the global mode and then i discovered the failure
<niemeyer> rogpeppe: Looking
<niemeyer> rogpeppe: Thanks for the lbox fix
<rogpeppe> niemeyer: yw
<Aram> <rogpeppe> TheMue: (i don't see many proposal emails sadly)
<Aram> I see too many
<Aram> like 3 for each proposal or something
<Aram> rogpeppe: would you take some of mine?
<niemeyer> Aram: Yeah, sorry about that.. haven't yet found a way to have comments in both Launchpad and Rietveld without that overload
<fwereade_> rogpeppe, pong, sorry, my sister just called to inform us that she is now engaged to be married
<fwereade_> so, yay :)
<rogpeppe> Aram: that used to be true for me too
<rogpeppe> fwereade_: cool
<rogpeppe> fwereade_: i don't think i pinged you actually
 * fwereade_ reads back and observes that he appears to be on crack
<rogpeppe> TheMue: you've got a review
<niemeyer> rogpeppe: cloudinit done too
<rogpeppe> niemeyer: thanks
<niemeyer> rogpeppe: np
<TheMue> rogpeppe: thx, appreciated it, will add it after dinner.
<niemeyer> fwereade_: Did you figure what was up there?
<fwereade_> niemeyer, apparently TheMue has proposed a fix
<fwereade_> niemeyer, I'm holding off on merging anything for the moment in case it's trickier than it looks and demands a revert or something
<niemeyer> fwereade_, TheMue: What was the bug?
<niemeyer> fwereade_, TheMue: As far as I recall, the changes to the global port were all optional, and shouldn't have broken tests
<TheMue> niemeyer: the dummy provider test includes jujutest, but here the global mode failed because it didn't handle it
<TheMue> niemeyer: now the dummy provider supports the global mode too
<niemeyer> TheMue: Okay, so I guess we should implement support on it/
<niemeyer> TheMue: This CL (https://codereview.appspot.com/6635043/) has changes to the firewaller too, in addition to the dummy provider
<niemeyer> TheMue: Can we fix this bug without touching the firewaller?
<TheMue> niemeyer: i can split it, yes
<niemeyer> TheMue: Sounds quite sensible, thanks
<TheMue> niemeyer:  i detected it when extending the firewaller
<niemeyer> TheMue: That's the way it always goes.. that's the point to stop a CL and do another one
<TheMue> niemeyer: yes, would have been better
<niemeyer> TheMue: Then, can we talk about the theory behind the firewaller change?
<TheMue> niemeyer: will propose it after dinner, family calls
<niemeyer> TheMue: Sounds sensible too, thanks
<niemeyer> rogpeppe: You had a review that looked sensible on TheMue's branch.. would be interested in taking the patch on that one file and pushing something you're happier with, so we can fix trunk?
<rogpeppe> niemeyer: i can do, but i'm mid-flow currently, and i'd prefer not to lose context until i've fixed this test, if that's ok
<niemeyer> rogpeppe: Absolutely, thanks
<niemeyer> fwereade_: Ah, I can reproduce the failures now.. my branch lacked updates
<fwereade_> niemeyer, jolly good :)
<rogpeppe> niemeyer: smallish CL. https://codereview.appspot.com/6639043
<niemeyer> rogpeppe: Handling trunk now
<rogpeppe> niemeyer: (currently verifying that live tests pass)
<rogpeppe> niemeyer: are you fixing trunk? that's great thanks. if not, i can move on to it now.
<niemeyer> rogpeppe: I am
<TheMue> niemeyer: you asked what the firewaller CL has to do with the open task. this one nothing, but while thinking about the firewaller i dicovered that i can't simply close a port in global mode because another service may need it. so i added the open port counter.
<niemeyer> TheMue: What happens if the firewaller is restarted?
<TheMue> niemeyer: have to check it again, but as all ports are re-opened then the counter gets new initialized.
<niemeyer> TheMue: All ports are re-opened!?
<TheMue> niemeyer: as i said, have to recheck it. but please let me do the dummy CL before
<niemeyer> TheMue: I'm already working on it.. doesn't solve the problem by itself
<niemeyer> TheMue: We need more careful consideration of how this is going to work before implementing further ode
<niemeyer> code
<niemeyer> TheMue: Please don't worry about it for now, though.. it's late for you
<TheMue> niemeyer: yes, it is, but it's an interesting topic ;)
<niemeyer> TheMue: So this may sound interesting: https://codereview.appspot.com/6615063
<niemeyer> rogpeppe: In case you want to check as well, given your prior review: https://codereview.appspot.com/6615063
<TheMue> niemeyer: *click*
<rogpeppe> niemeyer: LGTM
<rogpeppe> niemeyer: unfortunately i seem to have broken trunk too. i must have live tested the wrong branch, dammit.
<rogpeppe> niemeyer: am just testing the (obvious) fix, then will propose
<rogpeppe> niemeyer: (it's only the live tests that break)
<niemeyer> rogpeppe: Cool, thanks
<TheMue> niemeyer: LGTM, like the ports map idea.
<rogpeppe> niemeyer: 1 line fix: https://codereview.appspot.com/6635045
<fss> niemeyer: can we get those two CLs merged? :-)
<niemeyer> rogpeppe: Makes sense, LGTM
<niemeyer> fss: Yeah, will look at them in a sec
<rogpeppe> niemeyer: sorry about that. i must work out some system for juggling my branches and tests better.
<rogpeppe> niemeyer: you're holding the lock on juju-core currently
<niemeyer> rogpeppe: Trying to submit
<rogpeppe> niemeyer: ah, cool
<rogpeppe> niemeyer: that's not quick!
<fss> niemeyer: thanks
<rogpeppe> niemeyer: ... acquired 12 minutes, 12 seconds ago.
<niemeyer> rogpeppe: Trust me, I want to submit as much as you do
<rogpeppe> niemeyer: i was commiserating, not complaining...
<rogpeppe> niemeyer: i've got to go, but i'll put my submit into a loop until it succeeds
<niemeyer> rogpeppe: Don't worry, I can submit it once Launchpad works
<rogpeppe> niemeyer: that would be great, thanks.
<niemeyer> rogpeppe: np
<rogpeppe> see y'all tomorrow
<niemeyer> Launchpad simply doesn't work
<niemeyer> Apparently I can't commit anything to LP
<niemeyer> I wonder if it's just me or if it's a general issue
 * niemeyer hops onto #lp
<niemeyer> <beuno> niemeyer, yeah, same here
<niemeyer> Today is one of those days when the simple things seem unnaturally complex :)
 * niemeyer prepares some chimarrÃ£o to relax a bit
<niemeyer> Blowing away my local repository and starting over worked
<niemeyer> fwereade_: ping
<niemeyer> If someone is up for a simple one: https://codereview.appspot.com/6642046
<niemeyer> No semantic changes
 * fwereade_ needs to restart to complete updates... but is going to subvert power relationships by shutting down instead. ha!
#juju-dev 2012-10-09
<davecheney> hey - awesome - my CI machine on canonistack was deleted
<davecheney> that's fantastic
<niemeyer> davecheney: Oops :)
<niemeyer> davecheney: Welcome to the Clouds!
<niemeyer> :)
<davecheney> it's a good thing there wasn't anything important on it
<niemeyer> davecheney: The schema package, which we use with configs, take care of the type-enforcing logic
<niemeyer> Today is Launchpad-EOF day
<davecheney> niemeyer: it sure bloody is
<davecheney> niemeyer: yeah, I didn't think the int64/int problem would be a real problem
<davecheney> only when constructing faux test data
<davecheney> niemeyer: [LOG] 41.35831 SYNC Cluster 0xf840582210 is stopping its sync loop.
<davecheney> ... Panic: command failed: bzr commit -m Imported charm. (PC=0x4114F3)
<davecheney> happens on a fresh precise machine
<davecheney> ^ store
<niemeyer> davecheney: Does it say what the error message was?
<niemeyer> davecheney: The bzr error, that is
<davecheney> that is it
<niemeyer> davecheney: That's the command run, not its output
<davecheney> niemeyer: http://paste.ubuntu.com/1268515/
<niemeyer> davecheney: Would you mind to tweak the message so we get an idea?
<niemeyer> davecheney: ?
<davecheney> i think i have looked into this before
<niemeyer> davecheney: I trust you :)
<davecheney> i have a branch somwhere
<davecheney> that did add extra debugging
<davecheney> i remember being annoyed that bzrDir.Commit panic'd
<davecheney> will fix
<niemeyer> davecheney: Btw, any news on the ec2 signature issue?
<davecheney> it's on the cards for today
<niemeyer> davecheney: Sweetest
<davecheney> i have a nasty feeling there is a limit to the number of machines we can specify in that url
<davecheney> going to do some spelunking in the aws forums
<niemeyer> davecheney: I don't doubt, but it'd be a surprisingly bad error message if nothing else
<davecheney> it's actually a 403
<niemeyer> davecheney: Isn't that a forbidden?
<davecheney> which smells like a generic 'hmm, i don't like that, better tell you to get stuffed'
<davecheney> niemeyer: panic(fmt.Sprintf("command failed: bzr %s\n%s", strings.Join(args, " "), output))
<davecheney> ^ this is how that tests tries to capture the output
<davecheney> no idea why it isn't working
<davecheney> will try compbined output
<davecheney> bingo
<davecheney> ... Panic: command failed: bzr commit -m Imported charm.
<davecheney> bzr: ERROR: Unable to determine your name.
<davecheney> Please, set your name with the 'whoami' command.
<davecheney> E.g. bzr whoami "Your Name <name@example.com>"
<niemeyer> davecheney: We should show the error as well
<niemeyer> Ah, okay
<niemeyer> Combined output
<niemeyer> We cannot use that, unfortunately.. :(
<davecheney> i tried that, but it breaks the test for others that expect Run to only handle stdout
<niemeyer> We should at least display the rest of the output
<niemeyer> davecheney: Right.. it fixed a real bug
<niemeyer> It used to be combined
<davecheney> is there a flag we can pass to bzr to force an identity
<davecheney> ?
<niemeyer> davecheney: Doesn't it respect $EMAIL?
<davecheney> niemeyer: no idea, let me try
<davecheney> niemeyer: $EMAIL works
<davecheney> i'll fix the test to pass that in
<niemeyer> Sweet, thanks
<davecheney> niemeyer:  https://codereview.appspot.com/6631051
<niemeyer> davecheney: LGTM
<davecheney> niemeyer: ty
<davecheney> niemeyer: http://docs.amazonwebservices.com/AWSEC2/latest/APIReference/ApiReference-query-TerminateInstances.html
<davecheney> no mention of a limit for n
<davecheney> and nothing obvious on the googles
<niemeyer> davecheney: It's likely an error in the signature logic
<davecheney> i will look there for something length related
<niemeyer> davecheney: I'd try to find something that can sign that request properly, like the Python's boto, and comparing the signatures
<niemeyer> davecheney: and perhaps most interestingly, comparing the payloads
<davecheney> will do
<niemeyer> Why you love me not, Launchpad
<davecheney> https://codereview.appspot.com/6642048/
<fwereade> wrtp, heyhey
<wrtp> fwereade: yo!
<wrtp> fwereade: how's tricks?
<fwereade> wrtp, ah, not bad, and you?
<wrtp> fwereade: not too bad. just trying to keep myself oriented in the sea of tiny CLs that i'm doing for this change. sometimes i think that it's better to do larger CLs, just to keep the mental overhead down (plus less testing overhead)
<fwereade> wrtp, I know the feeling
<TheMue> morning
<fwereade> TheMue, heyhey
<TheMue> heya fwereade
<TheMue> fwereade: any idea on how we can detect if a machine fails the hard way (not by stopping it manually)?
<fwereade> TheMue, sorry, explain the situation more
<fwereade> TheMue, are yu talking about an actual instance disappearing?
<TheMue> fwereade: yes.
<fwereade> TheMue, I think we expect the firewaller to notice that the provider's not reporting it any more in the Instances list
<TheMue> fwereade: if we remove it watchers will get notified. but what happes if there's a hard stop?
<fwereade> TheMue, wait, we don't have an instances watcher do we?
<TheMue> fwereade: afaik not
<fwereade> TheMue, "machine" and "instance" are different -- which are we talking about here
<fwereade> ?
<Aram> morning.
<fwereade> Aram, heyhey
<TheMue> Aram: morning
<TheMue> fwereade: let's start with instances, the hardest part. ;)
<fwereade> TheMue, AFAIK the only way to tell is by polling the provider :/
<fwereade> TheMue, IIRC the python had a separate thing running once a minute to do that, does that ring a bell?
<TheMue> fwereade: i also had polling in my mind, only wanted to get sure. thanks for the py hint, i'll look there.
<fwereade> TheMue, I *think* that's what we do anyway :) been a little while since I looked...
<TheMue> fwereade: i could integrate such a mechanism in the firewaller, one poller per instance. and if it fails i notify the main loop to react <thinkking/>
<fwereade> TheMue, one poller per instance sounds a bit off... consider N=100000
<fwereade> TheMue, surely this is provisioner more than firewaller?
<fwereade> TheMue, (arguably a whole separate task...)
<TheMue> fwereade: that's a scaling problem of the current firewaller, even w/o polling. it already runs goroutines for all machines and units
<fwereade> TheMue, that's no reason to make it worse :p
<TheMue> fwereade: that hasn't been meant as a reason, only that we have to rethink the fw for large clouds
<TheMue> fwereade: maybe a kind of partinioning
<TheMue> arg
<TheMue> partitioning
<fwereade> TheMue, yeah, makes sense... but surely this is even more reason to separate out the restarting for now
<fwereade> TheMue, although... hm
<fwereade> TheMue, it took users about a year to figure out that that functionality even existed in Python, iirc
<fwereade> TheMue, I'm really just idly wondering if that bit is the highest possible priority right now, assuming you're in sync with niemeyer just ignore me :)
<TheMue> fwereade: first step is only to recognize dead instances to keep the ports state in the fw up-to-date
<fwereade> TheMue, ah, hmm, I see
<fwereade> TheMue, actually, sorry, no I don't...
<fwereade> TheMue, why do we need to close ports on dead instances?
<fwereade> TheMue, ok, the more I think, the more I feel like I've missed something big/important
<TheMue> fwereade: not to close them, but to know that they have to be opend when an instance becomes availble again
<fwereade> TheMue, I *thought* that you were changing to an everything-open model in preparation for per-machine FWing?
<fwereade> TheMue, was that just some fever dream of mine? :P)
<TheMue> fwereade: that's optional, and brings other problems we still have to think about. think about multiple services needing the same port, so you can't just close it for the first one but for the last one.
<fwereade> TheMue, isn't the logic completely identical?
<fwereade> TheMue, (but ok the answer to my question is "no" -- so, sorry, I'm out of the loop: what is your current goal?)
<TheMue> fwereade: no, today we tell the instance to close ports. in case of only one global security group it would do it immediately for all, even if other services need that port.
<TheMue> fwereade: but my current goal is only to get aware of real dying instances to keep the state in the firewaller up-to-date
<TheMue> fwereade: the global firewaller mode is a different topic
<fwereade> TheMue, with per-machine FWing we'd leave the global group open all the time, wouldn't we?
<fwereade> TheMue, sorry I'm derailing again
<fwereade> TheMue, ok, I think I am being dense though: please explain again why you need to close ports on an instance that doesn't exist?
<fwereade> TheMue, ahhhhhh sorry: is it because next time we start an instance for that machine, it will be started with the security group of the original instance?
<TheMue> fwereade: pls forget security groups
<TheMue> fwereade: different topic and currently not interesting from the firewallers perspective, because the API is neutral to how the default and the global modes are implemented
<fwereade> TheMue, ok then, back to your original explanation: "not to close them, but to know that they have to be opend when an instance becomes availble again"
<fwereade> TheMue, I still don't follow
<fwereade> TheMue, if a new instance shows up for that machine
<fwereade> TheMue, surely the last thing we want is open ports?
<TheMue> fwereade: the ports will be opened for the units. but if the firewaller "thinks" that a port is already open, then it wouldn't open it.
<TheMue> fwereade: technologically it is closed, the firewaller still thinks it is open
<fwereade> TheMue, possible strawman: all we need to do is, whenever a machine's instance-id changes, we should close all ports on all units for that machine
<fwereade> TheMue, because opening the ports that state thinks are open, on a new instance, is surely Bad and Wrong?
<fwereade> TheMue, ISTM that you're trying to figure out how to implement a security hole ;p
<fwereade> TheMue, but then I may be missing context and repeating previous discussions..?
<TheMue> fwereade: trying to follow you, don't see the sec hole
<TheMue> fwereade: the ports aren't opend by default, indeed not
<fwereade> TheMue, well, we shouldn't open ports until the charm tells us to, right?
<fwereade> TheMue, so if we have instance X running u/0, with a bunch of ports open
<TheMue> fwereade: yes, and when the firewaller in this moment "thinks" it is already open, it won't open it
<fwereade> TheMue, and instance X dies hard, and u/0 is redeployed to instance Y
<fwereade> TheMue, and you then open a bunch of ports on Y before the unit tells yu they shoudl be opened
<TheMue> fwereade: pls read, i will not open anything automatically
<TheMue> fwereade: only on demand of a deployed unit
<fwereade> TheMue, right
<TheMue> fwereade: BUT!
<TheMue> fwereade: the firewaller still "THINKS" !!! that the port is already open (he has not gone aware that the instance has been gone)
<TheMue> fwereade: so he won't open the port, even if needed, because he currently thinks there is nothing to do
<TheMue> fwereade: your hint with the instance id may be a good one
<TheMue> fwereade: can we be sure, in every environment, that they are always new?
<fwereade> TheMue, I'm not sure :/
<fwereade> TheMue, so actually it shouldn't be the FW
<fwereade> TheMue, I think the only thing that makes sense if for the UA to reset its port state when it installs a charm
<fwereade> TheMue, which then solves your problem... right?
<TheMue> fwereade: sounds reasonable
<fwereade> TheMue, or maybe not, sorry, I need to think it through again, I'm somewhat unfamiliar with the guts of the firewaller
<TheMue> fwereade: we only have the gap between the moment of the dying instance and the redeployment of the unit
<TheMue> fwereade: the fw knows about machines to map this information to instances and about services (are they exposed) and units. the queue unit -> machine -> instance controls, which ports are to open/close.
<TheMue> fwereade: in fast words ;)
<wrtp> TheMue: can't you just remove ports from an instance when a machine's instance gets changed?
<wrtp> TheMue: yes, that would mean that a dying instance would still keep some global ports open, but i think that's reasonable.
<fwereade> wrtp, as pointed out, I *think* that we can't be sure that a new machine will have a new instance id
<TheMue> wrtp: see fwereade
<wrtp> fwereade: i don't think that matters.
<TheMue> wrtp: how do you detect, that the instance is new?
<fwereade> wrtp, ah, ok, if the provisioner does that it could work...
<wrtp> TheMue: i don't think you need to
<fwereade> wrtp, you suggested that we could remove ports when the instance is changed... but you're saying we don't need to detect new instances to detect this?
<wrtp> TheMue: in fact, i think we have to be able to assume that an instance id isn't repeated.
 * fwereade has a confuse
<fwereade> wrtp, well, you can't do that, can you?
<fwereade> wrtp, I'm pretty sure EC2 will sometimes repeat instance ids in really rather quick succession
<wrtp> fwereade: if an instance id might be repeated, then we have no way of knowing for sure when a machine has been assigned to a new instance
<TheMue> wrtp: you think of watching the instance id of a machine? every set, even with equal values, is a change.
<fwereade> wrtp, except for the fact that *we* do the assigning...
<wrtp> fwereade: yes, but we're watching the instance id on the machine. that's all we know of the new instance.
<wrtp> fwereade: and if an old instance has gone away, we might allocate a new instance and set the machine's instance id.
<fwereade> wrtp, right -- but the FW has no way to detect this, does it?
<TheMue> fwereade: with the help of a watcher it may be done
<wrtp> fwereade: and this could be a problem.
<wrtp> TheMue: i don't believe it can
<TheMue> wrtp: yes, if it doesn't change we will not see it :(
<wrtp> fwereade: it's actually not a problem for real, because we actually never store fw settings per instance
<wrtp> fwereade: we store them per machine.
<fwereade> wrtp, yeah, which is a problem... isn't it?
<wrtp> fwereade: despite the API in Environ.
<wrtp> fwereade: well, it's kind of odd. we can have a situation where we cannot change the port settings for a given machine, because its instance has gone away
<fwereade> wrtp, (a problem, because, we don't want to have open ports on an instance until code running on that instance asks for them to be open)
<fwereade> wrtp, I assert that ports are entirely about instances and only coincidentally to do with machines
<wrtp> fwereade: with global ports, we do want that to be true
<wrtp> fwereade: when we start an instance, we wipe its security group AFAIR
<wrtp> fwereade: it would be nice if they were, but the implementation doesn't do that
<fwereade> wrtp, then isn't it just a matter of the MA, on first run, clearing all ports for all assigned units, and then we're done?
<wrtp> fwereade: we pretend that the ports are set per instance, but they're not - they're per machine
<wrtp> fwereade: possibly. i seem to remember suggesting that before.
<fwereade> wrtp, I think that if the clearing is already in place that that is the right thing to do
<TheMue_> wrtp: and with a global firewall mode they are globally. but how does a machine knows about which ports other machines need?
<TheMue> wrtp: so there has to be a port manager for the global mode
<wrtp> phone call, sorry
<fwereade> TheMue, sorry, why would any machine need to know about any other machine's ports?
<fwereade> TheMue, except in the very limited sense in which, yes, the FW is running with a machine agent
<wrtp> back
<wrtp> fwereade: wouldn't the correct place to clear the ports for a given unit be when we reassign that unit to a new machine?
<fwereade> wrtp, when does that happen?
<wrtp> fwereade: never, currently AFAIK. i think this is a fairly sketchy area.
<TheMue> fwereade: no, you got me wrong, a machine does not need to know about other machines
<fwereade> wrtp, the issue I *think* I am talking about is when a machine is reprovisioned, with all its units, and this does happen -- at least in python :)
<TheMue> fwereade: but if a machine clears all ports for all assigned units, then in a global mode a port may be closed for a different machine that still needs that port.
<wrtp> fwereade: ok, so perhaps that's the time that the ports should be cleared.
<wrtp> fwereade: (by the provisioner)
<wrtp> TheMue: a machine can't clear the ports in the environment - only the firewaller can do that
<TheMue> wrtp: yes
<wrtp> TheMue: a machine agent could clear the ports in any units assigned to that machine though, but i'm not convinced that's the best place for it.
<fwereade> TheMue, I think there's something I'm missing about this "global" mode
<TheMue> wrtp: i answered "<fwereade> wrtp, then isn't it just a matter of the MA, on first run, clearing all ports for all assigned units, and then we're done?"
<fwereade> TheMue, er
<TheMue> wrtp: no, i would like to have the control in the firewaller
<fwereade> TheMue, how does a machine agent calling ClosePort() on a unit affect the global set of opened ports?
<TheMue> fwereade: currently the fw doesn't know about that global mode
<wrtp> TheMue: i don't think the firewaller should be changing a Unit's open ports.
<fwereade> TheMue, if the FW is closing a port just because one unit had a port closed then it's just crack, surely?
 * TheMue thinks we are turning here a bit
<fwereade> wrtp, +1
<fwereade> TheMue, s/closing a port/closing a global port/
<TheMue> reset<CR>
<TheMue> fwereade: again, the fw today doesn't know about the global mode
<wrtp> fwereade: where's the code that's doing the refcounting of ports then?
<fwereade> TheMue, please don't criticise my suggestions in the context of theis global mode and then tell me the global mode is irrelevant
<wrtp> s/fwereade/TheMue/
<fwereade> wrtp, I have no idea... tbh the idea of a global FW sounds like complete crack to me
<wrtp> TheMue: the plan is to make the firewaller aware of global mode, right?
<fwereade> wrtp, shitty security, and a whole load of complexity
<wrtp> fwereade: i think it's a reasonable pragmatic security
<TheMue> fwereade: no, i'm not telling you it's irrelevant, i only want to sort the ideas a bit
<wrtp> s/a reas/reas/
<fwereade> wrtp, so every machine opens the intersewction of everything that might be open?
<wrtp> fwereade: union
 * fwereade had a brain once but then he left it somewhere
<fwereade> wrtp, ok... but, yeah, isn't that completely crackful from a security POV?
<wrtp> fwereade: it means we will be able to scale in ec2 without opening all ports.
<wrtp> fwereade: actually, i think the thing that's crackful is that we're pretending that it's all per-instance when it's not.
<TheMue> wrtp: i made a proposal with ref counting, but niemeyer meant we have to discuss about it, because the  environments can act differently. see https://codereview.appspot.com/6635043/ and http://irclogs.ubuntu.com/2012/10/05/%23juju-dev.html at 14:23
<wrtp> TheMue: i think niemeyer's concerns there could be addressed by having the provisioner clear out the ports of machine's units when it detects that the machine's instance is dead (or when it reprovisions a machine)
<TheMue> wrtp: that has been my initial question: how do we detect a dead instance?
<fwereade> TheMue, by polling, in the provisioner, like I said
<TheMue> fwereade: it's already doing so? you sounded like that's still open and in py it's an external program.
<fwereade> TheMue, I dunno, I'm afraid I didn't implement it
<fwereade> TheMue, if the provisioner is not polling for dead instances then, to match py, I think it should be
<fwereade> TheMue, and if it is, then we have the information we need available right there, and can take action as needed
<TheMue> fwereade: yes, i didn't found anything in the provisioner code. but i hoped i only may be blind. ;)
<fwereade> TheMue, and if we decide that is not the right place for it we can then put it somewhere else
<fwereade> TheMue, I thought I saw a TODO in there back in oakland :/
<TheMue> fwereade: ok
<TheMue> fwereade: imho it would be ok if the provisioner detects it but it notifies the firewaller to keep control over the ports (as it has its own representation of its world)
<fwereade> TheMue, why can't the provisioner make the appropriate state changes?
<wrtp> fwereade: +1
<wrtp> TheMue: the provisioner only needs to set the units' ports
<fwereade> TheMue, if that feels wrong, then I don't really mind who does make the state changes, except that it shouldn't be the FW... surely?
<fwereade> TheMue, I think that the FW sounds complex enough without adding feedback loops to it ;)
<TheMue> fwereade: yes, state changes are a kind of notification too. i mean that only opening and closing ports should be done in the firewaller to keep the internal representation up-to-date
<wrtp> i'm wondering if the environment global port changes should have a different entry point; e.g. Environ.OpenPorts
<TheMue> wrtp: that sounds reasonable
<fwereade> wrtp, I shouldn't probably be getting into this, but the everything-opens-the-union-of-ports-needed-in-the-whole-deployment approach really does still sound crazy to me
<wrtp> fwereade: and the alternative is?
<fwereade> wrtp, it feels no different in spirit from "meh, just open everything"
<wrtp> fwereade: i think it's quite different
<wrtp> fwereade: a small set of ports vs everything
<wrtp> fwereade: most installations will only open a very small number of ports, i believe
<fwereade> wrtp, still doesn't seem sane to me that service not-a-big-deal can open ports for very-important-db-service
<wrtp> fwereade: ah, well if we're talkin' malicious services, you're probably right.
<fwereade> wrtp, but there is clearly something I just Do Not Get here
<TheMue> fwereade: maybe one global group has the same fault than one per machine. one per service could make more sense.
<wrtp> fwereade: that something is that without this change, we *cannot* scale under ec2
<fwereade> wrtp, haven't we known since day 1 that the FWing is crack, and the only sane solution is getting the MA to handle it?
<fwereade> wrtp, well, yeah, because of the known-stupid approach
<wrtp> fwereade: yeah, that would be much better
<wrtp> fwereade: do we know how to do that?
<fwereade> wrtp, I was under the impression that we have iptables everywhere... and the logic for knowing what ports should be open on a machine does already exist
<wrtp> fwereade: i tend to agree that we're adding complexity for no particularly good reason, and it's complexity that we want to throw away as soon as possible
<wrtp> fwereade: is iptables sufficient?
<Aram> a trivial: https://codereview.appspot.com/6637050
<wrtp> fwereade: after all, do we want to allow a dubious charm to manipulate a machine's iptables and thereby exposed ports that shouldn't be exposed
<fwereade> wrtp, perhaps not -- I am no expert -- but I haven't heard anyone suggesting it isn't, and I've heard plenty of people suggesting it should
<fwereade> wrtp, we have this tool called "open-port"
<wrtp> Aram: LGTM
<wrtp> fwereade: yes, but open-port does nothing if the service is not exposed.
<fwereade> wrtp, right, and?
<fwereade> wrtp, in theory, at least, units are containerised
<fwereade> wrtp, and the MA will be responsible for the FWing
<wrtp> fwereade: yeah.
<fwereade> wrtp, I don't see how the situation here is any different from anything else the unit could or could not do
<wrtp> fwereade: i think part of the difficulty is we don't know what we're doing just to make do, and what's going to be around for a while.
<wrtp> fwereade: because currently, units are not containerised
<wrtp> fwereade: with the current scheme, a unit can't expose itself without someone from outside explicitly deciding to do so.
<wrtp> fwereade: i'm not saying that using iptables is a bad thing, just that it does change the security model slightly.
<TheMue> wrtp, fwereade: how do different provider handle this? does openstack has security groups too?
<wrtp> fwereade: i do think that the effort that's going into doing this global ports stuff (and adding complexity to the core that will take effort to remove later) we'd be better implementing an on-machine firewaller
<fwereade> wrtp, yeah, I'm just reacting to a feeling that we're putting a *lot* of effort into a provider-specific solution that is kinda crap even for that provider
<wrtp> TheMue: at least some other providers don't implement firewalling at all AFAIK
<wrtp> fwereade: agreed
<fwereade> wrtp, but, well, I have my own worries in other bits of the codebase :/
<TheMue> wrtp: expected it, yes. iptables should work everywhere, don't they? how about lxc?
<wrtp> fwereade: and things like tests for config.FirewallerMode spreading around the code make me unhappy
 * fwereade slopes off for a pre-meeting ciggie, brb
<TheMue> wrtp: yes, the topic is larger than thought in the beginning
<Aram> TheMue: containers can each have a different network stack, so yes.
<niemeyer> Yo
<wrtp> niemeyer: yo!
<Aram> hi
<wrtp> niemeyer: G+?
<wrtp> Aram: ^
<niemeyer> davecheney: ping
<wrtp> niemeyer: this was the CL i meant to propose last night but accidentally pushed it onto a previous branch instead: https://codereview.appspot.com/6639043/
<fss> niemeyer: good morning
<fss> niemeyer: is launchpad happier today?
<niemeyer> fss: Haven't talked to it yet, but I'm hoping so :)
<niemeyer> wrtp: Cheers, will check it
<fss> niemeyer: nice, let's hope so :-)
<niemeyer> Woohay broken pipe, literally
<niemeyer> And it wasn't just any pipe.. it was a *huge* pipe, with wild pressure
<niemeyer> OK: 42 passed
<wrtp> niemeyer: this should make the authentication problem easier to deal with in tests: https://codereview.appspot.com/6643049
<niemeyer> wrtp: That doesn't quite work..
<wrtp> niemeyer: oh
<wrtp> niemeyer: it *seems* to...
<niemeyer> wrtp: Yeah, but just seems :)
<wrtp> niemeyer: ok, so how is it broken?
<niemeyer> wrtp: The login information is cached.. mgo manipulates connections by itself.. you've rendered the login information it is using invalid by killing the user
<wrtp> niemeyer: the login information is sent with each request?
<niemeyer> wrtp: No, it's properly handled internally
<wrtp> niemeyer: FWIW if this logic is wrong, then i think the logic that i previously had in the bootstrap-state test might have been wrong too
<wrtp> niemeyer: i'd like to understand the mgo auth model a little better
<niemeyer> wrtp: If you were killing the user that juju logs in with, then yes, it's wrong
<wrtp> niemeyer: so you can't authenticate and then delete the user?
<niemeyer> wrtp: It's pretty straightforward.. you give it a user, and it uses it
<wrtp> niemeyer: and a session is associated with a user?
<niemeyer> wrtp: If you remove the user under it, it may blow up next time
<wrtp> niemeyer: so, i'm trying to think of a way that i can set the db into authenticated mode, set an admin user, and still be able to set the db back into unauthenticated mode afterwards, without knowing what the admin user was set to.
<wrtp> s/admin user was set/admin password was set/
<wrtp> niemeyer: i think you're saying that a session becomes invalid if the user it was logged in as is removed. but removing the user is the only way to get the db to work without logging in.
<wrtp> niemeyer: or rather, having *no* admin user is the way to do that
<niemeyer> wrtp: There's nothing complex there.. don't remove the user you want to operate as..
<niemeyer> wrtp: otherwise we'll see random failures when it attempts to authenticate a connection.. that's all
<wrtp> niemeyer: because there's no a one-to-one correspondence between session and connection?
<wrtp> s/no a/not a/
<wrtp> niemeyer: i have to say i thought there was, but i think i understand better now.
<niemeyer> wrtp: yeah, there's a good reason why it's called session rather than connection :)
<niemeyer> wrtp: A session abstracts away the communication with the whole cluster
<niemeyer> wrtp: The primary may shift and you may not notice
<niemeyer> (you may notice as well, though, depending on what was in progress by then)
<wrtp> niemeyer: so perhaps a better approach might be to set the admin password to "" rather than removing the user, and always attempt to log in even if the password is "".
<niemeyer> wrtp: My suggestion is to keep evolving logic for a bit without refactoring
<niemeyer> wrtp: Let's try to get this stuff working
<wrtp> niemeyer: my reason for doing this is i couldn't work out a good way of writing a particular test in another branch
<wrtp> niemeyer: i'll show you the test, one mo
<wrtp> niemeyer: http://paste.ubuntu.com/1269312/
<wrtp> niemeyer: it's in the juju package
<wrtp> niemeyer: given that Environ.Bootstrap sets the admin password (as it should), how can i make the test revert to the previous admin-passwordless state if it fails half-way through?
<wrtp> niemeyer: if the password is changed, that might invalidate the session too, right?
<niemeyer> wrtp: Reset the database if the test fails, for example
<wrtp> niemeyer: yeah
<wrtp> niemeyer: you mean restart the mgo server?
<wrtp> niemeyer: so i think that means that my defer st.SetAdminPassword("") lines in the state tests are bogus
<wrtp> s/that/this/
<niemeyer> wrtp: I mean that's one way of doing it
<niemeyer> wrtp: Resetting the password can also work, as you've noticed
<wrtp> niemeyer: how can we reset the database?
<niemeyer> wrtp: ?
<wrtp> niemeyer: we don't have an authenticated connection any more, so we can't manipulate the db to reset it or change the password
<niemeyer> wrtp: It's our database.. our files.. our machine :)
<wrtp> niemeyer: ok, so how do we do it? can i go underneath mgo and manipulate its files directly?
<wrtp> niemeyer: i'm not sure how many abstractions i need to break here :)
<wrtp> niemeyer: perhaps restarting mgo is a reasonable approach. if we get an auth failed error when trying to reset the db, we could kill the server and start a new one. this should never happen in the normal course of events, so it won't slow down tests.
<niemeyer> wrtp: Right
<wrtp> niemeyer: i think it feels right to make the db reset work regardless of what the test has done. the current SetAdminPassword defers are unnecessary (and wrong, as i think you've demonstrated)
<wrtp> niemeyer: so are you ok with the above approach? (restarting mgo on auth failure)
<niemeyer> wrtp: I think it's fine if the test passes
<niemeyer> wrtp: No reason to slow down the test
<wrtp> niemeyer: which test?
<niemeyer> wrtp: The same one you're talking about
<niemeyer> Woohay.. Launchpad responded after 1h trying to submit
<wrtp> niemeyer: i agree there's no reason to slow down the test, hence my suggestion (restarting mgo on auth failure when resetting), which is what i'm checking you're ok with
<wrtp> niemeyer: jeeze
<niemeyer> fss: Answering your question, no, Launchpad isn't much happier today
<niemeyer> wrtp: The point is that you need the defers on the success case
<wrtp> niemeyer: ah, i see
<fwereade> been on since 8ish, have run out of brain... might be back a bit later, but provisionally calling it a day
<niemeyer> fwereade: Have a good time then, provisionally :-)
<wrtp> niemeyer: ok, that makes sense, as the password is in a known state at the end of the test. it means we probably don't need to make it a defer either - we can just call SetAdminPassword at the end of the test, which makes the logic more straightforward.
<niemeyer> wrtp: Right
 * niemeyer => lunch
<wrtp> fwereade: i just saw this uniter test failure: http://paste.ubuntu.com/1269425/
<fwereade> wrtp, hum, that is relevant to my interests, would you make a bug please?
<wrtp> fwereade: k
<wrtp> fwereade: https://bugs.launchpad.net/juju-core/+bug/1064476
<wrtp> niemeyer: PTAL https://codereview.appspot.com/6643049
<fss> niemeyer: :-(
<fss> niemeyer: sorry for the huge delay, I was out for lunch
<niemeyer> fss: No worries
<wrtp> niemeyer: passwords used for real: https://codereview.appspot.com/6632049
<niemeyer> wrtp: Thanks
<niemeyer> wrtp: I'll have to give the other branches some attention
<wrtp> niemeyer: np
<niemeyer> fwereade: ping
<niemeyer> fwereade: Oh, sorry, you're in relax mode already
<wrtp> i'm off for the evening.
<niemeyer> wrtp: Cheers
<wrtp> niemeyer, fwereade, Aram: night all
<niemeyer> wrtp: Thanks, have a good night too
<wrtp> niemeyer: will do, thanks
<wrtp> niemeyer: and you
<niemeyer> Aram: https://codereview.appspot.com/6595064/ reviewed
<niemeyer> Aram: Sorry it took a day to get to it
<fwereade> niemeyer, ping
<niemeyer> fwereade: yo
<fwereade> niemeyer, I guess that was actually a pong really :)
<niemeyer> fwereade: Ah, don't worry then, it's all good
<niemeyer> fwereade: I've been doing reviews, so you'll have some ideas to look at/branches to merge tomorrow
<fwereade> niemeyer, yeah, I see the one you don't like
<niemeyer> fwereade: It's not entirely that I don't like.. I think it's more about a previous debate having some evidence than about the branch content itself
<niemeyer> fwereade: I think we should debate, but I wouldn't mind that the shift of convention was done a tip, for example
<fwereade> niemeyer, the trouble is, on a brief reading, I can't see any cases where the error return doesn't make the code more complex
<niemeyer> fwereade: Interesting, I see exactly the opposite
<niemeyer> fwereade: I see a different convention handling cases we're used to
<fwereade> niemeyer, in every case, you seem to be asking me to switch `if x() { y() }` into `if a, err = y(); err != nil { if err != someSpecificError {return err} } else { a.b() }
<niemeyer> fwereade: and pretty much no case where it's not the good-old if err != foo { ... }
<fwereade> niemeyer, well, in every case, I have to handle a nonsensical extra branch
<niemeyer> fwereade: Is it 100% guaranteed that if we see a relation id in RelationIds, Relation(id) will necessarily work for it?
<fwereade> niemeyer, because those methods actually would only return one error ever
<fwereade> niemeyer, well, yes...
<niemeyer> fwereade: Interesting. How can we guarantee it?
<fwereade> niemeyer, it should not be in any way dependent on external state
<fwereade> niemeyer, well, we need to ebmed meaning into the interface above and beyond that explicitly stated in the code
<fwereade> niemeyer, in the same way that, say sort.Strings() makes the guarantee that it won;t launch nuclear missiles
<fwereade> niemeyer, IYSWIM
<fwereade> niemeyer, this is a straight replacement of a struct with an interface
<fwereade> niemeyer, if it's ok to do `if X != ""`, why is it not ok to do `if X() != ""`?
<niemeyer> fwereade: I thought the review was clear
<niemeyer> fwereade: I'm simply showing evidence that something looks odd
<niemeyer> fwereade: I'm not hand-waving that this is bad
<niemeyer> fwereade: If you're doing Has+Get, Has+Get, Has+Get, Has+Get consistently, it *seems* to me that the interface is fragile.. because tomorrow non-William will come here and put Get, and blow it up
<niemeyer> fwereade: I accepted to wait until later to see if we'd do that or not.. your branch does exactly that so far
<niemeyer> fwereade: In pretty much all cases but one or two
<niemeyer> fwereade: I'm still talking, though
<niemeyer> fwereade: Rather than enforcing anything
<fwereade> niemeyer, ok... perhaps I am taking it the wrong way
<niemeyer> fwereade: What if we took away all of those Has methods and used methods that have a second (..., ok bool) result?
<niemeyer> fwereade: Does that solve your concern?
<fwereade> niemeyer, probably 99%, yes
<niemeyer> fwereade: Cool, it solves mine as well
<fwereade> niemeyer, it's the introduction of fake error paths that's mandated by the error return that bugs me
<fwereade> niemeyer, ok, and that's fewer methods in the interface too, nice :)
<niemeyer> fwereade: Because it forces both the consumer and the producer of that interface to acknowledge the fact the data may not be availble
<fwereade> niemeyer, indeed -- it still feels somewhat heavyweight, tbh, but maybe by just the right amount considering the different expectations of an interface and a struct
<niemeyer> fwereade: My feeling is that it's actually both less work and less code than the current implementation
<niemeyer> fwereade: If you're happy to move that way, as I mentioned, I'm happy to have that done at the tip
<fwereade> niemeyer, agreed :)
<fwereade> niemeyer, but tbh if we're agreed on a direction I'm perfectly happy threading it through... feels cleaner
<niemeyer> fwereade: Sounds good.. my LGTMes are still valid if you decide to refactor on the way
<fwereade> niemeyer, cool, thanks
<fwereade> niemeyer, depending on whether or not cath wakes up, I should be able to run this branch past you again pretty soon
 * niemeyer sings for cath
<fwereade> niemeyer, if I give them named return values, ie (r ContextRelation, ok bool), ISTM that that makes the convention pretty clear without explicit documentation... sane?
<niemeyer> fwereade: ok is kind of ambiguous.. if we name it "found" I guess it'd be okay
<niemeyer> fwereade: ambiguous in the language, I mean
<niemeyer> fwereade: ok is of course meaningless in that regard :-)
<fwereade> niemeyer, ok, cool -- I'm mainly asking because I can't find the right words for the doc comment :)
<niemeyer> fwereade: appending "if it was found and whether it was found" to the end of the first sentence of those methods should do the deal, I think
 * fwereade peers critically at the sentences... yeah, LGTM
<fwereade> niemeyer, cheers
<fwereade> niemeyer, and, yeah, the code's way nicer too
<fwereade> niemeyer, https://codereview.appspot.com/6633043 reproposed
<niemeyer> fwereade: Woot
<niemeyer> fwereade: LGTM, thank hyou
<fwereade> niemeyer, cool, thanks
<fwereade> niemeyer, sorry this bit was difficult... I had a surprisingly violent adverse reaction to the error returns over the weekend, though... I felt my code was made of lies, in some way, and it really bugged me :)
<niemeyer> fwereade: No worries.. I think the end result is better than either of the original ideas we had
<niemeyer> fwereade: So the brainstorming was worth it
<fwereade> niemeyer, definitely :)
#juju-dev 2012-10-10
<davecheney> WHAT
<davecheney> % juju status
<davecheney> error: Get https://s3.amazonaws.com/juju-4218c46225cb7856d9d5ca3c1a685cd87b647996/provider-state: lookup s3.amazonaws.com: no such host
<davecheney> internets is whack!
<davecheney> amazon is sooooo slow today
<davecheney> niemeyer: % error: cannot assign unit "tomcat7/0" to machine: cannot assign unit "tomcat7/0" to unused machine: duplicate key insert for unique index of capped collection
<davecheney> happens a lot
<davecheney> is this related to the version of mongo i'm running
<SpamapS> davecheney: shouldn't you be hitting ap-southeast-1 ?
<davecheney> SpamapS: doesn't make any difference
<davecheney> s3 is still slow as shit
<niemeyer> davecheney: Yeah, new mgo solves that
<davecheney> niemeyer: hmm, maybe i didn't go get
<davecheney> will double chekc
<davecheney> no, i must have had too, otherwise it wouldn't compile
<niemeyer> davecheney: It wouldn't?
<davecheney> didn't you introduce a new symbol in the latest mgo ?
<davecheney> and we use that in juju
<niemeyer> davecheney: I introduced a symbol before the fix to the problem above was released
<davecheney> ok
<davecheney> niemeyer: i've recompiled
<davecheney> thanks
<niemeyer> davecheney: and mgo will still compile file even if juju lacks the fix to the problem above
<davecheney> niemeyer: welcome to the unspoken dependency problem in go
<niemeyer> davecheney: Yeah, it's kind of a general problem
<niemeyer> davecheney: Also one of the reasons why I don't do interim announcements/releases
<davecheney> niemeyer: yeah, there is no good way for people to _know_ they have the right release
<davecheney> i've worked around it in juju by adding tests which break when we've added something to a dep, like the --format= stuff in gnuargs
<davecheney> but that isn't really a solution
<niemeyer> davecheney: Yeah, not a general one at least.. we can easily add a test against a specific version, though
<niemeyer> Assert(mgo.Version, Equals, Version)
<niemeyer> davecheney: Assuming mgo.Version exists, which it doesn't at the moment
<niemeyer> Anyway, I'm in serious need of some sleep
<davecheney> niemeyer: later
<niemeyer> It was a long day, both with great stuff and with unusually awkward stuff
<niemeyer> davecheney: Have a good day there
<davecheney> niemeyer: will have the charm audit done by this arvo
<davecheney> am down to s
<davecheney> with a larger sample size, the numbers are looking better
<davecheney> fewer charms broken as a % of the total
<davecheney> so, some intersting problems with subordinate charms
<davecheney> they tend to clobber each other's apt-get requests
<davecheney> and fail
<davecheney> and this is what happens when you start too many instances
<davecheney>   20:
<davecheney>     instance-id: pending
<davecheney>   21:
<davecheney>     instance-id: pending
<davecheney>   22:
<davecheney>     instance-id: pending
<davecheney> lucky(~) % grep -c -- '- failed' ~/charm-audit.txt
<davecheney> 24
<davecheney> lucky(~) % grep -c -- '- started' ~/charm-audit.txt
<davecheney> 54
<davecheney> lucky(~) % grep -c -- '- unknown' ~/charm-audit.txt
<davecheney> 5
<davecheney> will post full report tonight.
<SpamapS> davecheney: I would not expect subordinates to install in parallel
<fwereade> davecheney, how are you installing subordinates in the first place? :)
<davecheney> fwereade: juju deply
<davecheney> there are a few charms that don't do anyting
<davecheney> the juju charm for example
<davecheney> but others happily cohabitate
<fwereade> davecheney, still don't see how that'd work -- what's deploying them?
<davecheney> fwereade: try it yourself
<davecheney> happy uniter is happy
<fwereade> davecheney, that is pleasing, but... the uniter can't deploy new units yet, afaik, and the MA shoudln't be deploying subordinate units... should it?
<davecheney> fwereade: maybe it can't tell
<fwereade> davecheney, but juju deploy exits early, before adding units, with subordinate charms
<davecheney> fwereade: ahh
<davecheney> i was mistaken
<davecheney> juju is a subordinate
<fwereade> davecheney, and Service.AddUnit refuses to add them anyway :)
<davecheney> but charms like hadoop-master and hadoop-slave are cohabitatin
<wrtp> davecheney, fwereade: morning
<fwereade> davecheney, ah, cool... so you have hacked placements or something?
<fwereade> wrtp, heyhey
<davecheney> fwereade: nope
<davecheney> that is just what happens
<fwereade> davecheney, I has a suspicious
<fwereade> davecheney, but well, hey, at least we're running them, even if it's not clear tome how ;p
<SpamapS> sounds like the placement policy of "unused" machines isn't being respected
<fwereade> SpamapS, yeah...
<SpamapS> which would actually make quite a few people very happy ;)
<fwereade> SpamapS, but then lots of things *do* end up on their own machines (right, davecheney?)
<SpamapS> (even if it is dangerous and error prone)
<fwereade> SpamapS, haha, yeah
<davecheney> SpamapS: fwereade it could be related to the charm names
<davecheney> it only happens to charms that have the same prefix
<fwereade> davecheney, whoa, wtf?
<davecheney> fwereade: try it
<fwereade> davecheney, so cs:precise/hadoop-master and cs:precise/hadoop-slave end up cohabiting?
<davecheney> yup
<davecheney> two unit agents
<davecheney> running at the same time
<davecheney> fighting over the apt lock
<SpamapS> http://blog.ancientlasers.com/wp-content/uploads/2012/03/Mind-Blown.jpg
<fwereade> davecheney, hmm, this feels rather like a bug with a capital B, and probably capital U and G as well
<SpamapS> davecheney: the apt thing is a problem when python unit agents co-habitate as well.. charms tend to be written assuming they're "masters of their own domain"
<davecheney> SpamapS: intersting
<davecheney> isn't their a flag to apt to say 'please wait, forever if that is what it takes'
<SpamapS> not that I know of
<SpamapS> but I believe you can use aptd for that
<davecheney> maybe i'm thinking of yum
 * davecheney slaps himself
<SpamapS> davecheney: either way, its not just apt-get ... there may be other ordering issues
<SpamapS> davecheney: if two things want to rearrange the services running right now.. the stops/starts may happen in a very wrong order.. for instance
<SpamapS> if you're going to have more than 1 unit per "container", you have to serialize all hook execution
 * SpamapS really should go try to sleep again
<davecheney> SpamapS: more investigation is called for
<SpamapS> davecheney: I'm still quite surprised that charms with similar names are being chosen to live on one machine
<davecheney> SpamapS: it is possibly user error
<fwereade> wrtp, I think I have a fix for lp:1065576 (the git problem you found yesterday)... can you remind me what the standard I-fixed-a-bug summary line should look like?
<wrtp> fwereade: pkgname: fix git so it doesn't do X
<wrtp> fwereade: Fixes bug NNN
<fwereade> wrtp, ah, I thought there was meant to be a bug ref in the summary
<fwereade> wrtp, cheers
<wrtp> fwereade: i don't think so
<wrtp> fwereade: but you can use --bug with lbox
<wrtp> fwereade: (and you probably should)
<fwereade> wrtp, last time I tried that it overwrote the description so I steered well clear
<wrtp> fwereade: fairy nuff
<fwereade> wrtp, https://codereview.appspot.com/6636056 should be a trivial, I think
<fwereade> bbs
<wrtp> fwereade: it looks plausible, but i don't really know what's going on here. what does "a hook has run and is being committed by git" mean? how are hooks committed by git?
<Aram> moin.
<fwereade> wrtp, b
<fwereade> Aram, morning
<fwereade> wrtp, we snapshot dir state at the end of each hook
<wrtp> fwereade: which dir?
<fwereade> wrtp, so git can be already running something in the uniter when we try to do status
<fwereade> wrtp, the charm dir
<wrtp> fwereade: wow. isn't that a bit... heavyweight?
<fwereade> wrtp, on discussion a good while ago, it seemed to generally be considered to be a good thing
<wrtp> fwereade: doesn't that mean that if you've got a large charm, it's read in its entirety on each hook?
<wrtp> fwereade: i'm sure i'm missing something. what's this in aid of?
<fwereade> wrtp, git has always seemed to me to be pretty efficient
<fwereade> wrtp, what is it you're worried about?
<wrtp> fwereade: i suppose if we trust mtimes, it won't be too bad.
<wrtp> fwereade: i thought the charm directory was usually immutable actually
<fwereade> wrtp, huh? the charm dir is anything but
<fwereade> wrtp, IMO this is a serious drawback
<wrtp> fwereade: you mean the directory containing the charm hooks &c ?
<fwereade> wrtp, yeah, the deployemtn dir
<wrtp> fwereade: interesting. so what does this enable?
<wrtp> fwereade: i hope we don't have charms that put large database files in the charm directory.
<wrtp> fwereade: because that could make things seriously slow (and we could use lots of disk space too)
<fwereade> wrtp, mainly it's intended to help us figure out wtf has been going on when things go wrong
<fwereade> wrtp, AIUI it's charm state that tends to get put in there
<fwereade> wrtp, I don't think people are going to be installing postgres inside the charm dir, for example
<wrtp> fwereade: if they do, we'll be in for a nasty surprise :-)
<wrtp> s/we/they/
<fwereade> wrtp, if they do they're fucked whatever we do
<fwereade> wrtp, because charm upgrades will do the snapshots anyway
<fwereade> wrtp, this way, if people are messing with the charm dir other than in hooks, at least we find out fast
<wrtp> fwereade: there's a difference between snapshotting once on upgrade, and snapshotting on every hook
<wrtp> fwereade: what's wrong with messing with the charm dir other than in hooks?
<fwereade> wrtp, how do you tell a hok isn't messing with the charm dir itself?
<wrtp> fwereade: when i wrote a charm before, the charm dir seemed to be the only piece of space that was the charm's own. so it seemed logical to put mutable data files there.
<wrtp> fwereade: i think you're saying that's frowned on
<fwereade> wrtp, depends how they're mutated
<wrtp> fwereade: it didn't depend on anything before AFAIK
<fwereade> wrtp, if you've got, say, a cron job messing with the charm dir then, yeah, that's not sensible
<wrtp> fwereade: why not?
<wrtp> fwereade: do we document that?
<fwereade> wrtp, I have no idea
<fwereade> wrtp, (why we don't document it)
<wrtp> fwereade: it makes me a little anxious
<fwereade> wrtp, (*whether* we document it)
<wrtp> fwereade: because we're saying "of course you shouldn't do *that*", but there's nothing that ever told anyone that it might be a bad idea (and it certainly seemed like a reasonable idea to me back when)
<fwereade> wrtp, I don't see how it could ever be considered sane to mutate the charm dir from outside a hook at the same time a hook is running
<fwereade> wrtp, not knowing when a hook is running, which you don't, would STM to preclude the sanity of touching the charm dir other than through juju
<wrtp> fwereade: why not? say you're implementing a subordinate. you run a database of some kind; where do you put the data files for it? putting them in the charm dir seems kinda reasonable to me.
<fwereade> wrtp, woudl you consider it sensible to change DB files youself, from outside the DB, while the DB was running?
<wrtp> fwereade: no, but the DB will change its files while the hooks are running and while they're not, right?
<fwereade> wrtp, how is that situation any different?
<wrtp> fwereade: there are files that matter to us in the charm dir, and files that don't
<fwereade> wrtp, I would personally put the data files somewhere in var
<fwereade> wrtp, oh really? how do we tell what isn't important?
<fwereade> wrtp, you seem to be assuming perfect knowledge of all current and future possible charm upgrades...
<fwereade> wrtp, if you want to keep some charm-related state in the charm dir, and manipulate it in the hooks, great
<fwereade> wrtp, if you want to install everything from the service in a non-standard place, on your own head be it ;)
<wrtp> fwereade: if the user creates a "data" dir in their charm dir and uses it for some running database (that's similar to what i did), i don't see that should have untoward effects
<wrtp> fwereade: but we're going to break that
<fwereade> wrtp, I don't see how that's possible
<fwereade> wrtp, you can't do that at all ever anyway, can you?
<wrtp> fwereade: huh?
<fwereade> wrtp, (without it being risky/broken, anyway)
<wrtp> fwereade: why's that?
<fwereade> wrtp, how do you handle concurrent writes to the charm dir?
<fwereade> wrtp, or writes/reads, whatever
<wrtp> fwereade: how does who handle it?
<wrtp> fwereade: the running database?
<wrtp> fwereade: or juju?
<fwereade> wrtp, how do you propose to implement this scheme whereby juju, and something-else, are both writing concurrently to the same dir without errors?
<wrtp> fwereade: two things can happily write to two independent files without errors. i'm not sure i see the problem.
<fwereade> wrtp, and how do you propose to ensure this?
<wrtp> fwereade: i must be missing something here. we all share the same filesystem.
<wrtp> fwereade: are you thinking about upgrades in particular here?
<fwereade> wrtp, I'm thinking about charm upgrades, and every hook
<fwereade> wrtp, all these things can write arbitrary things into the charm dir
<fwereade> wrtp, you're apparently saying that we should be able to guarantee that *anything* else can *also* write arbitrarily to the charm dir, at any time, without anything ever goind wrong
<wrtp> fwereade: the things written by hooks are under the control of the charm author. as are things written by programs running in the bg, started by the hooks.
<wrtp> fwereade: not really. i'm saying that the charm dir is just a dir. we can have two independent things writing to files in it with no problem, just the same way that many programs can write to independent files under the global root.
<wrtp> fwereade: the difficulty here is that we're assuming that juju has entire control of the charm dir, but i'm not sure that's necessarily the case.
<fwereade> wrtp, I think you're saying that you can guarantee that the written files will be independent
<fwereade> wrtp, which STM to be an unsupportable assertion
<wrtp> fwereade: if i start a database and give it a file to write to, or a directory that i've created for it to put its data files in, i know that it's not going to write outside that file or dir.
<wrtp> fwereade: how is the charm dir different from, say, /tmp ?
<fwereade> wrtp, the charm dir is owned by juju, and juju expects to have control over it?
<wrtp> fwereade: (FWIW, i'm not saying that what we're doing now with git snapshots is *wrong*, but i think it's worth exploring the implications)
<fwereade> wrtp, would you install mysql inside postgres' data dir?
<wrtp> fwereade: the problem is that the charm author might feel *they* have control over the charm dir
<fwereade> wrtp, they have control over it, strictly mediated by juju
<wrtp> fwereade: after all, they've created the files and populated the directories
<wrtp> fwereade: it's not strictly mediated...
<wrtp> fwereade: and i wouldn't be surprised if lots of charms used it for general scratch space
<fwereade> wrtp, this STM like a documentation issue, fundamentally
<wrtp> fwereade: and if they do, i we may have a problem
<wrtp> fwereade: sure
<wrtp> fwereade: it's also a potential backward-compatibility issue
<fwereade> wrtp, IMO it's a latent-charm-bug issue
<wrtp> fwereade: out of interest, is a charm allowed to change its hooks (e.g. to add a new relation-joined hook) on the fly?
<fwereade> wrtp, I don;t think we ever claimed remotely that it was safe to modify juju's dirs while juju is running
<wrtp> fwereade: it's only a bug because we've made it a bug
<fwereade> wrtp, nothing stopping it from doing so
<wrtp> fwereade: so it doesn't cache the current hooks then?
<fwereade> wrtp, assuming that it's fine to mess with something's runtime state, just because nobody told you not to, doesn't seem very sensible to me
<fwereade> wrtp, no, it just executes whatever's there
<wrtp> fwereade:  we never claimed remotely that it *wasn't* safe to modify juju's dirs while juju is running AFAIK :-)
<fwereade> wrtp, what makes you think that is a reasonable assumption?
<wrtp> fwereade: it's not *something's* runtime state. it's a *charm's* runtime state. so it seems not unreasonable for a charm to assume that it can change it whenever it likes.
<wrtp> fwereade: why not?
<fwereade> wrtp, 5 seconds of thinking would surely reveal that juju has the power to make arbitrary changes to that dir, and lead one to look for mechanisms by which that operation can be made safe, and to conclude from the lack of such mechanisms that it is in fact not safe
<fwereade> wrtp, details like "no, you cannot run juju tools outside hooks" seem to support this
<wrtp> fwereade: is there not a hook that's run before an upgrade?
<fwereade> wrtp, nope
<fwereade> wrtp, that might also be considered a hint
<wrtp> fwereade: that seems wrong actually
<fwereade> wrtp, hmm, how would it help?
<wrtp> fwereade: ISTM that a charm ought to be able to have the opportunity to shut things down cleanly before an upgrade if it wants to
<fwereade> wrtp, whoa, why would upgrading the charm remotely justify screwing up the service?
<wrtp> fwereade: depends on the service.
<fwereade> wrtp, if, say, a config change causes us to want to change *service* version, then we can deal with that
<fwereade> wrtp, conflating service version and charm version seems to be a fundamental error
<wrtp> fwereade: there is a *post*-upgrade hook, right?
<fwereade> wrtp, yes
<fwereade> wrtp, AIUI this is essentially to give you the opportunity to rearrange the files inside the charm dir if required
<wrtp> fwereade: well, you might also need to restart services, etc
<fwereade> wrtp, what does a charm change matter to the service?
<wrtp> fwereade: i'm talking about daemons, not juju services, sorry
<fwereade> wrtp, yeah: why would a charm change justify such an action?
<fwereade> wrtp, a config change might
<wrtp> fwereade: what if the charm change is to make a daemon start with different command-line arguments?
<wrtp> fwereade: or to use an alternative daemon for something
<wrtp> fwereade: there are many possible reasons a charm might be upgraded
<fwereade> wrtp, ok, fair enough -- then you can do that just fine in the post-upgrade hook with a minimum of downtime
<fwereade> wrtp, what good would a pre-upgrade hook do?
<fwereade> wrtp, although, more likely, you'll just do that in the subsequent config-changed anyway
<wrtp> fwereade: ISTM that people may have daemons that look at files in the charm dir. if the charm dir can change without warning, that's not a good idea.
<fwereade> wrtp, I agree that that is not a good idea
<wrtp> fwereade: it's probably not a good idea anyway, but i bet lots of people do it
<fwereade> wrtp, that is why people should not do it
<fwereade> wrtp, the charm dir is for the charm
<fwereade> wrtp, the service already has a standard place for everything, surely?
<wrtp> fwereade: i see that. but what *you* think of as the charm is not what others might think of as the charm
<wrtp> fwereade: what place is that?
<wrtp> one mo, gotta do the bins
<fwereade> wrtp, service-dependent, surely?
<fwereade> wrtp, but the general philosophy is that we should be able to completely nuke juju, and leave the service running unhindered
<fwereade> wrtp, if the service knows about juju, the charm is being bad
<fwereade> wrtp, brb ciggie
<wrtp> fwereade: is there a way for a charm to find out its service name?
<wrtp> fwereade: because you might easily have two of the same charm deployed in the same container. then each one needs some space for itself. the charm dir might seem like a good place for that, but i see why you think it's not.
<wrtp> fwereade: i wonder if we should provide each charm with its own temporary directory that it can do whatever it likes to.
<wrtp> fwereade: then deprecate the whole idea of writing to the charm dir, whether inside or outside a hook.
<fwereade> wrtp, +1 on that, but I think it's independent of the don't-let-the-service-know-about-the-charm issue
<wrtp> fwereade: i was responding to your "the service already has a standard place for everything, surely?" - that's not necessarily true
<wrtp> fwereade: but i agree with your sentiment in general
<fwereade> wrtp, by convention, if nothing else
<fwereade> wrtp, if you're installing X, you tend not to put X's data files inside Y's data dir
<wrtp> fwereade: what if you're installing two Xs?
<wrtp> fwereade: we really really really need to write some decent docs
<fwereade> wrtp, agreed there :)
<wrtp> fwereade: that was a genuine question above BTW; is there a way for a charm to find out its service name?
<fwereade> wrtp, not directly, but it knows its unit name
<wrtp> fwereade: ah, that's good enough then.
<wrtp> fwereade: how does it find its own unit name?
<fwereade> wrtp, JUJU_UNIT_NAME I think
<fwereade> wrtp, yep
<fwereade> wrtp, what would you expect to need it for?
<fwereade> wrtp, (the service name, I mean)
<wrtp> fwereade: it's a useful disambiguator. if you're a subordinate service, you want to choose a place to store your charm-external state that doesn't clash with other subordinates that are potentially using the same charm.
<fwereade> wrtp, I'd expect the relation id to be the disambiguator, but maybe that's not right
<wrtp> fwereade: there may be no relation
<fwereade> wrtp, how do you deploy a subordinate without a relation?
<wrtp> fwereade: i suppose for subordinates, there's always a relation
<wrtp> fwereade: but do you know that relation id when you're executing the install hook?
<fwereade> wrtp, shouldn't the install hook just, er, install?
<wrtp> fwereade: it might need to install to some charm-external directory. same applies to the start hook.
<fwereade> wrtp, I'm not sure that we're trying to solve the run-multiple-versions-of-the-same-software problem here are we?
<wrtp> fwereade: it's not hard to run multiple versions of the same software; all we need is a directory that each instance can call its own.
<wrtp> fwereade: we allow several subordinate services in a given container that might use the same charm. i can imagine cases where that might be very useful.
<wrtp> fwereade: (i'm imagining a subordinate service that provides some fairly general functionality, determined by the config settings)
<fwereade> wrtp, then, in that case, surely the unit name is a reasonable disambiguator there
<wrtp> fwereade: indeed
<fwereade> wrtp, and the relation id is good for handling multiple relations run by the same charm
<wrtp> fwereade: yeah
 * fwereade once again fails to successfully use wrtp's fly-killing techniques
<fwereade> but it was *close* that time
<wrtp> fwereade: you'll get there :-)
<wrtp> fwereade: did it escape through the top or the bottom?
<fwereade> wrtp, bottom
<wrtp> fwereade: keep trying...
<fwereade> wrtp, really impressively sharp turn away actually, it probably deserved to live
<wrtp> fwereade: you're selecting for canny flies
<wrtp> fwereade: a few more generations and you'll never get 'em
<fwereade> wrtp, I'm not saying I won't try again next time it comes in range ;)
<wrtp> crack crack crackity crack
 * wrtp has just found out why his tests were passing when they couldn't possibly be passing
<wrtp> niemeyer: morning
<wrtp> !
<niemeyer> Good morning!
<wrtp> niemeyer: my use-passwords branch last night was a bit crackful i'm afraid. i forgot to pass the --auth flag to the mongod started by environs/cloudinit.
<fwereade> niemeyer, heyhey
<wrtp> niemeyer: i thought of something this morning and could not for the life of me think how the tests were passing
<wrtp> niemeyer: and that's the reason...
<niemeyer> wrtp: Hah
<niemeyer> fwereade: yo
<wrtp> niemeyer: i thought it all went a bit too smoothly!
<niemeyer> Aram: ping
<Aram> yo
<Aram> hello
<Aram> niemeyer: ^ ^
<niemeyer> Aram: Heya!
<niemeyer> Aram: Good point regarding the check
<niemeyer> Aram: I suspect we'll need to verify if it's already in the pending list or not in those removal detection branches then
<Aram> niemeyer: perhaps it's still a good idea to check if pending is required at the begining of the logic.
<Aram> so we do it only once
<Aram> and then depending on that boolean and the rest of the logic determine what to actual returnm
<Aram> does that make sense?
<Aram> alteratively, we could do it once at the end.
<Aram> but I think logic is simpler if we do it at the begining.
<niemeyer> Aram: I think having this logic in a small method would be as nice, and perhaps more clear
<Aram> good idea.
<niemeyer> Aram: hasString and hasInt can probably be shared by all watchers
<Aram> yes
<fwereade> hmm, maybe I should eat something, bbiab
<niemeyer> fwereade: Sounds wise
<wrtp> niemeyer: fairly trivial: https://codereview.appspot.com/6641053
<wrtp> oops, the branch is misnamed
<wrtp> niemeyer: renamed and reproposed
<wrtp> https://codereview.appspot.com/6635062
<niemeyer> wrtp: On a call with flacoste.. back in a bit
<wrtp> niemeyer: np
<niemeyer> wrtp: Very ncie
<niemeyer> nice
<wrtp> niemeyer: thanks
<wrtp> niemeyer: another fairly small one: https://codereview.appspot.com/6604061
<niemeyer> wrtp: LGTM
<wrtp> niemeyer: lovely, thanks
<niemeyer> Connection Timeout: disconnecting client after 300.0 seconds
<niemeyer> bzr: broken pipe
<niemeyer> Am I the only one facing those issues?
<fwereade> niemeyer, I think so -- I did notice slightly greater than usual flakiness earlier this week but that's it
<niemeyer> fwereade: Weird.. I can't really guess what could be causing it
<niemeyer> Everything else seems to work normally
<wrtp> Aram: i just saw this test failure. looks like something's not sorting the unit names as expected: http://paste.ubuntu.com/1271309/
<Aram> wrtp: sorry, my fault.
<Aram> will fix right away
<wrtp> Aram: thanks!
 * wrtp only discovered this morning that you can do tail -f * and it works how you'd want.
<wrtp> niemeyer: next step: https://codereview.appspot.com/6615071
<Aram> wrtp: quick trivial so I fix the build: https://codereview.appspot.com/6625075/
<wrtp> Aram: LGTM
<Aram> wrtp: sorry for the breakage, in my defense I always run the state tests 20 times before submitting, and it worked here.\
<fwereade> niemeyer, can you think of any reason for me not to move uniter.HookContext to hook.Context? I've come to the conclusion that RelationContext is actually fine as it is -- it's Relationer that really needs the big changes, and it shouldn't be affected by this
<niemeyer> wrtp: done
<niemeyer> fwereade: RelationContext or ContextRelation?
<fwereade> niemeyer, ha, ContextRelation, I'm pretty sure I've changed the name throughout, but my brain hasn't synced up yet
<fwereade> niemeyer, (ContextRelation would also move)
<niemeyer> fwereade: I don't have the context to tell whether a package makes sense, but I can try to help: why do you think a package makes sense? You've been going back and forth in other cases; what's the criteria you've used on those?
<fwereade> niemeyer, I'm planning to move it into the existing hook package
<niemeyer> fwereade: Is that an answer to the above question?
<fwereade> niemeyer, not really, still composing
<fwereade> niemeyer, I moved it to uniter originally because I was sure that was "closer" to the "right place" for it, because there was clearly some overlap between Context and Relationer
<fwereade> niemeyer, it's probably not justified yet though
<niemeyer> fwereade: My feeling is that Context will be strongly bound to the uniter data, but I don't really know yet
<fwereade> niemeyer, I *think* that with something like the following, uniter doesn't need to know about Context at all -- does this just look awful to you? http://paste.ubuntu.com/1271366/
<fwereade> niemeyer, the trouble is that putting that in uniter seems wrong, but I can't really put it in hook if Context (et al) aren't there
<wrtp> niemeyer, fwereade: what do you think about log.Printf messages, and whether/how they should be prefixed?
<wrtp> the first log messages i did (in environs/ec2, for example) are prefixed with the package name.
<wrtp> TheMue, Aram: any thoughts?
<fwereade> wrtp, last time I prefixed normal log messages with the package, niemeyer suggested I remove them; but I'm not sure we have much in the way of conventions
<TheMue> wrtp: my personal logging package has the package es prefix in braces, and that's helpful
<wrtp> i think that some prefix is nice to have, as it makes it obvious where the messages are coming from
<TheMue> wrtp: so i don't have to think about, it's done automatically
<wrtp> fwereade: my thought has been: error messages have no prefix; printed messages get a prefix.
<wrtp> TheMue: that's a reasonable approach actually
<wrtp> TheMue: assuming we remove the launchpad.net/juju-core/ part of the package path
<fwereade> wrtp, doesn't sound unreasonable, but don't really have strong feelings
<fwereade> brb
<TheMue> wrtp: my package currently shows the full path, but yes, that should be configurable (as well as the logging level)
<TheMue> wrtp: see https://code.google.com/p/tcgl/source/browse/applog/applog.go
<TheMue> wrtp: Debugf() also shows filename, function name and line number
<wrtp> TheMue: tbh i think that's a bit heavyweight for what we want.
<TheMue> wrtp: sure, that's my personal approach. but the automatic prefix could help
<wrtp> TheMue: i'm happy with seeing the prefix in the message itself - then it's very obvious how the message is being made
<wrtp> TheMue: i dunno, i could go either way
<TheMue> wrtp: yes, doing it by convention may be enough
<TheMue> wrtp: sometime i fall back in my mainframe history *lol*
 * TheMue has to step out, wife needs me for shopping. bbl.
<niemeyer> fwereade, wrtp: We had jointly decided to drop these prefixes.. my reviews were merely attempting to put in place what we agreed
<wrtp> niemeyer: ah, i don't think i saw that discussion
<fwereade> niemeyer, likewise -- but not bothered by it :)
<niemeyer> wrtp: You were part of it for sure.. it's just been a while
<wrtp> niemeyer: personally, i think the prefixes are useful, particular when the messages from different workers are mixed up in the same file
<niemeyer> fwereade, wrtp: We can change the convention.. I just don't want to be going back and forth pretending we have a convention when we don't
<wrtp> niemeyer: ok, yeah, i definitely thought we had a convention to have prefixed for log messages but not for error messages.
<wrtp> s/prefixed/prefixes/
<niemeyer> wrtp: I hadn't seen that distinction yet, but happy to have it
<wrtp> niemeyer: i'm slightly wondering if you're thinking about the error-message prefix conventions
<niemeyer> wrtp: Yes, I'm thinking of those too
<wrtp> niemeyer: ah, i see
<wrtp> niemeyer: i think there's a definite difference.
<wrtp> niemeyer: error messages are usually printed as a suffix to something else
<wrtp> niemeyer: log messages are printed by themselves.
<niemeyer> wrtp: Not really
<niemeyer> wrtp: We don't have that convention today in any meaningful way.. logged messages are all over the place
<wrtp> niemeyer: all the log messages i've written have a prefix, i think
<niemeyer> wrtp: "That's the way I do" != "That's the convention we use"
<wrtp> niemeyer: indeed. i tried to make a convention by writing log messages that way, but we never discussed it.
<niemeyer> wrtp: Can you please just send a message to juju-dev so that it's actually an agreed convention?
<wrtp> niemeyer: ok, will do
<niemeyer> wrtp: Thanks!
<wrtp> niemeyer: done
<niemeyer> fss: ping
<niemeyer> Aram: ping
<Aram> niemeyer: pong
<niemeyer> Aram: Re. "Addressed review points. Not submitting because I'm waiting for changes
<niemeyer> in firewaller and machiner, so I don't break the build."
<niemeyer> Aram: What changes?
<Aram> niemeyer: the firewaller and the machiner use the old, API incompatible version of the watcher
<Aram> (the one that returns entities, not ids).
<niemeyer> Aram: Would you mind to cover that as well so that we can integrate that change?
<Aram> I'll take a look.
<niemeyer> Aram: I think you can have an idea for the change by diffing.. hmmm
<niemeyer> Aram: Within the firewaller directory: bzr diff -r 600..601 . | vi -
<niemeyer> Aram: It should be fairly simple.. my suggestion is to, right below those case unit, ok := <-ud.watcher.Changes():" lines, actually load the unit
<fss> niemeyer: pong
<niemeyer> Aram: The rest will remain mostly untouched
<niemeyer> fss: Hey
<niemeyer> fss: I'm trying to merge your changes
<niemeyer> fss: Both look good
<niemeyer> fss: iam-deleteuser is conflicting with the previous branch, though
<fss> niemeyer: =/
<niemeyer> fss: Can you please merge lp:goamz into iam-deleteuser, fix conflicts, run tests, and re-propose?
<niemeyer> fss: I'll then submit
<fss> niemeyer: sure
<niemeyer> fss: Cheers!
<fss> niemeyer: I'll do it in a second
<fss> niemeyer: if I get in any trouble using bzr, I may ask for your help :-)
<niemeyer> fss: Sure thing
<fwereade> out for a while, have good evenings all
<wrtp> fwereade: have fun
<wrtp> niemeyer: does it matter that anything logged into the database can change the password for any other entity?
<wrtp> niemeyer: i'm kinda presuming not, but it surprised me slightly.
<niemeyer> wrtp: I mentioned it a few times when were brainstorming on the concept
<wrtp> niemeyer: that must've gone over my head, sorry
<niemeyer> wrtp: We won't be playing games with database permissions at the moment
<niemeyer> wrtp: np
<niemeyer> wrtp: Just saying it's expected
<wrtp> niemeyer: cool
<niemeyer> wrtp: The reason for the apparently silly care we're taking now is because this is the same logic we'll need once we have the API
<niemeyer> wrtp: Once we get the API, we can easily cut down that kind of cross-talking
<wrtp> niemeyer: i just wrote another test in state to verify that it works, because i thought it didn't :-)
<niemeyer> wrtp: It surely does.. the authentication is against the database.. the user data is inside the database
<wrtp> niemeyer: yeah - i guess i thought there was a difference between admin users and other users
<niemeyer> wrtp: There is in fact
<wrtp> niemeyer: ah yes, i remember
<niemeyer> wrtp: We add the admin to the admin database, which means it can manipulate everything
<wrtp> niemeyer: but it doesn't make any difference to us, because we only have two dbs
<wrtp> niemeyer: i was thinking that only the admin could add users
<niemeyer> wrtp: Certain commands can only be run against the admin database
<niemeyer> wrtp: So there's actually some practical implication in addition to hat
<niemeyer> that
<niemeyer> wrtp: Only admins can add users to all databases
<wrtp> niemeyer: uh huh
<niemeyer> wrtp: We shouldn't worry too much about that, though.. our primary goals on that round is getting basic authentication and transport security in place
<niemeyer> wrtp: We can split down capabilities on the next round, once we get the API stuff going
<wrtp> niemeyer: take it or leave it, i don't mind. https://codereview.appspot.com/6636057
<niemeyer> wrtp: Hmm, that's pretty interesting
<niemeyer> wrtp: I suspect we'll want to enable the machine to set a unit's password, *if* the unit is assigned to the machine
<wrtp> niemeyer: yeah, that's a good point.
<niemeyer> wrtp: So that we can deploy the unit within the machine with a password that is locally defined and thus known by the machine so that it can provide --initial-password
<niemeyer> wrtp: It shouldn't be able to set the password for an unrelated unit, though
<wrtp> niemeyer: agreed
<wrtp> niemeyer: the test should probably assign the units to the given machine
<wrtp> niemeyer: but maybe it's nice to have a test that will fail in an interesting way when we add appropriate capability checking
<niemeyer> wrtp: Not sure.. I'd prefer to not waste time writing tests we know are wrong
<wrtp> niemeyer: ok. i can easily do that for this test if you like; or i can just drop the CL for now.
<niemeyer> wrtp: I sometimes do that when I'm not in charge of implementing the feature, just so I can tell when upstream has changed, but that's not the case.. we know exactly when that will be done, and know it won't be a trivial change
<niemeyer> wrtp: I'm happy to have the CL with the Assign change.. it's already written, and this would verify intended behavior
<wrtp> niemeyer: ok, will do
 * niemeyer sips some nice coffee
<niemeyer> wrtp: Thanks man
<wrtp> niemeyer: unit passwords: https://codereview.appspot.com/6632061
<niemeyer> wrtp: Checking
<niemeyer> wrtp: done
<wrtp> niemeyer: thanks
<niemeyer> wrtp: np
<wrtp> niemeyer: i've fixed the password test: https://codereview.appspot.com/6636057
<niemeyer> wrtp: LGTM.. would be nice to clarify the commit message
<wrtp> niemeyer: will do. in fact, i'll submit tomorrow, as i hear a voice from below...
<wrtp> niemeyer: have a good rest-of-day
<wrtp> night all
<niemeyer> wrtp: Thanks, and have a good one too!
<fss> let the merge begin
<niemeyer> fss: Good to go?
<niemeyer> Hmm.. not yet
<fss> lbox proposing :-)
<fss> niemeyer: ptal
<niemeyer> fss: Looking
<niemeyer> fss: On the way
<fss> niemeyer: \o/
<fss> niemeyer: later today i'll start sending iamtest package
<niemeyer> fss: Sweet!
<fss> niemeyer: https://codereview.appspot.com/6631063
<niemeyer> fss: Looking
<fss> niemeyer: thanks :-)
<fss> niemeyer: notice that I didn't use iamtest_test in tests, so it was easier to test the calls, accessing private members of Server directly. What do you think?
<niemeyer> fss: Being able to ask the test server for which users exist sounds useful
<niemeyer> fss: I'd prefer to have that as part of the public interface, and have the test not poking at internals
<fss> niemeyer: I see
<niemeyer> fss: Also, we really need the integration test in the same style we have within ec2
<niemeyer> fss: So that someone can run tests for goamz/iam and have an idea that Amazon is actually happy with the implementation
<fss> niemeyer: hmm
<niemeyer> fss: The existing stuff within ec2i_test.go should serve as inspiration
<fss> niemeyer: I will add these integration tests
<fss> niemeyer: then provide GetUser support on iam, and use it to test iamtest's CreateUser and DeleteUser
<niemeyer> fss: Right, that's a great way to wire things together
<fss> niemeyer: actually, I think I'll need GetUser for CreateUser and DeleteUser integration tests too
<niemeyer> fss: Then you know that: a) goamz/iam works agains the real interface; b) goamz/iam works against the fake interface; Thus c) the real and the fake interface look alike
<niemeyer> fss: Absolutely
<niemeyer> fss: And you can do that with a single suite
<fss> niemeyer: a single suite?
<niemeyer> fss: Have a look at ec2i_test.go, and the relationship between ClientTests and AmazonClientSuite
<fss> niemeyer: ok, thanks :-)
<niemeyer> fss: Then, check out ec2t_test.go, and check how LocalServerSuite is defined
<fss> niemeyer: cool, thanks for the help. Time for some fun
<niemeyer> fss: np
<fss> niemeyer: here is GetUser, I will use it in integration tests: https://codereview.appspot.com/6631064
<fss> niemeyer: now I will start to work on the integration suite
<niemeyer> fss: Looking
<niemeyer> fss: Looks good.. just a trivial on the doc
<fss> niemeyer: opsss
<fss> niemeyer: shame on me
<fss> niemeyer: ptal
<niemeyer> fss: No worries, and thanks
<niemeyer> fss: That's in
<fss> niemeyer: thanks, I'm looking ec2i_test and ec2t_test
<fss> niemeyer: the integration suite already paid itself, the error treatment is broken. Should I send the fix with the integration tests cl, or in a separate CL?
<niemeyer> fss: It's fine to be in the same CL in that case.. it's really the test for the fix
<fss> niemeyer: ok
<fss> niemeyer: do you have any trouble with lbox + vim?
<fss> niemeyer: whenever I use vim, I get "error: Change summary is empty.". It's like vim is not actually saving the temporary file in the disc
<niemeyer> fss: Hmm.. nope.. works fine here
<niemeyer> fss: How does $EDITOR look like?
<fss> EDITOR=/usr/local/vim
<fss> I set it to nano, so I can send the cl
<fss> /usr/local/bin/vim, sorry
<fss> niemeyer: here we go: https://codereview.appspot.com/6625077
<niemeyer> fss: Looking
<niemeyer> fss: Are you sure you're saving the file? I really can't imagine what might be going on
<fss> ec2t_test.go is the test suite for the ec2test package?
<fss> niemeyer: yes, I can `cat` it
<niemeyer> fss: Very strange.. if you find what's up, please let me know
<niemeyer> fss: On the previous question, yes
<fss> niemeyer: I've compiled /usr/local/bin/vim by hand, I'll try /usr/bin/vim later and see if the problem persists
<niemeyer> fss: Suite is great, thanks. Submitting
<fss> niemeyer: okay, I'll add iamt_test in the previous iamtest CL
<niemeyer> fss: Ah, please add to a new one
<niemeyer> fss: I'm moving this one forward
<fwereade> niemeyer, heyhey
<fss> niemeyer: this one? https://codereview.appspot.com/6631063/
<niemeyer> fss: Ah, sorry, I misunderstood, that's great
<niemeyer> fwereade: Heya
<fwereade> niemeyer, re HookContext.service: perhaps I've been making bad assumptions re service config -- I had assumed that we'd be able to get the config for the unit's current charm without directly involving the service at all... but now I seem to recall a discussion in which you were planning to keep all a service's configs in a single document
<fwereade> niemeyer, well, not necessarily planning
<fwereade> niemeyer, I can't remember whether you were for or against it
<niemeyer> fwereade: That's probably because I wasn't either for or against it :-)
<fwereade> niemeyer, that would explain my inability to recall one or the other :)
<niemeyer> fwereade: Hehe :)
<niemeyer> fwereade: I'm still wondering, actually.. I leaning slightly towards different documents
<fwereade> niemeyer, so, with that half-baked assumption in mind, it seemed clear that the service was enitrely transitory data
<fwereade> niemeyer, I think my brain had drifted into a similar configuration
<fwereade> niemeyer, it had crossed my mind that it would be convenient to key them on service name + charm url, which can be constructed trivially from a unit
<fwereade> niemeyer, and so service seemed like a sort of unwanted appendix that didn't really deserve a field
<niemeyer> fwereade: service.Config(charmURL) is perhaps the best candidate at the moment
<niemeyer> fwereade: With optional charmURL, so service.Config(nil) returns the tip
<fwereade> niemeyer, my heart says unit.ServiceConfig(), but I expect you have good reasons for your choice... yeah, it's more orthogonal
<niemeyer> fwereade: That sounds slightly awkward.. unit.ServiceCharm()? unit.ServiceCharmURL()? :-)
<fwereade> niemeyer, just so long as the unit is up to date, anyway -- but, yeah, at least the service wouldn't need a refresh
<fwereade> niemeyer, it's not my heart: it's my lazy fingers that want it, and my brain has now overruled them
<niemeyer> fwereade: LOL
<fwereade> niemeyer, => service is an expected long-term feature of HookContext => it stays
<fwereade> niemeyer, cheers
<niemeyer> fwereade: Cool, I see your original reasoning as well, thanks
<fwereade> niemeyer, btw, possible uniter weirdness
<fwereade> niemeyer, given that .unit (and, I guess, .service ;)) are long-lived, I'm conflicted about when they should be being refreshed
<niemeyer> fwereade: If we don't know it probably means we don't have to refresh (yet)
<fwereade> niemeyer, ha, I have just looked at the state code... I'm sure I remember a time when we weren't always updating the local doc with the change we made
<niemeyer> fwereade: Ah, we should, I think
<fwereade> niemeyer, (was there some idea at one stage that a local state doc should always reflect the actual state of the remote document at some point in time, rather than being a palimspest?)
<niemeyer> fwereade: I think we were unsure until the first sprint in Lisbon
<fwereade> niemeyer, must have just missed more recent conversations
<niemeyer> fwereade: That's when we explored the options in details, and agreed on doing what we have now
<niemeyer> fwereade: I don't think we've changed the approach since then
<fwereade> niemeyer, great
<niemeyer> fwereade: That doesn't mean it's perfect, but at least so far things to be holding together
<niemeyer> erm
<niemeyer> things seem to be
<fwereade> niemeyer, yeah, so far, digging deeper has always reassured me
<fwereade> niemeyer, oh, btw
<fwereade> niemeyer, filter has a config watch, which is only used in ModeAbide (but will probably also be used in ModeAbideDying, which wil emerge at some stage)
<fwereade> niemeyer, however, I am in the position of adding a relations watch which is only going ot be useful to a single mode
<niemeyer> fwereade: Cool
<fwereade> niemeyer, and it's feeling a little bit off-kilter
<niemeyer> fwereade: Because we're watching things we can't always act upon?
<fwereade> niemeyer, partly... but more because ModeAbide, as a consumer, will be acting on uniter-wide state in response to these changes
<niemeyer> fwereade: Hmm.. I don't yet see the light
<fwereade> niemeyer, and it is, I think, flat-out *wrong* for any other mode to use those events and mess with state in response to them
<fwereade> niemeyer, unless they do exactly what ModeAbide does
<niemeyer> fwereade: Sure, but that sounds a bit of a general rule
<niemeyer> fwereade: It's flat out wrong for any mode to watch state and do changes they shouldn't
<fwereade> niemeyer, I think what I'm trying to say is that it's legitimate for different modes to respond to (say) resolved events in different ways
<niemeyer> fwereade: I think I understand what you're saying, but IMO what you describe is one of the benefits we intended with the filter method
<niemeyer> fwereade: To me it's a huge simplification the fact we can simply allow modes to care about stuff they should, and ignore the stuff (including events) they shouldn't act upon
<fwereade> niemeyer, ok: (drags self laboriously towards point) I am conflicted over whether or not I should implement a WantRelationsEvent on filter
<niemeyer> fwereade: I suggest not implementing it until there's a need for the logic
<niemeyer> fwereade: It won't save us from someone using the method when they shouldn't, and it'll mean we'll have to write and read the logic for the whole thing
<fwereade> niemeyer, well -- I'm not sure why it is a good idea to add complexity to filter -- which is pretty clean, but not unworthy of careful treatment
<fwereade> niemeyer, when I can be pretty sure that only ModeAbide should ever be messing with those events
<fwereade> niemeyer, especially considering that ModeInit and ModeAbideDying will themselves be expected to independently manipulate runtime relation state
<niemeyer> fwereade: I don't understand how the two things link together
<niemeyer> fwereade: Yes.. only ModeAbide should be receiving those events, so...?
<fwereade> niemeyer, sorry, I'm rather thinking out loud
<niemeyer> fwereade: Seems to be tightly related to everything I said above
<fwereade> niemeyer, so it is bad and scary and wrong to let anyone else see them
<niemeyer> <niemeyer> fwereade: I think I understand what you're saying, but IMO what you describe is one of the benefits we intended with the filter method
<niemeyer> <niemeyer> fwereade: To me it's a huge simplification the fact we can simply allow modes to care about stuff they should, and ignore the stuff (including events) they shouldn't act upon
<niemeyer> <niemeyer> fwereade: It won't save us from someone using the method when they shouldn't, and it'll mean we'll have to write and read the logic for the whole thing
<fwereade> niemeyer, primarily because they are not recoverable in the way that, say, a resolved event is
<fwereade> niemeyer, if there is only a single correct way to consume these events, and we cannot recover history, it seems unwise to leave them lying around on filter where not-William (;)) can use them
<niemeyer> fwereade: Understood.. recoverable as in we can't request the same event to be sent again
<niemeyer> fwereade: Right?
<fwereade> niemeyer, yes
<niemeyer> fwereade: Cool, so WantConfig would be fine it seems
<niemeyer> fwereade: Except, do we want to run that at all times?
<fwereade> niemeyer, WantConfig is ok because (1) it's recoverable and (2) it's used by 2 modes, even though one of them doesn't exist
<fwereade> yet
<niemeyer> fwereade: I don't think I get it.. WantConfig would mean we'd *always* get such an event
<fwereade> niemeyer, yeah, that's exactly the idea
<niemeyer> fwereade: and my understanding is that we don't want to run a config-changed hook unless the config changed, in normal situations
<niemeyer> fwereade: which is what ModeAbide is about, in my understanding
<fwereade> niemeyer, except when we enter ModeAbide -- we agreed this was a clean way to guarantee that start and upgrade-charm were always succeeded by config-changed
 * fwereade twitches as an idea clicks
<fwereade> niemeyer, huh, I don't need it in ModeAbideDying
<fwereade> niemeyer, so now we have relations more like config than anything else, and neither of them really deserves to dirty up the filter
<niemeyer> fwereade: Won't we get into ModeAbide when we get out of an error state? Or out of a relatoin changed state? Etc?
<fwereade> niemeyer, we discussed this, and yu were keen on it; the primary justification, which I was very happy with, was that if *any* hook *had* to be idempotent anyway, it was config-changed
<niemeyer> fwereade: I totally agree with that
<niemeyer> fwereade: I don't think I agreed we should run config-changed whenever we sneeze, though :-)
<fwereade> niemeyer, the only extra time it's run is after ModeHookError AFAIR
<niemeyer> fwereade: ModeAbide is the steady state.. we'll get into it everytime we get out of any other mode
<fwereade> niemeyer, easy to change, happy to do it (queued hooks are already a thing), will be nicer :)
<fwereade> niemeyer, at this stage the only reason to do it is historical
<niemeyer> fwereade: Cool
<fwereade> niemeyer, the uniter's shed a lot of complexity since then :)
<niemeyer> fwereade: Indeed
<fwereade> niemeyer, hum, ok
<fwereade> niemeyer, wait
<fwereade> niemeyer, we *also* run config-changed whenever we start up and first come into modeabide
<fwereade> niemeyer, and that is necessary
<niemeyer> fwereade: Right, after start
<fwereade> niemeyer, because we don't have any permanent record of service config version
<fwereade> niemeyer, no, any time we're bounced
<niemeyer> fwereade: bounced?
<fwereade> niemeyer, process death
<fwereade> niemeyer, or even just the UA restarting the uniter
<fwereade> niemeyer, I did think this was crazy
<niemeyer> fwereade: Hmm.. why is it crazy?
<fwereade> niemeyer, because it forces us to run one whenever we enter ModeaBIDE :)
<niemeyer> fwereade: The two things sound pretty unrelated
<fwereade> niemeyer, we're entering ModeAbide, and we might need to run a config-changed hook
<fwereade> niemeyer, how do we tell?
<niemeyer> fwereade: One is a high-level requirement, the other is an implementation detail
 * niemeyer looks at fwereade
<niemeyer> fwereade: Okay, it's pretty late there.. I'll help.. how can we tell when is the first thing we do something inside a process or type?
<niemeyer> s/first thing/first time
<fwereade> niemeyer, ha :/
<fwereade> niemeyer, seems somehow wrong to have to store that state on the uniter itself
<niemeyer> fwereade: There are many ways to do it, but it's far from a OMG case :)
<niemeyer> fwereade: Sorry, I have to run a quick errand.. I'll be back in ~15mins
<fwereade> niemeyer, quite so... it feels weird to make the uniter stateful like that
<fwereade> niemeyer, np, will probably be around ;)
<niemeyer> fwereade: LOL.. just ponder for a moment about that last statement regarding how it's weird to make the uniter stateful :-)
<fss> niemeyer: here is it: https://codereview.appspot.com/6631063/
<fss> niemeyer: I've implemented iamt_test.go running full ClientTests suite, since they're born together
<niemeyer> fwereade: Around again
<niemeyer> fwereade: So,
<niemeyer> fwereade: back to your point, the uniter is already pretty stateful.. we have a full state machine on it
<fwereade> niemeyer, yeah, indeed
<niemeyer> fwereade: (a pretty nice one, actually!) :-)
<fwereade> niemeyer, I'm implementing it now, it's interesting
<niemeyer> fwereade: I'm not sure if I misunderstood the heart of your point, though.. maybe it's becoming stateful in a way it wasn't?
<fwereade> niemeyer, well, the state machine is really in the modes, which aren't in uniter fields, but I take your point
<fwereade> niemeyer, I think the code will illustrate my perspective better than anything I can say, and I*think* it's nearly there
<niemeyer> fwereade: There are actually other stateful bits too.. filter being a notable example
<fwereade> niemeyer, yeah, indeed, I actually don;t think it's a bad thing -- I've just got very used to Modes not directly using uniter for state, except as a proxy for disk state
<fwereade> niemeyer, I have a nervous feeling about this code, because I'm not sure whether or not you'll like it -- it's relatively neat, but I felt the need to comment to explain what it's doing in one place, which is always a bit of a worry
<fwereade> niemeyer, ready to propose in a sec
<fwereade> niemeyer, then I should go, need to be up tomorrow (bah)
<niemeyer> fwereade: Thanks a lot, looking forward to checking it out
<fwereade> niemeyer, ok, looking at it in codereview, I think it's pretty good :) https://codereview.appspot.com/6632062
 * fwereade suspects these words will come back to haunt him
<niemeyer> fwereade: Hehe :-)
#juju-dev 2012-10-11
<davecheney> le sigh
<davecheney> charm deployments all broke overnight
<davecheney> was working yesterday
<davecheney> now nothing deploys
<fwereade> davecheney, ouch :(
<fwereade> davecheney, what was it?
<davecheney> no idea
<davecheney> fwereade: i thought it was just the charms I was testing
<davecheney> or some ec2 weirdness
<davecheney> but I went back and testing charms i knew worked
<davecheney> and they all jam with the same error
<davecheney> or
<davecheney> waiting for the error state to clear
<TheMue> morning
<fwereade> TheMue, heyhey
<fwereade> davecheney, huh, what's the error?
<TheMue> fwereade: heya
<wrtp> morning all
<TheMue> wrtp: hi
<wrtp> davecheney: i hope that's not my fault
<wrtp> TheMue: hiya
<wrtp> davecheney: do you see the uniter version in the status?
<TheMue> wrtp, fwereade: here's a trivial one https://codereview.appspot.com/6649044/
<wrtp> TheMue: can't we get rid of FwDefault completely?
<fwereade> TheMue, yeah, I was just about to ask about that :)
<TheMue> as far as i understood niemeyer he wants to keep it
<wrtp> TheMue: i'd prefer to have a method on Environ that returns the environment's default firewall mode.
<wrtp> TheMue: as it is, FwDefault doesn't mean anything - we don't know what semantics to expect.
<TheMue> wrtp: please add it as comment so that i can discuss it with niemeyer
<wrtp> TheMue: ok, will do. i'm afraid that means that this branch can't be taken to be trivial and submitted immediately.
<TheMue> wrtp: see also point 2 in his mail
<TheMue> wrtp: it is only the first one in a row of 6
<wrtp> TheMue: reading his email, it seems like he wants to get rid of FwDefault too
<wrtp> TheMue: "
<wrtp> We can't ever use FwDefault outside of the
<wrtp> provider
<wrtp> "
<TheMue> wrtp: today we have no EnvironProvider.Validate. imho that is intended to to validate in the mode and in case of FwDefault return the providers default mode, instance or global
<TheMue> wrtp: that's my interpretation of point 2
<wrtp> TheMue: ah, i think i understand now.
<wrtp> thinking
<TheMue> wrtp: so different environments can return different defaults
<wrtp> TheMue: i'm not sure i understand why we need an explicit default value. can't ValidateConfig see that the firewall mode is missing from the config, and substitute its default mode?
<wrtp> TheMue: that is: we already have a good way of specifying default values - we leave the value unspecified
<TheMue> yep
<fwereade> whoops, doctor's appointment, gtg
<fwereade> see you shortly
<Aram> moin.
<TheMue> Aram: moin moin
 * TheMue has to step out to play taxi driver for his daughter. bbiab
<wrtp> fwereade: it looks to me as if the cloud-init script runs as root. can you think of a reason to retain the sudos in environs/cloudinit ?
<fwereade> wrtp, not offhand, no
<wrtp> fwereade: i think they're just there as a legacy of the python
<wrtp> fwereade: i'll remove 'em and see if anything breaks
<fwereade> wrtp, that's probably the easiest way
<wrtp> fwereade: BTW i think we should probably not be running our agents as root
<fwereade> wrtp, except, wait, is everything broken since last night?
<wrtp> fwereade: it seems to work for me. but i haven't tried deploying a charm
<fwereade> wrtp, that would be the ideal but I'm not sure I see a good way round it
<wrtp> fwereade: oh, because we need to be root to start LXC containers?
 * fwereade wonders if he broke config somehow, really ought to actually take a look at it, it's just the contextswitching...
<fwereade> wrtp, I thought we did, yeah
<wrtp> fwereade: hmm, that makes sense :-(
<wrtp> fwereade: i'm just trying to deploy a charm now. will paste you the log output if it fails
<fwereade> wrtp, debug output would probably be helpful if that can be arranged
<wrtp> fwereade: i think debug output is always enabled currently
<wrtp> fwereade: it certainly *seems* to be deploying ok
<wrtp> fwereade: hmm, it seems to have executed the install and start hooks, but the status is still "pending"
<wrtp> fwereade: http://paste.ubuntu.com/1273059/
<wrtp> fwereade: current status: http://paste.ubuntu.com/1273062/
 * fwereade looks
<fwereade> wrtp, huh, it seemed on dave's that he was always failing start hooks, that looks like the hook is fine
<wrtp> fwereade: yeah, but shouldn't the status show something other than "pending"?
<fwereade> wrtp, yeah, so somehow it seems ModeAbide is wedged, very early on
<fwereade> wrtp, which is strange... I suspect it's waiting for a config event which should be guaranteed
<wrtp> fwereade: is this unexpected? "2012/10/11 12:07:34 JUJU cannot read hook output: read |0: bad file descriptor"
<fwereade> wrtp, isn't that your hookLogger? ;p
<wrtp> fwereade: probs
<fwereade> wrtp, I haven't seen it but AIUI that sort of thing is not entirely unepected is it?
<wrtp> fwereade: that's maybe what happens when the fd isn't closed properly, i can't remember.
<fwereade> wrtp, if it had debug output I would be able to tell what the filter was doing, I think
<wrtp> fwereade: ah, is debug output not enabled by the global debug flag?
<fwereade> wrtp, (even though I would also have to wade through the state debug stuff as well... sigh :))
<fwereade> wrtp, hmm, not sure, haven't looked at our cloudinits for a while, didn't actually know it was doing the debug flag
<wrtp> fwereade: ah, container doesn't enable the debug flag, grr
<wrtp> niemeyer: yo!
<niemeyer> Morning!
<wrtp> fwereade, niemeyer: trivial CL: https://codereview.appspot.com/6654043
<niemeyer> wrtp: LGTM
<wrtp> niemeyer: thanks
<fwereade> niemeyer, heyhey
<niemeyer> fwereade: Heya
<wrtp> niemeyer: another small one: https://codereview.appspot.com/6655043/
<niemeyer> wrtp: LGTM
<wrtp> niemeyer: thanks
<wrtp> niemeyer: (the quick reviews are really appreciated!)
<niemeyer> wrtp: My pleasure
<niemeyer> fwereade: When you have a spare moment to continue yesterday's brainstorm, please ping
<fwereade> niemeyer, should be good in just a sec
<fwereade> niemeyer, ping
<niemeyer> fwereade: Yoyo
<niemeyer> fwereade: Can we have a quick call? I think we can save time
<fwereade> niemeyer, sgtm
<fwereade> niemeyer, https://plus.google.com/hangouts/_/044d83e4e11954e90f12cb6c2a82bf2e8ed1c724?authuser=0&hl=en#
<niemeyer> fwereade: Thanks.. funny that you can paste the URL and my phone is ringing before I manage to open the G+ page
 * fwereade is quick like ninja, on very rare occasions :)
<fss> niemeyer: morning
<niemeyer> fss: Heya
<TheMue> niemeyer: heya, did you read rogers comment about point 3 of the firewall mode changes? he prefers to put the global open/close/ports at Environ instead of EnvironProvider. i like that idea, because it feel more natural work on the concrete environment. what do you think?
<niemeyer> TheMue: He's totally right.. it was my mistake
<TheMue> niemeyer: fine, will do so
<wrtp> niemeyer: the final piece, i believe: https://codereview.appspot.com/6655044
<niemeyer> wrtp: Looking
<niemeyer> wrtp: done
<wrtp> niemeyer: thanks
<niemeyer> wrtp: np
<wrtp> niemeyer: the _ for the state was deliberate BTW. it means i can return nil, err and still see if the state was set.
<wrtp> niemeyer: i'd prefer not to use bare returns
<niemeyer> wrtp: Let's please name it properly as usual
<wrtp> niemeyer: ok, i'll rename the variable further down i guess
<niemeyer> wrtp: I don't understand why you have to
<wrtp> niemeyer: if i don't do that, then my defer won't work
<wrtp> niemeyer: because it won't be able to see that st!=nil
<niemeyer> wrtp: You don't need the defer either
<niemeyer> wrtp: See the rest of the review
<wrtp> niemeyer: ah, ok
<wrtp> niemeyer: i'm not sure about putting the password in the agent dir itself. do we guarantee that the agent dir is mode 700 ?
<wrtp> niemeyer: mind you, i suppose not much information can be gleaned from the password length as it's always the same.
<niemeyer> wrtp: No, just the file
<wrtp> niemeyer: PTAL https://codereview.appspot.com/6655044
<fwereade> niemeyer, lbox appears to be panicking when I try to authenticate -- 3/4 times so far (the 4th time it complained about an incorrect password, which I'm ... pretty sure I typed right, but ofc cannot verify)
<fwereade> niemeyer, is this reminiscent of anything known to you?
<fwereade> niemeyer, this is the (interesting bit of, i think) the panic: http://paste.ubuntu.com/1273426/
<niemeyer> fwereade: No, there was an issue with auth when Go changed in an incompatible way between releases, but it shouldn't be happening now
<fwereade> niemeyer, google seems to agree that my password is what I think it is... I think it is moderately unlikely that I would have typed it wrong on all 4 occasions
<niemeyer> fwereade: Try to drop your auth details (rm ~/.lbox*)
<fwereade> niemeyer, don't seem to have any
<niemeyer> fwereade: Sorry, that should be .lpad
<fwereade> niemeyer, is that connected to the google auth?
<niemeyer> fwereade: Sorry, I'm clearly on crack... that's ~/.goetveld*
<fwereade> ha, now I'm getting a blank auth page on launchpad :/
<fwereade> niemeyer, bah, same ol' panic
 * fwereade resorts to the universal panacea
<niemeyer> fwereade: Let me try to kill my auth and see if I can repro
<fwereade> niemeyer, yeah, if I do a deliberate wrong password it fails nicely
<fwereade> niemeyer, correct password has panicked every time
<niemeyer> wrtp: LGTM
<fwereade> niemeyer, which is *insane* because I proposed earlier today
<wrtp> niemeyer: thanks!
<niemeyer> fwereade: Did you upgrade in between?
<fwereade> niemeyer, hum, I think I probably did
<fwereade> niemeyer, not sure what from
<niemeyer> fwereade: I hope it's not an interim breakage again
<wrtp> niemeyer: and with that, i think our current authentication stuff is done. woo.
<wrtp> niemeyer: i'm gonna push a few trivial changes that i've been putting off for a while, but if you have something urgent that i should move on to, let me know.
<niemeyer> wrtp: Cool, the main thing is really to keep pushing on that front, testing etc.. but trivial changes as a break sounds good too
<niemeyer> fwereade: I'm facing EOF issues with LP before even getting there
<niemeyer> fwereade: The cloud gods are mad
<wrtp> niemeyer: it seems to be working ok for me
<niemeyer> fwereade: Worked fine, but I think I was using a locally built lbox.. let me make sure I'm getting the one from the PPA
 * fwereade wonders what he did :/
<niemeyer> fwereade: Boom
<fwereade> niemeyer, ha:
<fwereade> 2012/10/11 17:54:23 RIETVELD Login on https://codereview.appspot.com successful.
<fwereade> 2012/10/11 17:54:23 RIETVELD Login failed: Get http://example.com/marker: redirect blocked
<niemeyer> fwereade: Launchpad's lbox must be compiled with the broken tip
<niemeyer> fwereade: I'll fix that after lunch
<niemeyer> fwereade: Meanwhile, go get launchpad.net/lbox will get you going
<fwereade> niemeyer, lovely, thanks
<niemeyer> Hmm
<niemeyer> It's compiling against stable, actually, which seems to indicate that it was released broken? Uh oh
 * niemeyer => lunch
<fwereade> niemeyer, yeah, the freshly built version seems to fall over the same way
<fwereade> niemeyer, I will also be back later
<fwereade> niemeyer, (ty for the advice re config-changed, infinitely cleaner now)
<TheMue> fwereade: it catched me too, lbox panicked *sigh*
<niemeyer> This really sucks
<niemeyer> I anticipated the bug, reported, made sure it was fixed, and even then it got released without the fix.. :(
<niemeyer> TheMue: go get launchpad.net/lbox
<niemeyer> I'll remove the Launchpad package
<niemeyer> Or perhaps just make it build against tip
 * niemeyer checks it
<TheMue> niemeyer: got it, same error here. funnily the package one already worked twice today for me
<niemeyer> TheMue: Ah, you probably have go 1.0.3 installed
<niemeyer> TheMue: So hold off a bit until the package bulids
<niemeyer> TheMue: The broken one will work fine until you have to auth again
<TheMue> niemeyer: yes, 1.0.3. so i'll wait
<wrtp> niemeyer, fwereade, TheMue, Aram: this CL makes log messages consistent, as talked about on juju-dev. it's a large CL, but pretty trivial. https://codereview.appspot.com/6654044/
<TheMue> rogpeppe: *wow*
<niemeyer> rogpeppe: It's huge indeed, and in a quick sampling it doesn't feel so great
<niemeyer> rogpeppe: Comments sent
<rogpeppe> niemeyer: thank you
<niemeyer> rogpeppe: It'd be good to have a more careful evaluation
<rogpeppe> niemeyer: i thought the HOOK output was perhaps not so good
<rogpeppe> niemeyer: i went with a literal interpretation of the rules to start with
<niemeyer> rogpeppe: Â»       Â»       fmt.Fprintf(os.Stderr, "%s: %v\n", filepath.Base(os.Args[0]), err
<niemeyer> rogpeppe: What did the literal rules say about fmt.Fprintf?
<rogpeppe> niemeyer: i interpreted that as a kind of log output. perhaps that's a stretch.
<rogpeppe> niemeyer: even if that doesn't get done in this CL, i think it's worth doing.
<niemeyer> rogpeppe: Yeah, *log* and *output* are not the same thing
<rogpeppe> niemeyer: saying "error:" for every command is not great.
<rogpeppe> niemeyer: the unix standard is to print the name of the command
<niemeyer> rogpeppe: Good old let's-derail-the-conversation-until-we-disagree
<rogpeppe> niemeyer: ok, sorry, i thought it was pretty uncontroversial.
<rogpeppe> niemeyer: i'll rewind those changes
<niemeyer> rogpeppe: It's *very* uncotroversial to bikeshed widely about log messages, send a message to the mailing list so we agree, explicitly state that it's about log.Printf/Debugf for *clarity*, then send a 100 files change that does something else entirely
<rogpeppe> niemeyer: i'm sorry
<rogpeppe> niemeyer: as a separate CL, might you agree in principle that changing the "error:" messages is a good idea?
<niemeyer> rogpeppe: No, I want to keep moving forward and stop the bikeshed immediately
<rogpeppe> niemeyer: when i see shell script output, it's useful to know what commands have printed the messages.
<rogpeppe> niemeyer: i've run across this a few times so far.
<niemeyer> .
 * rogpeppe thinks this is more than just a bikeshed colour. It adds to juju usability.
<rogpeppe> but i'll stop there.
<niemeyer> Thanks, let's make it work
<rogpeppe> perhaps i should abandon all the log printf changes too, if it's just bikeshedding.
<niemeyer> rogpeppe: Messages are inconsistent, we discussed this on IRC, and in the mailing list, and agreed on something. If you wanna drop it, someone else can pick it up later, no worries.
<niemeyer> rogpeppe: That's unrelated to continue fiddling with output messages, though.
<rogpeppe> niemeyer: do you agree with dropping the cmd/ prefix for commands BTW?
<niemeyer> rogpeppe: Is it ok to have cmd/juju and juju prefixed by the same thing?
<rogpeppe> niemeyer: hmm, good question.
<rogpeppe> niemeyer: probably not. that's a good call.
<niemeyer> TheMue: lbox rebuilt.. wanna give it a try?
<TheMue> niemeyer: yes
<TheMue> niemeyer: yep, it works, thanks a lot
<niemeyer> TheMue: Phew, np
<niemeyer> fwereade: ^
 * TheMue is stepping out, dinner time
<niemeyer> TheMue: Enjoy
<TheMue> niemeyer: first three CLs for the global mode are in
<fwereade> niemeyer, having supper now, but https://codereview.appspot.com/6632062 reproposed
<niemeyer> fwereade: Sweet
<niemeyer> fwereade: Thanks much
<rogpeppe> niemeyer: this should be better i hope: https://codereview.appspot.com/6654044
<rogpeppe> i'm off now, night all
<niemeyer> rogpeppe: Thanks, have a good one
<niemeyer> fwereade: That looks very nice indeed
<rogpeppe> niemeyer: and you
<hazmat> niemeyer, fwiw we've had a few people today run  into a nil memory ref in lbox..  https://pastebin.canonical.com/76339/
<niemeyer> hazmat: Yeah, it's fixed already
<niemeyer> hazmat: unfortunately 1.0.3 got released with a bug
<hazmat> niemeyer, awesome re fix, thanks
<niemeyer> hazmat: Go 1.0.3 that is.. we've fixed the issue before it was out, but the release manager missed the fix
<hazmat> niemeyer, ah.. i remember you mentioning that a while back re bug in go http client.
<hazmat> cool
<niemeyer> hazmat: Exactly.. we've rushed to fix that shortly after the bug was introduced, back in july.. sadly the bug was merged onto the release and the fix wasn't
<fwereade> niemeyer, cool, thanks
<niemeyer> fwereade: ping
 * niemeyer => doc.. back soonish
<davecheney> % juju bootstrap --upload-tools
<davecheney> error: cannot upload tools: cannot write file "tools/juju-0.0.1-precise-amd64.tgz" to control bucket: We encountered an internal error. Please try again.
<davecheney> thanks, amazon
<fwereade> niemeyer, pong
<fwereade> davecheney, heyhey -- I never got to investigating your problems from yesterday, which I felt I kinda should have, but... well, er, I didn't
<fwereade> davecheney, can I be of any assistance now though?
<davecheney> fwereade: hey
<davecheney> i'm trying to have a look now as part of adding juju remove-unit
<davecheney> but I can't get an environment to bootstrap in any zone
<fwereade> davecheney, ha, ouch :(
<davecheney> us-east-1 is screwed
<davecheney> ap-southeast-1 is broken
<davecheney> trying us-west-1
<davecheney> if I can get an env going
<davecheney> who sort of debugging will be useful ?
<davecheney> % juju bootstrap -e us-west-1 --upload-tools
<davecheney> lucky(~/src/launchpad.net/juju-core/cmd/juju) % juju deploy -e us-west-1 -n 5 mysql
<davecheney> error: cannot put charm: cannot make S3 control bucket: Your previous request to create the named bucket succeeded and you already own it.
<davecheney> that is fucking great
<davecheney> every non us-east-1 env now won't bootstrap
<fwereade> davecheney, oof
<fwereade> davecheney, well, I'm not sure that it would be, necessarily, with the logs you sent -- but rog was having a problem that seemed potentially kinda similar, that might have been helped by it
<fwereade> davecheney, I just mean running the agents with --debug
<davecheney> yeah, i saw he made that change last night
<davecheney> i've merge from trunk
<davecheney> so when I do get something deployed, it will be in --debug
<fwereade> davecheney, I appear to have something bootstrapping in eu-west-1
<davecheney> fwereade: seet
<davecheney> sweet
<davecheney> while I have you
<davecheney> i've implemented conn.RemoveUnits(units ...)
<fwereade> davecheney, I was planning to try to deploy mongodb and see what happened :0
<davecheney> as setting unit.EnsureDying()
<davecheney> and leaving it as that
<davecheney> am i correct that the UA will detect that, do whatever, and set the unit to Dead ?
<fwereade> davecheney, yeah, that should be enough
<fwereade> davecheney, it would also be good to implement --force
<fwereade> davecheney, which would call EnsureDead
<davecheney> that was my initial attempt
<davecheney> it didn't work
<fwereade> davecheney, which will be handy right now for removing horribly borked deployments
<davecheney> because it stepped around the UA
<fwereade> davecheney, blocked by something? or nothing responded?
<fwereade> davecheney, it shouldn't be the default but it should be possible, I think
<davecheney> unit went away
<davecheney> but the machine didnt
<niemeyer> fwereade: Yo
<niemeyer> davecheney: Morning!
<fwereade> davecheney, machines shouldn't go away until we terminate-machine
<niemeyer> davecheney: Isn't that super early? :)
<fwereade> niemeyer, heyhey
<niemeyer> fwereade: I was going to ask you about the service config watcher
<fwereade> niemeyer, oh yes
<davecheney> niemeyer: i was going to go for a ride, but it is pissing down
<niemeyer> fwereade: I'm wondering about what logic to implement for the multi-config world
<davecheney> so, best to strike while the iron is hot
<niemeyer> fwereade: Pondering if it would be fine to do per-charm-url watching
<fwereade> niemeyer, I think it would
<davecheney> aaaaaaaaaaaargh!
<davecheney> % juju status
<davecheney> error: We encountered an internal error. Please try again.
<davecheney> us-east-1 is screwed
<niemeyer> fwereade: and assume that the filter would be kind enough to re-watch so the modes/etc don't have to care
<niemeyer> fwereade: What do you think?
<fwereade> niemeyer, the filter is already in a position to know what the unit's current charm is, so it should be near-enough trivial
<niemeyer> fwereade: Awesome! That makes things pretty easy
<niemeyer> fwereade: Now that we have explicit settings management, the next step is almost trivial
<fwereade> niemeyer, the semantic change is that you say something like u.f.NotifyCharm(url, mustForceUpgrade), and assume that all config and upgrade events are filtered relative to that state
<davecheney> awesome
<davecheney> now i can't deploy at all
<davecheney> 2012/10/11 22:07:16 JUJU state: opening state; mongo addresses: ["localhost:37017"]
<davecheney> 2012/10/11 22:07:16 JUJU machiner: unauthorized access
<niemeyer> fwereade: Neat
<fwereade> niemeyer, I just need to be slightly careful about not resetting the config watch when the charm didn't change so I don;t poo extra events into the stream
<niemeyer> davecheney: Uh
<davecheney> that is on machine 0
<davecheney> it's got itself locked out
<niemeyer> davecheney: I guess rogpeppe didn't really test it live :(
<fwereade> davecheney, ey up
<fwereade> william@diz:~/code/go/src/launchpad.net/juju-core$ juju deploy mongodb
<fwereade> error: cannot put charm: cannot make S3 control bucket: Your previous request to create the named bucket succeeded and you already own it.
<niemeyer> davecheney: Do you have access-secret set up in your environment?
<niemeyer> davecheney: Perhaps it is assuming it is set
<davecheney> niemeyer: nope
<niemeyer> davecheney: Try to set it in the environment config
<davecheney> niemeyer: i have an admin-secret
<davecheney> is that the same thing ?
<niemeyer> davecheney: Erm, sorry, that's the one
<davecheney> fwereade: i get that error in every non us-east-1 env
<davecheney> 2012/10/11 21:58:58 JUJU machiner: machine agent starting
<davecheney> 2012/10/11 21:58:58 JUJU state: opening state; mongo addresses: ["localhost:37017"]
<davecheney> 2012/10/11 21:58:58 JUJU machiner: unauthorized access
<davecheney> 2012/10/11 21:59:01 JUJU machiner: rerunning machiner
<davecheney> the machiner/PA never starts
<niemeyer> davecheney: Okay, so we'll need to debug it, or wait until rogpeppe addresses it
<fwereade> davecheney, ah balls, isn't that that the namespace is global or something, just a mo
<niemeyer> davecheney: How does cloud-init look like?
<davecheney> fwereade: yes, all buckets live inthe same namespace
<davecheney> don't just copy the bucket config from one env to another
<davecheney> but even then, it doesn't help
<davecheney>   ap-southeast-1:
<davecheney>     type: ec2
<davecheney>     control-bucket: juju-7
<davecheney> niemeyer: checking cloud init now
<fwereade> davecheney, bah, yeah, I appear to be just as screwed as you
<davecheney> niemeyer: bootstrapping now
 * fwereade slopes back off to kick relations around some more
<davecheney> niemeyer:
<davecheney> 2012/10/11 22:20:46 JUJU:DEBUG watcher: loading new events from changelog collection...
<davecheney> 2012/10/11 22:20:46 JUJU storing no-secrets environment configuration
<davecheney> 2012/10/11 22:20:46 JUJU bootstrap-state initial password ""
<davecheney> jujud-machine-0 start/running, process 10711
<davecheney> and then
<davecheney> 2012/10/11 22:20:46 JUJU machiner: machine agent starting
<davecheney> 2012/10/11 22:20:46 JUJU state: opening state; mongo addresses: ["localhost:37017"]
<davecheney> 2012/10/11 22:20:46 JUJU machiner: unauthorized access
<davecheney> 2012/10/11 22:20:49 JUJU machiner: rerunning machiner
<davecheney> 2012/10/11 22:20:49 JUJU machiner: machine agent starting
<niemeyer> davecheney: That looks wrong
<niemeyer> davecheney: It should have an initial password
<davecheney> niemeyer: let me dig into cloudinit on our side
<niemeyer> davecheney: Or just have a look at what it looks like in the server
<niemeyer> davecheney: I'm pretty sure rogpeppe addressed that already in theory
<davecheney> niemeyer: ironically, juju status works
<davecheney> so _i_ can connect to the state
<davecheney> just none of the agents
<niemeyer> davecheney: That empty initial password is a good hint of what's wrong
<niemeyer> davecheney: It should not be empty
<fwereade> niemeyer, offhand, do you know what refcounts exist at the moment in state?
<fwereade> niemeyer, I am at the point of needing to have a sane relation lifecycle in place, and probably service too
<niemeyer> fwereade: I don't think we have any yet
<fwereade> niemeyer, am I right in recalling agreement that relation existence (with any life) should block service removal?
<fwereade> niemeyer, sorry, service Dyingness even
<niemeyer> fwereade: THe thing we were going to add was machine units in the machine, but we ended up with the units themselves there
<niemeyer> I mean, unit names
<niemeyer> fwereade: Yes, that's my understanding as well
<fwereade> niemeyer, great
<fwereade> niemeyer, except, hm, when I say removal, I actually *mean* EnsureDying
<niemeyer> fwereade: I hadn't thought of that, but I guess it makes sense
<fwereade> niemeyer, so: EnsureDying demands that no relations exist; AddRelation demands that the services be alive; therefore I think I need a relation-count on service
<fwereade> niemeyer, sane-sounding?
<fwereade> niemeyer, then, similarly, relation.EnsureDead demands that no units are in scope, while unit.EnterScope demands that the relation not be... I *think* Dead, but this reasoning also applies to Dying... so I think I need a units-in-scope-count on relation
<niemeyer> fwereade: Hmm.. I'm think it there's a chance we could do without, but I guess we need it if we want that semantics
<fwereade> niemeyer, and then, I suspect, service.EnsureDead requires no units, while service.AddUnit demands that the service be Alive
<fwereade> niemeyer, yeah, I am a little upset that there are so many, but I think that they are the sane way to manage the transactions
<niemeyer> fwereade: What would happen if we allowed EnsureDying on the service?
<niemeyer> fwereade: EnsureDying in general is a sign that the given entity is dying, with no practical consequences other than things getting into a termination procedure
<fwereade> niemeyer, essentially we'd take down all the relations with the service, and I have a vague recollection that we've been requiring explicit relation removal before allowing people to remove a key piece of infrastructure
<niemeyer> fwereade: It actually doesn't sound to bad to allow it, I think, assuming the practical consequences are positive
<fwereade> niemeyer, (I think that if we want remove-service => remove-many-relations, things potentially get somewhat complex as well -- I'm not sure that I can compose a transaction that will successfully remove every relation on a service reliably)
<fwereade> niemeyer, (oh, wait, refcount=N; docExists x N)
<fwereade> niemeyer, so we don't have to worry about handling it in the uniter
<fwereade> niemeyer, I think I would prefer that such drastic action at least require a --force, though
<niemeyer> fwereade: Sorry, I missed the line there
<fwereade> niemeyer, sorry, not quite sure what you saw, let me repaste
<fwereade> <fwereade> niemeyer, (I think that if we want remove-service => remove-many-relations, things potentially get somewhat complex as well -- I'm not sure that I can compose a transaction that will successfully remove every relation on a service reliably)
<fwereade>  niemeyer, (oh, wait, refcount=N; docExists x N)
<fwereade>  niemeyer, so we don't have to worry about handling it in the uniter
<fwereade>  niemeyer, I think I would prefer that such drastic action at least require a --force, though
<niemeyer> fwereade: I mean I'm missing your line of thinking there, regarding refcount and doc-exists.. how does that affect things?
<fwereade> niemeyer, if we're going to set a service to Dying, I think we should also set all its relations to Dying too, so that the counterpart services clean up nicely
<fwereade> niemeyer, I think that if we do a transaction involving setting the service and N relations to Dying, we can assert that:
<fwereade> niemeyer, service.relation-count == N
<fwereade> niemeyer, and, N times over, that the respective relation document exists
<fwereade> niemeyer, and I think be protected against anyone adding a relation to a dying service (assuming they assert service Alive ofc)
<niemeyer> fwereade: We can't assert that service and relation are both dying on the same transaction
<fwereade> niemeyer, ...oh
<niemeyer> fwereade: Because nothing prevents a new relation from being inserted right before we start to execute that transaction
<fwereade> niemeyer, wait, that wasn't what I wanted to assert
<niemeyer> fwereade: <fwereade> niemeyer, I think that if we do a transaction involving setting the service and N relations to Dying, we can assert that:
<niemeyer> fwereade: That's what I was referring to
<fwereade> niemeyer, I'm setting, not asserting, Dying; is that relevant?
<fwereade> niemeyer, actually, I'm not at all sure that I can do that
<fwereade> niemeyer, so, ok, if we want to take down all relations with a service (which feels somewhat alarming to me), I think we have to have the uniter handling relation-killing when it detects a dying service -- just as it kills the unit. right?
<niemeyer> fwereade: I don't yet have a strong opinion on it going either way, so I'm happy to explore the option you feel most comfortable with
<niemeyer> fwereade: One thing to note, perhaps as a guideline for us,
<niemeyer> fwereade: is that the interface we offer to the user, doesn't have to match precisely the internal details
<niemeyer> fwereade: So, let's say we refcount
<niemeyer> fwereade: and at least for the moment, let's say we prevent people from killing services with relations
<fwereade> :)
<niemeyer> fwereade: the at least for the moment is based on the above idea.. we can go this way even if later we decide to change
<niemeyer> fwereade: Because we can make the remove-service command deal with the manipulation
<niemeyer> fwereade: and break down in case someone else runs a race, for example
<niemeyer> fwereade: Either way, continuing
<niemeyer> fwereade: When do we *allow* the service to be EnsureDying'ed?
<niemeyer> fwereade: When all relations are Dying, or when all relations are Dead?
<fwereade> niemeyer, IIRC we did a long time ago agree that it was when none even existed -- but I think that translates effectively to all Dead
<fwereade> niemeyer, however I do not think this is optimal
<niemeyer> fwereade: Okay, that seems a bit unfriendly
<fwereade> niemeyer, Dying, to me, makes a lot more sense
<niemeyer> fwereade: That's better, but still makes me ponder
<fwereade> niemeyer, and this ofc changes the underlying assumptions of what I said above, I think
<niemeyer> fwereade: Okay, here is a strawman
<niemeyer> fwereade: We can do both:
<niemeyer> 1) Add refcounting to the service, so we can implement force/non-force logic and error out if we please
<fwereade> niemeyer, dying service requires dying relations; dead service requires dead relations?
<davecheney> % bzr conflicts
<davecheney> Conflict adding file juju/deploy.go.BASE.  Moved existing file to juju/deploy.go.BASE.moved.
<davecheney> Conflict adding file juju/deploy.go.OTHER.  Moved existing file to juju/deploy.go.OTHER.moved.
<davecheney> Conflict adding file juju/deploy.go.THIS.  Moved existing file to juju/deploy.go.THIS.moved.
<davecheney> none of those six files exist
<fwereade> niemeyer, ah no forget I said anything
<fwereade> niemeyer, please continue
<niemeyer> 2) Implement logic that supports dying service with non-dying relations, and terminates properly
<niemeyer> fwereade: This means, in a way, we have the cake and can eat it as well.. to terminate setting service to Dying is enough, but we can tweak the API to our desires, so we can blow it up or not (perhaps supporting --force)
<niemeyer> davecheney: bzr conflicts reports what happened
<niemeyer> davecheney: Run "bzr resolved" to notify you've dealt with it
<davecheney> niemeyer: bzr couldn't fix it
<davecheney> i copied my changes away and did a revert
<davecheney> it was caused by swiching branches with uncomitted changes
<fwereade> niemeyer, ok, this is the return of the corrective agent :)
<niemeyer> fwereade: Oh, hopefully not
<niemeyer> fwereade: Why would we need it?
<fwereade> niemeyer, well, I don;t think we can put it in the unit agent, because we can't know that any units will actually exist to run that logic, I think
<fwereade> niemeyer, so I was imagining that that would have to handle (2)
<niemeyer> fwereade: Who sets service to Dead?
<niemeyer> fwereade: and then removes it?
<fwereade> niemeyer, the unit agent, assuming one exists -- or, I presume, the client, if it can be sure that such action is warranted by reason of there being no units
<niemeyer> fwereade: Exactly.. the whoever removes the last unit, is probably the half-answer
<niemeyer> fwereade: This is the place, and the code path, to delete the relations
<fwereade> niemeyer, I don't think we can sanely set service to Dead until *after* all relations are Dead (or gone), can we?
<fwereade> niemeyer, otherwise we have a live relation referencing a dead service, and that's not going to end well
<niemeyer> fwereade: How can we have live relations if all units of one side are gone?
<niemeyer> fwereade: Well, I guess we have to wait until the other side acknowledges
<fwereade> niemeyer, the relation scope may be unoccupied, but that doesn't imply anything about the relation's Life, does it?
<niemeyer> fwereade: And that sounds fine.. we can backtrack the service removal to the last thing that acks the release of a resource attached to that service
<fwereade> niemeyer, yeah, that sounds right
<niemeyer> fwereade: Hmm..
<niemeyer> fwereade: We can probably implement something simpler
<niemeyer> fwereade: Way simpler, in fact
<fwereade> niemeyer, this would make me happy
<niemeyer> fwereade: We can in fact set service and all its relations as dying at once
<fwereade> niemeyer, that was what I suggested originally
<niemeyer> fwereade: Right, and I thought that wasn't possible, but it is
<niemeyer> fwereade: We have to loop, though
<niemeyer> fwereade: To avoid races
<niemeyer> fwereade: We have to assert on the number of relations
<niemeyer> fwereade: Hmm.. which is perhaps wrong
<niemeyer> fwereade: Yeah, we can't do that in fact.. :(
 * niemeyer thinks about alternatives
 * niemeyer => whiteboard
 * fwereade looks up at clock
<fwereade> niemeyer, sorry to abandon you, but I should call it a night
<fwereade> niemeyer, I look forward to continuing the discussion tomorrow
<niemeyer> fwereade: Okay, it's doable
 * fwereade cheers, and decides to stick around for a mo
<niemeyer> fwereade: We can take the refcount into account in the transaction
<niemeyer> fwereade: Sorry
<niemeyer> fwereade: The txn-revno
<niemeyer> fwereade: To guarantee that the refcount hasn't been incref'd and decref'd in-between
<niemeyer> fwereade: So we loop, and assert txn-revno is the same, and include service and all relations in the same transaction
<niemeyer> fwereade: This guarantees that either service dies with all its relations, or we get an ErrAbort
<niemeyer> fwereade: by die I mean become Dying
<fwereade> niemeyer, yeah, I think that makes sense
<fwereade> niemeyer, it was what I had being fumbling for with the docExistses
<niemeyer> fwereade: If we get ErrAbort, we start over and ensure we're still doing a sane transaction
<niemeyer> fwereade: The problem is that we can assert on non-existence
<niemeyer> fwereade: Erm, cannot
<niemeyer> fwereade: So we have to:
<niemeyer> 1) Grab service
<niemeyer> 2) Grab all its relations
<niemeyer> 3) Do a transaction with relations from 2, and include an assertion ensuring that service hasn't changed since (1)
<fwereade> niemeyer, yep
<niemeyer> That means (2) observed everything (1) knew about
<niemeyer> and thus (3) is consistent
<fwereade> niemeyer, yep, exactly
<niemeyer> fwereade: You can sleep well, I think :-)
<fwereade> niemeyer, indeed :)
<niemeyer> fwereade, davecheney: Btw,
<niemeyer> fwereade, davecheney: Tomorrow is a national holiday here
<niemeyer> fwereade, davecheney: And I'll try to relax a bit.. but I'll be around at some point. There's at least one meeting I need to attend at 1:30
<fwereade> niemeyer, cool
<niemeyer> Which is 16:30 UTC
<niemeyer> mramm: ^^
<fwereade> niemeyer, I will surely see you around then :)
<niemeyer> fwereade: Yeah
<fwereade> niemeyer, btw, I would be most grateful if you would take a very quick look at https://codereview.appspot.com/6650043/ before you go
<fwereade> niemeyer, blast, I should repropose, it's on top of the old config-changed bits
<niemeyer> fwereade: Will do
<fwereade> niemeyer, just want to propose another first though, just because all it's waiting on is a full test run
#juju-dev 2012-10-12
<fwereade> niemeyer, that one should be a trivial, it's the State.Relation(...Endpoint) -> State.EndpointsRelation(...Endpoint) + state.Relation(int)
<fwereade> niemeyer, that IIRC we agreed on a few days ago
<niemeyer> fwereade: +1
<mramm> niemeyer: cool
<davecheney> niemeyer: % juju bootstrap --upload-tools
<davecheney> error: cannot start bootstrap instance: cannot make user data: invalid machine configuration: initial password is empty
<davecheney> submitting branch now
<davecheney> well, for review
<fwereade> niemeyer, how pleasing, a palindromic CL id: https://codereview.appspot.com/6601066
<niemeyer> fwereade: Wow, beautiful :)
<niemeyer> davecheney: Hmm, ok.. curious about the cause
<davecheney> niemeyer: looking for it now
<niemeyer> fwereade: Looking already
<davecheney> at least we know where the problem is now
<fwereade> niemeyer, great -- and https://codereview.appspot.com/6650043 shouldn't be too controversial either
<niemeyer> fwereade: nicely straightforward
<niemeyer> fwereade: Cool, looking
<davecheney> niemeyer:         password := e.Config().AdminSecret()
<davecheney>         if password != "" {
<davecheney>                 password = trivial.PasswordHash(password)
<davecheney>         }
<davecheney> ec2/bootstrap
<niemeyer> davecheney: Hmm.. that looks okay..?
<davecheney> AdminSecret() must be returning ""
 * fwereade calmly resists the temptation to submit tonight, and really really goes off to bed, he can look forward to it in the morning :)
<niemeyer> fwereade: Hmm.. one last question? :)
 * fwereade hasn't really gone
 * fwereade is sadly predictable :)
<niemeyer> fwereade: ROTFL
<davecheney> niemeyer: ahh, found it
<davecheney> i was missing an admin-secret key for one of my environments
<davecheney> but the schema didn't complain
<niemeyer> fwereade: Why is it not sending the plain slice it gets?
<niemeyer> fwereade: It seems to be sending all relations it ever received at all times
<fwereade> niemeyer, it clears them out when it sends
<fwereade> niemeyer, f.relations = nil
<fwereade> niemeyer, then it collects a set of ids it's seen until someone receives again
<niemeyer> fwereade: Ahh, I see
<fwereade> niemeyer, I didn;t want to just collect by appending because dupes would be icky
<niemeyer> fwereade: I suggest keeping the form as a list of ids still
<niemeyer> fwereade: But that's minor.. it looks pleasantly straightforward too
<davecheney> niemeyer: from ec2/config.go
<davecheney>                 "admin-secret":   schema.String(), // Unused. Here just for compatibility.
<niemeyer> fwereade: Have a good sleep!
<davecheney> ^ clearly, this isn't true
<niemeyer> fwereade: and thanks
<fwereade> niemeyer, all I think I'll ever want to do is iterate over it
<fwereade> niemeyer, and it feels a bit icky to (potentially) merge into a list without dupes every change
<niemeyer> davecheney: Indeed, but that shouldn't, I think, be affecting anything
<fwereade> niemeyer, what's the advantage of a list?
<davecheney> niemeyer:                 "admin-secret":  schema.Omit,
<niemeyer> fwereade: I'm willing to bet this is orders of magnitude faster and consumes less space in the real cases
<davecheney> means AccessSecret always retuns a value
<davecheney> and that value is ""
<fwereade> niemeyer, I guess this is indeed not really one of the N=100000 cases
<niemeyer> fwereade: not only that, but real cases will generally see no appends at all, and allocating and handling a sequence of bytes is pretty friendly to the hardware
<fwereade> niemeyer, all true -- thanks :)
<niemeyer> fwereade: np, and have a good night! :-)
<fwereade> niemeyer, consider it done... but tomorrow ;)
<fwereade> niemeyer, davecheney, gn
<davecheney> fwereade: gn
<niemeyer> davecheney: Isn't that the "default" value?
<davecheney> yup, found it in a few places
<davecheney> which means a valid config can have a "" for the inital password
<niemeyer> davecheney: It can, but that's not related to this logic
<niemeyer> davecheney: and I thought you had said you didn't have an empty admin-secret?
<davecheney> niemeyer: that was my fault
<davecheney> i didn't read my environments.yaml closely enough
<davecheney> that key was missing from some envs
<niemeyer> davecheney: Ah, okay, so without the admin secret indeed it will fail I suppose
<davecheney> just confirming now
<davecheney> ok, yes, with an admin-secret it is working
<niemeyer> davecheney: We should prevent that, but ec2 is not the right place.. that should happen in environs/config/config.go
<niemeyer> davecheney: Woohay! That's great news.
<davecheney> niemeyer: already fixing
<davecheney> niemeyer: and i've added double underpants in cloudinit
<davecheney> niemeyer: before I get too far into this
<davecheney> is this statement correct -> admin-secret is now a required configuration item ?
<niemeyer> davecheney: I think so
<niemeyer> davecheney: Assuming it all works well
<davecheney> niemeyer: ok, i'll propose this and we can discuss it with rog tonight
<davecheney> well, once I get all the tests passing
<rogpeppe> davecheney: morning!
<TheMue> morning
<davecheney> morning
<TheMue> davecheney: not yet time for dinner? ;)
<davecheney> TheMue: soon
<davecheney> still fighting with tests
<davecheney> https://codereview.appspot.com/6659048
<TheMue_> hmmmp, network
<davecheney> rogpeppe: i've hit a roadblock with admin-secret
<davecheney> niemeyer is pretty sure it is now a required configuartion item
<rogpeppe> davecheney: oh dear. tell me about it
<rogpeppe> davecheney: fuck, yes it is
<rogpeppe> davecheney: alternatively i *could* change things so it would work without it
<davecheney> rogpeppe: https://bugs.launchpad.net/juju-core/+bug/1065750
<davecheney> ^ long version
<rogpeppe> davecheney: is there some problem you're having even when you *do* specify admin-secret?
<davecheney> nope, it works then
<davecheney> but without something liek  https://codereview.appspot.com/6659048 (unfinished)
<rogpeppe> davecheney: cool, i'll just make it mandatory and all problems will go away :-)
<davecheney> you can happily deploy a non working config
<davecheney> rogpeppe: yes, but if you make it manditory, then things like BootstrapConfig stop working
<rogpeppe> davecheney: yeah, it was my intention to make it mandatory - it slide off my todo list, i'm sorry.
<rogpeppe> davecheney: why's that?
<rogpeppe> s/slide/slid
<davecheney>         // We never want to push admin-secret to the cloud.
<davecheney>         delete(m, "admin-secret")
<davecheney>         m["agent-version"] = tools.Number.String()
<davecheney>         return config.New(m)
<davecheney> if m is lacking "admin-secret"
<davecheney> config.New() will fail
<rogpeppe> hmm
<rogpeppe> so it's mandatory at one level, but not at another
<davecheney> unpossible
<rogpeppe> i'd forgotten that wrinkle
<rogpeppe> i think probably juju.NewConn (?) should fail if there's no admin secret,
<rogpeppe> davecheney: but config should continue to work without it
<davecheney> well, my first effort was https://codereview.appspot.com/6659048/patch/2001/3003
<davecheney> but that just spiraled out of control trying to make all the tests pass
<rogpeppe> davecheney: yeah, that won't work
<rogpeppe> davecheney: you've gotta stop the problem at source
<rogpeppe> davecheney: i'm happy to fix this problem if you haven't done so already. it's all my fault.
<davecheney> rogpeppe: have a crack, i'm done for hte day
<davecheney> the diff is just getting longer and longer
<rogpeppe> davecheney: ok, thanks for taking a look
<rogpeppe> davecheney: can you push your latest attempt?
<davecheney> rogpeppe: i just did
<rogpeppe> davecheney: cool, so https://codereview.appspot.com/6659048 is it?
<davecheney> y
<fwereade__> rogpeppe, TheMue: I imagine, but would like to confirm, that none of you are implementing service or relation lifecycle stuff?
<rogpeppe> fwereade__: correct in my case at least
<rogpeppe> fwereade__: morning, BTW!
<fwereade__> rogpeppe, cheers
<fwereade__> rogpeppe, heyhey :)
<TheMue> fwereade__: yes, i'm doing firewaller stuff right now
<fwereade__> TheMue, thanks
<TheMue> fwereade__, rogpeppe: morning
<fwereade__> TheMue, (morning :))
 * fwereade__ has just spotted where he broke the uniter
<fwereade__> rogpeppe, am I to understand that we can't deploy things atm for authy reasons?
<rogpeppe> fwereade__: no.
<rogpeppe> fwereade__: but you do need to add an admin-secret to your environ config
<fwereade__> rogpeppe, ah, ok, I've always had one of them lying around anyway ;p
<rogpeppe> fwereade__: i'm just trying to work out where to put the checks to ensure that it's set
<rogpeppe> fwereade__: config.Config is unfortunately *not* the right place
 * fwereade__ has always maintained that an input config and an internal config are not the same thing, and that the hassle of pretending they are is not wrth it
 * rogpeppe tends to agree, but we have what we have now.
 * fwereade__ wishes rogpeppe good luck
<rogpeppe> fwereade__: part of the problem is that it's quite hard to enable passwords for local tests... but maybe it's ok now since my "restart mongo on unauthorised-access-failure" hack
<rogpeppe> TheMue: hiya
<TheMue> rogpeppe: ;)
<Aram> moin.
<fwereade__> Aram, heyhey
<fwereade__> TheMue, I'm a bit confused, would you confirm for me what security groups I should be seeing with a bootstrap + deploy mongodb?
<TheMue> fwereade__: today one named "juju-<groupname>" and one per machine named "juju-<groupname>-<machineid>".
<fwereade__> TheMue, ok, cool -- but I am confused, because none of them seem to have any rules; and yet I can connect to state just fine: is it immediately apparent that I am being stupid?
<TheMue> fwereade__: the juju-group afaik has all ports open between each other and 22 to the world. the other group should have those ports worldwide open which are needed.
<fwereade_> rogpeppe, can I get a trivial LGTM on https://codereview.appspot.com/6656051
<rogpeppe> fwereade_: looking
<TheMue> yummy, had a good lunch, kraut
<rogpeppe> fwereade_: not *that* trivial :-)
<fwereade_> rogpeppe, the code change really really is -- and, bother, I thought the test changes would be clear
<rogpeppe> fwereade_: the code change really is (although i'm not sure i see the implications - why didn't it crash before?); the test changes need a little thought though.
<fwereade_> rogpeppe, it did, I think
<fwereade_> rogpeppe, I believe this was what was hitting davecheny 2 nights ago
<rogpeppe> fwereade_: ah. have you got a test that failed before and now passes?
<fwereade_> rogpeppe, yes; the expanded config-changed test
<fwereade_> rogpeppe, in it, config-changed now records the output of config-get, and we verify that we see what we should
<rogpeppe> fwereade_: by "verify lack of hooks" do you mean that you're verifying lack of hook *executions* ?
<fwereade_> rogpeppe, ha, yes
<rogpeppe> fwereade_: ok, that makes more sense
<rogpeppe> fwereade_: i'm still really enjoying your little test DSL BTW
<fwereade_> rogpeppe, yeah, it makes me happy too :)
<rogpeppe> fwereade_: who needs LISP, eh? :-)
<fwereade_> rogpeppe, haha :)
<rogpeppe> fwereade_: what does node.Update do?
<rogpeppe> fwereade_: scratch that
<rogpeppe> fwereade_: i just read the code, doh!
<fwereade_> rogpeppe, ha, yeah, I find myself doing that :)
<rogpeppe> fwereade_: i mean i actually read the code for changeConfig.step properly and noticed the little "s" arg to Update...
<fwereade_> rogpeppe, ha, yeah
<rogpeppe> fwereade_: so config-get never worked at all before?
<rogpeppe> fwereade_: anyway, LGTM
<rogpeppe> fwereade_: i'll leave it up to you to judge whether it's trivial enough to submit now
<fwereade_> rogpeppe, it did until I did a half-baked (agreed) post-LGTM change the other day
<rogpeppe> fwereade_: ahhh
<fwereade_> rogpeppe, since it fixes my own monstrous howler, I feel it is very important that it be sumbmitted asap :)
<fwereade_> rogpeppe, and I'm preety sure the test changes are all positive
<rogpeppe> fwereade_: go for it
<fwereade_> rogpeppe, cheers
<rogpeppe> fwereade_: a similarly almost-trivial CL: https://codereview.appspot.com/6601067
<fwereade_> rogpeppe, LGTM -- need to be off for lunch now though
<rogpeppe> lunch too
<Aram> I need some information about the firewaller and machiner.
<Aram> I need to understand better what to replace change.Remove with.
<Aram> Dying, Dead, or (Dead OR Removed).
<Aram> rogpeppe: fwereade_: TheMue ^ ^
<rogpeppe> Aram: for which watcher?
<Aram> rogpeppe: Machine(Principal)?UnitsWatcher.
<Aram> I used Dead OR Remove, and tests pass.
<Aram> but I am not sure if that's right or not.
<Aram> actually I used Dead OR Remove OR Unassigned
<TheMue> Aram: i would prefer dead to keep it consistent with the lifecycle
<fwereade_> Aram, AIUI that is what we need there, yes
<fwereade_> Aram, but I had a vague feeling that we wanted to implement all the life-group watchers as sending lists of ids?
<rogpeppe> fwereade_: Dead OR Remove OR Unassigned seems about right
<Aram> fwereade_: that's what I did, I reimplemented the units watcher to send list of ids.
<fwereade_> rogpeppe, agree, I think, haven't 100% thought through unassigned
<fwereade_> Aram, ah, excellent -- I thought the idea was that we should just be sending every life change, rather than filtering those at the watcher level?
<Aram> fwereade_: I'm not sure I follow. you want the watcher to return map[Life][]string instead of []string?
<fwereade_> Aram, not at all
<fwereade_> Aram, above, I got the impression that you were not sending changes from alive to dying; was that wrong?
<Aram> this is the behavior now:
<Aram> / MachinePrincipalUnitsWatcher notifies about assignments and lifecycle
<Aram> / changes for all principal units assigned to a machine.
<Aram> /
<Aram> / The first event emitted contains the unit names of all principal
<Aram> / units currently assigned to the machine, irrespective of their life
<Aram> / state. From then on, a new event is emitted whenever a unit is assigned
<Aram> / to or unassigned from the machine, or the lifecycle of a unit that is
<Aram> / currently assigned to the machine changes.
<Aram> /
<Aram> / After a unit is found to be Dead, no further event will include it.
<Aram>  
<Aram> so the watcher fires both when life is changed, and when is a change in assignment, reglardless of lifecycle.
<rogpeppe> fwereade_: i'm having second thoughts about that earlier CL. here's an alternative: https://codereview.appspot.com/6601068/
<rogpeppe> fwereade_: i'm seriously considering adding a NoBootstrap field to JujuConnSuite, but fear it might be condemned as crackful.
<fwereade_> rogpeppe, not especially keen myself, I think I liked the earlier one more
 * Aram loved $CDPATH
<Aram> s
<rogpeppe> fwereade_: the problem is i need to do exactly the same thing in CmdSuite
<rogpeppe> fwereade_: and it seems a bit silly to duplicate *almost* all of JujuConnSuite's functionality in two places, when we actually want something only slightly less than it does
<fwereade_> rogpeppe, hmm, good point -- I think I'd prefer a NoBootstrap field to what you have there
<fwereade_> rogpeppe, but I can't predict niemeyer's response
<rogpeppe> fwereade_: ok, that's useful thanks
<rogpeppe> fwereade_: maybe i should just bite the bullet and do the duplication
<fwereade_> rogpeppe, I suspect that will give yu the smoothest path through review
<rogpeppe> fwereade_: done: https://codereview.appspot.com/6660048
<Aram> holy cow, bazaar really doesn't have the option of reverting a commit.
<Aram> you need to do some merge magic
<rogpeppe> Aram: bzr uncommit ?
<Aram> rogpeppe: that only removes the last commits, and it really removes them, I don't want that, I want to undo an arbitrary change bu applying the reverse diff and commit the new undo operation.
<rogpeppe> Aram: yeah, you're out of luck there.
<rogpeppe> Aram: i've used patch for that before
<Aram> you can do a merge . -r -X..-X+1
<rogpeppe> Aram: i didn't know merge accepted revno ranges
<Aram> rogpeppe: yeah, it worked just fine
<Aram> rogpeppe: fwereade_: TheMue: PTAL at the firewaller and machiner changes: https://codereview.appspot.com/6625069
<Aram> firewaller and machiner tests pass with those changes and the new watcher.
<TheMue> Aram: in firewaller, when are units added after watcher changes?
<TheMue> Aram: ah, found, is ok
<TheMue> Aram: missed the line
<Aram> Amazon doesn't like the diacritical marks in my name.
<Aram> wtf.
<Aram> I can't set a name if it's not ASCII
<Aram> "We could not verify the address you have provided. If you are sure this is a correct address, please try again."
<Aram> how is amazon veryfying my address and why?
<Aram> fuck you.
<Aram> already spent to much time on this.
<Aram> this should not have taken more than 30 seconds.
<rogpeppe> fwereade_, Aram: trivial (code movement only): https://codereview.appspot.com/6651062/
<rogpeppe> Aram: i'm sorry i'm not sure i'll have time to go through the watcher changes today. i'll try though.
<Aram> rogpeppe: don't worry.
 * Aram is done for today.
<Aram> cheers everyone, have a great weekend.
<rogpeppe> fwereade_: can i get a quick LGTM on the above branch please?
<rogpeppe> niemeyer: ^
<rogpeppe> https://codereview.appspot.com/6651062/
<niemeyer> rogpeppe: On holiday, and on a call
<rogpeppe> niemeyer: np at all
<rogpeppe> niemeyer: hope you're having a good day!
<niemeyer> rogpeppe: Hello, btw :)
<rogpeppe> niemeyer: likewise
<rogpeppe> TheMue: ping
<TheMue> rogpeppe: pong
<rogpeppe> TheMue: any chance of a quick LGTM on the above CL?
<rogpeppe> TheMue: code movement only, no code changes
<TheMue> rogpeppe: already looking
<TheMue> rogpeppe: you got it ;)
<rogpeppe> TheMue: ta!
<rogpeppe> TheMue, fwereade_: here's another one if you fancy having a look: https://codereview.appspot.com/6653050/
<TheMue> rogpeppe: thumbs up
<rogpeppe> TheMue: thanks
<TheMue> so, i'm off. have a nice weekend
<rogpeppe> me too, have a great weekend everyone!
<niemeyer> mramm: I have a good chuckle when I'm in a phone meeting with Mark S. and then there's that message saying "The LEADER has left the conference!" and all goes silent at the end..
<niemeyer> rogpeppe: Have a good one
 * niemeyer steps out for more family activities..
<niemeyer> Have a good weekend all
<mramm> niemeyer: have a great weekend
<mramm> niemeyer: I never saw it that way, but it is great.
<SpamapS> would be a great prank to have IS change that message to 'Elvis has left the building'
#juju-dev 2013-10-07
<jpds> Anyone awake?
<axw> jpds: I am awake, something I can help with?
<achandra> hello. I appear to be having the toughest time trying to connect to my swift object-store on devstack via juju
<achandra> ive pulled up the url of the object-store via keystone catalog
<achandra> however, once i create the juju-dist and upload the meta-data
<achandra> i still get an un-authorised connection
<achandra> ideas?
<rogpeppe> mornin' all
<rogpeppe> hmm, i left juju running the local provider over friday and saturday and ended up with a 2.1GB log file
<rogpeppe> ha, it's all my fault
<rogpeppe> it panicked 47717 times before i killed it yesterday
<axw> morning rogpeppe
<axw> heh :)
<rogpeppe> axw: hiya
<rogpeppe> axw: i was just glancing at  https://codereview.appspot.com/14465045/
<rogpeppe> axw: where are the sftp urls coming from?
<axw> rogpeppe: environs/sshstorage
<axw> its URL method returns sftp:// URLs
<rogpeppe> axw: ah, because the URLs were only expected to be used from a shell script?
<axw> rogpeppe: I was expecting them to not be used at all (or only for logging)
<rogpeppe> axw: ah, i see
<axw> but yeah, they get used by wget or http.Get in some cases
<rogpeppe> axw: also, not quite sure what you mean by "problematic for the trunk implementation"
<rogpeppe> axw: do you just mean the trunk branch of juju?
<axw> rogpeppe: yeah, sorry, badly worded
<axw> I just mean that the current code is broken
<axw> and this change fixes it by wrapping it in httpstorage
<rogpeppe> axw: np, i *thought* so, but i wasn't sure
<axw> thus hiding the invalid URL method away
<rogpeppe> axw: right, now i think i have enough info to review the change properly :-)
<axw> :) thanks
<rogpeppe> axw: hmm, i'm getting a chunk mismatch error
<rogpeppe> axw: could you repropose please?
<axw> yup, just a moment
<axw> rogpeppe: do you think it'd be worthwhile using gotype instead of "go build" in .lbox.check? might speed things up a bit ... can you think of any downsides?
<axw> maybe it's just my crappy laptop's the problem
<rogpeppe> axw: is there a gotype command?
<axw> (repropose is still chugging along)
<axw> rogpeppe: yeah, somewhere in go.tools
<rogpeppe> axw: you're running go tip then, right?
<axw> code.google.com/p/go.tools/cmd/gotype
<axw> rogpeppe: nope, why?
<axw> go/types works on 1.1
<axw> it didn't for a bit, but rsc fixed that
<rogpeppe> axw: hmm, what go version are you using?
<axw> rogpeppe: 1.1.1
<rogpeppe> axw: axw: i'm pretty sure it's not go build that takes all the time
<rogpeppe> axw: hmm
<rogpeppe> axw: on my machine it's go vet that takes forever
<axw> rogpeppe: I didn't even verify. okay, probably not worthwhile then
<rogpeppe> axw: but...
<rogpeppe> axw: i thought that was only since go vet became external (since go 1.1.1) and started using go types
<rogpeppe> axw: it takes about 30s to run on my machine
<axw> rogpeppe: minutes here, but I have a crappy laptop
<axw> rogpeppe: propose finished
<rogpeppe> axw: i mean "go vet" takes 30s
<rogpeppe> axw: propose can take minutes for me too
<axw> hmmk
<axw> I suppose it doesn't matter too much, forces me to think a bit harder before proposing junk ;)
<rogpeppe> axw: i really wish it was faster
<rogpeppe> axw: BTW i'm *still* seeing a chunk mismatch :-|
<axw> ack
<axw> :(
<axw> hmm
<TheMue> morning
<axw> rogpeppe: where does this chunk mismatch show up?
<axw> morning TheMue
<rogpeppe> axw: https://codereview.appspot.com/14465045/diff/1/provider/null/environ.go
<rogpeppe> axw: one mo, maybe i didn't reload the main CL page
<rogpeppe> axw: ignore me
<axw> rogpeppe: I think so
<axw> nps
<rogpeppe> axw: it works fine
<rogpeppe> axw: does anything use sshstorage other than the null provider?
<axw> rogpeppe: nope
<axw> only the null provider, only for bootstrap
<rogpeppe> axw: oh yes, one thing that's been niggling at me for a while: how do you ensure that the bootstrap storage object is eventually closed?
<rogpeppe> axw: as doesn't it contain a live ssh connection?
<axw> rogpeppe: you don't/can't  at the moment
<axw> yep
<rogpeppe> axw: ah, so it just leaks
<axw> rogpeppe: yes, but this is only ever done from the command line
<rogpeppe> axw: currently...
<axw> hmm
<rogpeppe> axw: we shouldn't assume that though - it's ok for people to use this stuff directly from Go.
<axw> rogpeppe: if this is moved to something non-interactive, I think we'll have bigger problems
<axw> like, how to input sudo passwords
<rogpeppe> axw: ha, yes
<axw> but yeah I understand
<rogpeppe> axw: although that's not a problem if it's already running as root, i guess
<axw> true
<axw> actually
<axw> not true
<axw> it's the target machine that matters
<axw> and you typically wouldn't allow root ssh
<rogpeppe> axw: how does the target machine know you're running as root?
<axw> rogpeppe: anyway, I guess this would be solved by adding a Close method to Environ
<axw> rogpeppe: ?
<rogpeppe> axw: yes, but that's actually a very big change
<axw> rogpeppe: null provider bootstrap takes two parameters: host, and login user
<rogpeppe> axw: well, actually, it would probably need a close method on both Environ and Storage
<axw> rogpeppe: login user is the current user on the source machine by default, but you'd probably need to change that to something non-root
<rogpeppe> axw: sure
<rogpeppe> axw: we could probably use a comment in the code just to let people know that this has been thought about
<axw> rogpeppe: okey dokey, will add one
<mattyw> morning folks, any reason why the maximum length of a summary using lbox is so low?
<rogpeppe> mattyw: i think it's an lp restriction
<rogpeppe> mattyw: ridiculous, ain't it?
<rogpeppe> axw: just to clarify: what code is actually using the storage URL?
<axw> rogpeppe: I didn't follow all the way to the end, but it's related to generating simplestreams metadata
<axw> if the tools exist on the target, but the metadata doesn't, it grabs the tools off the target and generates the metadata from them
<rogpeppe> axw: ah! so it uses the URLs inside the metadata
<axw> rogpeppe: it uses them to obtain the tools, to generate the metadata
<rogpeppe> axw: probably the other way around, no? it doesn't need to obtain the tools to generate the metadata, but the metadata will contain URLs which need to point to the tools
<rogpeppe> axw: (but i haven't looked at the code...)
<axw> rogpeppe: I think the juju CLI is fetching them off the target, generating the metadata at the CLI end, then uploading the metadata
<axw> because there's nothing running on the target necessarily
<axw> (in this case there isn't)
<axw> the metadata includes information about the tools like file size and SHA256 sums
<axw> so the tools need to be obtained to generate it
<rogpeppe> axw: hmm, so how can tools exist on the target without metadata? i thought we'd moved away from legacy tools storage...
<axw> rogpeppe: I could only surmise that it was a botched bootstrap
<axw> it got part way through downloading the tools, but didn't get as far as generating/uploading the metadata
<axw> rogpeppe: then a second bootstrap found the tools on the remote machine, but no metadata
<axw> so it decided to fetch them and generate the metadata then (and failed)
<rogpeppe> axw: hmm
<rogpeppe> axw: perhaps that's the logic that needs to be fixed
<axw> rogpeppe: perhaps, though I think it's prudent to hide invalid URLs away anyway
<axw> something else may come along wanting to use it
<rogpeppe> axw: well, the difficulty i have with this is that URLs are supposed to be usable from other machines
<axw> though TBH, it's still flawed - they're only valid for the lifetime of the current process
<rogpeppe> axw: and this only makes sense if the URL is only used locally
 * axw nods
<mattyw> fwereade, I'm about to set that mp as needing review, are there any bugs or blueprints I should link it to?
<fwereade> mattyw, nothing for that work specifically -- IIRC there's an "identity" blueprint somewhere that the followups will probably apply to though
<fwereade> mattyw, if you'd like to capture your requirements as a bug and link that it can't hurt though
<fwereade> mattyw, always nice to be able to look back and figure out why we write the code in the first place ;)
<mattyw> fwereade, ok will do
<rogpeppe> axw: i'm having difficulty finding the logic that fetches the tools off the target, generating the metadata at the CLI end
<rogpeppe> axw: istm that to do that it would need to List the storage, but there are very few places that do that
<axw> rogpeppe: I'll have a hunt
<axw> rogpeppe: in the mean time, https://codereview.appspot.com/14483043
<rogpeppe> axw: argh, darn chunk mismatch again
<axw> ah wtf
<axw> rogpeppe: the copying/listing/etc. is in environs/sync
<axw> brb
<mattyw> fwereade, https://codereview.appspot.com/14389043/ thanks :)
<axw> rogpeppe: in environs/tools/simplestreams.go, generateMetadata
<axw> "        if fetch && t.Size == 0 {"
<rogpeppe> axw: yeah, i'm seeing it now
<rogpeppe> axw: in sync.SyncTools
<fwereade> mattyw, tyvm
<mattyw> fwereade, no, thank you
<axw> rogpeppe: ideally that code should use the Storage
<axw> rogpeppe: but that would require a means of getting from a URL back to a storage name
<rogpeppe> axw: hmm, yeah
<rogpeppe> axw: unless a tools.List contained an expanded version on Tools which also held the original name, i guess
<axw> yeah, or that
<axw> rogpeppe: I reproposed that CL
 * fwereade breakfast
<rogpeppe> axw: istm that part of the problem is that we're in a half-way house between two schemes - we've got two forms of metadata (the simple streams stuff and the storage list) and the latter doesn't have enough information to fulfil the former's requirements
<rogpeppe> axw: that is, the real problem here is that toosl.ReadList isn't returning information on the hashes & sizes of the contents of the storage
<axw> rogpeppe: yeah, because that would incur quite a lot of overhead for the usual case where the metadata already exists
<axw> but yes, there's information loss there
<axw> the storage names in particular
<rogpeppe> axw: what would happen if we just passed in fetch==false ?
<axw> rogpeppe: the metadata would be written out with Size=0, and no hash. I *think* jujud gets unhappy about that
<rogpeppe> axw: that's what i was wondering
<rogpeppe> axw: it's good to think through stuff in this area, i think, especially because we want to solve lp:#1219582
<_mup_> Bug #1219582: bootstrap is slow, lookups tools multiple times <canonistack> <juju-core:Triaged> <https://launchpad.net/bugs/1219582>
<rogpeppe> it's kind of ironic that s3 does actually provide hash and size info, although i think it's only the md5 hash
<rogpeppe> axw: does it actually make a difference if the metadata exists on the target machine?
<rogpeppe> axw: it looks to me as if the logic in SyncTools ignores any metadata in the target
<axw_> rogpeppe: it ignores it for copying the tools over, but it's used to decide which tools tarball to install into an instance... not sure what happens if it's not there
<rogpeppe> axw: which means that we'll always read all the tools in both the target and source storage; i may have missed something though.
 * axw_ tries
<axw> rogpeppe: that doesn't sound right, let me check
<mattyw> fwereade, ping?
<axw> rogpeppe: SyncTools only does a storage.List on the source and target, and copies across what's missing from source. It grabs metadata from target if it exists, then generates metadata for what's missing by fetching the tools back off the target
<rogpeppe> axw: ah, no WriteMetadata does it, i see
<rogpeppe> axw: we really need some sort of local storage cache
<fwereade> mattyw, pong, sorry, got distrcted half way through
<axw> rogpeppe: yeah, that could certainly help
<mattyw> fwereade, no problem, just going through the next stage of the id stuff you said about adding the owner as an argument to AddService
<fwereade> mattyw, ah, yes
<mattyw> fwereade, that seems to break A LOT of stuff (all just mechanical changes because now there's a new argument) so I just wanted to check that this was what you wanted
<fwereade> mattyw, I can't think of another way to do it, I'm afraid
<mattyw> fwereade, ok no problem, just wanted to check we had the right idea before we fix all the things we broke
<fwereade> mattyw, yeah, thanks for checking :)
<mattyw> fwereade, I guess this is one reason for it being the next step - the number of lines in the diff is likely to be huge
<fwereade> mattyw, indeed, probably best to do as little as possible beyond that
<rogpeppe> mattyw, fwereade: this may be a silly idea, but i just thought i'd mention the possibility:
<rogpeppe> mattyw, fwereade: rather than add "owner" arguments to lots of methods in State, we could potentially have a facade in front of State that knows the current owner
<fwereade> rogpeppe, we're not planning to give anything except services owners at the moment
<rogpeppe> fwereade: even still - i just wonder if it might be a more economical approach, given that any given API client will be acting on  behalf of a particular user
<fwereade> rogpeppe, and fwiw that facade is really the api server, anyway, isn't it?
<rogpeppe> fwereade: sure, but we're about to change many many calls into state.State.
<fwereade> rogpeppe, expand a little maybe? ISTM that it's going to be even harder to do that
<rogpeppe> fwereade: type StateWithOwner {*state.State; User string}
<rogpeppe> fwereade: then StateWithOwner.AddService calls State.AddServiceOwnedBy
<fwereade> rogpeppe, so we s/State/StateWithOwner/ everywhere?
<fwereade> rogpeppe, not 100% sure that wins us anything
<rogpeppe> fwereade: i'm not sure either; worth thinking through though, i think.
<fwereade> rogpeppe, ISTM that the only benefit of what you propose is that we need to make fewer test changes
<fwereade> rogpeppe, but that each of those tests gets a bit more abstracted from what it's testing
<rogpeppe> fwereade: test changes are a big cause of churn and time taken
<fwereade> rogpeppe, I think that's an argument for a better way to define test fixtures, not an argument for messing with the code itself
<rogpeppe> fwereade: there are 231 calls to AddService
<fwereade> rogpeppe, how many places is AddService really used? once? twice?
<rogpeppe> fwereade: once
<rogpeppe> :-)
<fwereade> rogpeppe, ensuckening the code to accommodate shitty tests is not a win IMO
<fwereade> rogpeppe, I think the actual answer is in an AddService params struct
<mgz> jam: HEY
<mgz> er, caps
<jam> hey mgz
<mgz> mumble?
<fwereade> rogpeppe, because I don't think this will be the only change like this
<fwereade> rogpeppe, eg it's crazy that we create a service and set its config in different txns
<fwereade> mattyw, ^^ - since you have to hit them all anyway, a params struct might not be a bad idea
<mattyw> fwereade, yeah just thinking about that, most of the changes seem to be in tests that don't need to know about an owner, this would be one way of getting around that
<rogpeppe> fwereade: a params struct seems reasonable
<fwereade> mattyw, mmmmm I worry a little that we actually *do* want an owner on every service
<fwereade> mattyw, the other possibility is to replace all of those with a helper in state/testing that adds a service with an implicit user-admin owner
<mattyw> fwereade, but in the tests where we don't need to test against an owner it seems odd to pass one in
<rogpeppe> fwereade: i'm thinking along those lines
<rogpeppe> fwereade: i'm wondering about something like testing.Deploy
<rogpeppe> fwereade: which would do the AddTestingCharm, add some number of units and return the results.
<rogpeppe> fwereade: which would hopefully match reasonably well to the requirements of lots of tests
<TheMue> fwereade: you pushed the ec2 and openstack Prepare() in. shall i continue now with the others?
<fwereade> mattyw, rogpeppe: I'd really not like to use the Deploy terminology anywhere we don't have to, it's pretty much crack
<fwereade> TheMue, that would be awesome, yes please
<rogpeppe> fwereade: really?
<TheMue> fwereade: ok
<fwereade> rogpeppe, yeah -- AddService and AddUnit are sane, promoting AddService+AddUnits to a primitive of its own is just silly at the model level
<fwereade> rogpeppe, fine, it's nice at the UI level
<rogpeppe> fwereade: istm that it's one of the best know juju commands and has reasonably well known semantics, so when we see it in a test we know what's happening
<fwereade> rogpeppe, but the command itself is about the user experience, not the underlying model
<fwereade> rogpeppe, when we're testing the model I would prefer that we not mix in concepts from higher levels
<rogpeppe> fwereade: i'm interested in making our test code smaller here
<rogpeppe> fwereade: how many testing AddService calls are *not* followed by an AddUnit ?
<rogpeppe> fwereade: and how many are not preceded by an AddTestingCharm call?
<fwereade> rogpeppe, the charm case is somewhat stronger than the unit one, I think
<rogpeppe> fwereade: i sometimes toy with the idea of allowing tests to set up stuff with some kind of textual DSL
<fwereade> rogpeppe, and I would prefer that mattyw implement a small step in a helpful direction than that he get lumbered with the burden of analysing and fixing additional context 230 tims
<rogpeppe> fwereade: sure, sorry, i've diverted here
<fwereade> rogpeppe, something like that would be great, I agree
<rogpeppe> axw: how have you been reproducing lp#1235717 ?
<_mup_> Bug #1235717: null provider bootstrap fails with error about sftp scheme <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1235717>
<mattyw> fwereade, rogpeppe sorry - internet went down
<fwereade> mattyw, no worries
<mattyw> (maybe not the whole thing - but certainly my view of it)
<rogpeppe> mattyw: np - you only missed my ramblin
<rogpeppe> g
<rogpeppe> mattyw: i think the conclusion is: add a params struct
<fwereade> rogpeppe, mattyw: wait a mo
<fwereade> rogpeppe, mattyw: a params struct is, independently a somewhat good idea, but I think I'd rather see a trivial AddService helper in state/testing, that takes the same args and just sets user-admin
<rogpeppe> fwereade: seems reasonable - then we can make the call itself panic if it gets an empty owner, perhaps
<fwereade> rogpeppe, mattyw: I wouldn't say no to a params struct as well ofc
<fwereade> rogpeppe, yeah, something like that
<fwereade> rogpeppe, mattyw: we *will* need to make sure juju still works on services with empty owner fields
<fwereade> rogpeppe, mattyw: but the StateSuite(?) has references to all those collections anyway, so we can mess them up in the DB and check they still work
<fwereade> rogpeppe, mattyw: however once service owners are part of the model I think it would be foolish to allow them to remain unset in normal use
<fwereade> rogpeppe, matty: probably not a panic though, this stuff will be happening in the api server
<rogpeppe> fwereade: unless, for backward compatibility purposes, we say that blank means admin
<fwereade> rogpeppe, internally we do
<fwereade> rogpeppe, no excuse for muddying up the interface
<rogpeppe> fwereade: agree
<mattyw> fwereade, rogpeppe ok, just want to make sure I understand this right: First step is to make a helper in state/testing for addservice, this will call the addservice function as is but also set the admin user inside the service?
<rogpeppe> mattyw: i think so, yes
<mattyw> and all the calls to addservice in the tests should use this new helper? or only the ones that need the owner set?
<rogpeppe> mattyw: all the calls, i think
<rogpeppe> fwereade: looking at the null provider, it's a pity that "null" needs to be quoted in environments.yaml. i wonder if we should've gone with the "nil" provider instead...
<axw> rogpeppe: sorry, was afk
<axw> yes, I have reproduced
<rogpeppe> fwereade: do you think we should print admin-secret in the output of juju init?
<fwereade> rogpeppe, ha, we should be dropping those really, shouldn't we
<rogpeppe> axw: i wanted to reproduce for myself - was wondering how you did
<axw> 1. bootstrap. 2. stop juju processes; remove /etc/init/juju*, everything in /var/lib/juju *except* storage/tools/releases/*; bootstrap again
<rogpeppe> fwereade: well, it's potentially useful
<fwereade> rogpeppe, who for?>
<rogpeppe> fwereade: but then again there are probably other sources for info on provider settings
<axw> rogpeppe: going to make/have dinner, I'll write down the steps properly a bit later
<rogpeppe> fwereade: anyone that wants to have a known admin secret login
<rogpeppe> axw: thanks
<rogpeppe> fwereade: for example if i want to use the web GUI interface on one of my environments currently, i have to type in something like c1c4cb509044b524b35585f0fb259646 which isn't ideal
<fwereade> rogpeppe, pasting it in isn't so hard though :)
<jam> dimitern: TheMue: standup?
<jam> https://plus.google.com/hangouts/_/calendar/am9obi5tZWluZWxAY2Fub25pY2FsLmNvbQ.mf0d8r5pfb44m16v9b2n5i29ig
<mattyw> rogpeppe, fwereade does the helper want to go in state/testing or in juju/testing in the JujuConnSuite (alongside addtestingcharm for example)
<rogpeppe> mattyw: the latter
<mattyw> rogpeppe, make more sense, ok thanks
<dimitern> fwereade, rogpeppe: https://codereview.appspot.com/14486043
 * TheMue => lunch
<natefinch> so, are the dev releases of juju released to the public or are they internal only?
<jam> natefinch: dev releases are public
<jam> natefinch: 1.15.1 is in the public s3 bucket, the hp storage container, tec.
<jam> etc
<natefinch> jam: interesting.... I haven't built a windows installer for it, so I'm assuming the installer on the website is still 1.14
<dimitern> natefinch, that's still the "stable" release
<natefinch> dimitern: ahh, ok, so the dev releases are public, but don't replace the latest stable release
<dimitern> natefinch, I think so yeah
<natefinch> dimitern: basically wanted to make sure I wasn't falling down on the job by not creating an installer for 1.15.
<dimitern> natefinch, I think there still should be an installer for every release, even dev ones
<dimitern> jam, fwereade ? ^^
<natefinch> dimitern: that would seem to make sense.
<jam> natefinch: we would still want a 1.15 installer, otherwise we can't test that it is suitable for being a stable release.
<jam> ideally it would just be part of the release process
<jam> but the need for a Windows machine makes it slightly tricky
<jam> natefinch: if you could easily set it up with an automated "ec2" sort of script, then someone like curtis could kick it off after cutting the tarball
<natefinch> jam: You mean a script to create the installer?  I certainly could, but I don't know about "easily" :)  It's sort of an 8 step process if you 're starting from a bare machine and need to build both the juju client and the installer from it.
<jam> natefinch: so the idea would be you *could* do a Windows ec2 instance so that anyone on the team could have access to it.
<jam> It would be possible to bundle the final image
<jam> so you don't have to do all the setup yet-again
<jam> I used that in the past for other projects and it was ~ok, not great, but reasonable
<natefinch> jam: ahh, yeah that would certainly be easier than scripting the install of Go and all that :)
<natefinch> jam: dang, I was thinking I should just write a charm to do it... but of course that won't work on windows :)
<rogpeppe> hmm,
<rogpeppe> axw: one problem is: tools.WriteMetadata actually ignores any metadata that it finds that corresponds with tools already in the target storage
<rogpeppe> axw: if we fixed that, the problem would disappear for almost all the time
<rogpeppe> axw: and make sync tools faster too
<dimitern> rogpeppe, fwereade, review poke
<rogpeppe> dimitern: sorry, looking
<axw> rogpeppe: hmm yeah, I guess it should check if size/sha256 are initialised or not, and use that to decide to blat over the top
<rogpeppe> axw: it only appends to toolsList
<dimitern> axw, size/sha256 are never initialized if you upgrade from 1.14 btw
<rogpeppe> axw: whereas it actually want to update it if there are some entries in existingMetadata that are also in toolsList already
<rogpeppe> s/want/wants/
<rogpeppe> axw: "Merge in existing records" is a bit of an overstatement
<rogpeppe> axw: i wonder if things might work fine if we just ignored the List entirely.
<rogpeppe> axw: and just merged the metadata
<axw> rogpeppe: I don't really get it. In the scenario I've been investigating, len(existingMetadata) == 0
<rogpeppe> axw: really? interesting.
<axw> rogpeppe: i.e. the tools tarballs are in storage, but no the metadata json
<axw> not*
<rogpeppe> axw: i tried a scenario where there *is* existing metadata, but we still have the same problem
<axw> ah
<axw> rogpeppe: can you please describe it?
<rogpeppe> axw: i bootstrapped with the null provider (to localhost)
<rogpeppe> axw: (the bootstrap failed because i'm already running the local provider, but it doesn't matter in this case)
<rogpeppe> axw: then i tried bootstrapping again, and saw the "sftp:" problem
<rogpeppe> axw: and i can verify that the metadata is all there
<axw> ah ok
<axw> rogpeppe: that's probably a more plausible explanation for what Bjorn reported :)
<rogpeppe> axw: and looking at WriteMetadata, i can't see how any hash/size metadata for existing tools in the target storage would get into the metadata without re-reading them
<rogpeppe> axw: because the toolsList argument won't have that metadata and the "Merge in existing records" loop doesn't change any entries in that argument
<rogpeppe> axw: yes - it'll happen every time
<axw> rogpeppe: ahh, because it's appending, not replacing...
<rogpeppe> axw: so i'm wondering whether a better approach is just to ignore tools in the target bucket if they don't have metadata
<rogpeppe> axw: yes
<axw> ok
<axw> I understand now, thanks
<rogpeppe> axw: it appends only if it doesn't find any existing entries
<rogpeppe> axw: the only down side i can currently see about ignoring tools with no metadata is that we may redo work, re-copying tools that were already copied
<rogpeppe> axw: which will happen if the upload process was aborted half way through
<hazmat> g'morning (relatively) i think the  recent simplestreams/tools rejigger broke what used to be a  correct stackoverflow answer that's still being referenced/used, if anyone has a minute it would be worth correcting http://askubuntu.com/questions/327177/cannot-bootstrap-due-to-precise-images-in-regionone-with-arches-amd64-i386
<rogpeppe> dimitern: reviewed
<dimitern> rogpeppe, cheers
<dimitern> fwereade, poke
<dimitern> rogpeppe, I wouldn't mix tags into state - they're an api concept
<axw> rogpeppe: seems to me that SyncTools should be calling WriteMetadata with only "missing", not targetTools+missing
<axw> and whatever's where already should be assumed to be correct already
<axw> s/where/there/
<axw> i.e. only update metadata for things that are copied
<rogpeppe> dimitern: Tag is implemented on State objects
<dimitern> rogpeppe, but not used otherwise for queries and such
<rogpeppe> dimitern: does that matter here? this an error message for users
<rogpeppe> dimitern: i guess users don't really see tags either though
<dimitern> rogpeppe, well
<dimitern> rogpeppe, yeah I suppose it'll reduce the code a bit
<rogpeppe> dimitern: yeah, i think just a simple list of agents would work just as well
<rogpeppe> axw: i've been wondering about the "abort half way through problem" and perhaps this is a stupid idea, but what if we took the metadata as primary, and always copied that *first*?
<rogpeppe> axw: and avoid the use of Storage.List entirely
<rogpeppe> axw: well, we couldn't avoid it entirely
<rogpeppe> axw: but we'd only use it to find tools that the metadata mentions that don't show up in the list
 * dimitern => lunch
<rogpeppe> axw: if we do things that way, the first thing SyncTools would do would be to merge the metadata
<rogpeppe> axw: and then it would copy across the actual tools data as necessary
<axw> rogpeppe: if it then fails while copying tools, we're going to have a bad time, no?
<axw> next time we do a sync
<axw> it'll think the target has everything
<rogpeppe> axw: no, because we make sync actually check the List
<rogpeppe> axw: at least, i *think* it could work ok
<rogpeppe> axw: it'll potentially be a problem if we *haven't* done the sync
<axw> rogpeppe: it sounds viable to me. I'd probably want to talk to wallyworld before doing it tho
<rogpeppe> axw: for the time being, i think we should just ignore the target tools list
<rogpeppe> axw: i'm really not very keen on the localhost storage wrapper solution
<axw> rogpeppe: I'm not keen on the fact that URL() is there at all
<rogpeppe> axw: it's there because we need it
<rogpeppe> axw: it's our way of telling a remote cloud-init shell script how to get the juju tools
<axw> rogpeppe: yeah I know the general use case, but I mean, it doesn't make sense for sshstorage
<axw> since it's transient
<rogpeppe> axw: perhaps you could make the method return an error?
<axw> so I suppose you're right
<axw> yeah
<axw> it should return an error, and we should change this stuff to never call it
<rogpeppe> axw: yeah
<rogpeppe> fwereade: do you know if there's still a case where we do any kind of "best-fit" matching to the available tools? even juju bootstrap now picks an exact match, no?
<axw> rogpeppe: by "ignore the target tools list", do you mean only update metadata for copied tools?
<rogpeppe> axw: i mean that we make our copy decisions based on the source and target metadata only
<rogpeppe> s/we make/we should make/
<rogpeppe> axw: that way we ensure that we always have the hash and size metadata
<fwereade> rogpeppe, I don't think there's any ore guessing,no
<rogpeppe> fwereade: eh?
<rogpeppe> fwereade: ah, i see
<rogpeppe> more
<fwereade> rogpeppe, ah, yeah, sorry
<rogpeppe> fwereade: i'm just pondering the implications of the change i was discussing with axw above
<axw> rogpeppe: I'm clearly missing something (possibly wine related). I think we should just make this change: in SyncTools, in the call to WriteMetadata, pass "missing" instead of "targetTools"
<rogpeppe> fwereade: to copy metadata *first*, then the tools themselves
<fwereade> rogpeppe, that sounds bad?
<axw> rogpeppe: because WriteMetadata seems to expect toolsList to contain only changed tools
<axw> rogpeppe: as in, "newToolsVersions" being one-to-one with toolsList
<rogpeppe> axw: looking again
<fwereade> rogpeppe, offhand it seems that we'd be able to read metadata for missing tools if we did that
<rogpeppe> fwereade: we would, but i'm not *sure* that's a bad thing
<rogpeppe> fwereade: i'll just look for axw, back in a bit
 * fwereade is becoming vexed by these blasted daywalking mosquitos
<rogpeppe> axw: i don't think that works
<rogpeppe> axw: because missing is derived from Storage.List, and hence contains no metadata
<rogpeppe> axw: i mean, no hash/size metadata
<axw> rogpeppe: no, missing is copied, and the copy generates the size/hash
<rogpeppe> axw: oh, that's sneaky and underdocumented :-)
<axw> ;)
<rogpeppe> axw: hmm, that *might* have the effect i'm after; still thinking
<rogpeppe> axw: it means the argument to WriteMetadata now means something quite different
<axw> rogpeppe: I'm not so sure - WriteMetadata seems to treat it as "changed" tools
<axw> I think the name is just a bit generic
<rogpeppe> axw: "The metadata from toolsList is already present"
<rogpeppe> axw: hmm, but i guess it is present *now*
<rogpeppe> axw: um, no
<rogpeppe> axw: it's *some* of what's present now
<axw> rogpeppe: yeah, some is present, some is new
<rogpeppe> axw: it's what we've just copied, but none of what was there originally
<axw> but going down the function, it then treats the whole lot as being new?
<axw> rogpeppe: it currently includes what's there already
<axw> SyncTools calls WriteMetadata(targetTools+missing)
<rogpeppe> axw: yes - and that's very deliberate
<rogpeppe> axw: the logic is trying to create a metadata file that includes both what we've just copied *and* what was in the bucket before
<rogpeppe> axw: and our problem is that it can't know what was in the bucket before without reading it
<rogpeppe> axw: rather, it can't know the metadata for previous bucket contents without reading them
<rogpeppe> axw: heh, istm that WriteMetadata could pass metadataStore to generateMetadata and then we wouldn't have any of this problem
<rogpeppe> axw: you said earlier that we don't know the tools storage name given the Tools struct
<rogpeppe> axw: but of course that's not true - we do!
<axw> rogpeppe: assuming the URLs are all relative to the same tools dir we do
<rogpeppe> axw: we don't need to use the URL
<axw> ah
<axw> right
<axw> it's statically defined
<axw> :)
<rogpeppe> axw: it's tools.StorageName(tools.Version)
<axw> StorageName
<axw> forgot about that
<axw> rogpeppe: I'm still convinced that WriteMetadata should only be called with missing. "The metadata from toolsList is already present" means, to me, size/sha256 is precomputed for the tools in the list
<axw> which is clearly not the case for the existing tools
<rogpeppe> axw: and we should probably change sync.copyTools to avoid using URL too
<rogpeppe> axw: if that's the case, then the current logic ignores any tools that are present before sync-tools is called, AFAICS
<rogpeppe> axw: the question is "already present"... where?
<axw> rogpeppe: howso? newToolsVersions will only contain things that were copied; existingMetadata may contain additional things
<rogpeppe> axw: i'm thinking about the case where we've aborted a bootstrap half way through
<axw> rogpeppe: that currently won't copy over again on the second sync, right?
<rogpeppe> axw: (and i've also just realised why it's a really bad idea to re-read the target storage to work out its hash and size, but that's another story)
<rogpeppe> axw: currently it won't copy over again on the second sync, but it *will* generate metadata for the files it copied across last time
<axw> yeah. if "missing" were passed through, that'll still happen due to the "fetch"
<rogpeppe> axw: really?
 * rogpeppe looks again
<rogpeppe> axw: istm that fetch will only affect stuff that's in toolsList already
<axw> rogpeppe: but toolsList is appended to
<axw> with things in existingMetadata
<rogpeppe> axw: yes, but i was talking about tools that do not currently exist in the target storage metadata
<rogpeppe> axw: because we copied the tools but didn't get around to updating the metadata
<rogpeppe> axw: and currently we *will* generate metadata for those tools, even though they don't have metadata, i think
<axw> rogpeppe: yes, we do
<axw> rogpeppe: in the copy
<rogpeppe> axw: sorry, we do what?
<axw> rogpeppe: we do generate metadata for them
<axw> as you said
<rogpeppe> axw: and changing to use missing will mean that we don't, right?
<axw> rogpeppe: it will if it's not in the target storage's metadata
<rogpeppe> axw: really?
<axw> ah wait
<axw> no
<axw> I get it now
 * axw sighs
<rogpeppe> axw: i think part of the difficulty with analysing this is that WriteMetadata mixes concerns quite a bit
<rogpeppe> axw: did you just lose contact?
<axw_> rogpeppe: I think, instead, we should change newToolsVersions to map[string]*tools.Tools, and store each item from toolsList in that
<axw_> yeah
<rogpeppe> axw_: last thing i saw from you was "axw sighs"
<axw_> rogpeppe: then replace any item in it if Size==0
<axw_> <rogpeppe> axw: i think part of the difficulty with analysing this is that WriteMetadata mixes concerns quite a bit
<axw_> <axw> rogpeppe: and the wine
<axw_> <axw> but yes
<rogpeppe> lol
<axw> rogpeppe: something like this: http://paste.ubuntu.com/6205035/
<rogpeppe> axw: BTW it's fine to use a version.Binary as a map key
<rogpeppe> axw: (i know the original didn't, but just saying)
<axw> rogpeppe: thanks, will keep it in mind
<rogpeppe> axw: that looks plausible, but i had in mind a slightly more radical solution; one mo
<rogpeppe> axw: something like this (as primitives) http://paste.ubuntu.com/6205057/
<rogpeppe> axw: then most of the awkward logic in WriteMetadata could be abstracted out to where it belongs, in SyncTools
<rogpeppe> axw: because there is no reason AFAICS why WriteMetadata should be so bound up with merging stuff
<axw> rogpeppe: that sounds like a good idea to me
<axw> it's a little complicated at the moment
<rogpeppe> axw: here's a version with some doc comments, to give a slightly better idea of what i'm thinking of: http://paste.ubuntu.com/6205080/
<rogpeppe> axw: FromList could probably be MetadataFromList
<natefinch> jam, rogpeppe, mgz, fwereade, dimitern, TheRealMue: anyone up for an easy one?  https://codereview.appspot.com/14326044/
<rogpeppe> natefinch: looking
<natefinch> arosales: ping
<rogpeppe> axw: would you mind going forward with that? i know it's a serious diversion from your original fix :-)
<axw> rogpeppe: yep, I'm happy with that
<axw> getting pretty late now, I'll pick it up in the morning
<rogpeppe> fwereade: i'd appreciate it if you took a glance at my above paste, as applied to tools.WriteMetadata
<rogpeppe> axw: thanks
<rogpeppe> axw: it was useful you were able to stay on
<axw> rogpeppe: thanks for enduring my nonsense :)
<rogpeppe> axw: not at all
<rogpeppe> axw: i was nonsensical too
<arosales> natefinch, pong
<axw> alrighty, I'm off to prod llgo a bit. ttyl
<natefinch> arosales: it has come to my attention that's there.s a 1.15.1 release for Juju but no one has asked me for a windows installer for it... that seems like something we should fix
<natefinch> rogpeppe: I actually should update dependencies.tsv as well, since that change to ec2 requires a change to goamz (which is already in goamz)
<rogpeppe> natefinch: you should :-)
<natefinch> rogpeppe: done, committed, proposed
<arosales> natefinch, I think we are only doing osx and windows installs for stable releases
<arosales> s/installs/clients/
<natefinch> arosales: as jam pointed out to me this morning, we can't know if it's stable if we don't test the dev release :)
<arosales> natefinch, good point :-)
<arosales> natefinch, could you sync with sinzui on the process for creating windows clients and testing them?
<natefinch> arosales: sure thing
<arosales> natefinch, thanks.
<sinzui> natefinch, we can talk later today. I know how you build it.
<natefinch> sinzui: sure thing.  Just wanted to make sure it didn't fall through the cracks, especially given we've occasionally had windows-only problems.
<arosales> natefinch, sinzui I expect it to be a little ad hoc atm until sinzui's team can get some time to build out the testing matrix with tooling.
<rogpeppe> natefinch: reviewed
<sinzui> arosales, I got Lion + xcode 5 + 1.14.0 to install on my mac, but I cannot get 1.14.1 to install. I do reproduce the same error seen in my untested pull request. We need to look at vars.go and learn what changed between 1.14.0 and 1.14.1 I also confirm 1.15.1 fails the same way for me: https://github.com/mxcl/homebrew/pull/22762
<sinzui> natefinch, I did test azure deploys and upgrades. That is why the release went out at 3AM my time
<arosales> sinzui, thanks for working on  osx.  are you able to build locally?
<arosales> juju-core devs any hint on the error, "src/launchpad.net/juju-core/juju/osenv/vars.go:40: undefined: Home"
<arosales> when building juju-core
<sinzui> arosales, 1.14.0 can. 1.14.1+ cannot.
<arosales> sinzui, same error when building locally?
<fwereade> rogpeppe, sorry, is that paste just about primitives with which to rewrite WriteMetadata?
<mgz> arosales: you need vars_linux.go in that directory?
<rogpeppe> fwereade: yes, pretty much
<rogpeppe> fwereade: and in the process fix a current problem with it
<mgz> arosales: should have a func Home() in it
<rogpeppe> fwereade: (as well as the fact that it's very hard to reason about currently, because it does about 4 different things at once)
<fwereade> rogpeppe, yeah, I am having that problem as I try to parse exactly how they'd be used
<fwereade> rogpeppe, they do look sensible at first blush though :)
<rogpeppe> fwereade: they'd be called from SyncTools, which would use MergeMetadata as appropriate
<rogpeppe> fwereade: at the moment a significant portion of WriteMetadata is really about syncing tools
<fwereade> rogpeppe, that sounds like a good way to go offhand
<rogpeppe> fwereade: thanks. i think it's preferable to papering over the problem, cf. https://codereview.appspot.com/14465045/
<fwereade> rogpeppe, heh, the description there surely raises an eyebrow or two
<fwereade> rogpeppe, don't we have a filesystem storage implementation anyway if we're just on localhost?
<rogpeppe> fwereade: indeed
<rogpeppe> fwereade: my brain hurts
<rogpeppe> fwereade: erm, no we don't have a filesystem storage implementation for the bootstrap storage on the remote machine
<rogpeppe> fwereade: that CL uses a local HTTP listener to talk to the remote storage via ssh
<fwereade> rogpeppe, heh, so presumably we don't have one for the local provider either?
<rogpeppe> fwereade: i think we do
<fwereade> rogpeppe, we should probably be using that where it makes sense then tbh
<rogpeppe> fwereade: this is only an issue when bootstrapping the null provider, i think
<fwereade> rogpeppe, hmm, I could have sworn we added file:// url handling for exactly that case
<rogpeppe> fwereade: sorry, not quite sure where you're coming from here
<rogpeppe> fwereade: how would that help when the files we're referring to are the other end of an ssh connection with no juju binaries at the other end?
<rogpeppe> s/are the/are at the/
<mattyw> fwereade, I'm wondering If I should track the juju id stuff as a blueprint to help us keep track of it?
<mattyw> (sorry to interupt)
<fwereade> rogpeppe, I was just thinking of what happens purely at the far end -- so we do surely need the SSH storage to write to it, but unless it's actively *easier* we should probably just be using the filesystem to get them when we're actually running on the remote machine
<rogpeppe> fwereade: that's what we do, i think
<fwereade> mattyw, just a sec let me have a quick look, I couldhave sworn there wasone already
<fwereade> rogpeppe, ok, then I completely misunderstood something
<rogpeppe> fwereade: the URL in this case is only used locally as a result of WriteMetadata and SyncTools being a bit crackful
<rogpeppe> fwereade: my suggestion is to make SSHStorage.URL return an error
<rogpeppe> fwereade: (to make it clear that there really is no URL)
<rogpeppe> fwereade: and to change the SyncTools logic so it never uses the storage URL (it really doesn't need to)
<fwereade> rogpeppe, ah, ok, if getting URLs out of SyncTools solves the problem I'm 100% behind that
<rogpeppe> fwereade: i'm pretty sure it does, yes
<fwereade> rogpeppe, and indeed it seems there really is no sensible URL
<fwereade> rogpeppe, great, thanks
<rogpeppe> fwereade: hence the refactoring suggestion above. axw had a suggestion for tweaking the existing logic, but i really would prefer to clean things up in this area
<rogpeppe> fwereade: as it's taken both of us about a day to understand the problem, and i know that i won't understand the code when i go back to look at it again tomorrow
<fwereade> rogpeppe, that sounds smart to me -- I guess axw will be looking into it tonight?
<rogpeppe> fwereade: yes, he said he'd take it forward
<sinzui> Does anyone have a  1.14.1 and 1.15.1+ setup that they can test upgrade-juju on openstack. My testing of this always fails
<arosales> mgz, thanks for the suggestion. Is this a manual add or something we are missing in out builds (re vars_linux.go)
<mgz> no. I'm just trying to help you diagnose why your tree is broken
<jamespage> sinzui, I'm trying to test that; but I'm getting confused about how 'juju-tools' endpoint should be configured in our keystone instance
<jamespage> 1.14.1 works with a juju-tools URL "http://10.98.191.34:8080/v1/AUTH_79699f6f71e245b186720f1e2bc03cf0"
<jamespage> 1.15.1 does not like that
<sinzui> jamespage, I tried tools-url: https://swift.canonistack.canonical.com/v1/AUTH_526ad877f3e3464589dc1145dfeaac60/juju-dist/tools
<sinzui> ^ that is per wallyworld.
<sinzui> I can see the released tools are found on canonistack and hp by appending juju-dist/tools to the public-bucket url
<jamespage> hmm
<natefinch> mgz, arosales: about the broken code.... he's talking about building on OSX - which I think we don't have an implementation of Home for
<jamespage> sinzui, nope
<arosales> mgz, ah gotcha
<jamespage> juju fails to find them and tries to revert back to s3
<jamespage> which is not accesible in my env
<mgz> natefinch: seems unlikely, unless there's some weird build stuff, the logic is just if "windows" else linux
<arosales> natefinch, mgz, I thought sinzui said he also saw the error locally . .  .
<arosales> mgz, natefinch I would have to defer to sinzui on the error, but sinzui may be busy atm with juju-tools
<natefinch> mgz: There's a vars_windows.go and a vars_linux.go .... there's no var_darwin.go
<mgz> there's no reason it should just use linux though
<natefinch> mgz: the implementation is the same, yes.  It's a bug in the code
<arosales> mgz, sorry for losing the context
<arosales> mgz, here is the background https://github.com/mxcl/homebrew/pull/22762
<arosales> mgz, roughly trying to build on osx via brew.
<mgz> seems like that just needs fixing then
<arosales> natefinch, do you see a similar error for windows client builds?
<sinzui> jamespage, Does this look like a sane listing? I think it is. https://pastebin.canonical.com/98610/
<mgz> easy enough for anyone with a mac to test on
<natefinch> arosales: no, it's OSX-only
<arosales> huh that is interesting
<jamespage> sinzui, yeah - thats what mine looks like
<natefinch> arosales: it's a very clear cut problem... we have code for this Home function that compile sfor Windows and for OSX, but we forgot about OSX
<arosales> natefinch, ah
<natefinch> arosales: where we == me
<arosales> :-)
<sinzui> jamespage, I places the tools there with "juju sync-tools -e public-tools-canonistack --dev"
<arosales> natefinch, mgz sinzui or myself can test on osx if you have a something you would like us to try
<sinzui> jamespage, maybe we need a trailing / in the tools url?
<natefinch> arosales: I can make a fix... should I make it based on 1.14?  Or should I just make it based on 1.15 and we can make sure it gets into 1.16?
<arosales> natefinch, my preference would be 1.15.1 based so we can target 1.16
<natefinch> arosales: mine too :)
 * arosales guesses it would need to fit trunk and then also be applied to the 1.15.1 branch
<sinzui> mgz, sorry, just caught up to want you are looking at . I just annotated the module. I can test, but I need to reboot :( The change does explain why 1.14.1+ is broken, though why osx/darwin/bsd/posix doesn't behave like gnu/posix is puzzling
<arosales> sinzui, I think natefinch is going to work on a fix
<natefinch> sinzui: it's just a build tag on one of the files that says "compile this file only for linux" when it should have said "compile this file for any OS that isn't Windows"
<sinzui> natefinch, fab, I am schooled.
<natefinch> sinzui: I managed to add support for windows while simultaneously effectively removing support for OSX ;)
<natefinch> mgz: is there magic I need to do to rename a file so bzr keeps the revision history?
<sinzui> natefinch, I suspected that since win is the main difference between 1.14.0 and 1.14.1.
<mgz> natefinch: `bzr mv`?
<natefinch> mgz: ahh of course
<natefinch> mgz: https://codereview.appspot.com/14496043
<mgz> looking
<mgz> what is that magic...
<mgz> natefinch: so, why was it failing, given the two files didn't have build constraints before
<natefinch> mgz: the _<os>.go is effectively a build tag... so before we had _linux and _windows
<natefinch> mgz: now I replaces the _linux one with an in-file build tag that specifies !windows
<mgz> that's all to magic
<mgz> I am not liking the build package docs I'm reading...
<mgz> *too
<natefinch> mgz: standard go way of doing build tags ... I actually really like the _<os>.go because it makes it very clear from just the name of the file where it'll build
<natefinch> mgz: like _test.go
<natefinch> natefinch: I sorta wish _!windows.go  would work
<mgz> ugh
<natefinch> mgz: ^^ heh, tagging myself
<natefinch> mgz: not really... but almost
<natefinch> mgz: it would also be more clear if rietveld made the name change more obvious
<mgz> the launchpad diff is clear enough
<mgz> (I checked that)
<natefinch> mgz: yeah, that one's much better in this case
<sinzui> rogpeppe, Is the status of this issue correct? Are we committing to fix it by 1.16.0? I think the landing implies it is no longer critical https://bugs.launchpad.net/juju-core/+bug/1233278
<_mup_> Bug #1233278: juju test suite is not isolated <juju-core:In Progress by rogpeppe> <https://launchpad.net/bugs/1233278>
<rogpeppe> sinzui: i don't think we're committing to fix it by 1.16 - it no longer seems to be causing quite so many problems
<rogpeppe> sinzui: i think we can lower the pri to High, but please check with fwereade
<sinzui> ^ fwereade what say you? ^
<mattyw> rogpeppe, fwereade I'm nowhere near done but is this the kind of thing you guys had in mind for the addservice helper? https://code.launchpad.net/~mattyw/juju-core/add-owner-to-service/+merge/189654
<rogpeppe> mattyw: yeah, looks reasonable. i *think* i'd be happier with the owner last in the params, as it seems the least important thing, but that's all
<mattyw> rogpeppe, in the actual call to addservice?
<rogpeppe> mattyw: yeah
<mattyw> rogpeppe, probably a good idea, I've just shoved it there at the moment so I can get the compiler to generate a list of things I need to fix :)
<rogpeppe> mattyw: i think it would do that anyway
<rogpeppe> mattyw: as you're adding an extra arg
<mattyw> rogpeppe, I just meant I just put it anywhere in the list without thinking about order
<rogpeppe> mattyw: ah, no probs
<jamespage> sinzui, http://paste.ubuntu.com/6205614/
<jamespage> I can't get my tools to pickup using tools-url
<jamespage> sinzui, I can't get the tools-url to override anything afaict
<sinzui> jamespage, thank you for confirming it sin't just me. I reported a bug and will continue to look for a configuration that is accepted
<jamespage> sinzui, the other problem is that a keystone juju-tools entry that was compat with 1.14.1 is not compat with 1.15.1
<jamespage> and vica versa
<sinzui> jamespage, When I tested this *before* release. I uploaded tools to a testing area. There were no aws tools at the time. I could deploy using the deb I built. You can still see my testing files at https://region-a.geo-1.objects.hpcloudsvc.com/v1/60502529753910/juju-dist
<jamespage> sinzui, I'm wondering if because there is a juju-tools in my keystone catalog, the local client won't use the tools-url in the environments.yaml
 * sinzui ponders :443 as a factor
<jamespage> sinzui, yeah - I don't use https for swift for this openstack deployment
<jamespage> sinzui, actually my streams data looks like fud one second
<jamespage> sinzui, hmm
<jamespage> sinzui, well thats interesting
<jamespage> sinzui, I juju destroy-environment'ed again
<jamespage> and it was OK
<jamespage> odd
<sinzui> jamespage, you can upgrade-juju on an openstack?
<jamespage> sinzui, I'll have to rollback to confirm that - one second
<jamespage> sinzui, I can upgrade a bootstrap node
<jamespage> my bug from the other day is back where the bootstrap node does not start any more units...
<jamespage> still don't know how exactly that resolved itself
<sinzui> jamespage, I have seen just the bootstrap node upgrade. The agents failed though.
 * sinzui looks for log and updated bug about that
<natefinch> fwereade: care to talk about your comments on my rootdisk branch?
<natefinch> dimitern, rogpeppe, mgz, fwereade: just got what looks to be a sporadic test failure on the bot - FilterSuite.TestUnitRemoval  from inside the uniter
<mgz> natefinch: can you file a bug and requeue?
<natefinch> will do
<dimitern> natefinch, I think I've seen that before, please make sure there's no existing bug about it
<natefinch> dimitern: I saw another bug about a sporadic failure in uniter, but the tests and logs seemed different, so I filed a new bug.
<dimitern> natefinch, ok, thanks
<natefinch> Anyone know if we're allowed to expense parking fees at the airport for travel to the sprint?
<rogpeppe> fwereade, natefinch, dimitern: https://codereview.appspot.com/14395043
<rogpeppe> natefinch: sorry, i don't know
<rogpeppe> that's me for the night
<rogpeppe> g'night all
<natefinch> rogpeppe: I'll take a look.  g'night
<rogpeppe> natefinch: thanks
<fwereade> natefinch, heyhey
<natefinch> fwereade: howdy
<fwereade> natefinch, about root-disk
<fwereade> natefinch, it's not a problem with your branch per se, just a problem I noticed while I was looking at it ;)
<natefinch> fwereade: I gathered as much. I'm fine with fixing it, was just unclear what the fix was
<fwereade> natefinch, I *think* it's just to drop the fields that aren't always set, and move the hardwareCharacteristics method out into a func that acceptsinstanceType, rootDisk, (more?)
<fwereade> natefinch, just so it doesn't look like hardwareCharacteristics is a meaningful operation on ec2Instance
<fwereade> natefinch, the only place we create them is in StartInstance anyway, so it shouldn't be onerous
 * fwereade waits to hear what he's missed :)
<fwereade> natefinch, am I blithering?
<natefinch> fwereade: ha, I missed the fact that hardwareCharacteristics is only called from inside StartInstance. You're right, it's crack to store that data on the ec2Instance only to return it one line down in the same function
<fwereade> natefinch, I think it's a hangover from when we thought it would be reasonable to ask an instance for the details of its hardware, but it's fiddlier than one might hoe
<fwereade> er, hope
<natefinch> fwereade: totally makes sense.
<natefinch> fwereade: well that's an easy fix
<fwereade> natefinch, cool, consider that LGTM with the change then
<natefinch> fwereade: cool
<fwereade> natefinch, oh, a thought
<fwereade> natefinch, possibly we should keep the existing minimum size in play?
<natefinch> fwereade: like don't let people set disk less than 8G?
<fwereade> natefinch, I'm in no way wedded to the idea, but I'm a bit worried people might use =0 as shorthand for "I don't really care" without meaning "I *really* don;t care"
<fwereade> iyswim
<fwereade> natefinch, we guarantee we'll do at least as well as they ask
<fwereade> natefinch, so an 8G result does match anything less than that
<fwereade> natefinch, but actively giving them much less might cause avoidable problems
<natefinch> fwereade: I can see someone wanting to keep their root disk as small as possible and then using the spare for a different partition (or something... ).. The image itself only uses like 1.5G or so from what I've seen
<fwereade> natefinch, OTOH, if someone *does* ask for 4G they probably mean it
<fwereade> indeed
<natefinch> fwereade: I think treating 0 as 8G is fine
<fwereade> natefinch, great, sgtm
<fwereade> thanks
<natefinch> fwereade: but if they ask for 1G... we should do our best and fall over if that turns out to be impossible
<natefinch> fwereade: cool
<fwereade> +1
 * fwereade is happy about the new place... it has a CRT TV,and it emerges that my dreamcast still works, as does the lightgun
<natefinch> haha awesome.  Loved the dreamcast :)
<fwereade> house of the dead 2 remains flat-out awesome :)
<fwereade> sometime I will see if MSR really is better than any of the project gotham games
<fwereade> I'm a bit afraid in case it isn't
<natefinch> I don't think I ever played it.  Played one of the gotham ones... forget which
<fwereade> basically the same deal
<fwereade> except they left the traffic islands in where they took them out of the gotham games
<natefinch> huh funny
<fwereade> I thought they were great, wasn't the same without them
<natefinch> we played a ton of NHL2k*, NFL2k* and soul calibur
<natefinch> which is sorta funny, since I don't even particularly like watching sports
<fwereade> natefinch, the only sports game I ever really got into was... nhlpa on the megadrive maybe?
<natefinch> fwereade: nice.
<natefinch>  fwereade: that was pretty much the last time I played sports games.  American Football is surprisingly interesting to play, though in video games it often comes down to "give the ball to the guy who can run faster than anyone else"
<natefinch> fwereade: though probably games have come a long way in the past decade ;)
<fwereade> natefinch, heh, I am always a little bit embarrassed that I've never remotely figured out american football
<fwereade> natefinch, probably just takes the right videogame though
<natefinch> fwereade: there's a ton to it, but most of it is "Get the ball past the other line of guys and dance around in their home base"
<fwereade> natefinch, clear goals are important ;)
<natefinch> fwereade: I'm getting a problem with livetests.go .. can't tell if it's my fault or something external:
<natefinch> [LOG] 85.45774 DEBUG juju.environs.simplestreams fetchData failed for "http://127.0.0.1:56349/test-bucket/tools/streams/v1/index.sjson?AWSAccessKeyId=x&Expires=1696711806&Signature=6T43LEGnZYvE04zTygifGAi5uxA%3D": The specified key does not exist.
<natefinch> [LOG] 85.45776 DEBUG juju.environs.simplestreams cannot load index "http://127.0.0.1:56349/test-bucket/tools/streams/v1/index.sjson?AWSAccessKeyId=x&Expires=1696711806&Signature=6T43LEGnZYvE04zTygifGAi5uxA%3D": invalid URL "http://127.0.0.1:56349/test-bucket/tools/streams/v1/index.sjson?AWSAccessKeyId=x&Expires=1696711806&Signature=6T43LEGnZYvE04zTygifGAi5uxA%3D" not found
<natefinch> /home/nate/code/src/launchpad.net/juju-core/environs/jujutest/livetests.go:874:
<natefinch>     c.Assert(err, gc.ErrorMatches, ".*missing machine nonce")
<natefinch> ... error string = "no \"raring\" images in test with arches [amd64]"
<natefinch> ... regex string = ".*missing machine nonce"
<natefinch> OOPS: 36 passed, 6 skipped, 1 FAILED
<natefinch> --- FAIL: TestEC2 (8.01 seconds)
<fwereade> natefinch, heh, I have a horrible feeling that those were a casualty of the simplestreams change... are they all doing UploadFakeTools?
<fwereade> natefinch, except waitno
<fwereade> natefinch, they still ought to "work" with fake tools, they'll just go wrong on the instances
<fwereade> natefinch, regardless I'm a bit surprised it's looking for raring... does the test specify that explicitly?
<natefinch> fwereade: trunk works, must be something my change broke
<fwereade> natefinch, ok, so,  those log messages shouldn't be the problem
<fwereade> natefinch, we don't expect .sjson in this context
<fwereade> natefinch, yeah, some confirmation: ec2.TestImagesData does indeed lack raring-amd64
<fwereade> natefinch, but I am growing more and more confused about where raring is coming from
<natefinch> fwereade: merging code in from trunk fixes the problem
<fwereade> natefinch, ah cool :)
<fwereade> natefinch, bbs
<natefinch> fwereade: interesting: ./juju bootstrap --constraints root-disk=4G
<natefinch> ERROR cannot start bootstrap instance: cannot run instances: Volume of size 4GB is smaller than  snapshot 'snap-5c0cb05f', expect size >= 8GB (InvalidBlockDeviceMapping)
<fwereade> natefinch, ha
<natefinch> fwereade: I guess that answers that
 * fwereade pretends he was all foresighted and clever earlier
<natefinch> haha
<adam_g> hi.. the release notes for 1.15.0.1 WRT logging are a bit unclear.   where is the correct place to look for logs on a unit now?  or are hooks no longer logged locally if no log config has been set pre-deploy?
<natefinch> adam_g: you're on at an awkward time for Juju devs. Our New Zealand guy who is usually on now is on vacation this week. I'm about to head out, and I'm not sure about the answer to your question.  There's a couple Australian guys that should come on in the next few hours to answer your question - axw and davecheney
<natefinch> adam_g: fwereade is european time, so it's pretty late for him, but he might be able to answer that question... he was afk last I knew, though
<adam_g> natefinch, thanks. found this: https://lists.ubuntu.com/archives/juju/2013-September/002998.html which explained it well. might be worth linking that from any release notes out there
<natefinch> adam_g: ahh, cool
<natefinch> adam_g: heh, that's from the New Zealand guy who is on vacation.  I must have missed that email.
<adam_g> right
<adam_g> me too :)
<natefinch> Gotta run, dinner's ready.  Glad you found the answer to the question.
<bigjools> morning!
<davecheney> yay end of daylight savings
<davecheney> now we get a sensible crossover with the US
#juju-dev 2013-10-08
<bradm> davecheney: had a chance to look at those charm reviews of mine?  anything I need to do to help with it?
<davecheney> bradm: i'm still waiting for someone to take my charm training wheels off
<davecheney> i can review but not commit
<davecheney> i will re ping everyone, about everything
<bradm> davecheney: thanks
<bradm> davecheney: I figure my 2 squid charm merges will be good ones, since they're so trivial - I can understand my moin merge taking a while, its a tad larger
<axw> hey wallyworld, have a good break?
<wallyworld> axw: not bad :-) no internet for a week and a busted laptop so i've only got back online today. seems like there were some hiccups with the release
<axw> wallyworld: there were a few very last minute bugs
<axw> but mostly okay I think.
<axw> wallyworld: I'm doing a little bit of refactoring in environs/tools/simplestreams.go around WriteMetadata
<wallyworld> ok
<wallyworld> i'm addi8ng in support for signing
<axw> there's a bug in the null provider caused by a subtle bug in WriteMetadata
<wallyworld> bug number?
<axw> if existing tools metadata has size/sha256, it gets ignored when doing sync
<axw> just a sec
<axw> #1235717
<axw> https://bugs.launchpad.net/juju-core/1235717
<axw> eh
<axw> that's not right
<axw> https://bugs.launchpad.net/juju-core/+bug/1235717
<_mup_> Bug #1235717: null provider bootstrap fails with error about sftp scheme <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1235717>
<wallyworld> i need to read the code to understand the cause/effect i think
<axw> wallyworld: what ends up happening is, WriteMetadata finds tools and metadata in the target; doesn't do any copies, but thinks it needs to fetch the tools to compute metadata again
<axw> wallyworld: there's a simple fix, but rogpeppe thought it might be a good idea to refactor this function as it's doing quite a lot more than writing metadata
<axw> and I agree after getting thoroughly confused last night
<wallyworld> yeah, it's purpose has grown
<davecheney> "nova-cloud-controller/1:identity-service-relation-joined:1963941934992934910"
<davecheney> are relation id's sequential ?
<jam> axw: if you know the number you can just do bug #1235717 and mup will give the right URL
<_mup_> Bug #1235717: null provider bootstrap fails with error about sftp scheme <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1235717>
<axw> thanks jam
<axw> forgot the shortcut
<jam> I think it doesn't like #1235717
<_mup_> Bug #1235717: null provider bootstrap fails with error about sftp scheme <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1235717>
<jam> ah, I guess it likes it just fine
<jam> bug 1235717 also works, IIRC
<_mup_> Bug #1235717: null provider bootstrap fails with error about sftp scheme <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1235717>
<axw> maybe because I had it on its own line in the chat
<jam> #1235717
<_mup_> Bug #1235717: null provider bootstrap fails with error about sftp scheme <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1235717>
<axw> :/
<jam> axw: Seems to work for me, maybe it was taking a nap
<axw> or mup just doesn't like me
<jam> try again?
<axw> #1235717
<_mup_> Bug #1235717: null provider bootstrap fails with error about sftp scheme <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1235717>
 * axw shrugs
<jam> axw: best guess, it was loading it and just slow, but I don't really know
<axw> wallyworld: is there any reason not to fetch tools from storage, as opposed to via its URL, when computing sha256/size?
<axw> wallyworld: sshstorage doesn't have real URLs :)
<jam> wallyworld: a very concerning bug is #1236446 I'm trying to sort it out, but upgrading 1.14 => 1.15 is broken right now.
<_mup_> Bug #1236446: Cannot upgrade-juju from 1.14.1 to 1.15.1 on openstack <openstack> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1236446>
<wallyworld> axw: the tools can be fetched from anywhere. i think the url may have been used because that's all that is known at that level of the code, not sure now
<axw> wallyworld: cool. it's possible to get from storage by using StorageName, just wanted to check if there was another reason
<wallyworld> axw: so long as storage is accessible
<axw> wallyworld: well, this is in the code that writes the metadata to storage, so it has to be
<wallyworld> ah ok. i see what you mean now
<wallyworld> jam: i've not had internet access for a week, so am wading through the emails. the upgrade is still broken then. is it only on openstack?
<jam> wallyworld: it might just be a configuration/uploading tools to the right location issue. According to the bug from Curtis EC2 and Azure are working
<wallyworld> ok, reading the bug now
<jam> wallyworld: the primary email I'm going off of is "1.15.1 release summary" direct from Curtis to canonical-juju sometime yesterday"
<wallyworld> ok
<jpds> jam: Ping.
<jam> jpds: pong
<jam> walla walla ding n
<jam> dong
<jam> foiled!
<jpds> jam: So, how do I make  work?
<jpds> Err: https://bugs.launchpad.net/juju-core/+bug/1202163
<_mup_> Bug #1202163: openstack provider should have config option to ignore invalid certs <cts> <cts-cloud-review> <papercut> <Go OpenStack Exchange:Fix Committed by jameinel> <juju-core:Fix Released by jameinel> <https://launchpad.net/bugs/1202163>
<jam> jpds: you should be able to set "ssl-hostname-verification: false" in your environments.yaml for a particular provider. Note it has only been implemented for Openstack at this point
<jam> I imagine MaaS is another important target
<jpds> jam: Yep, doesn't work.
<jam> jpds: with what specific version of juju and what does it actually do ?
<jam> jpds: We don't actually have a service that uses self signed certs to check, so I was mostly going off of code auditing
<jpds> jam: 1.15.1, bootstrap says: certificate is valid for CN, not machine-5.maas.
<jam> jpds: "jam: Note it has only been implemented for Openstack at this point
<jam> (9:52:29) jam: I imagine MaaS is another important target"
<jpds> jam: This is Openstack, within MAAS.
<jpds> machine-5.maas is just my swift-proxy.
<jam> jpds: can you run "juju bootstrap  --debug" and paste the result somewhere?
<jpds> jam: One moment.
<jpds> jam: Meh, public pastebin is broken.
<jpds> jam: https://pastebin.canonical.com/98639/
<jpds> ssl-hostname-verification":true
<jam> jpds: yep
<jam> I think I know the problem
<jpds> I can see it's totally ignoring my setting.
<jam> 1.15.1 has 2 config files
<jam> ~/.juju/environments.yaml
<jam> and ~/.juju/environments/<ENV>.jenv
<jam> once the latter exists, it ignores the former (IIRC)
<jam> rogpeppe: ^^ the first instance of someone getting confused by this situation (as I mentioned last week :)
<jpds> I changed .juju/environments/openstack.jenv.
<jpds> Still doesn't work.
<jam> jpds: same debug result (it is set to true?) the only things I can think of off hand are to paste your config and see if there are typos, etc.
<jpds> jam: Set to false, same error: https://pastebin.canonical.com/98640/
<jam> jpds: odd that it is https on 8080, but I'll dig a bit more
<jam> jpds: certainly having that set is a prerequisite for anything else working, so we are one step closer
<jpds> jam: Object store: https://machine-5.cloud:8080/v1/AUTH_01fe93f0573849ffa6ed03d082f26c7c
<jam> jpds: sure, I'm not saying it is wrong, I was just surprised.  "port 80 is http, 8080 is where people put private http usually, and this is ssl enabled". but regardless, I'm trying to understand which bit of code is trying to read from that
<jam> I have an idea
<jam> given it is saying "cannot find provider-state"
<jam> but I would still have thought that it would be using the disabled hostnames fetch
<fwereade> jpds, jam: in general, we *always* ignored environments.yaml, and we still ignore the .jenv file, whichjust contains a record of what you bootstrapped with
<fwereade> jam, jpds: you will never ever be able to doanything useful by changing local config for a bootstrapped enviuronment
<jam> fwereade: but bootstrap hasn't actually succeeded yet for ssl-hostname-verification: false
<fwereade> jam, ah sorry, I'm still reading through the past
<jam> fwereade: so while I understand your point, it doesn't apply here, because he hasn't succeeded yet
<fwereade> jam, that seemed important enough to mention straight off
<jam> fwereade: sure, and I can see where it may have been relevant, as we create .jenv during bootstrap, so if it had succeeded but was failing on the server side, we couldn't change attributes there.
<jam> fwereade: is there an obvious way for a client to change the env setting? I think "juju set" is for services, is there a "juju set-env" ?
<jam> ah yes
<fwereade> jam, there is indeed
<jam> wallyworld: one option for the "unable to upgrade" is that we add a "juju set-env tools-url: ..." as advice for people upgrading on HP
<fwereade> jam, but if we haven't bootstrapped that won't help
<jam> It is *far* from ideal, but if we can come up with a workaround, it isn't as critical of a problem
<jam> fwereade: right. I'm trying to understand why bootstrap at all isn't working, as I did do some client level testing (by moving /usr/share/certs) but I'll try it again for jpds
<fwereade> jam, jpds: that paste seems to show ssl-hostname-verification set to true
<fwereade> jam, jpds: so that bit may wellbe working properly, and we need to figure out why it's set to that in the first place
<jam> fwereade: ugh, renaming /usr/share/ca-certificates doesn't *do* anything, you have to tweak /etc/* and then run "update-certificates".
<jam> fwereade: the first paste had "true" the second had "false" after he fixed the .jenv
<jam> fwereade: but *my* testing of it wasn't actually correct, because there is a cache of what actual certs are valid
<jam> ffs, I can't get certs to stop being trusted :(
<jam> "Updating certificates in /etc/ssl/certs... 0 added, 150 removed; done."
<jam> but 'wget' still happily dovnloads from an https: location
<fwereade> jam, gaah, bad luck
<fwereade> jam, and, I see, sorry poor reading comprehension
<jam> ugh.... UbuntuOne forces an extra Go_Daddy cert into the system so that U1 works, which is, of course, the cert that I want to use for Canonistack testing
<fwereade> jam, ouch
<jam> fwereade: but I can brute force it. mv /etc/ssl/certs{,.hidden}
<jam> works
<jam> I get an invalid cert trying to wget from canonistack
<jam> well, now it fails because it has 0 root certs, but *maybe* it won't care as long as you have SkipInsecureVerify
<jpds> fwereade: The second paste shows it set to false.
<fwereade> jpds, yeah, I seem to have reading problems, it's probably best to ignore everything I say first thing in the morning
<jpds> Guys, I can't even use the local provideR:
<jpds> 2013-10-08 07:28:25 ERROR juju supercommand.go:282 Get http://10.0.3.1:8040/provider-state: dial tcp 10.0.3.1:8040: connection refused
<jpds> All I did was: sudo apt-get install juju-core lxc mongodb-server on a 13.04 machine.
<jpds> With ppa:juju/devel.
<jam> jpds: you should be able to "apt-get install juju-local" which has the right dependencies
<jam> though maybe we don't have juju-local in the devel ppa
<jpds> E: Unable to locate package juju-local
<jam> fwereade: interesting bug wrt ~/.juju/environments/*. It would seem that "juju destroy-environment" deletes the file there
<jam> is that intentional?
<fwereade> jam, absolutely
<jpds> Right, so looks like I have the dependencies, devel is just borked.
<fwereade> jam, unintended consequence?
<jam> fwereade: given that tweaks have to be done in that file
<jam> and then you nuke it
<jam> was a bit surprising
<jam> I was abusing the 2-layer so that I didn't have to upset my environments.yaml
<jam> anyway
<fwereade> jam, I don't quite get why it needed to be tweaked, to be fair -- shouldn't it have just been trashed in the first place?
<jam> fwereade: if you bootstrap with ssl-hostname-verification: true, it tries to do the bootstrap, it prepares, but can't launch an instance
<jam> so you then tweak the file
<jam> and can bootstrap again
<jam> but then destroy-env
<fwereade> jam, it's fair enough to do so if bootstrap didn't work, I guess
<jam> and then bootstrap
<jam> doesn't work
<jam> and if you *just* tweak ~/.juju/environments.yaml
<jam> then just "bootstrap" doesn't work
<jam> because the .jenv is still around
<fwereade> jam, I'd prefer in general to destroy-environment before continuing if bootstrap goes weird
<jam> jpds: so I can't reproduce what you are seeing here. If I nuke my certificates correctly, but set ssl-hostname-verification: false it succeeds in bootstrapping a canonistack instance.
<jam> I'll try one more thing
<jam> fwereade: so "juju destroy-environment" can't delete the env file because it fails to connect to the server...
<fwereade> jam, aw hell
<fwereade> rogpeppe, ^
<jam> fwereade: so in this case, if you have a self-signed service, you try to "juju bootstrap" it fails because of ssl-hostname-verification, you should then edit *both* files to fix it, so that you can destroy and then bootstrap again later
<TheMue> morning
<fwereade> TheMue, heyhey
<jam> morning TheMue
<jam> fwereade: I have to go sign paperwork for house-stuff in about 1 hour. I may or may not make it back to the standup. What I'd *like* to do for code-review this week is to do a 5-why's on what's going on with the release. Does that sound reasonable to you ?
<fwereade> jam, yes, that sounds very sensible
<jam> jpds: so. I can get a bootstrapped environment using the tools from ppa:juju/devel after nuking my certs and setting the right settings in config
<fwereade> jam, better than focusing on any specific minutiae
<jam> fwereade: k, if I don't make it to the standup, can you let people know?
<jam> jpds: the keys seem to be: delete ~/.juju/environment/ENV.yaml, set ssl-hostname-verification: false in ~/.juju/environments.yaml
<jam> then, for me, juju bootstrap -e canonistack --debug is able to properly connect to everything.
<jam> jpds: my initial thought is that stuff might be hosed if you had something bootstrapped before, and you are trying to connect to something that should already be running but isn't
<jam> especially for local provider, I believe we changed how the environment storage worked
<jam> axw: ^^ can you confirm?
<jam> fwereade: maybe you know. Should "juju-1.14 bootstrap -e local" and then "juju-1.15 status" work?
<jam> I don't know if it does or not (I haven't tested it)
<axw> jam: it should work as before
<fwereade> jam, I think it should work
<jam> axw: (11:28:49) jpds: Guys, I can't even use the local provideR:
<jam> (11:28:51) jpds: 2013-10-08 07:28:25 ERROR juju supercommand.go:282 Get http://10.0.3.1:8040/provider-state: dial tcp 10.0.3.1:8040: connection refused
<fwereade> jam, assuming 1.15 has the right env set ofc ;p
<axw> ehh
<jam> fwereade: my question is if you're enviroment was 1.14 and then you try to do something with 1.15
<jam> o
<jpds> jam: Yeah, ppa:juju/stable works fine.
<jam> axw: I'm grasping here, but didn't we use disk storage for stuff, and then switch to http storage ?
<axw> jam: it was always http, it just got split
<axw> it used to http&disk in one
<axw> I'll try 1.14->1.15 in a sec
<jam> jpds: so stable should just be 1.14.1. which shouldn't work with self-signed certs. (but might be working for your local provider stuff)
<jam> jpds: but I did confirm that with ppa:juju/devel I was able to get up and running after nuking /etc/ssl/certs
<jam> which *didn't* work if i didn't set ssl-hostname-verification: false
<jpds> jam: Nuking ssl/certs sounds fun.
<jpds> But yeah, local works fine with stable, not devel.
<jam> jpds: well "mv /etc/ssl/certs /etc/ssl/certs.hidden"
<jam> not a full nuke :0
<jpds> I'll try that.
<axw> jam: I just bootstrapped with 1.14.1, status works (in my env) with trunk
<jam> axw: makes me wonder if "raring" is involved here
<jam> (13.04 machine)
<axw> jam: I'm on raring
<jam> jpds: so you shouldn't have to do anything with /etc/ssl/certs. I just did it because I don't have a Cloud with invalid certificates I can try to deploy to
<jam> jpds: I *think* the trick is to configure ~/.juju/environments.yaml with ssl-hostname-verification: false, then delete ~/.juju/environments/env.yaml and try to "juju bootstrap"
<jamespage> jam: uploading tools 1.14.1 for maas was awkward but I did it using the --dev option with the older client
<jpds> jam: OK, now I have what looks like a swift error.
<jamespage> jam:  seeing alot of polling of the MAAS API now - bug 1236734
<_mup_> Bug #1236734: juju 1.15.1 polls maas API continually <juju-core:New> <https://launchpad.net/bugs/1236734>
<jam> jamespage: that appears to be cycling over all the nodes 1 at a time and then back around again ?
<jamespage> jam: I think so - it never stops
<jamespage> .12 is the bootstrap node
<jam> jamespage: well it hits 742b and then doesn't hit it again for about 6 requests
<jam> do you have 5-6 nodes in the env ?
<jamespage> jam: 6 physical servers and 8 lxc containers
<jam> jamespage: do you have any juju logs to correlate ?
<jamespage> jam: I can
<jam> jamespage: you may also need to do: juju set-env 'logging-config=<root>=DEBUG'
<jamespage> jam: pasted to the bug - but an obvious correlation
<jamespage>  cannot get addresses for instance "/MAAS/api/1.0/nodes/node-0d121d8c-4527-11e2-ba10-2c768a4f56ac/": Requested array, got <nil>.
<jam> jamespage: those are provisioned (deployed to ) machines, right? not ones that are sitting idle? It does look like we're trying to set the addresses for the various machines but can't find the actual address
<jam> not sure the "Requested array" stuff is
<jamespage> jam: they are all provisioned - this was an upgrade to an existing environment
<jam> jamespage: can you do just a "wget" to /MAAS/api/1.0/nodes/?id=node-742baf7e-4527-11e2-9188-2c768a4f56ac&op=list and add the info there as well?
<jam> It may be that we were expecting to see a list of IP addresses
<jam> in a field and we can't find it now
<jam> and rogpeppe added a poll to find updates to addresses, though I thought it was polling 1/min not 1/s
<rogpeppe> mornin' all
<TheMue> rogpeppe: morning
<jam> morning rogpeppe
<jam> you ears must be burning :)
<rogpeppe> jam, TheMue: hiya
<rogpeppe> jam: that's probably why i got lost in the woods on my morning bike ride :-)
<fwereade> rogpeppe, jam: arrrrrgh does Addresses() actually work on maas?
<jamespage> jam:
<jamespage> curl "http://10.98.191.11/MAAS/api/1.0/nodes/?id=node-0d121d8c-4527-11e2-ba10-2c768a4f56ac&op=list"
<jamespage> Unrecognised signature: GET list
<jamespage> this is raring maas
<rogpeppe> fwereade: the implementation looked plausible, but i didn't try it live i'm afraid
<jam> jamespage: might be a POST, let me dig up my MaaS knowledge a bit
<jamespage> jam: its a GET in the apache log from juju as well
<jam> jamespage: k, I do see "When a machine has no address it will be bolled at ShortPoll == 1s until it does"
<rogpeppe> jam: it should probably back off after a while
 * rogpeppe reads back through the log
<jamespage> jam: I suspect that is dependent on a newer maas maybe?
<jamespage> jam: in the WebUI I don't see any addresses associated with the servers
<jam> the error message is from gomaasapi in "failConversion(wantedType string, ob JSONObject)"
<jam> I'm not sure what API we are requesting yetd
<jamespage> http://paste.ubuntu.com/6208383/
<jam> jamespage: mi.maasObject.GetMap()["ip_addresses"].GetArray() is expected to be the list of IP addresses for a machine...
<jam> jamespage: it appears to have been added in MaaS rev 1521
<jamespage> jam: so this is a backwards compat issue?
<jam> "raphael badin: Add API method to fetch the IP addresses attached to a node"
<jamespage> 1461 is in raring
<jam> so, I see the same call to find IP Addresses in 1.14.1
<jam> we just didn't poll it in the past
<rogpeppe> jam, fwereade: about destroy-environment: perhaps we should put a "bootstrapped" flag into the .jenv file; if there are bootstrap attributes in the file and that's not set, then juju destroy-environment could forgo trying to connect to the environment before deleting the file.
<jam> mgz: MaaS IPAddresses call is unreliable, and we seem to prefer it to DNSName (aka hostname) in the Addresses code
<jam> jamespage: so it looks like 1.14.1 *could* have looked for the "IP Addresses" field, but generally preferred the "hostname" field
<fwereade> rogpeppe, I think I'd prefer an omitempty NotBootstrapped than to have that cluttering up the file
<jam> 1.15.1 is now expecting the ip_addresses field
<jam> which apparently doesn't exist in raring
<jam> rogpeppe: it is a bit of an edge case for "ssl-hostname-verify" so I don't want us to go overboard fixing it, but something we should think about
<rogpeppe> fwereade: that seems reasonable
<jam> we've run into other problems with bootstrap failing and then the system requires destroying an environment that doesn't exist
<rogpeppe> jam: in fact i'm having second thoughts
<rogpeppe> jam: if bootstrap fails, some of the environment still does exist
<rogpeppe> jam: or can do, at any rate
<rogpeppe> jam: because we might have local storage which needs deleting
<rogpeppe> jam: and in the future, destroying an environment will probably be done by connecting to the API and getting the environment to destroy itself
<jam> jamespage, mgz: so we need to sort out the ip_addresses stuff. I'm thinking maybe we try ip_addresses and if that request fails just fall back to hostname
 * jam goes to sign some paperwork at the bank
<axw> fwereade: regarding this: https://codereview.appspot.com/14032043/diff/19001/juju/conn.go#newcode171
<fwereade> axw, ah yes
<axw> fwereade: the only con I had was, NewConn takes an Environ as input; so the output Conn would have a different Environ
<fwereade> axw, the initial Environ is *only* good for connecting
<axw> yep
<axw> fwereade: just might be surprising, but I suppose it could only be positively surprising
<fwereade> axw, there may be a case to be made that *that* env should be SetConfiged, but that feels a bit hairier to me
<axw> fwereade: agreed
<axw> fwereade: so I'll update it to SetConfig on a new env which gets set on the Conn
<fwereade> axw, you should be able to just create a new env with the latest config and set that on the conn
<fwereade> axw, no call for SetConfig there, I think
<axw> fwereade: yes sorry, what you said
<axw> fwereade: I just meant, the Conn that comes out will have a different (up to date) Env, rather than the input one
<fwereade> axw, great, sgtm
<rogpeppe> fwereade, axw: presumably it'll mean we'll have to add a mutex to the conn
<fwereade> mgz, so, maas instance Addresses -- should it just log an error and move onif ipAddresses fails?
<rogpeppe> fwereade, axw: and hide its Environ
<axw> rogpeppe: why do you say that?
<axw> rogpeppe: this is happening in NewConn
<rogpeppe> axw: ah yes, sorry
<rogpeppe> axw: i hadn't appreciated that
<jamespage> fwereade, sorry but bug 1236754 is going to cause problems
<_mup_> Bug #1236754: behaviour change: relation-get for unset attribute returns "" in 1.15.1 <juju-core:New> <https://launchpad.net/bugs/1236754>
<fwereade> jamespage, oh, hell, thanks for spotting that -- critical for 1.16, I guess
<fwereade> dimitern, looking at the history, my best guess is that ^^ is a consequence of the api changeover -- does anything spring to mind for you?
<dimitern> fwereade, sound like the map[string]interface{} -> map[string]string transition
<dimitern> fwereade, what should happen for unset settings and relation-get?
<fwereade> dimitern, yeah, I think you're right, we're doing that `value, _ = settings[key]` thing
<jamespage> rogpeppe, remember that problem I had where the bootstrap node would not talk to nova correctly?
<jamespage> rogpeppe, I figured out what it was
<jamespage> network fragmentation
<jamespage> actually jam might be interested in that as well ^^
<jamespage> it was due to the fact that the bootstrap node was accessing the API server from within a neutron hosted private tenant overlay network
<jamespage> and the MTU's where set to 1500 on the gateway node
<jamespage> which was causing fragmentation when the bootstrap node tried to access the API server
<jamespage> hanging instance creation
<rogpeppe> jamespage: interesting
<jamespage> I bumped the MTU on the physical server to resolve the issue
<jamespage> 1546 provides enought space for the instance to still operate at 1500 with the extra 46 bytes carrying the GRE overlay headers
<jamespage> rogpeppe, I feel I need to log that somewhere;
<jamespage> but not sure where
<jamespage> someone else is bound to hit it
<wallyworld> mgz: ping
<rogpeppe> jamespage: does the neutron API use UDP or something?
<jamespage> rogpeppe, no its TCP
<fwereade> dimitern, except I can't actually figure it out how we could have got blank output with the json formatter in the first place
<jamespage> rogpeppe, its this issue - http://techbackground.blogspot.co.uk/2013/06/path-mtu-discovery-and-gre.html
<dimitern> fwereade, interesting point
<rogpeppe> jamespage: interesting; i didn't realise that MTU wasn't negotiated correctly
<jamespage> rogpeppe, althought the bootnote about ovs 1.10 should apply - but I still see issues - interesting
<rogpeppe> jamespage: is there anything we can do in juju to help here?
<davecheney> rogpeppe: on most wan links the MTU is usally adjusted down, to leave room for the GRE encapsulation
<jamespage> rogpeppe, I'm still thinking about that
<jamespage> davecheney, yeah - thats the other way to fix this - drop the mtu on the instances themselves
<jamespage> that can be done using DHCP options
<davecheney> jamespage: IMO it's the more common approach
<davecheney> for you cant guarentee jumbo frames
<jamespage> davecheney, its not 100% reliable - not all dhcp clients use that option
<davecheney> jamespage: indeed
<davecheney> one of the many problems with GRE encapsulation
<fwereade> jamespage, is it possible that in https://bugs.launchpad.net/juju-core/+bug/1236754 it actually used to return `null` rather than ``?
<_mup_> Bug #1236754: behaviour change: relation-get for unset attribute returns "" in 1.15.1 <juju-core:Triaged by fwereade> <https://launchpad.net/bugs/1236754>
<jamespage> fwereade, possibly
<jamespage> fwereade, null -> None as well
<fwereade> jamespage, that's the only explanation I can see that makes sense
<jamespage> fwereade, but I remember raising a bug about this
<jamespage> in 1.12
<dimitern> so it's not the api
<fwereade> dimitern, yeah, it's the type change down in RelationGetCommand.Run
<rogpeppe> fwereade, TheMue: i'd appreciate a review of this, please. https://codereview.appspot.com/14395043/
<TheMue> *click*
<rogpeppe> fwereade: if Addresses doesn't work on maas, that kinda stuffs the API caching
<fwereade> rogpeppe, indeed -- have you seen mgz today?
<rogpeppe> fwereade: nope
<fwereade> rogpeppe, because it *looks* like we just need to log and ignore errors from the ipAddresses method
<rogpeppe> fwereade: yeah, we've already got one address, so returning an error seems wrong
<wallyworld> fwereade: fyi, you can't bootstrap on hp cloud using the shared credentials cause we are out of security groups. there's a whole lot of nec and yjp ones but i'm not sure if any of those can be deleted
<fwereade> davecheney, do you know if we can do anything about the above?
<davecheney> fwereade: i'm not sure I understand the problem
<davecheney> fwereade: about MTUs ?
<TheMue> rogpeppe: you've got a review
<rogpeppe> TheMue: thanks
<rogpeppe> TheMue: "I dislike the 0 postfix" - what would you use instead?
<rogpeppe> TheMue: personally i think it works ok when you've got two views of the same object -
<fwereade> davecheney, sorry, I mean about nec/yjp security groups in hpcloudas referenced by wallyworld
<rogpeppe> TheMue: the zero implies an original
<wallyworld> davecheney: you can't juju bootstrap on hp cloud right now
<wallyworld> it errors with a 400 code, too many sec groups
<wallyworld> nova sec-group-list shows a lot of them for sure
<davecheney> wallyworld: wha
<davecheney> this is news to me
<TheMue> rogpeppe: as i said, only a personal thing. i would write agentConfig, err := NewAgentConfig()
<davecheney> are you using firewall-mode: global ?
<rogpeppe> TheMue: and what about the second variable, which is also an agent config, and refers to the same object?
<wallyworld> i haven't set that anywhere
<wallyworld> whatever the default it
<wallyworld> is
<davecheney> wallyworld: the default is not to use that
<davecheney> and by default you get 25 security groups *PER TOP LEVEL ACCOUNT*
<rogpeppe> TheMue: (the variable currently named "config")
<davecheney> ie, if you're sharing some account Antonio gave you
<wallyworld> yeah i am
<davecheney> you're going to have to share the security groups
<davecheney> two options
<TheMue> rogpeppe: here i'm fine with config, internalConfig is already the type, so cannot use it
<davecheney> 1. firewall-mode: global or GTFO
<davecheney> 2. scream bloddy murder at the HP support guys and try to get the limit increased
<davecheney> it's such a retarted limit
<davecheney> thye just picked an arbitary number
<rogpeppe> TheMue: so we've got agentConfig vs config - i don't think that shows the association between the two as well as config0 and config, tbh
<wallyworld> davecheney: option 1 seems the quickest for now to get going. i'm trying stuff on canonistack right now but will revisit hp cloud later. thanks for the advice
<TheMue> rogpeppe: then uncastedConfig and config :D
<rogpeppe> TheMue: not keen
<fwereade> rogpeppe, if you want to imply "original", how about "originalConfig"?
<TheMue> rogpeppe: people have different preferences, i only told you mine. but that doesn't prevented me from an lgtm. just a comment. feel free to let it as it is.
<rogpeppe> TheMue: thanks
<TheMue> rogpeppe: yw
<rogpeppe> fwereade: that seems too weighty for me for something that's just a throwaway name and the reason for it should be evident from looking at the only place it's used, two lines below
 * TheMue never liked numbers in identifiers if it is possible to avoid them
<rogpeppe> fwereade: if anything, i might go for "configInterface", but again, it seems a bit mich
<rogpeppe> much
<rogpeppe> fwereade: if you have a few moments, i'd still appreciate a once-over of that CL, BTW.
<fwereade> rogpeppe, I would say that explicit beats implicit in general ;)
<fwereade> rogpeppe, ok, I'm running some tests, quick link please?
<rogpeppe> fwereade: https://codereview.appspot.com/14395043
<mgz> fwereade: ipAddresses in the maas provider? that did get reviewed by the red squad after nate wrote it, but it's easy enough to change
<mgz> can just make l65 err check log err and return addrs, nil
<mgz> bug 1236734 complains about the maas api being polled at all though, and that is deliberate
<_mup_> Bug #1236734: juju 1.15.1 polls maas API continually <juju-core:New> <https://launchpad.net/bugs/1236734>
<fwereade> rogpeppe, reviewed
<rogpeppe> fwereade: thanks
<fwereade> mgz, rogpeppe: what are the addressupdater polling timings again?
<rogpeppe> fwereade: currently 1s until there are addresses and 1m thereafter
<rogpeppe> fwereade: i'm considering backing off the initial timing, say at 10% each time, until it reaches the longer time
<rogpeppe> fwereade: and perhaps the longer time could be 30m
<mgz> hm, so the main fault is still the ip_addresses not existing in raring maas and our error condition there
<rogpeppe> fwereade: but suggestions for values very welcome
<rogpeppe> fwereade: they're totally arbitrary currently
<fwereade> rogpeppe, 1s seems really excessive -- I'd expect something more like 1m in the first place
<rogpeppe> fwereade: instances usually get an address within a few seconds on ec2, at least
<fwereade> rogpeppe, I guess the problem is that it's never stopping polling really
<rogpeppe> fwereade: that's why i'm suggesting backing off
<rogpeppe> fwereade: with a small exponent
<fwereade> rogpeppe, not unreasonable, indeed, but feels rather low-value compared to just fixing Addresses
<rogpeppe> fwereade: we should probably do both
<fwereade> rogpeppe, right, but the failing Addresses STM like it's critical for 1.16
<rogpeppe> fwereade: it's not
<rogpeppe> fwereade: because nothing uses the addresses from the state
<rogpeppe> fwereade: the only critical thing that i can see is the log file spam
<fwereade> rogpeppe, the log file spam is pointing out, quite correctly, that we're broken
<rogpeppe> fwereade: obviously we should fix Addresses, but if there's a long term problem with an environment, polling continually at the same rate seems a bit fruitless
<mgz> rogpeppe: we do call a non-existent maas api every second, which is pretty poor
<rogpeppe> fwereade: sure, we're broken, but it's not going to break anything else in the system, is it?
<rogpeppe> mgz: agreed, it's poor, but is it a critical problem?
<mgz> or, the api exists, but a field we need didn't get added till saucy it seems
<dimitern> fwereade, rogpeppe, updated https://codereview.appspot.com/14486043
<fwereade> rogpeppe, I think it is critical, primarily because we'll be a fucking laughing-stock if we release something that ham-fisted
<fwereade> rogpeppe, it's not like nobody will notice
<fwereade> rogpeppe, it's already been noticed in about 24h
<rogpeppe> fwereade: ok, sure
<rogpeppe> fwereade: it depends how we define critical
<fwereade> rogpeppe, doing something retardedinternally is prbably not
<fwereade> rogpeppe, once it leaks out into the rest of the world, I think it is
<rogpeppe> fwereade: we could just remove the log statement then :-)
<fwereade> rogpeppe, Ithink we're just lucky that maas isn't rate-limiting us out of existence ;)
<rogpeppe> fwereade: (not serious)
<fwereade> rogpeppe, ;)
<rogpeppe> fwereade: if it's easy to fix the maas call, then that's great.
<dimitern> jam, rogpeppe, standup
<dimitern> fwereade, will you manage the g+?
<dimitern> fwereade, will you try to join again?
<fwereade> dimitern, I have been
<dimitern> fwereade, man you should call melita and give them some piece of your mind :)
<fwereade> dimitern, yeah, I think I will be, this has got far beyond piss-taking level
<dimitern> fwereade, and you're even on supposedly better bandwidth than mine
<fwereade> dimitern, yeah, this "60Mbps" is... er... decidedly *not*
<rogpeppe> fwereade: ha ha - 60Mbs... to the router box
<dimitern> fwereade, rogpeppe, review poke
 * TheMue => lunch
<jam> mgz: so who are we actually assigning bug #1236734 to? I can probably pick it up, but I want to make sure we have assignees for all the Critical bugs
<_mup_> Bug #1236734: juju 1.15.1 polls maas API continually <juju-core:Triaged> <https://launchpad.net/bugs/1236734>
 * fwereade is going to have some breakfast, and maybe follow it up with lunch; in the meantime: https://codereview.appspot.com/14537043
<teknico> what does it mean when "juju bootstrap" replies this?
<teknico> error: build command "go" failed: exec: "go": executable file not found in $PATH;
<teknico> what's there to build?
<jam> teknico: it sounds like you are running a dev release that can't find 'jujud' tools and so is trying to build them for you
<jam> teknico: (a) can you file a bug about us trying to build from something installed from a package ?
<jam> teknico: it was meant to make it easier  for developers to ensure they had tools matching the client they are testing, etc. but clearly it leaked into the devel release
<teknico> uhm, I remember adding the dev PPA, but I can't find it in the apt config anymore
<teknico> I'm using 1.14.1-0ubuntu1
<teknico> jam, is that ^^ a devel release?
<jam> teknico: it shouldn't be, but you can still file a bug about us falling back to trying to build tools in a released juju-core
<jam> teknico: you could probably run "juju bootstrap --debug" to get more info about why it might be trying to do so
<teknico> jam: http://pastebin.ubuntu.com/6209041/
<jam> teknico:  DEBUG juju.environs.tools build.go:210 copy existing failed: write /tmp/juju-tools259801668/jujud: no space left on device ?
<teknico> yeah, df says: overflow            1024       0      1024   0% /tmp
<jam> teknico: so it would appear that it *found* the jujud it wanted, but couldn't copy it into a tarball, so it thought it should treat that as falling back to building from source
<jam> so still a potential for a bug, but I imagine cleaning out your tmp will get you going :)
<teknico> does it need more than one meg? :-)
<teknico> there's nothing in /tmp, I wonder why so small
<jam> teknico: I would imagine it needs about 10M or so
<teknico> and how it worked before :-)
<jam> teknico: if I was being tight, maybe it only needs 2-4MB but certainly it has always needed more than 1MB
<teknico> jam: https://bugs.launchpad.net/juju/+bug/1236824
<_mup_> Bug #1236824: boostrap tries to build jujud <juju:New> <https://launchpad.net/bugs/1236824>
<mgz> landing bot is back up and should be working happily, tell me if anyone has issues
<jam> mgz: I was unable to update the environment (juju set) to include a stanza for now lp:juju-core/1.16 I did set it manually
<jam> mgz: I did upload the change to swift, though it is basically just copy the 1.14 lines and put it into the 1.16 lines.
<jam> but we need to call "juju set --config" so that a reboot will leave us *close* to working
<mgz> jam: is there any issue with just doing that?
<dimitern> fwereade, rogpeppe, hate to be a bother, but please https://codereview.appspot.com/14486043
<teknico> ok, it was due to the main filesystem filling up previously
<rogpeppe> dimitern: sorry, just finishing off proposal for critical bug fix
<rogpeppe> dimitern: ok, swap ya: https://codereview.appspot.com/14438049
<rogpeppe> fwereade: the above CL adds exponential backoff to the address updater
<dimitern> rogpeppe, looking
<rogpeppe> dimitern: reviewed
<dimitern> rogpeppe, you've got a review too
<rogpeppe> dimitern: thanks!
<jam> mgz: I tried to do it myself, and just got "waiting for ip address for machine-0"
<fwereade> dimitern, a few more notes from me there
<fwereade> rogpeppe, need another review? (btw, is someone handling maas.instance.Addresses()?)
<dimitern> fwereade, thanks
<rogpeppe> fwereade: yes, mgz is on it
<rogpeppe> fwereade: more eyes always good when landing on release...
<fwereade> rogpeppe, great, thought so, just wasn't sure I'dbeen explicit
<mgz> jam: wat?
<mgz> I have no idea what that error is
<rogpeppe> dimitern: for the 1.1 value - it's pretty arbitrary
<jam> mgz: not an error, just and indefinite hang I think. I was using juju 1.15 and the initial error it gave was "not bootstrapped". Presumably because there was no .jenv file
<mgz> aaaa
<jam> mgz: so I'm guessing something got out of sync with your config
<mgz> I haven't touched the config, and I'm not sure if dimitern has either
<rogpeppe> dimitern, mgz, jam, fwereade, jamespage: different suggestions for a backoff exponent welcome. 10% is pretty arbitrary.
<mgz> it's more likely just an issue from the juju version being different?
<jam> rogpeppe: I think the most common exponential backoff is 2x, but it doesn't really matter much
<rogpeppe> jam: i think 2x slows down too quickly
<jam> rogpeppe: it seems to be the recommended value from AWS: http://docs.aws.amazon.com/general/latest/gr/api-retries.html
<jam> rogpeppe: 1s, 2s, 4s, 8s, 16s doesn't seem that bad for what you're doing. The idea is to cap it at the LongPoll time anyway, right?
<rogpeppe> jam: yes
<jam> rogpeppe: so the nice thing about 2x, is that you wait as long as you've tried so far
<jam> well, 2^n-1 I guess.
<jamespage> 2x sounds good to me
<mgz> this is additional wait time, not interval though
<jam> theoretically if you did 1s, 1s, then 2,s 4s, 8s, 16s
<jam> each time you would be waiting as long as the total time before that
<rogpeppe> jam: if the average wait time for a new address is 10s, then we'll wait 6s longer than we need to
<jam> rogpeppe: and the world will crumble in dispair from waiting 1.6x longer than we need to :)
<jam> despair
<jam> rogpeppe: so there are lots of ways of doing it, try 10x, then backoff, etc etc. If we have a strong feeling about the expected time to get the first one
<rogpeppe> jam: yeah
<jam> you could play with the exponent to try to get exactly that value
<jam> but really, I don't think it matters
<jam> you're trying to get the right Order of Magnitude
<jam> which pretty much any exponent will get you to
<rogpeppe> jam: it feels a little bit different to retrying arbitrary API errors
<jam> and 2x has the nice property that people will understand it
<rogpeppe> jam: indeed
<rogpeppe> jam: yeah. 2x is probably fine for the usual case where an instance takes ages to start - the address will be ready by the time the instance is up
<rogpeppe> jam: although i'm still tempted by a closer fit :-)
<jam> rogpeppe: so if I wasn't doing 2x, I would probably do 1.5x, but that doesn't line up nicely, but as mgz says, it isn't setting a deadline so the logs won't show it evenly spaced anyway (because you have to add ping time to each request)
<jamespage> fwereade, mramm: for reference bug 1236622 is going to be a blocker as well IMHO
<_mup_> Bug #1236622: Unable to upgrade from 1.14.1 to 1.15.1 on maas environment <juju-core:New> <https://launchpad.net/bugs/1236622>
<rogpeppe> jam: not quite sure what you mean by "doesn't line up nicely"
<jam> rogpeppe: if you do 2x, then when you look at a log file, you can usually visually estimate 1s, 2s, 4s, 8s. If you use 1.5x, then the gaps are 1s, 1.5s, 2.25s, etc
<jam> 3.375, 5.0625
<jam> not whole numbers
<rogpeppe> jam: ha, i see
<fwereade> rogpeppe, jam: let's go with x2
<rogpeppe> fwereade: was doing that
<fwereade> rogpeppe, great :)
<dimitern> rogpeppe, thanks for the clarification
<rogpeppe> dimitern: np
<fwereade> jam, so for the maas/openstack upgrade problems -- am I right in understanding that they're *all* in cases where tools had been set up manually originally, and so were set up manually again, and so couldn't be found because sync-tools is not writing to the old locations?
<fwereade> jam, or have I missed some subtlety?
<jam> fwereade: so the maas one is that, the original issues for 1.15 were around that. The specific thing that Curtis mentions would have to be something else
<jam> it might be a fallback issue, it might be something else
<jam> fwereade: I'm trying to see if I can reproduces
<jam> reproduce
<jam> fwereade: but yes, the general upgrade problem is because we used to put tools in location A, 1.15 wants them in location B, but *upgrade* is still looking for them over in A
<jam> because it is the 1.14 code that is finding the tools
<mramm> jam: that makes sense -- do we need to put tools in two places for this release?
<fwereade> mramm, yes; and I thought we'd already sorted out that we did have to, and that we were doing that already
<mramm> fwereade: ok
<fwereade> mramm, I think we missed the manual case, though
<mramm> I will let you guys keep working on it
<mramm> fwereade: ahh
<mramm> fwereade: gotcha
<fwereade> mramm, syncing tools will not work
<fwereade> mramm, but it looks like there's *something* else we haven't quite figured out
<fwereade> hey, wtf
<fwereade> I can't seem to stop debug-log with ctrl-c any more
<fwereade> would someone who's got an environment up confirm please?
<fwereade> so, arrgh
<fwereade> rogpeppe, are you working on something critical at the moment?
<rogpeppe> fwereade: i'm working towards API caching, but no
<rogpeppe> fwereade: just proposing a pretty trivial branch that standardises the output of juju init (and makes it a little more readable)
<rogpeppe> fwereade: is there something you'd like me to do?
<fwereade> rogpeppe, would you take a quick look at what changed in the ssh command lately please? I LGTMed it, looked perfectly sane, but I suspect it of causing problems
<fwereade> rogpeppe, unless axw_ is around, but I don't think it's a sociable hour for him
<rogpeppe> fwereade: what kind of problems do you suspect?
<fwereade> rogpeppe, ctrl-C seems not to work any more in debug-log
<rogpeppe> fwereade: ah, ok
<rogpeppe> fwereade: i'll take a look
<fwereade> rogpeppe, and when I exit the debug-hooks window I need to exit again before I'm returned to my local shell
<TheMue> fwereade: would you mind to take a quick look at https://codereview.appspot.com/14527044/? no review, only a discussion if the approach is what you had in mind.
<mgz> lbox doesn't want my branch reviewed...
<TheMue> mgz: had some trouble a few moments ago too, just waited longer until the command finished
<mgz> just did it!
<mgz> so, https://codereview.appspot.com/14543043
<mgz> MAAS it up
<TheMue> mgz: looking
<fwereade> TheMue, commented
<TheMue> fwereade: thx
<dimitern> fwereade, I don't get how using SetTransactionHooks can be useful there
<fwereade> dimitern, change environ config irrelevantly, check it retries; change it so it's already correct, check no error; repeatedly change it, check excessive contention
<rogpeppe> natefinch: this is your kind of change :-) https://codereview.appspot.com/14426046/
<dimitern> fwereade, you mean in the tests, no the actual impl
<fwereade> dimitern, yeah, indeed
<dimitern> fwereade, ok, will try
<fwereade> dimitern, just gives us coverage of the weird situations we used to have to "test" by inspection
<dimitern> fwereade, so the idea is: 1) change some setting in a Before hook, assert err is nil, 2) change agent-version to the new one, assert err is nil, 3) not sure how to repeatedly call a before hook
<natefinch> rogpeppe: nice, looking
<fwereade> dimitern, check through export_test in state, there are a few useful variants of STH
<fwereade> dimitern, there are a couple of tests somewhere already that do exrcise the excessive contention checks
<fwereade> rogpeppe, would you agree there's something funny happening in the ssh commands?
<rogpeppe> fwereade: i would
<rogpeppe> fwereade: i was just trying to replicate in the local provider
<rogpeppe> fwereade: and i find that i can't ssh in, and debug-log fails too
<rogpeppe> fwereade: that may be a separate issue tho
<rogpeppe> fwereade: i will try bootstrapping with ec2 and seeing if the same thing applies
<fwereade> rogpeppe, great, thanks
<fwereade> jam, are we about to lose you? can we address some of it by getting someone, independently, to hack up sync-tools to put things in the old location as well (temporarily)?
<jam> fwereade: you've already lost me :). I tested hp, and with public-bucket-url set when bootstrapping 1.14.1 (which was required for hp back then), upgrade-juju --dev works just fine. My guess it is a case of tools getting copied to his private bucket, and thus not getting 1.15 tools in there.
<jam> fwereade: so MaaS needs a fix
<jam> but HP and Canonistack both work
<jam> I guess arguably you didn't *have* to test public-bucket-url in 1.14 because we would copy the tools in there for you and *that* circumstance seems to be broken to upgrade to 1.15 if you don't copy the tools in
<fwereade> jam, ok -- but for anyone who synced tools already, a new sync tools will surely not work?
<jam> fwereade: I think so
<jam> if you have sync-tools with 1.14 then sync-tools with 1.15 will not allow an upgrade
<jam> because it doesn't copy it to both places
<sinzui> jam, when you called juju-upgrade, was tools-url in your config?
<jam> sinzui: no
<fwereade> jam, isn't the maas issue that there *is* no public bucket, so *everyone* is hitting the bad-sync-tools issue?
<sinzui> for HP
 * sinzui replays that
<jam> sinzui: looking at your log, it looks *strongly* like you had 1.14.1 tools in your private bucket
<jam> so it wasn't using public-bucket-url to find the tools where we updated them for 1.15
<sinzui> I will use another provate bucket then
<sinzui> private
<jam> fwereade: as in, we can't set things up for them in MaaS, yes
<jam> sinzui: you can use "swift delete" if you want, but make sure "juju bootstrap" with 1.14 to start things off *doesn't* copy the tools, if it does, then you're back in this situation.
<jam> fwereade: so yeah, 1.16 should probably copy to both location for "sync-tools" if it sees an existing environment
<jam> (or just punt and always do it)
<fwereade> jam, I must say I'm a little bit tempted to always do it
<jam> fwereade: it makes releasing the tools easier :)
<jam> but all those places have 1.14 tools
<fwereade> jam, sorry I don't follow the last thing you said
<jam> fwereade: if the rule was "if we see the fallback location has tools, copy them there" then it does what we want in "all" locations.
<jam> the official buckets need that, as do people who are upgrading
<jam> but people who are just starting don't
<fwereade> jam, ah, yes -- and we *should* need that just on 1.16, right?
<jam> fwereade: right, since 1.16 will only look at the new location, etc.
<sinzui> fwereade, Both my dev > 1.17.0 and stable > 1.16.0 branches failed to merge. Both failed the same way reporting that Juju cannot bootstrap because no tools for add_machine. The bootstrap test says ... Panic: no reachable servers
<jam> sinzui: can you link? I only just set up the 1.16 branch on tarmac bot
<jam> found it: https://code.launchpad.net/~sinzui/juju-core/inc-stable-1.16.0/+merge/189688
<sinzui> yep
<jam>     c.Check(findToolsRetryValues, gc.DeepEquals, test.expectedAllowRetry)
<jam> ... obtained []bool = []bool{false}
<jam> ... expected []bool = []bool{false, true}
<jam> looks suspiciously like what happened when you uploaded 1.15 originally
<sinzui> yeah, I am looking for the merge that fixed that
<jam> sinzui: well, IIRC the *original* fix for that was to fix the permissions on the public bucket
<jam> but I don't see the code actually connecting to s3... anymore
<sinzui> oh. I was thinking of this: https://code.launchpad.net/~rogpeppe/juju-core/428-skip-TestStartInstanceOnUnknownPlatform/+merge/188393
<sinzui> not the same thing
<jam> sinzui: ugh, just changing the "version" string does, indeed, break the bot run. If I "cd cmd/juju" and then run "go test -gocheck.f Bootstrap" it passes with version = "1.15.1" but fails with "1.16.0" in that slot.
<jam> on the tarmac bot
<jam> I'm going to test it locally as well
<sinzui> okay
<jam> sinzui: fails locally
<sinzui> :(
<jam> sinzui: I'm guessing you might not have checked, but did it pass for you?
<jam> certainly *I* wouldn't have tested that a version number change would break all of bootstrapping.
<dimitern> fwereade, updated https://codereview.appspot.com/14486043 - it got much nicer now
<fwereade> dimitern, great, thanks
<jam> sinzui: I know what it is... but it still makes me sad
<jam> sinzui: if you are running a development version of juju, it automatically will set "--upload-tools" if it doesn't find them
<jam> but 1.16 is a *release* version
<jam> well, stable but non-dev
<jam> sinzui: my guessi is line 186 of cmd/juju/bootstrap.go "if err ... && version.Current.IsDev() {"no tools found, so attempting to build and upload new tools"}
<sinzui> hmm
<jam> sinzui: If I manually add "c.Fatalf()" to TestAllowRetries I *see* that line in the test suite run w/ 1.15.1 but I *don't* see it in 1.16.0
<sinzui> jam, Okay. I see slight differences in the error logs https://code.launchpad.net/~sinzui/juju-core/inc-stable-1.16.0/+merge/189688
<fwereade> dimitern, reviewed
<dimitern> fwereade, thanks
<jam> sinzui: oh ffs. The other problem I *think* for BootstrapTwice test. Is that --upload-tools always updates the Version.Build value (so you have 1.16.0.1) but a version with build != 0 implies that it is a Dev version
<jam> and we don't match Dev versions by default
<jam> so, AFAICT the tests are just broken wrt stable series because they are expecting Dev behavior
<fwereade> jam, that automatic upload-tools looks pretty appalling regardless
<sinzui> ahh! jam that has always puzzled about why I see that happen in testing
<sinzui> there should be a me in there I think
<jam> fwereade: fwiw, I've always been against auto-sync-tools behavior. I can understand where it is convenient, but it is wrong almost as often as it is right (misconfigured tools-url, etc), and the workaround for people that actually want it is to just add "juju sync-tools" before they bootstrap
<fwereade> jam, is that all just for the local provider?
<jam> We added it for MaaS where you always have to sync-tools
<fwereade> jam, uploading has *nothing* to do with syncing tools
<jam> fwereade: well, the first automatic --upload-tools was for the local provider
<fwereade> jam, yeah
<jam> then someone saw we were doing it, and said "hey, if I have a dev version, might as well auto set it"
<fwereade> jam, I should have pitched a massive fit about it at the time :(
<jam> I think that was part of "forcing to minor version matching"
 * TheMue is currently building mongo on os x. first time seeing all 8 ht at 100% load, nice.
<fwereade> TheMue, the provider tests are the really important bits here fwiw
<fwereade> TheMue, so if you can spare half a core to look into those it would be great ;)
<jam> sinzui: so I can reasonably calmly say "BootstrapTwice" is broken because it expects automatic uploading. Adding "--upload-tools" to the first bootstrap lets it get farther, but it still fails because it crosses Dev vs NonDev behavior
<jam> sinzui: TestUpgradeJujuWithRealUpload has a similar problem
<jam> in that a 1.16 juju binary won't try to use a 1.16.0.1 tool that it finds
<TheMue> fwereade: *rofl*
<TheMue> fwereade: should work
<jam> Bootstrap.testAllowRetries is likely also a Dev vs NonDev, and all the other tests pass
<jam> and I really need to get back to family time, but I wish I had better answers here.
<jam> sinzui: it isn't your patch, the tests are just bad
<jam> though I'm pretty sure you knew that
<sinzui> small comfort
<dimitern> fwereade, sorry about that, updated https://codereview.appspot.com/14486043/ again
<TheMue> fwereade: several failures, have to analyze. first impression is that it is stream related
<teknico> is this a known problem? https://bugs.launchpad.net/juju/+bug/1236900
<_mup_> Bug #1236900: tar: unrecognized option ''--numeric-uid'' <juju:New> <https://launchpad.net/bugs/1236900>
<dimitern> fwereade, I realize I should've started to implement it like it is now (for loop encompassing the whole thing, abort handling, etc.) - I guess it's been a while since I wrote state code
<fwereade> dimitern, sorry meeting, but I know the feeling
<fwereade> teknico, that is not something I've seen before
<fwereade> teknico, fwiw it's probably better reported against juju-core
<teknico> fwereade: stub just pointend out #1236726
<teknico> https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1236726
<teknico> I'll mark my bug as duplicate of that one
<fwereade> teknico, phew, not us :)
<teknico> sorry for the false alarm :-)
<fwereade> teknico, I like false alarms compared to the alternatives ;)
<teknico> fwereade: I'm sure we all do :-)
<dimitern> fwereade, i need to step out for a while, but please take a look after the meeting is over
<sinzui> mgz, I want to mark https://bugs.launchpad.net/juju-core/+bug/1236446 invalid. I do see a proper upgrade with a clean control bucket and only using public-bucket-url
<_mup_> Bug #1236446: Cannot upgrade-juju from 1.14.1 to 1.15.1 on openstack <openstack> <regression> <juju-core:Triaged by gz> <https://launchpad.net/bugs/1236446>
<mgz> sinzui: that seems reasonable
<fwereade> mgz, sinzui: ok, I guess -- but the maas bug is still real and applies to openstack in certain circumstances, right?
<mgz> ? the mass bug is double-fixed on trunk
<mgz> unless you mean something else?
<mgz> *maas
<fwereade> mgz, https://bugs.launchpad.net/bugs/1236622
<_mup_> Bug #1236622: Unable to upgrade from 1.14.1 to 1.15.1 on maas environment <juju-core:Triaged> <https://launchpad.net/bugs/1236622>
<sinzui> fwereade, yes, the mass bug is really about supporting old and new locations for a transition
<mgz> ah, haven't followed that one at all
<fwereade> mgz, I think the heart of it is "sync-tools must also copy tools to old locations (if the old location has any tools (?))"
<sinzui> mgz, you can see what I have been doing in release-public-tools to build a tree then deploy it for old and new. I image you have done the same
<fwereade> mgz, sinzui: and I *think* that exactly the same applies to anyone who's got tools in their private bucket, basically
<mgz> I think that may need some Ian input
<sinzui> mgz, I was going to talk to wallyworld tonight. I wanted to talk to him about integrating the future public key we will use for signing
<fwereade> sinzui, mgz: I was going to ask ian to look at that tonight on the basis that he's most likely to spot weird corner cases
<sinzui> okay
<fwereade> sinzui, and I have natefinch looking into the tests that fail for release versions now
<rogpeppe> fwereade: FWIW both juju ssh and debug-log seem to working fine for me under ec2
<sinzui> fwereade, fab. I was going to report that as a bug. Do we have a procedure problem because branches are landing but we have not inced the version?
<fwereade> sinzui, I don't *think* so, because in my mind it's not 1.16 until the version we finally release
<fwereade> sinzui, but this may be weird and non-standard?
<fwereade> sinzui, since you're handling the releases I would like us to follow a model you're comfortablewith
<sinzui> oh good, then this might save me an email to the list. I forked the stable branch, but since it is unchanged we can fork again or possibly merge a group of revisions, such as 1955-1960 from dev into stable
<sinzui> fwereade, I want to be certain 1.16.0 has ever revision we care about, as 1.14.0 did not
<fwereade> rogpeppe, ok... and now it seems to work for me on ec2
<rogpeppe> fwereade: i need approval for this to be landed on 1.16, BTW: https://codereview.appspot.com/14540044/
<fwereade> rogpeppe, how about local?
<rogpeppe> fwereade: local is buggered for me
<rogpeppe> fwereade: i can't ssh or debug-log at all
<mgz> I also have a 1.16 cherrypick to be rubberstamped
<rogpeppe> fwereade: i'm looking into that
<fwereade> mgz, would you link me your branch and try to repro rog's issue while I do those then please? :)
<mgz> https://codereview.appspot.com/14546044
<fwereade> mgz, (sorry, unless I'm forcing a big context switch on you_
<fwereade> rogpeppe, hold on though
<mgz> I'll spin up a saucy instance and try the local provider
<fwereade> rogpeppe, does local provider work for you at all? or are you seeing teknico's issue?
<fwereade> rogpeppe, mgz, because there's https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1236726
<mgz> fwereade: it's odd, it appeared to work, but there was no evidence of any actual lxc containers
<rogpeppe> fwereade: it works fine for me apart from ssh and debug-log
<rogpeppe> fwereade: it really *was* working - i could juju status and everything
<fwereade> rogpeppe, debug-hooks too?
<rogpeppe> fwereade: i've never used debug-hooks
<rogpeppe> fwereade: let me have a look
<mgz> no, and I'm not sure you started another machine?
<mgz> did you get as far as making a wordpress or anything?
<fwereade> rogpeppe, it's pretty cool actually :)
<rogpeppe> fwereade: do i need to run that in a vt100 emulator?
<mgz> I think lxc was probably just borked as in that bug maybe
<fwereade> rogpeppe, try it and see
<mgz> rogpeppe: also I think I now realise what the ssh problem was
<rogpeppe> mgz: oh yes?
<mgz> bet it was trying to ssh to a 10. address that was bridged back to localhost
<mgz> not a container at all.
<mgz> the state server stuff all just runs uncontained on your machine
<mgz> hence talking to that worked
<fwereade> mgz, LGTM
<rogpeppe> mgz: ah yes, that's indeed the case
<mgz> so, `juju ssh 0` on local is likely just non-operational
<fwereade> guys, I have to dash out to the shops for the sake of familial harmony,bbs
<rogpeppe> fwereade: k
<mgz> (well, `juju ssh MACHINE` is borked on local provider anyway still I think)
<fwereade> mgz, oh *crap* ofc it is, DNSName doesn't work, doesit?
<mgz> fwereade: drink deep of the sake of familiy history
<fwereade> haha
<mgz> *familial harmony even
<rogpeppe> mgz: i'm trying to deploy a service in the local provider, and it seems stuck in pending. that may be because it's downloading a precise image, but it may be borked, and i'm not sure how to tell the difference
<mgz> that is a good question
<mgz> look at network traffic? :)
<rogpeppe> mgz: ha, just tell me if i'm being stupid, but i *think* the test in local.localInstance.DNSName is just backwards
<rogpeppe> mgz: join in the fun on the hangout if you like
<mgz> I'm there
<jamespage> davecheney, your juju dev PPA is using my personal golang-backports which is old and buggy
<natefinch> am I the only one who hates tests that match based on error strings?  expected "tools not found", expected "no matching tools available" .  Is that failing because someone changed the error string, or because it's the wrong error type?  I can't tell from the test output.
 * jamespage fixes that
<natefinch> er expected <foo> obtained <bar>, obviously
<jamespage> davecheney, fixed and updated bug 1226902
<_mup_> Bug #1226902: ppa builds are built without cgo <juju-core:In Progress by dave-cheney> <https://launchpad.net/bugs/1226902>
<TheMue> natefinch: in my private code i'm using an error type with an error code that can be tested (additionally message and payload if wanted)
<TheMue> so, have to step out
<TheMue> good night, cu tomorrow
<fwereade> natefinch, I believe that if we *don't* match on error strings we impose horrible ones on users -- that'snot to say we shouldn't match types too though
<natefinch> fwereade: I don't know that matching error strings really makes the error strings better.  Most of the time the code is looking to make sure the right error is returned.  A test can't tell you if the error string makes sense.
<rogpeppe> fwereade: so... mgz and i have been delving into the local provider stuff
<rogpeppe> fwereade: there are a few interesting bits
<rogpeppe> fwereade: for example, you can't use lxc's AllInstances unless you're running as root
<rogpeppe> fwereade: although it doesn't return an error...
<rogpeppe> anyway, i'm done for the day
<rogpeppe> g'night all
<natefinch> rogpeppe: g'night, thanks for looking into that stuff
<fwereade> rogpeppe, thanks
<fwereade> and mgz also :)
<hazmat> rogpeppe, re pending or not sudo lxc-ls --fancy will give some info into the container status
<fwereade> natefinch, I dunno, I've found that most of our really awful error messages are the ones that have been hiding away behind .* matches
<hazmat> rogpeppe, if your on saucy, and the container is up you can just do lxc-attach -n container_name /bin/bash to enter into the container
<natefinch> fwereade: My policy is never to show error messages to a user.  Error messages are for log files and devs.  If there's a point where you need to show a message to a user, create a user-visible message right there.   There's often a huge different in requirements for a useful dev message and a useful user message
<natefinch> fwereade: granted, our tool is used by people a lot more technically inclined than most, so the difference is less huge
<fwereade> natefinch, I subscribe to the utopian notion that at least *some* of the users will tell you what the error message actually was, so the line's a little blurred, but... yeah, good point in general I think
<fwereade> natefinch, I feel it more strongly wrt log output vs user output
<natefinch> fwereade: definitely we should have both very useful log output and very useful user output.   And I don't know a good way to ensure that the error messages in either case are "good"
<natefinch> fwereade: btw, question - is there a way to run just one test through gocheck?  I know go test has test.run="regex"  but that doesn't seem to work with gocheck. Am I messing it up, or does using gocheck negate that functionality?  It seems to always run zero tests when I do that
<fwereade> natefinch, -gocheck.f (forfilter, i think)
<mramm> hey all
<fwereade> natefinch, I never tried -test.run so I don't know how perfectly it matches, but I imagine it'd be pretty close
<mramm> anybody know what is happening with this bug: https://bugs.launchpad.net/juju-core/+bug/1236622 ?
<_mup_> Bug #1236622: Unable to upgrade from 1.14.1 to 1.15.1 on maas environment <juju-core:Triaged> <https://launchpad.net/bugs/1236622>
<fwereade> mramm, heyhey
<mramm> I see it is critical and not assigned to anybody
<natefinch> mramm: we dropped maas support in 1.15.1
<mramm> and am trying to field questions about our release....
<fwereade> mramm, I'm going to get ian to do that overnight but wanted to talk to him
<fwereade> natefinch, haha
<natefinch> heh
<mramm> natefinch: hahahahahahaha
<mramm> hahahahahaha
<mramm> hahahahahahahah
<mramm> fwereade: cool
<fwereade> mramm, I'm pretty sure we have a decent handle on it, but ian's almost certainly the best person for it
<mramm> fwereade: cool
<mramm> assigned it to him
<fwereade> mramm, good idea
<fwereade> natefinch, any sanity apparent in those bootstrappy tests?
<natefinch> fwereade: not yet.  Mostly just looked like it can't find the tools for some reason.  trying to figure out what that reason is... it's just sort of 6 layers up in the code away from the tests
<fwereade> natefinch, I'm not sure it's related, but ISTM that bootstrap.go:180 is total crack
<fwereade> natefinch, ie (1) it's a lie and (2) if we make it true it's complete insanity because... oh, no, it's *probably* right but depends completely on weirdspecialpleading
<natefinch> fwereade: you mean the toolsSource?   The toolsSource that is then never used after that line?
<fwereade> natefinch, ah, but it turns out it actually *is* if you read further into what happens with the SyncContext
<fwereade> natefinch, it's just that that bit of code magically knows that that's the one that will be used :-/
<natefinch> fwereade: haha I see
<mramm> jorge is having some trouble deploying stuff to HP
<mramm> https://bugs.launchpad.net/juju-core/+bug/1237011
<_mup_> Bug #1237011: Can't deploy to HP Cloud due to region error <juju-core:New> <https://launchpad.net/bugs/1237011>
<jcastro_> if someone has a working hp stanza I'd like to check it out
<natefinch> Not me, Jorge.  Not sure who does the HP testing in dev
<rogpeppe> fwereade: how are we supposed to submit branches to 1.16?
<rogpeppe> fwereade: i was told there was no bot running, so presumably just approving won't work
<rogpeppe> fwereade: and i just tried lbox submit and it said "readonly transport" which presumably implies i haven't got push rights
<fwereade> rogpeppe, approve seemed to work for me earlier today...
<rogpeppe> fwereade: ok, i'm trying that
 * rogpeppe is off to play tunes
<rogpeppe> g'night all, again :-)
<fwereade> rogpeppe, have fun
<fwereade> natefinch, so I *think* what is going on is that bootstrap is working as intended -- ie it will not attempt to upload tools for release versions
<natefinch> fwereade: so is the problem just that we don't have 1.16 tools out anywhere for it to download?
<fwereade> natefinch, but I cannot remotely understand what the hell the justification is for *ever* auto-building tools
<fwereade> natefinch, developers sometimes forget to do it? tough shit
<natefinch> fwereade: haha yeah
<fwereade> natefinch, no excuse for fucking up the code :(
<fwereade> natefinch, if lack of tools elsewhere is a problem, that implicates poor test isolation
<natefinch> fwereade: that doesn't seem to be the problem, because I get the same problems when I set the version to 1.14.0
<natefinch> which I presume should otherwise work
<natefinch> but I also don't have a clear mental model on exactly what the tests are expecting to have where.
<fwereade> natefinch, yeah, I think the trouble is that it's working as intended for extremely confused and myopic values of intended
<fwereade> natefinch, I think the root of all of this is the desire to have a local env that doesn't try to sync tools from outside your laptop
<fwereade> natefinch, which is not an ignoble goal in itself
<fwereade> natefinch, but it was used to justify severe abuse of upload-tools, and I *think* we're seeing the distant but direct consequences
 * fwereade ciggie, think
<natefinch> heh
<fwereade> mgz, ping
<natefinch> fwereade: so.... it's definitely a problem that we have different code paths for dev builds vs. stable builds
<natefinch> fwereade: it means you can never really know that a dev build is stable, because the stable build could use different code paths
<fwereade> natefinch, yep, it's completely fucked up
<natefinch> fwereade: if it were me, I'd take out IsDev() entirely... it's just a bad idea
<fwereade> natefinch, and someone has deliberately added code to allow tests to run without isolation too, which makes me want to set fire to things
<fwereade> natefinch, IsDev has at least one legitimate(?) purpose -- to prevent accidentally upgrading a release version to a dev version
<fwereade> natefinch, (just cmd.Context, but *still*)
<natefinch> fwereade: that's one spot, and I would hide away that code so that no one else can easily get to it....  by making it a public function, people now are tempted to use it for nefarious (or at least questionable) purposes.
<fwereade> natefinch, that said, I think it's an isolation problem again actually
<fwereade> natefinch, different paths for dev/release are fine so long as people *test* them
<natefinch> fwereade: yeah,  I think you're right. It just bugs me that it only shows up in release builds
<fwereade> natefinch, it's not hard to patch out the current version
<natefinch> fwereade: no effort > minimal effort.  Devs are lazy. :)
<natefinch> fwereade: and busy.  and at times forgetful.
<fwereade> natefinch, indeed, we all are :(
<natefinch> fwereade: I have to get going. Family duties, unfortunately.  I'd like to try to understand better what the code is supposed to be doing vs what it is doing... I'll look at it in the morning.
<fwereade> natefinch, no worries, enjoy :)
<davecheney> thumper ?
<wallyworld> davecheney: holidays
<davecheney> bzzr
<wallyworld> sinzui: so, if i do a utility *just* to do the signing, you are happy to generate the tree locally using sync-tools with --source and --destination and then call "sign-tools" after that?
<davecheney> ok, will log a bug
<davecheney> does anyone know if the logging gripe that I had last week was fixed, or a bug raised ?
<davecheney> http://paste.ubuntu.com/6211472/
<wallyworld> not sure sorry as i was away last week
<wallyworld> fwereade: do you know ?  ^^^^^^^
 * davecheney looks at commit log
<davecheney> nope, doesn't look like it
<davecheney> will raise a bug
#juju-dev 2013-10-09
<sinzui> wallyworld, Yes, I think that would be lovely and "sign-tools" is clear
<wallyworld> ack
<wallyworld> will do that today after some other work - i'm making so that sync-tools always uploads to old /tools location until 1.16 is released
<wallyworld> should hopefully address maas issue where no public tools are available. i was really hoping that public repository would be done so that this would not have been necessary
<davecheney> https://bugs.launchpad.net/juju-core/+bug/1236662
<_mup_> Bug #1236662: environs/ec2: if the bootstrap node is restarted, all the agents in the environment cannot reconnect <juju-core:New> <https://launchpad.net/bugs/1236662>
<davecheney> https://bugs.launchpad.net/juju-core/+bug/1237151
<_mup_> Bug #1237151: set rpc logging to TRACE and agents back to DEBUG by default <papercut> <juju-core:Triaged by thumper> <https://launchpad.net/bugs/1237151>
<davecheney> https://bugs.launchpad.net/juju-core/+bug/1237163
<_mup_> Bug #1237163: juju deploys wrong machine when no machine matching constraints is available <juju-core:New> <https://launchpad.net/bugs/1237163>
<davecheney> sidnei: ping
<wallyworld> jam: i just kicked the landing bot in the bollocks. it seems that it had not run from about 08-10-13 15:20 UTC. i deleted the flock file in ~tarmac. what is a little confusing is that the lock file was dated jul 29.
<jam>  wallyworld: it isn't created, what is happening is a test run left a process running that has the file open
<wallyworld> why 29th july?
<jam> wallyworld: most of the time it is a rogue mongodb process
<jam> wallyworld: because we don't ever delete the file
<jam> it is just used with flock
<wallyworld> sure. but i would have thought it would have been dated around 8th oct or whatever
<jam> wallyworld: the "flock" model is that you open a file handle, and then flock it, and then all children that are spawned from that process have the file handle in their process
<jam> so until they all die
<jam> the lock is held
<jam> wallyworld: and when the process dies
<jam> it doesn't *delete* the file
<jam> we just keep using the same file
<jam> wallyworld: it isn't open(O_EXCLUSIVE)
<jam> it is flock(EXISTING_FILE)
<jam> though that file will be created if it doesn't exist
<wallyworld> ok
<jam> wallyworld: anyway, the *actual* fix is to kill 16262 whicth is a mongod process that has been running for over 6hours cpu time
<wallyworld> ah ok
<jam> wallyworld: so if the bot looks stalled, I generally bring up top, and look for a stale process
<wallyworld> ok, will do that next time
<dimitern> mornin' all
<fwereade> dimitern, heyhey
<fwereade> wallyworld, jam, axw: mornings
<wallyworld> hello
<jam> morning fwereade
<rogpeppe> fwereade, axw, jam, wallyworld, dimitern: mornin' all
<fwereade> rogpeppe, heyhey
<wallyworld> what he said
<axw> morning all :)
<dimitern> it's very jolly this morning it seems :)
 * fwereade was just looking at the constraints matching logic as referenced by davecheney, and What The Fuck
<fwereade> it looks like we ignore the heuristics to begin with
<fwereade> so ofc they don't get used
<fwereade> and if we fail we *then* just apply the heuristic and give them, eh, some shit, whatever, who cares
<fwereade> certainly nothing that matches the constraints though
<jam> fwereade: so I think the only heuristics we give are for mem, right? So you could argue we could check if there is a mem constraint, and if not set it.
<fwereade> yeah
<jam> fwereade: though what *should* we do if we can't find a match?
<fwereade> that's what we did when I wrote it
<jam> just fail, I guess
<fwereade> we should apply the heuristics *first*
<fwereade> try to get something matching user requests + our own preferences
<fwereade> (if there's no conflict)
<fwereade> if that doesn't work, just use the user requirements alone
<fwereade> if that doesn't work, FAIL
<axw> fwereade: so I just noticed today that the 1.16 release milestone is set for <24h from now
<axw> if you didn't realise, null provider is pretty hobbled at the moment
<axw> just proposing the last fix now, but I guess it's too late to get it in (as well as get the others approved)
<fwereade> axw, ok, can you give me the really high-level version please?
<axw> fwereade: sure, I'll just get the bug numbers to reference
<axw> fwereade: when bootstrapping the null provider, the prompt for "ssh sudo ..."  is hidden (https://bugs.launchpad.net/juju-core/+bug/1235716)
<_mup_> Bug #1235716: sshstorage does not display sudo prompt <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1235716>
<axw> fwereade: when bootstrapping into a machine that already has tools, bootstrap fails completely (https://bugs.launchpad.net/juju-core/+bug/1235717)
<_mup_> Bug #1235717: null provider bootstrap fails with error about sftp scheme <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1235717>
<axw> fwereade: if you try to bootstrap a machine whose series doesn't match default-series, it thinks it can't find tools (https://bugs.launchpad.net/juju-core/+bug/1236691)
<_mup_> Bug #1236691: null provider bootstrap fails if default-series does not match target <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1236691>
<axw> the sudo one is easy to fix easily, but maybe not if ctx.Stdin/Stdout etc. have to be threaded through juju
<axw> the others have fixes but are a bit more involved
<axw> fwereade: they all have workarounds, so they're not crippling, but do lead to poor user experience
<fwereade> axw, tbh I am not inclined to make those part of 1.16 -- we've still got the null provider documented as experimental, right?
<fwereade> axw, so if they have workarounds I really don't want to take the instability risk today
<axw> fwereade: we do, though that doc is still pending approval..
<axw> fwereade: fair call
<fwereade> axw, is that into juju-core/docs? sorry, I haven't really been following reviews there
<axw> fwereade: tho the sudo one is good risk/reward ratio I think
<axw> fwereade: yeah
<axw> fwereade: https://code.launchpad.net/~axwalk/juju-core/manual-provisioning-docs/+merge/188501
<fwereade> axw, for doc reviews that languish, it would probably be best to lean on evilnickveitch, marcoceppi_, or jcastro
<axw> fwereade: okey dokey, ta
<fwereade> wallyworld, I see you have signing ones up, I'm about to take a look at those (also need breakfast... :/) -- any joy re sync-tools to old location? or should I get someone else to take a look? must be late for you now
 * fwereade hears that a wonderful person has made him toast, goes and eats it, bbs
<evilnickveitch> axw, fwereade it is in the queue! there is some other complicated stuff landing at the moment
<axw> cool, thanks evilnickveitch
<axw> sync-tools is reviewed, fwereade
<fwereade> axw, sorry, which one? sync-tools-to-old-location-in-1.16?
<fwereade> axw, wallyworld: that looks like it's not fixed
<fwereade> axw, all I see is an MP for juju-core
<fwereade> wallyworld, ^
<axw> fwereade: ah sorry, just in juju-core
<fwereade> axw, wallyworld: nothing for juju-core/1.16
<fwereade> axw, wallyworld: that should surely *not* be in trunk at all, should it?
<axw> good point
<fwereade> axw, wallyworld: trunk is just for code >= 1.17, and 1.16 is the last time we need that code
<fwereade> wallyworld, can we back that out and retarget it please?
<wallyworld> fwereade: i thought trunk was still for 1.16
<wallyworld> have we branched
<fwereade> wallyworld, we branched from 1.15.1 to the 1.16 series
<wallyworld> oh ok. i didn't realise. was there an email
<fwereade> wallyworld, probably not, I'd thought it was already "normal" :(
<wallyworld> i would have thought we only branch when 1.16 is cut
<wallyworld> i guess there's two opinions on when to branch - before or after release
<fwereade> wallyworld, I'd prefer to keep putative releases isolated from the hurly-burly of trunk
<axw> wallyworld: I think 1.15.1 is kinda like 1.16.0 rc1
<wallyworld> fair enough. can be argued either way :-)
<fwereade> wallyworld, yeah, what axw said
<wallyworld> branching early means more work. branching later means more care needed
<fwereade> wallyworld, axw: (if we had a mechanism that allowed for things like "RC1" it would be great, wouldn't it)
<axw> ;)
<wallyworld> fwereade: so all my 4 or so branches i have up - i now need to land in two places :-(
<wallyworld> well, all except the one above
<axw> fwereade: I thought I had the 1.16.0 tests fixed, but apparently they fail on precise .. still looking into it
<fwereade> axw, yeah, I was looking at that and I'm afraid I... just don't really understand wtf is going on there
<axw> fwereade: something to do with the metadata, I don't know where it's coming from tho. anyway, just letting you know I'm still looking at it
<fwereade> axw, I see no evidence that what we're doing makes any sense in the first place in either dev or releasemode
<axw> fwereade: yeah... quite a burden for what we get out of it
<fwereade> axw, wallyworld: any idea why we have two changes to upload, called at different times depending on the provider kind (which the command should not know anyway)?
<fwereade> s/changes/chances/
<wallyworld> where is this in the code?
<fwereade> wallyworld, cmd/juju/bootstrap
<axw> fwereade: you mean how "local" always forces upload?
<fwereade> .go
<fwereade> axw, yeah, that is crack for a start
<fwereade> axw, and then later there's stuff to auto-upload *anyway*
<fwereade> axw, so it's not just a massive layering violation, it's also completely unnecessary
<fwereade> I *think*
<wallyworld> fwereade: the local provider hack was done by tim for a reason i can;t recall, but i think it had to do with making tools available for the lxc container
<wallyworld> maybe because we "know" there are never tools available for the local provider?
<wallyworld> so we need to force an upload?
<fwereade> wallyworld, it was done by tim because oscon, and never tidied up because I wasn't able to convince him it was crack until iom, at which point more things were on our plate
<fwereade> wallyworld, but it's not even upload-tools
<fwereade> wallyworld, he completely changed the meaning of upload-tools
<fwereade> wallyworld, so now upload-tools and sync-tools are just kinda different ways to do the same thing
<wallyworld> upload tools calls sync tools
<wallyworld> because sync tools knows how to deal with the metadata etc
<fwereade> wallyworld, right, but it's not actually syncing -- and while they used to be separate they arenow so entangled that I have no idea wtf to do about them
<wallyworld> i need to look at the code in more detail to re-grok it
<fwereade> wallyworld, anyway, I don't mean to whine at you, you have been a powerful force for sanity
<davecheney> what is the add-machine syntax to make a new lxc continaer ?
<TheMue> morning
<rogpeppe> davecheney: lxc:1 ?
<davecheney> rogpeppe: muchas gracias
<wallyworld> fwereade: no problem, feel free to vent :-)
 * fwereade breathes deeply for a moment ;)
<fwereade> wallyworld, ok, so, I will be reviewing your branches for both trunk and 1.16 I think
<wallyworld> fwereade: yeah, everything that's up now is intended for 1.16
<wallyworld> fwereade: i'm also doing the plugin juju_env one for davecheney
<fwereade> wallyworld, can you back the sync-tools fix out of trunk and get it into 1.16 now while I focus on those for a bit?
<wallyworld> sure. just about to have dinner though, so after that
<wallyworld> fwereade: i assume the bot handles proposals against 1.16 ?
<fwereade> wallyworld, yeah, it did for me yesterday anyhow ;p
<wallyworld> cool :-)
<davecheney> % juju add-machine 0
<davecheney> error: malformed container argument "0"
<davecheney> WHUT ?
<fwereade> wallyworld, https://codereview.appspot.com/14524043/ reviewed
<fwereade> davecheney, add-machine lxc:0
<fwereade> davecheney, "0" already exists, can't be added again
<wallyworld> fwereade: the empty key is moot because it won't be used cause there's no signed tools metadata
<wallyworld> fwereade: and i plan to get the key off curtis or ben howard tonight (i hope)
<fwereade> wallyworld, ha, ofc -- do we have one incoming from sinzui?
<fwereade> wallyworld, ok, cool
<fwereade> wallyworld, LGTM for both then
<wallyworld> \o/ thanks :-)
<wallyworld> i'll let you mark them as such and i'll land to trunk and 1.16 after dinner
<fwereade> axw, https://code.launchpad.net/%7Eaxwalk/juju-core/lp1235717-refactor-simplestreams-writemetadata/+merge/189767 is for one of the bugs you linked above, right? I'll be ignoring it for a bit then
<axw> fwereade: yes indeed
<fwereade> wallyworld, likewise https://codereview.appspot.com/14530043/ -- very clean, ty
<jam> davecheney: https://bugs.launchpad.net/bugs/1237151 thumper is away this week
<_mup_> Bug #1237151: set rpc logging to TRACE and agents back to DEBUG by default <papercut> <juju-core:Triaged by thumper> <https://launchpad.net/bugs/1237151>
<axw> fwereade: I've been leaving this https://codereview.appspot.com/14218044/ until I rework according to your comment
<axw> not necessary for 1.16.0 right?
<fwereade> axw, I don't think it is, AFAIK everything works since we took the daily streams out of azure
<axw> fwereade: yeah azure is fine. it's just a potential surprise for private cloud users
<axw> no wait..
<axw> it's fine for htem at the moment
<axw> never mind
<fwereade> axw, I *think* it's fine, yeah
<fwereade> axw, hmm, what do we need admin-secret and ca-private-key locally for?
<davecheney> jam: i wanted to take a crack this arvo
<davecheney> but got pulled into support
<jam> fwereade: we need admin-secret in order to connect to mgo for the client, don't we ?
<axw> fwereade: yeah, just for the cli
<jam> davecheney: had to look it up: http://www.urbandictionary.com/define.php?term=arvo
<jam> :)
<axw> s/davecheney/davo/
<fwereade> axw, given that we've connected already, can we not get away without them?
<axw> fwereade: heh, that would make sense :)    I only put it in there because it was failing... I'll take a closer look at why
<davecheney> jam:  i'm sorry, it must be my accent
<davecheney> oh crap
<davecheney> add-machine lxc:1 created a new container, the juju didn't use it !!
<jam> davecheney: is it a constraint issue?
<jam> (not enough mem in the instance, etc)
<axw> fwereade: 1.16.0 tests should be fixed now, do you want to review the CL before I land on 1.16 as well? https://codereview.appspot.com/14521054/
<fwereade> axw, I think I need you to explain how the change works, and what happens in dev vs release mode
<axw> fwereade: documented here https://bugs.launchpad.net/juju-core/+bug/1237123/comments/1
<_mup_> Bug #1237123: cannot set version to 1.16 <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1237123>
<fwereade> axw, all I know is that I don't like or trust what we do today and so I'm suspcious of tweaks to it ;)
<axw> if that doesn't explain it, lemme know
<fwereade> axw, ah great, I will try to understand :)
<TheMue> shit
<TheMue> wife just detected that a wheel of the car is flat :(
<fwereade> axw, // If Dev is already true, leave it alone. seems to be conditional on dev being *false*-- am I just misreading or something?
<fwereade> axw, ohwait I see
<axw> fwereade: sorry, did I add to the subtlety? :)
<axw> fwereade: you're completely right about admin-secret not being needed there. my test broke because I took admin-secret out of my environments.yaml... but that branch doesn't have the change needed to Prepare admin-secret
<fwereade> axw, I guess I still don't get what MinorVersion being set implies about devness
<fwereade> axw, ok,but Prepare should be completely irrelevant here, right? nothing fails without admin-secret exxceptbootstrap AFAIK
<axw> fwereade: yeah, my bootstrap command was failing because admin-secret was blank.
<axw> hmm
<axw> but why not when I copied it.. wtf
<axw> fwereade: umm, yeah not really sure why re the MinorVersion either tbh
<axw> that is the same as before
<fwereade> axw, yeah, I can't figure it out either, and that's what scares me about the whole business
<fwereade> axw, changing things without fully understanding them
<fwereade> axw, SyncContext looks like it's got altogether too many magical meanings for the different fields
<fwereade> TheMue, I don't suppose you can shed any light on this? (on your return, I guess...)
<axw> fwereade: I'll have a look thru the log to see if I can figure anything out...
<fwereade> axw, thanks
<axw> fwereade: wallyworld changed this at r1807
<axw> wallyworld: ... any hints on the rationale for tying SyncTools.Dev to MinorVersion?
<axw> fwereade: I think I must've had a stale .jenv file when I was testing the prechecker stuff
<axw> one with a admin-secret in it
<axw> so yeah... that code can/will go
<TheMue> fwereade: eh, sorry, didn't follow. now have to wait for a technician, so I can continue here.
<TheMue> fwereade: what's the problem I shall investigate?
<fwereade> TheMue, we don't really know what's going on with SyncContext and wondered if you knew about it
<fwereade> TheMue, https://codereview.appspot.com/14521054/patch/5001/6004 -- "syncContext.MinorVersion != -1"
<jam> wallyworld: I just feel pretty strongly after some time that juju-metadata *shouldn't* be a plugin
<rogpeppe> fwereade: i responded to your review here: https://codereview.appspot.com/14395043
<TheMue> fwereade: off the cuff not, but will take a look
<wallyworld> jam: i agree. i never wanted it as a plugin in the first place
<rogpeppe> jam: i agree too.
<rogpeppe> jam: i never really understood the "too many commands" rationale for making it a plugin
<wallyworld> axw: let me think - i can't recall off hand
<jam> rogpeppe: especially given it is installed by default, so we still have those command
<jam> s
<rogpeppe> jam: ha, yes
<wallyworld> jam: i think juju help commands was the reason
<wallyworld> people didn;t want the list of commands to be tool large
<jam> wallyworld: we could easily hide commands in help commands if we wanted to
<jam> seems a better fix
<rogpeppe> wallyworld: how do you mean?
<jam> rogpeppe: didn't want to overload people looking for help with X with too many commands that might be useful perhaps
<rogpeppe> jam: why would we want to hide it? it's a perfectly good command.
<jam> I think that falls into "juju help" being opinionated
<wallyworld> rogpeppe: well, juju help commands shows a list of all commands, and people didn't want that list to be too large
<jam> and "juju help commands" giving everything
<wallyworld> rogpeppe: i don't buy that argument. i was "forced" to make it a plugin
<wallyworld> axw: it had to do with the legacy tools search functionality and the change to exact minor version matching
<wallyworld> from memory
<wallyworld> so, the old functionality searched for all tools with major version = 1 (say)
<wallyworld> and this excluded dev versions i think
<jam> axw: MinorVersion == odd means it is a dev version
<jam> that has been the internal case for a long time
<jam> long before wallyworld came along (IIRC)
<wallyworld> and with the minor version matching, if we wanted a match for 1.13 (say), we needed to set dev = true
<axw> jam: the code is just checking MinorVersion != -1 (i.e. minor specified)
<jam> the build > 0 was a newer thing, but it was intended for "--upload-tools" to always generate a Dev version (vs oficial release)
<axw> wallyworld: ok, I think I understand that
<jam> axw: ah, see wallyworld's comment.
<axw> wallyworld: perhaps it should be checking if it's -1 and odd then.
<jam> if you want to match exact minor version, and you're running dev you have to search dev
<axw> !=-1
<jam> axw: -1 is always odd, right?  :)
<axw> err yeah :)
<wallyworld> -1 is the default
<wallyworld> arg value
<axw> right, so -1 is special
<wallyworld> to distiguish from 0
<axw> fwereade: answer is ^^
<wallyworld> cause someone could search for major.minor = 2.0 (say)
<fwereade> wallyworld, ah,I didn't see where -1was the default -- I saw a SyncContext somewhere that didn'tseemto set them,but then I see there's sometransformation thereof above
<fwereade> wallyworld, axw: but, OK, I think I'm convinced by wallyworld's LGTM
<wallyworld> fwereade: i did a LTGM on inspection only. it seems ok. but the proof is does it fix the bug and not cause regressions.
<fwereade> wallyworld, axw: I think there's some call for a tech-debt bug around bootstrap at least though, it shouldn't be this hard to follow
<wallyworld> +1000
<wallyworld> fwereade: it has all sort of "evolved" incrementally into a mess
<fwereade> wallyworld, yeah, I know how that happens ;)
<wallyworld> it needs to be properly refactored
<fwereade> wallyworld, and, hey, we got value from the tech debt, but we should pay it down before it cripples us
 * fwereade does a bit more deep breathing
<jam> axw: bug 1237295, are there prechecker tests that could actually become stale in the current system? Stuff like not supporting LXC won't change without a code change anyway
<_mup_> Bug #1237295: state.Prechecker/Environment in machine-agent may become stale <juju-core:Triaged> <https://launchpad.net/bugs/1237295>
<wallyworld> yep. the short term focus was saucy and simplestreams working
<axw> jam: no
<wallyworld> axw: i'd love it if the change were tested manually to make sure it all hangs together
<axw> not currently a problem
<fwereade> axw, jam: Prechecker isn't merged yet is it?
<axw> fwereade: no
<axw> that's in preparation for my repropose
<fwereade> axw, jam: it *would* be nice to get that into 1.16 actually, because someone already got confused creating a container on ec2
<axw> wallyworld: I have done a brief test, I'll do some more in a bit
<fwereade> axw, jam: for some reason I am a little bit scared of it, but I can't figure out why
<fwereade> axw, thanks, +100
<jam> fwereade: it does feel a bit like something that could screw up and not let you do something that you *should* be able to do
<axw> fwereade: probably because it hunkers down into state ;)
<jam> so I'm reasonably ok giving it some more ongoing testing in a 1.17 series
<axw> it does seem a bit late to put it in imho
<jam> fwereade: so here's a destroy-environment issue
<fwereade> jam, oh yes...
<jam> "juju bootsrap" tries to create a private bucket, fills in that detail into the .jenv. But you're on a private cloud so you have to upload your own image metadata
<jam> so you shove it in your private bucket
<jam> and then bootstrap again
<jam> but bootstrap doesn't work because of the empty provider-state file left around the first time
<jam> so you "juju destroy-environment"
<jam> but now it lost what private bucket you were using
<jam> so bootstrap can't work
<fwereade> TheMue, ahh, here's dstroppa, would you have a word with him about the OS X build? and how practical it actually is to run non-statey provider tests on non-ubuntu?
<jam> because it won't be able to find the metadata you just uploaded to your private bucket
<davecheney> fwereade: juju deploy --to lxc:1 mysql just did this too me
<davecheney> http://paste.ubuntu.com/6213065/
<davecheney> but I cannot reproduce the problem
<davecheney> fwereade: can you think how this is possible
<davecheney> and if I should
<davecheney> a. raise an issue
<davecheney> b. pretend it didn't happen
<jam> davecheney: the syntax "--to lxc:1" means to create a new LXC on machine 1 and deploy to it
<jam> davecheney: "--to 1/lxc/1" is the way to deploy to the first LXC container on machine 1
<jam> sorry
<jam> 1/lxc/0
<jam> davecheney: so *as designed*
<jam> though confusing, I understand
<davecheney> jam: explain this
<davecheney> http://paste.ubuntu.com/6213071/
<jam> "lxc:1" is always "a new LXC on 1"
<davecheney> ^ destoryed env
<davecheney> ran it again
<davecheney> got this
<davecheney> noce 1/lxc/0
<davecheney> note
<jam> davecheney: local provider?
<davecheney> jam: hells no
<davecheney> ec2
<jam> bug #1237259
<_mup_> Bug #1237259: juju status reports 'missing' instance-state when not run as root <juju-core:New> <https://launchpad.net/bugs/1237259>
<davecheney> jam: false
<jam> ah, should have been clear about the ec2* addresses
<davecheney> jam: look at the machine id's
<jam> davecheney: "juju-machine-1" ?
<davecheney>       mysql/0:
<davecheney>         agent-state: pending
<davecheney>         machine: 1/lxc/0
<jam> davecheney: that is what it is supposed to be
<davecheney> yes, but the first time it wasn't
<jam> 1/lxc/0 is the first LXC instance on macihne 1
<davecheney> http://paste.ubuntu.com/6213065/
<jam> davecheney: so the first time you did "juju add-machine lxc:1" which means create a new LXC on machine 1 and add it
<jam> davecheney: then you did "juju deploy --to lxc:1"
<jam> which means "create *another* lxc and deploy to it"
<fwereade> davecheney, so if you ran exactly the same commands and got those different results there is surely a problem -- but what exactly did you run?
<jam> if you did "juju deploy --to 1/lxc/0" it would have used the existing container
<davecheney> fwereade:
<davecheney> juju bootstrap
<davecheney> juju add-machine
<davecheney> juju deploy --to lxc:1
<davecheney> juju deploy --to lxc:1 mysql
<davecheney> juju status
<davecheney> it's like sometimes juju doesn't find the container it expected
<davecheney> and creates another one
<fwereade> davecheney,  `--to lxc:1` always means "create a new container"
<davecheney> fwereade: yes, so mysql/0 should have been created on 1/lxc/0
<davecheney> but sometimes machine 1 gets a second container, 1/lxc/1
<fwereade> davecheney, you were talking about doing explicit `add-machine lxc:1` at one stage,I thought?
<davecheney> fwereade: i'm trying to reconfirm tat
<davecheney> in which case
<davecheney> add-machine lxc:1 and deploy --to lxc:1 mean _different_ things ?
<jam> davecheney: they *both* mean create a new LXC and use it for this operation
<jam> create a *new* LXC
<fwereade> davecheney, they mean exactly the same thing -- create a new lxc container on machine 1
<jam> is key here
<davecheney> oh
<davecheney> riht
<davecheney> i understand now
<davecheney> lxc:machine-number
<davecheney> not lxc:container-number
<fwereade> davecheney, yeah
<davecheney> right
<davecheney> sorry for the noise
<davecheney> am trying to trace a problem that jpds hit deploying mysql into a container
<TheMue> sorry guys, i'm off due to the defect for some time, will continue later :(
<davecheney> got sidetracked
<fwereade> davecheney, it's the specific form of a more general <pseudo-provider>:<location> syntax
<fwereade> davecheney, that should always imply new machine creation
<davecheney> fwereade: right
<davecheney> thanks
<fwereade> davecheney, np, it's nice to find things that aren't bugs ;)
 * davecheney goes back to tracing real bugs
<jam> wallyworld: fetchToolsHash assumes it can just http.Get any URL, it doesn't go via any of the DataSource objects
 * wallyworld looks
<jam> so it (a) breaks with ssl-hostname-verification: false and (b) will fail if we don't have readable stuff, but I guess tools always need readable stuff
<rogpeppe> jam: axw is on the road to fixing that
<jam> rogpeppe: fixing what part ?
<rogpeppe> jam: the fact that SyncTools uses http.Get at all
<wallyworld> jam: it was assumed that tools as used for calculating the hash would always come from public urls
<axw> rogpeppe: there's a CL up for that btw
<wallyworld> that was previously the case also with sync tools i think
<rogpeppe> axw: oh cool
<jam> wallyworld: right, even in "private-bucket" we have to set it up as world-readable so that any system can read the tools
<jam> *but*
<jam> when you have to set ssl-hostname-verification: false
<jam> that breaks your http.Get
<wallyworld> oh
<jam> wallyworld: if you used the DataSource associated with the URL I've configured it to have the right settings
<rogpeppe> wallyworld: i don't think that can ever have been the case - the fetchToolsHash logic sometimes works on the target bucket which could always have been private
<jam> rogpeppe: it can't
<wallyworld> jam: with private bucket, we don't need world readable as storage URL() is used which gives a world readable url
<rogpeppe> axw: link
<jam> rogpeppe: because you always have to be able to download the tools for a machine that doesn't have any creds yet
<axw> rogpeppe: un moment
<jam> wallyworld: not on openstack
<jam> because openstack
<rogpeppe> wallyworld: it doesn't for the local provider
<jam> doesn't have signed urls
<axw> rogpeppe: https://codereview.appspot.com/14527043/
<jam> wallyworld: because TemporaryURL isn't by-default enabled on all the Openstack instances we know of
<wallyworld> jam: yes, but private bucket is not intended to be used that way for openstack
<rogpeppe> wallyworld: SyncTools should not use URL
<jam> wallyworld: inspect the code, private-bucket is always made world readable for openstack
<axw> rogpeppe: this doesn't change the SyncTools bits (much, yet)
<wallyworld> jam: oh yeah
<rogpeppe> axw: looking
<jam> if you use "--upload-tools" it must be
<jam> because that's where we write them, IIRC
<wallyworld> rogpeppe: why? sync tools can get tools from a local directory if it wants
<rogpeppe> wallyworld: because it's got the Storage with a Get method - URL is intended for sending across networks, which we're not doing here.
<wallyworld> the data can come across a network
<wallyworld> but if we have a storage abstraction, then we can use that
<jam> rogpeppe: so we know that we will eventually need to get the data via URL
<jam> because we do "wget" in cloud-init
<rogpeppe> jam: that's not an issue for SyncTools though
<rogpeppe> jam: and in particular for the target storage of the null provider, we don't have any url
<davecheney> you know, if the cloud images came with lxc and the precise template already downloaded
<davecheney> things would bootup a lot faster
<rogpeppe> davecheney: +1
<wallyworld> fwereade: this one will make davecheney happy https://codereview.appspot.com/14499044
<jam> davecheney: well we don't actually support lxc on ec2 today anyway... :)
<davecheney> jam: you cna make machines
<davecheney> that bit works
<axw> davecheney: not for long :)
<davecheney> wallyworld: that *does* make me happy
<davecheney> axw: you poo
<jam> davecheney: axw is subitting a patch to disable it
<davecheney> hang on
<jam> since we can't make them routable yet
<davecheney> starting lxc conatiners on precise is ultra fail
<fwereade> wallyworld, LGTM
<davecheney> i don't know why I'm even trying
<jam> davecheney: 12.04.03 + cloud-tools should create nice LXC instances, I believe
<wallyworld> fwereade: that was for 1.16 as well as trunk
<davecheney> jam: creating hem works fine
<jam> davecheney: and the ec2 images have been updated for 12.04.03 (which defaults to the raring HWE kernel)
<davecheney> dealig with the kernel oopses is the tricky part
<axw> fwereade, wallyworld: sync-tools and --upload-tools both work fine in my testing
<fwereade> wallyworld, so I assumed :)
<davecheney> jam: ORLEY ?
<wallyworld> axw: awesome!
<axw> good to merge into 1.16.0?
<jam> davecheney: 12.04.03 defaults to the raring HWE kernel from reports of other people
<fwereade> axw, go for it
<davecheney> jam: testing
<axw> jam: indeed it does
<axw> I installed the other day
<axw> fwereade: ta
<davecheney> GOD DANMING
 * wallyworld has about 4 branches to merge into 1.16 and trunk, and another to back out of trunk and put into 1.16. sigh
<davecheney> EC2 iS SO FUCKING SLOW
<davecheney> WHY IS EVERYTHING SLW
<fwereade> axw, jam: about Prechecker... how bad would it be to merge a 1.16-only hack that just blocks containers in non-maas environments only? in the addMachine gubbins somewhere?
<davecheney> i feel my life draining away every day I use this product
<fwereade> axw, jam: heavily flagged as crazy and dangerous, and *not* merged into trunk?
<davecheney> jam: axw
<davecheney> i call bullsht
<davecheney> ubuntu@ip-10-248-78-68:~$ uname -a
<davecheney> Linux ip-10-248-78-68 3.2.0-54-virtual #82-Ubuntu SMP Tue Sep 10 20:31:18 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
<wallyworld> davecheney: could be worse, azure sucks balls
 * bigjools giggles
<davecheney> wallyworld: true, the universe can always invent a slower mousetrap
<jam> wallyworld: is that a bad thing?
<jam> I guess it depends what you are looking for from your provider
<wallyworld> jam: depends if you like having your balls sucked
<fwereade> davecheney, *we* can invent slower mousetraps any time you like! sometimes without even trying
<wallyworld> bigjools does
<bigjools> nom
<jam> fwereade: it seems like the whole reason Prechecker was created was to handle the container issue, not landing it just to land a hackish version of it doesn't sound great.
<fwereade> jam, it's bigger than that
<davecheney> anywa, axw, you're full of crap
<jam> wallyworld: which is why bigjools worked on the azure provider. It all fits together in the end.
<davecheney> no HWE kernel
<wallyworld> lol
<axw> davecheney: eh.. I'm sure it did
<jam> davecheney: it is possible that raw installs do, but EC2 doesn't
<fwereade> jam, PrecheckInstance is at least as important, I think, even if we haven't got much in the way ofimplementations for it
<jam> as in, they updated the image but only for tools and not for the kernel
<davecheney> jam: as I understand it an AMI is the root disk
<fwereade> jam, but if people read about containers and want to experiment with them, we currently accept anything and fail late
<axw> sorry fwereade family just came home... reading back
<davecheney> there is a separate file with the kernel
<davecheney> not sure what it is called, AKI or something
<davecheney> we don't get to control that
<jam> fwereade: sure, but if we have code to do it in a reasonably nice way, it seems a bit silly to dump it for a hack because we aren't sure if the nice way is fully vetted. It feels like the hack will be *even less* vetted. Though perhaps focused enough to make the trade off worthwile.
<davecheney> why does every container run 6 gettys ?
<fwereade> davecheney, I though an AMI implied an AKI but you could swap it out if you knew what you were doing? could be wrong
<davecheney> exactly which virtual console are people going to connect ot ?
<davecheney> fwereade: I think you can
<axw> fwereade: umm, if we were going to put in anything, I'd say what's already there
<davecheney> but looks like it hasn't been done
<axw> but... we've gone without it for this long...?
<fwereade> axw, jam: well, the trouble is that what's already there is still kinda off -- the Prechecker looks like it's working better than it necessarily is, and when people come across that code they'll assume it works properly
<fwereade> axw, jam: eg they'll use the environ's config to determine request sanity
<fwereade> axw, jam: despite that config being arbitrarily out of date
<jam> fwereade: if it is a *code in the future* problem, then we should just land it. If it is a "code is slightly wrong right now" then we shouldn't
<axw> fwereade: oh sorry, are you saying something to include something independent of prechecker?
<fwereade> jam, I dunno, I feel that past lack of attention to code in the future has put us where we are today
<fwereade> jam, axw: I teeter still
<jam> fwereade: is axw actively improving this situation ? The bug he reported seems to be so, and it still seems better to land something that is being actively developed than something that we want to get rid of as soon as possible.
<fwereade> jam, axw: and in particular, historically, "ok land it and fix it"  has come back to bite us because priorities shift and there's no time to fix
<jam> so yes, be concerned about future code not getting enough attention, but do it by bringing it up and focusing on it in this case
<axw> so, bzr/lp noob question: is this the right workflow for backporting a fix?    branch 1.16, merge in specific revnos, create MP and approve?
<axw> jam: I have started on fixing that bug, but only just
<axw> I'm somewhere between 0 and -1 on putting a precheck change into 1.16 at this stage
<fwereade> axw, jam: I'm just concerned that the "containers work on maas" messaging is perilously close to "containers work"
<fwereade> axw, jam: and a straight rejection is a much better UX
<fwereade> axw, jam: disappointing, but not enraging
<axw> fwereade: totally understand and agree with that
<axw> just slightly risky I think
<jam> fwereade: so I agree, the disagreement is about "we have this stuff to do that, but we're going to punt it for the stable release and do a quick hack before we land the stuff we care about in the unstable release".
<jam> It may be that scope means we need to
<jam> but it doesn't seem less risky then landing what we've been developing
<wallyworld> jam: so i have a branch which diverged from trunk after 1.16 was branched. i naiively put up a mp of that same branch but targetted to 1.16. but the diff had extra stuff. so i think my only choice is to cherry pick my changes and do a new branch stacked on 1.16 and then a mp from there?
<jam> wallyworld: yes
<wallyworld> sigh
<jam> wallyworld: "bzr branch lp:juju-core/1.16 .../my/local/path" bzr merge $EXISTING_BRANCH -r X..Y
 * wallyworld wishes we didn't branch 1.16 so soon :-(
 * wallyworld prefers to branch later
<wallyworld> jam: yeah, i know how to cherry pick :-P
<axw> jam: I'm doing a backport now too; do I just do that, then push to LP, create an MP and approve?
<fwereade> wallyworld, it was only like 5 days ago ;p
<jam> axw: you need to target to the lp:juju-core/1.16 branch, you can do so with "lbox propose -for lp:juju-core/1.16"
<wallyworld> fwereade: but there was still much to do for 1.16
<axw> jam: ta
<fwereade> wallyworld, most of that only became apparent in those 5 days
<fwereade> wallyworld, and I'd rather have specific fixes against a known quantity than deal with unrelated changes in trunk
<wallyworld> perhaps. it's a religious argument :-)
<wallyworld> my view is that you slow down trunk development and do testing in the week before
<wallyworld> so you only commit to one branch and have goo qa and it's all still controlled
<fwereade> that's reasonable too, indeed
<fwereade> let's argue over beers in sfo :)
<wallyworld> yep. i wouldnt be complaining if i didn;t have so many fucking branches to double land :-)
<wallyworld> and i guess i sorta knew there was still much to do
<wallyworld> eg signing metadata yada yada
 * fwereade ciggie before meeting
<axw> wallyworld, fwereade: can you please review this for juju-core/1.16 -- https://codereview.appspot.com/14579044
<axw> I'll bbl, going to have dinner
<wallyworld> ok
<fwereade> TheMue, standup?
<rogpeppe>  fwereade, jam: g+ is misbehaving on me
<jam> mgz: standup ?
<jam> https://plus.google.com/hangouts/_/867e754a0a06dd4a7c52dc4e27457cd018da3960
<jam> mgz: TheMue ^^
<jam> wallyworld: was https://code.launchpad.net/~wallyworld/juju-core/plugin-env intended for 1.16 ?
<wallyworld> yes, i have to do it in both
<wallyworld> doing trunk first
<jam> k, I targetted it
<jam> your trunk is marked as merged
<wallyworld> yes, i just merged it
 * dimitern => lunch
<natefinch-afk> how come babies only sleep *after* meetings are over? :/
<sidnei> davecheney: pong
 * TheMue is back, and very happy *grmpflx* lost time and 120 bucks :(
<jam> can anyone do a quick review for landing in 1.16 branch: https://codereview.appspot.com/14494048/
<jam> hi TheMue, welcome back
<TheMue> jam: yeah, thanks
<TheMue> at least now trying to solfe the os x troubles together with dstroppa
<TheMue> solve
<jam> mgz: are you looking back at the bot charm now, or are you finishing up stuff for 1.16 ?
<natefinch> TheMue - I'd like to hear more about that, since the code cross compiles for OSX just fine (on trunk at least)
<TheMue> natefinch: yeah, here too, but not for dstroppa. that's wondering me
<TheMue> natefinch: where I have massive problems is to run the tests. 1st my mongo has no ssl built in, but the second one is regarding simplestreams
<mgz> jam: planning on charm poking, unless something needs looking at for 1.16
<natefinch> TheMue: I kinda doubt running the tests would work with cross compilation.
<jam> mgz: k, then if you can investigate http://dave.cheney.net/2013/07/09/an-introduction-to-cross-compilation-with-go-1-1 while you're at it
<jam> mgz: but if you want' you can review: https://codereview.appspot.com/14494048/
<natefinch> jam: reviewed
<jam> thanks natefinch
<mgz> that was fast :)
<jam> mgz: it is a <1 line change
<natefinch> mgz: cross compilation is basically just "install go from source, run a script to build the tools for all GOOS and GOARCH, then use the correct tool for the target system"
<TheMue> natefinch: I'm compiling native here
<natefinch> TheMue: ahh, ok.  Well, it's nice to know it compiles on native OSX, at least on someone's machine
 * natefinch is tempted to set up a hackintosh VM for testing
<TheMue> natefinch: already tried it a few weeks ago, w/o running the tests, but then installing our typical wordpress/mysql on ec2
<TheMue> natefinch: e'thing fine
<natefinch> TheMue: nice nice
<natefinch> TheMue: I broke it when I made changes for Windows for 1.14.  unfortunately, no one noticed until 1.15 was out
<natefinch> jam: ^^  we released 1.14 to OSX.... are we just releasing straight code for OSX, or binaries?   (I don't know how homebrew works)  Curious how it got released completely broken like that
<TheMue> natefinch: I don't know how homebrew works, never used it
<TheMue> natefinch: I've simply branched, gogetted ;) all 3rd party stuff and then go install
<natefinch> TheMue: looks like homebrew is basically git to download zipped code that then runs ruby files to install stuff.  Which sounds handy, but incredibly dangerous.
<TheMue> natefinch: ic
<TheMue> natefinch: for missing unix software i'm using macports which also loads sources and builds them directly
<TheMue> natefinch: but imho the downloaded sources are from a special source code repository
<natefinch> TheMue: yeah, homebrew seems to be touting itself as a macports alternative.
<natefinch> TheMue: http://brew.sh/
<jam> natefinch: I'm pretty sure homebrew builds from a source file
<jam> but I don't know it either
 * rogpeppe goes to lunch and a dentist's appointment
<TheMue> rogpeppe: oh, had dentist early this morning too
<TheMue> rogpeppe: thankfully only 5 min checkup
<TheMue> rogpeppe: good luck
<axw> fwereade: can I please get a review on https://codereview.appspot.com/14579044/ ? exactly same as the other one, but for 1.16
<fwereade> axw, ah, sorry
<fwereade> axw, ha, sorry, I even had a tab open already with an unpublished LGTM
<fwereade> axw, done now
<axw> fwereade: thanks :)
<wallyworld> fwereade: rightio, my 5 branches have landed in trunk and 1.16 and i've reverted the legacy tools code in trunk. i just need to get the public key for tools metadata and commit that. let me know if anything else comes up
<sinzui> you have been very busy wallyworld
<wallyworld> yes
<wallyworld> be good when release is done
<wallyworld> sinzui: if you get tip of 1.16, then you will have metadata signing support
<sinzui> faboo
<wallyworld> i just need the public key now :-)
<sinzui> me too
<wallyworld> i hope we get it in time to make the release
<axw> wallyworld: has the bot been merging your MPs for 1.16?
<axw> mine's been sitting there approved for 37 minutes
<wallyworld> axw: yes :-)
<wallyworld> axw: did you propose against 1.16?
<axw> wallyworld: yup
<axw> wallyworld: https://code.launchpad.net/~axwalk/juju-core/1.16/+merge/190085
<wallyworld> link to mp?
<wallyworld> i'll take a look, it may just be queued behinds others
<axw> thanks
<wallyworld> axw: running now
<wallyworld> almost done
<axw> wallyworld: cool, ta
<wallyworld> weird branch name though :-)
<axw> wallyworld: yeah I forgot to give it a useful name
<wallyworld> figured as much :-P
<axw> luckily I only hae one to do ;)
<wallyworld> so you think :-)
 * fwereade is seriously tempted to delete environs/instances and start afresh :(
 * axw runs
<axw> third system syndrome? :)
<fwereade> axw, heh
<fwereade> axw, just premature abstraction
<fwereade> axw, it's got ec2-specific bits mixed all through
<fwereade> axw, and it's completely broken
<axw> fair enough, I was just being facetious :)
<fwereade> axw, I know :)
<axw> fwereade: I'd be keen to hear what things you don't like about it, maybe at SFO
<fwereade> axw, mainly that if it can't find an instance type it just gives you whatever
<fwereade> axw, which is just... broken
<axw> ? :(
<axw> is that the constraints thing you were talking about before?
<wallyworld> fwereade: i remember talking about that with you and as i recall we originally returned an error but i think we were told we had to return *something*
<wallyworld> unless i'm mistaken
<fwereade> wallyworld, I think you promised to fix it but we all got distracted, it was a stressful time, just before april ;p
<fwereade> wallyworld, and I rationalized that since we had to have it on openstack animplementation that nostly worked for both beat one that only worked for ec2
<wallyworld> i can't properly recall now. i do recall that all the code was originally for ec2
<wallyworld> we deal with provider specific attributes by allowing them to be nil
<fwereade> wallyworld, and you (quite defensibly) really wanted to avoid copying logic -- so I'm not even sure it turned out *worse* than it would have done if you'd heeded my urging to keep them separate, we know well how that ends up usually
<fwereade> wallyworld, I'm just grumpy because it's just come fully to my attention that it's been broken for 6 months
<hazmat> fwereade, re bugs, if you have a moment, i'd like a moment to discuss https://bugs.launchpad.net/juju-core/+bug/1174610 which originally got marked opinion.
<_mup_> Bug #1174610: unit ids should be unique <juju-core:New> <https://launchpad.net/bugs/1174610>
<hazmat> it recently got expired automatically by lp, just wanted to get it back into the queue/hopper
<wallyworld> i think the logic works ok - deal with a bunch of common attributes, and handle possibly nil ones
<fwereade> wallyworld, and I'm having some difficulty seeing how to tease it apart
<wallyworld> it get complicated because we have instances and instance types and they have to be smashed together
<wallyworld> instances come from simplestreams, instance types from flavours
<wallyworld> i think we read the possible instances first, then look up the instances types, and then see what is compatible and also satifying constraints
<fwereade> hazmat, ok, I see -- does it matter for services as well, or just for units?
<wallyworld> if no constraints satisfied, it uses heuristics to just pick something
<fwereade> wallyworld, no
<fwereade> wallyworld, it skips the heuristics it's meant to use
<fwereade> wallyworld, and then if it can't find a matching instance type it *then* uses the heuristics to pick something that *doesn't* match
<hazmat> fwereade,  just for units really
<fwereade> wallyworld, and all that stuff is tucked away inside a helper function
<fwereade> wallyworld, when the falling backought to be top-level
<wallyworld> the heuristics are only used if a match cannot be found
<hazmat> fwereade, the service case was explicitly stated as a desired overlap
<wallyworld> a constraints based match shoudl always be used first surely?
<hazmat> fwereade, ie.. if i delete wordpress svc, i can get the same db back on reconnect to mysql
<fwereade> wallyworld, because we may get a list of matches from constraints-with-heuristics, but find that none has a matching image, and need to fall back to try again without
<hazmat> fwereade, where as log aggregation, metrics against a unit is bad overlap.
<fwereade> wallyworld, a constraints based match ignores the heuristics
<wallyworld> yes, as it should?
<fwereade> hazmat, ok, yes, I see
<fwereade> wallyworld, no!
<wallyworld> if i say i want xyz, then that's what i should get
<fwereade> wallyworld, that's why we're getting 512M instances on canonistack
<fwereade> wallyworld, and if you don't say anything, we shouldn't just give you useless heaps of rust because you didn't specify you wanted something useful ;p
<wallyworld> that's the argument i had at the time - i didn't want to do that
<fwereade> hazmat, ok, I see your point there -- and I can see why it's unreasonable to expect people to use plain int ids (which don't themselves exist anyway)
<wallyworld> but i'm not getting why the system should not use my constraints if i provide them. maybe i'm just being thivk
<fwereade> wallyworld, it should
<fwereade> wallyworld, but it should also try to give you something that's likely to work *as well as* matching your constraints
<wallyworld> well, the user should know best
<fwereade> wallyworld, if the only things that satisfy your constraints are piles of junk, well, fair enough
<wallyworld> yes , that's my point
<fwereade> wallyworld, and we fall back to the least worst of those
<wallyworld> if constraints are satisfied, you get what you asked for
<wallyworld> maybe there was a requirements misunderstanding - none of this was documented as far as i know
<wallyworld> it was all sorta verbal
<fwereade> wallyworld, it's a pretty close match for python, which was documented actually -- but, yeah, the precise mechanismchanged
<wallyworld> but i do recall a disagreement on whether to return an error if a match could not be found. i sorta wish we did return an error instead of crap
<fwereade> wallyworld, the main point is that your approach gives the user instances that do *not* match the constraints they asked for
<fwereade> wallyworld, instead of returning an error
<wallyworld> i did want to return an error
<wallyworld> i could have sworn that i was not allowed to
<fwereade> wallyworld, I must have just completely failed to communicate then
<fwereade> wallyworld, I'm sorry
<wallyworld> me too
<wallyworld> i think the wires got crossed
<wallyworld> i do recall not being sure at the time that what was done was ok or not
<fwereade> wallyworld, yeah, I was not in a very good place at the time, and probably not being helpful
<wallyworld> well, we both are to blame :-)
<wallyworld> i knew a lot less back then
<wallyworld> some would say not much has changed
<fwereade> wallyworld, ehh, no worries, I'm a lot happier knowing you're not wedded to the approach
<wallyworld> oh not at all
<wallyworld> i just want it to work
<fwereade> wallyworld, and, fwiw, I think you're great
<wallyworld> but i do think that having provider specific things as nillable values can work ok
<wallyworld> it;s just the matching logic that needs tweaking
<wallyworld> constraints were implemented using the same approach
<fwereade> wallyworld, yeah, I'm not really bothered by that side of it, except inasmuch as it's another thing to keep track of while unpacking it
<wallyworld> yeah
<wallyworld> my view is the common logic and all the benefits there outweight the unpacking cost
<wallyworld> so, we need to get this fixed for 1.16 i assume
<fwereade> wallyworld, I kinda feellike we do, now I've looked at it
<wallyworld> i'm quite tired now, but can pick it up after a few hours sleep
<fwereade> wallyworld, but you're not allowed to do any more work today, get some sleep
<fwereade> wallyworld, if I don't get something useful done myself I can hand over when you come back
<wallyworld> ok.i think the only thing i have to do next is commit the signing key
<wallyworld> so hopefully this can be sorted out before the deadline.
<wallyworld> fwereade: i think you would add the best value here by writing the correct tests and we then make those work. ie true TDD
<wallyworld> clearly the tests cases are wrong
<wallyworld> if stuff is broken
<wallyworld> even if the implmenation has to be patched together a bit to make 1.16, at least the tests will tell us things will be ok
<wallyworld> and we can fix properly for 1.18
<wallyworld> and the tests will document how it should work
<fwereade> wallyworld, yeah, that's the plan :)
<wallyworld> \o/
<sinzui> wallyworld, check your email for the public key
<wallyworld> sinzui: great, thanks
 * wallyworld will do a little branch to get that in
<sinzui> fwereade, jam (if you are about and care). I have a choice to make with the 1.16 series branch. I need to do a lot of merged, and if I want things to look perfect, do them in a different order to get the version fix, then inc the version, then merge everything else. I could merge everything else as a single branch. I could just re-fork 1.16 at the revision I want. At the moment, I don't see anything that has landed after 1954
<sinzui>  that I don't want in the 1.16 branch
<fwereade> sinzui, I couldn't swear to that... and I think we've also been merging to 1.16, haven't we?
<fwereade> sinzui, what is there in trunk that's not in 1.16?
<wallyworld> sinzui: both 1.16 and trunk now have public key. so if you have private key you should be able to sign tools metadata and juju should be able to use it
<sinzui> fwereade, damn, I was supposed to be getting merge notification on this branch. My fault I am sure
<sinzui> fwereade, wallyworld I will retry the inc 1.16 merge then and review the changes
<fwereade> sinzui, so, just today, a constraints bug came up, that I *really* want to fix... but I think actually the right thing is to keep it out of 1.16, people have been putting up with it since forever
<sinzui> fwereade, I have used the same thinking with some of the other issues saw yesterday. Regressions need fixing. Blockers to get to the new version need fixing. I don't think we need to delay a release for a fix that will come available in a few weeks with the next release
<fwereade> sinzui, yeah
<fwereade> sinzui, my desire to fix it is just based on personal upset/embarrassment
<sinzui> fwereade, If the issue is fixed before we name the release revision, I am always happy to include
<sinzui> it
<sinzui> fwereade, I do check for fix committed bugs and make sure they have a milestone, and I propose them in the stable release if I think they are safe
 * sinzui does just that for natefinch's lp:~natefinch/juju-core/019-osx-fix
<fwereade> sinzui, I probably won;t get it done without rushing, I'm still feeling my way a bit
<sinzui> okay
<fwereade> jam, rogpeppe: heh, did we actually assign that log-levels bug to someone?
<fwereade> sinzui, when are you aiming to pull the trigger?
<rogpeppe> fwereade: the one we were talking about in the meeting?
<fwereade> rogpeppe, yeah
<rogpeppe> fwereade: i don't think so
<rogpeppe> fwereade: i had a brief glance at fixing the RPC logging to avoid logging pings; it's not that easy, sadly.
<fwereade> rogpeppe, bugger... and it sort of trailed off without consensus
<sinzui> fwereade, I see 7 bugs for 1.16 and 1 doesn't appear to be started. I don't think I can start the release for 8 hours. I really think I will be in 24 hours
<rogpeppe> fwereade: because at that level we're writing reply messages but don't necessarily know what the call is that we're replying to
<fwereade> rogpeppe, would you restate the objections to logging RPC at TRACE level?
<fwereade> rogpeppe, ISTM that, today, we don't get them anyway
<fwereade> rogpeppe, unless you change config
<fwereade> rogpeppe, so that's pretty much moot
<rogpeppe> fwereade: for me, seeing the RPC messages in the machine log has been indispensible in diagnosing issues
<fwereade> rogpeppe, understood -- but have you needed them when you haven't known you would? how often has there been a case you can't repro?
<rogpeppe> fwereade: almost always
<rogpeppe> fwereade: it's usually been when someone's had a problem and has pasted me the machine log
<fwereade> rogpeppe, ok, but, that doesn't work today, right?
<rogpeppe> fwereade: i guess it's changed
<rogpeppe> fwereade: if you bootstrap --debug, do you see it now?
<fwereade> rogpeppe, offhand, not sure
<fwereade> rogpeppe, but I would expect so
<rogpeppe> fwereade: so when are people seeing the RPC messages now?
<fwereade> rogpeppe, when they set DEBUG to look at the hook output, I think
<rogpeppe> fwereade: so can't they just set debug output for the hooks only?
<fwereade> rogpeppe, heh, probably -- but the other side of it is that they generally *do* want hook output it seems
<rogpeppe> fwereade: we could log hook output at Info level
<fwereade> rogpeppe, maybe the answer is just a slightly more sophisticated default config that includes DEBUG for ... whatever the uniter logger is called
<rogpeppe> fwereade: juju.worker.uniter
<rogpeppe> fwereade: i wonder whether we should have some logging modules that aren't named after juju packages
<rogpeppe> fwereade: oriented towards user logging requirements
<fwereade> rogpeppe, I have no objection to such things in principle, for sure
<fwereade> rogpeppe, at the moment we're only logging WARNING and above by default anyway
<rogpeppe> fwereade: then we could make sure that a given stream sees a set of coherent log messages, even if the messages happen to be generated in several different modules.
<rogpeppe> s/modules/packages/
<fwereade> rogpeppe, indeed, as pointed out by hazmat in http://www.mail-archive.com/juju-dev@lists.ubuntu.com/msg00214.html
<rogpeppe> fwereade: we could actually *design* the log messages that we want the user to see.
<hazmat> rogpeppe, there's a bug to adjust the root=DEBUG pinger=INFO
<hazmat> which will address both issues, the default of INFO is too quiet
<hazmat> well it addresses some of the issues, the issues in that email are still relevant
<rogpeppe> hazmat: unfortunately there's no "pinger" log module
<rogpeppe> hazmat: all the rpc messages are logged at transport level
<hazmat> rogpeppe, can that be changed?
<hazmat> rogpeppe, the pinger messages cause the logs to fill up, and have almost zero value outside of an rpc dev.. and even then its questionable.
<rogpeppe> hazmat: it's the easiest place to do that (and reliable because we get to see the underlying representation), but it may be possible to do something more flexible
<rogpeppe> hazmat: yeah, i agree
<hazmat> rogpeppe, the rpc messages are fine i think for debug transport, its just the pings, that obscfucate things.
<fwereade> hazmat, looks like the root default is WARNING anyway which is *definitely* too quiet
<hazmat> fwereade, rogpeppe for actual debug-log usage by end users the critical piece missing is both tagging log messages with their semantic location (unit id and machine id) instead of ip and then filtering capabilities at view time instead.. really adjusting the generation/collection to adjust the view is broken.
<rogpeppe> hazmat: what about the problem of filling up the log file?
<hazmat> cause needs change, but if an env don't have the data then the user is hosed.
<rogpeppe> hazmat: or is writing to that file considered "viewing"?
<hazmat> rogpeppe, writing to the log file is not viewings, its collection / aggregation. The pinger is the number one by far issue with that, and we should default to that off.. Moreoever its pretty damm common to use log rotation and compression here.
<rogpeppe> hazmat: if we're using compression then the pinger noise won't be an issue, will it?
<hazmat> which in the absence of pinger, means we only have messages that matter for archival, ie related to changes.
<rogpeppe> hazmat: (not that i'm saying we shouldn't switch it off if we can do so reasonably)
<hazmat> rogpeppe, yes.. its still an issue.. because its  A) not useful B) total obscures real info in viewing without filtering C) generates additional writes/exceess logs..
<hazmat> rogpeppe, ie. just because its compressing, doesn't mean we should loop garbage into it.
<rogpeppe> hazmat: for semantic location, i'm thinking about a particular log module which relays messages related to particular units or machines, tagged in a consistent way
<fwereade> rogpeppe, hazmat: OK, I am pretty convinced that (1) we don't want rpc traffic in there by default (2) we do want hook output in there by default (3) hook output as it stands is not so helpful because it's not usefully badged
<fwereade> rogpeppe, hazmat: with an option on (4) FFS filter debug-log at server side
<rogpeppe> fwereade: as i've said, i think at this relatively unstable stage of juju development, having rpc traffic (not pings) in there by default adds considerable value.
<rogpeppe> fwereade: particular as we move towards doing more and more stuff via the API
<rogpeppe> fwereade: i've just realised that filtering out ping logs might not be as awkward as i thought
<hazmat> rogpeppe, json encode the log records with multiple tags is the simplest way.. and what state of the art log aggregation typically does (heka/sensu/logstash)
<rogpeppe> hazmat: that's one option indeed
<hazmat> fwereade, re 4 .. what's FFS filter?
<rogpeppe> fwereade: "for fuck's sake" :-)
<fwereade> hazmat, For Something's Sake ;p
<rogpeppe> ahem, pardon my bad french translation
<fwereade> sorry brb
 * hazmat learns something new
<fwereade> ehh, I've been cussing and blinding all week,not sure why I got bashful about it there
<hazmat> fwereade, re .. 4.. i don't think it really matters if its server or client.. in terms of delivering the feature. yes it would be more efficient server side, but that's a bit more infrastructure. the feature itself in conjunction with 3, makes 1 and 2 effectively go away (assuming we switch default to root=debug).
<hazmat> because a user can then find what they care about.. ie. roughly similiar to what i stated later in that same thread. http://www.mail-archive.com/juju-dev@lists.ubuntu.com/msg00216.html
<hazmat> its still worthwhile to kill 1 though just due to its lack of usefulness (effectively looped garbage)
<rogpeppe> hazmat: it's only the pings which are garbage
<hazmat> rogpeppe, ah.. yes.. agreed.. i was specifically referencing the pings.. rpc traffic is useful.
<fwereade> rogpeppe, hazmat: ok, true that it doesn't matter where that much, and I'm not *really* against rpc traffic
<fwereade> rogpeppe, so do you have a clever plan for the pings? :)
<hazmat> isn't  it just  a separate log channel in that module for transport pings or removing the log messages from them entirely?
<rogpeppe> fwereade: i'm still diving in there. it would be easier to log everything in the rpc package rather than in the codec, but unfortunately we've lost information at that stage (for instance we won't see extra unrecognised fields in request bodies)
<rogpeppe> actually, i think i might have a cunning plan
<jcastro> I get " ERROR storage-auth-key: expected string, got nothing" when trying to bootstrap with the null provider
<jcastro> but that key isn't mentioned in the pending docs
<fwereade> jcastro, grar, I'll take a look quickly
<fwereade> jcastro, yeah, looks like it should be, it's there in the boilerplate config as{{rand}}
<jcastro> ack, I can fix it in the docs
<fwereade> jcastro, thanks
 * fwereade bbs again
<marcoceppi_> jcastro: http://pad.ubuntu.com/7mf2jvKXNa
<fwereade> rogpeppe, hazmat: ok, in terms of sheer usefulness (3) + (4) seem like the best way to spend my evening
<fwereade> rogpeppe, if you have something clever for (1) go for it
<rogpeppe> fwereade: i'd go for 3)
<rogpeppe> fwereade: i'm still on it, trying not to muck up the whole interface for this case
<rogpeppe> fwereade: still on (1), i mean
<fwereade> rogpeppe, understood :)
<jcastro> fwereade, null provider is running but it doesn't appear to be working, where does it log?
<jcastro> when trying to do a bootstrap
<jcastro> it appears like it didn't create anything in /var/lib/ like the command said it should have
<jcastro> also, do I need password-less sudo access on the remote machine?
<fwereade> jcastro, I think you do at the moment... that may be what you're hitting
<jcastro> ok
<jcastro> yeah I saw the command flash up in top but it didn't execute anything, and I was wondering if it was sitting there remotely waiting for a sudo password
<TheMue> yeah, one step further why os x tests fail
<fwereade> jcastro, there are several bugs open re null provider bootstrap, it's worth taking a look at those
<TheMue> step by step, di damm, di damm
<jcastro> fwereade, oh excellent, looking now
<smoser> $ juju init -w
<smoser> error: flag provided but not defined: -w
<smoser> i'm following output of 'juju help local'
<smoser> and it tells me that.
<smoser> shall i open a bug (juju 1.14.1-precise-amd64)
<jcastro> https://bugs.launchpad.net/juju-core/+bug/1235716 is what I was running into
<_mup_> Bug #1235716: sshstorage does not display sudo prompt <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1235716>
<jcastro> fwereade, do you know if this is something that we'll shoot for 1.16 or is it point release material?
<smoser> seems like its fixed in trunk
<smoser> jcastro, did you say yesterday that 'juju bootstrap' on local provider does not need to be root ?
<jcastro> it does need to be root
<fwereade> jcastro, I would *love* to... I had thought sinzui was planning to start a bit earlier than he suggested, so I will try to get some of those reviewed in the hope that axw can land them tonight
<jcastro> the idea is that it won't need to be someday
<fwereade> jcastro, I twitch a little re any changes now, but it does seem low-risk/high-value
<jcastro> fwereade, yeah it's a silly bug, once I got passed it it totally is doing the awesomeness though
<fwereade> jcastro, sweet
<jcastro> I'll link to docs on passwordless sudo and keyauth in the docs though, that way people can have a cleaner experience
<TheMue> fwereade: quick question. what do you think how simple it could be to use mongo w/o ssl in case of a test (depending on the os)?
<fwereade> TheMue, I doubt it'd be that hard, we set up mongo connections in relatively few places iirc
<TheMue> fwereade: ok, will investigate in that direction
<TheMue> fwereade: the two other changes that we would need to run the test on os x seem to be simple too
<TheMue> fwereade: one is that strip when building jujud needs the additional arg -x and the second is that the series on os x is "unknown"
<fwereade> TheMue, that first one sounds trivial, the second one might take some thinking
<TheMue> fwereade: btw, dstroppa can continue to work now, solved his troubles
<fwereade> TheMue, although a case could be made that it *shouldn't*, and anywhere it's causing trouble is evidence of poorly isolated testing
<fwereade> TheMue, you rock, tyvm
<TheMue> fwereade: *blush*
<TheMue> fwereade: your last sentence: it should not fail (the assert should be against the series returned on os x) or the returned series should not be "unknown"?
<TheMue> gna, network left me
<TheMue> ok, good night
<rogpeppe> fwereade: i'm not going to manage the ping-muting tonight, i'm afraid
<fwereade> rogpeppe, don't worry
<rogpeppe> fwereade: i think i know what to do, but it involves a bit of intricacy
<fwereade> rogpeppe, I'm off and on but should manage something sane for the other bits, and hidden nastiness is better than obvious nastiness ;p
<rogpeppe> fwereade: so that the codec can know what request is being replied to - easy in principal, but a couple of types need reworking to keep things cleanish
<rogpeppe> principle, dammit, i always get that wrong!
<rogpeppe> fwereade: g'night
<rogpeppe> and g'night to all
<sinzui> I see https://bugs.launchpad.net/juju-core/+bug/1237219 only has a merge into 1.16. Doesn't it also need to merge into trunk?
<_mup_> Bug #1237219: ssl-hostname-verification: false doesn't trigger if auth-URL is http:// <juju-core:In Progress by jameinel> <https://launchpad.net/bugs/1237219>
<fwereade> sinzui, it surely does, well spotted
<sinzui> fwereade, https://launchpad.net/juju-core/+milestone/1.16.0 is one bug away from being ready. I will talk to thumper about committing or deferring it
<fwereade> sinzui, thumper's on holiday, I'm looking into it myself at the moment, it's a little bit more involved than it looks
<sinzui> fwereade, thank you for taking the time
<fwereade> sinzui, no worries, thank you :)
<natefinch> fwereade: how do I get the bot to update goamz? Trying to merge in the ec2 rootdisk stuff, but the bot seems to have an outdated goamz
<fwereade> natefinch, is dependencies.tsv up to date?
<natefinch> fwereade: yeah, lemme double check that the revision I specified is the right onw
<fwereade> natefinch, if it's not paying attention to that, I'm not sure, I'm afraid
<natefinch> fwereade: yeah, it's correct.  So I guess the bot isn't paying attention, or has some other problem.
<natefinch> well boo
<sinzui> maybe we can update gobot to do the right thing. I think I have issue sorted out in my branch to create a tarball with the correct versions: https://codereview.appspot.com/14354043/
<natefinch> sinzui: that looks pretty good.   Reminds me how much I hate bash scripts, though :)
<sinzui> I despise bash
<sinzui> I vowed to never write it again after working on the charmworld charm.
<hazmat> so..maas tags aren't actually provider specific...
<natefinch> hazmat: tags as a constraint are currently only supported on MaaS, but I know at least amazon has support for tags, though that's about all I know about amazon's tags
<hazmat> natefinch, tags mean something totally different there
<hazmat> natefinch, tags in aws are for resources not for classes
<hazmat> in maas they reference an unallocated resource class.. in aws they reference an allocated group of resources
<natefinch> hazmat: yeah, they're quite different.  In theory we might still be able to use them with juju, but it's certainly a different beast
<hazmat> and it would be nice to validate user vocabulary..
<hazmat> er.. user supplied values
<natefinch> hazmat: what do you mean by validating their values?
<hazmat> natefinch, what happens with an typo'd tag value?
<hazmat> natefinch, pending forever?
<natefinch> hazmat: yep. That's pretty much how we handle all constraints that can't be fulfilled right now.  Not the best experience, for sure.
<hazmat> natefinch, except its not a constraint that can't be fulfilled... its a user input error
<natefinch> hazmat: there's no real way for us to tell that.  If the user asks for an instance with 10T of disk space, is that a typo, or just the user asking us for something we can't fulfill?  Maybe the tag was typoed and the user spelled it right in juju...  the reason for the mismatch between the constraint and what's available doesn't matter.
<natefinch> hazmat: regardless of why we can't fulfill the constraint... our current handling of it stinks.
<hazmat> natefinch, ignoring the generic constraints, these provider constraints are introspectable or static, and yes.. this was validation was previously done.
<natefinch> hazmat: it'll get fixed eventually, I'm sure.  At least we *have* constraints by maas tags now
<hazmat> agreed the reason doesn't matter, we just need to be better about reporting the mismatch as an unsolvable error upfront, rather then pending forever behavior
<natefinch> hazmat: absolutely
<hazmat> filed a bug 1237626
<_mup_> Bug #1237626: constraints should be validated <juju:New> <https://launchpad.net/bugs/1237626>
<fwereade> hazmat, fwiw constraint-prechecking is actually in progress but didn't make it in time
<fwereade> hazmat, and it's not pending forever... it's a error, communicated back in status, and the machine is then amenable to destruction
<hazmat>  natefinch fwiw core's support of maas tags is at a bare minimum of the features available from tags or previously available, filed bug  https://bugs.launchpad.net/juju-core/+bug/1237634 i don't think it matters atm to the users.
<_mup_> Bug #1237634: maas tags support should support full tag syntax <juju-core:New> <https://launchpad.net/bugs/1237634>
<hazmat> fwereade, cool, async provider status makes those errors much better
<fwereade> hazmat, python didn't do that, and last I heard nor did maas
<fwereade> hazmat, python *accepted* &|etc
<hazmat> fwereade, which part?
<hazmat> fwereade, right those are valid maas tags syntax
<hazmat> fwereade, core doesn't support that, it parses for a comma separated list only
<hazmat> hmm..
<fwereade> hazmat, last time I looked at python it accepted but ignored them
<hazmat> fwereade, it passed them through and validated the tag names
<hazmat> fwereade, if maas doesn't support it.. that's quite surprising, else i can't think of why support and tests for that case would have been added.
 * hazmat pokes through maas src
<fwereade> hazmat,     """Parser and validator for tags constraint expressions
<fwereade>     Tag names are extracted and checked against the list of known tags
<fwereade>     reported by the api.
<fwereade>     Currently tag constraints consist of just comma and/or whitespace tag
<fwereade>     names, all of are required for a match. Extending this to support full
<fwereade>     boolean expressions would be possible, and some forward compatibility
<hazmat> but perhaps in which case its an invalid bug
<fwereade>     is attempted.
<fwereade>     """
<hazmat> fwereade, hmm.. the logic in the method below explicitly strips of operators as it matches names
<natefinch> hazmat: right, I don't think it was ever implement
<natefinch> ed
<hazmat> fwereade, natefinch yeah.. i don't see any support for boolean operators in the maas src unit tests.
<hazmat> i'll mark the bug invalid
<hazmat> fwereade, but that brings up my original comment, if the maas tags support isn't a form of provider specific constraint, does that mean that supporting provider specific constraints is still blocked?
<natefinch> hazmat: there was some talk about using tags as a way to hack provider specific support
<fwereade> hazmat, not *exactly*, in that there is no technical obstacle to getting instance-type
<natefinch> tags=instancetype:m1.small
<fwereade> natefinch, no, certainly not
<hazmat> natefinch, as opposed to instance-type=m1.small?
<natefinch> fwereade: wouldn't be my first choice, no
<hazmat> fwereade, but that's also in the spirit of just globally adding them as a constraint of all providers ?
<natefinch> certainly adding an instance-type constraint is pretty trivial code
<hazmat> ie do we also do ec2-zone that way?
<fwereade> natefinch, hazmat: instance-type and zone are not yet there -- but zone when implemented will not I think be a constraint
<fwereade> hazmat, but rather a placement request
<hazmat> ie. this isn't really adding provider specific constraints, its just extending the global constraint vocabulary
<hazmat> fwereade, really it wants both
<hazmat> fwereade, the zone constraint because it does matter for various reasons (reserved instances, volume attach etc)
<fwereade> hazmat, well, I think zone is the wrong level for constraints actually -- constraints make more sense at a higher level, eg distribute units for safety
<hazmat> fwereade, the balance constraint/placement is good for ha as opposed to the current hot swap  on the constraint, but both are good
<hazmat> fwereade, there is more than ha as a reason for the zone constraint
<fwereade> hazmat, sure -- it's also good to put things near one another, but that is also IMO not best expressed as provider-specific zones
<hazmat> fwereade, part of the issue with juju is not being flexible enough for the real world, while automated features are great, alot of whats been the focus for the last cycle is on the flexibility for the real world.
<hazmat> fwereade, the zone constraint because it does matter for various reasons (reserved instances, volume location for attachment, global inbalance across zones (including non juju managed resources), etc)
<fwereade> hazmat, are you saying zones are important (I would not disagree) or that they should be expressed as constraints (I'm not sure I would)
<fwereade> agree that is
<fwereade> sorry english :/
<natefinch> fwereade: I'm guessing he doesn't care about the terminology as long as he gets the end result he wants :)
<hazmat> my name is tron.. i speak for the users ;-)
<fwereade> hazmat, the focus has been explicitly away from clever automation and towards the ability to explicitly specify placement
<fwereade> hazmat, lxc:3 and ssh:user@host are the first steps there
<hazmat> fwereade, constraint has been the only mechanism / vocabulary for that to date, if there's something else for placement that's great.. my concern is that such placement is policy not mechanism, and what's needed is mechanism.
<hazmat> fwereade,  exactly.. the way to use those for work is via --to which is mechanism not policy.. constraints are an even better mechanism as their state captured.
<hazmat> fwereade, i don't nesc. see why we should invent a new end user syntax for specifying service placement, when effectively constraints does the same
<hazmat> ie.. why invent --show-log  and deprecated  -v , when the latter is an established convention
<fwereade> hazmat, the distinction there is between service-level properties and the details of individual machines (units)
<fwereade> hazmat, and re show-log -- we would love to be able to draw a distinction between user output and logging, which are not at all the same thing, and part of that is trying to tease apart the way they've been glommed together
<hazmat> fwereade, re service level properties, that's valid if placement would actualize differently to each instance, but i think that to an end user the distinction is rather academic and the exposed interface is roughly the same, ie your referencing an implementation detail, else we're just inventing a new set of commands for the same service level operations.
<hazmat> fwereade, re logging.. again that's an implementation detail... to the end user we deprecated a rather standard unix cli option -v
<hazmat> and replaced it with some idiosyncratic syntax
<fwereade> hazmat, that *has* been the focus -- to have placement specified explicitly, without being clever or automatic
<fwereade> hazmat, it's in the service of getting -v/-q that are actually meaningful
<hazmat> fwereade, you mean not specified on the service per unit?
<hazmat> er.. not specified on the service but per unit for placement?
<fwereade> hazmat, yes
<hazmat> ah add-unit as the vertex shader..
<hazmat> fwereade, that still doesn't really invalidate the use of constraints here
<hazmat> constraints get captured at a unit level
<hazmat> hmm
<fwereade> constraints get captured at a unit level but have never been specifed/able at a unit level
<hazmat> fwereade, right, but is this really any different than that.. and how does this go back to the higher level automated policy of something like balanced?
<fwereade> hazmat, the idea is to make the tools available first, and to allow for smart automation thereof later
<hazmat> fwereade, the question i have is this really any different than specifying constraints with add-unit
<hazmat> ie. just an additional lookup level to the constraint amalgation
<fwereade> hazmat, perspective question, I guess: there is no *technical* reason these things *couldn't* be expressed as constraints... but I think that constraints become less and less valuable, and more confusing, as they delve deeper into provider-specificity
<fwereade> hazmat, as far as possible they should be specified in a juju-level language
<fwereade> hazmat, this is sadly leaky ofc
<hazmat> fwereade, well the start of this discussion was how to expose provider specific capabilties
<hazmat> ie. its not leaky.. its kinda of the point
<fwereade> hazmat, no argument that we do need to expose provider-specific capabilities
<fwereade> hazmat, just that those related directly to placement are so provider-specific that they don't fit well as constraints
<fwereade> hazmat, tags as global constraints (rather than maas-specific ones) *are* rather designed to allow an alternative backdoor into provider-specificity
<hazmat> fwereade, except i might want to specify them at a service level
<fwereade> hazmat, I think the line between *what* you run and *where* you run it is pretty clean
<hazmat> true.. but that's potentially subtle
<hazmat> fwereade, is specifying a spot instance a  where or what on
<hazmat> as an example
<fwereade> hazmat, spot instances are so far off they're over the horizon :/
<hazmat> its a what on.. but the distinction i think is subtle.. and having a new syntax to express is a implementation purity leaking to the end user... ie is there harm in simplifying the expression to the end user by combining the two.
<fwereade> hazmat, the potential harm is in defining a constraints vocabulary before we understand the set of use cases that people have and that we want to support
<hazmat> fwereade, that harm exists regardless of the xpression form
<fwereade> hazmat, a vocabulary for tactics doesn't restrict a future vocabulary for strategy, but defining a vocab for strategy today *does*
<hazmat> fwereade, putting them in the same book, doesn't restrict the words to describe either
<hazmat>  that's a valid point though
<hazmat> its just not clear that complex config here isn't something i wouldn't want specified at a higher level than add-unit, in which case its just as easily captured in an expanded notion of the constraint language
<fwereade> hazmat, it opens up a bunch of opportunities for conflict between expressions of related ideas at different levels
<hazmat> fwereade, not expressing as constraints opens up conflict opportunity even more
<hazmat> as you have orthogonal lanes of expression
<fwereade> hazmat, it's easier to explain that explicit placement directives override constraints -- just as --to does today -- than to enumerate how variousconstraints allplay together, especially when inherited constraints comeinto play
<natefinch> hazmat: if you have instance-type=m1.small cpu-cores=4 ... what do you do?
<fwereade> natefinch, hazmat: heh, I think we do what we did in python and complain that those just don't play well together
<hazmat> natefinch, error to the user on conflicted constraint
<hazmat> natefinch, thats a case of always conflict
<hazmat> fwereade, the different levels for constraint always get realized in a concrete and introspectable form that's self-consistent. placements aren't really overrides though, there additions to.
<hazmat> fwereade, and really i'd want to be able to specify them at the service level as well
<fwereade> hazmat, I think placements are end-runs around constraints
<fwereade> hazmat, did jitsu deploy-to honour constraints? ;)
<hazmat> fwereade, so as an end user.. it seems quite strange to specify effectively the same thing twice.. ie. what on and where.. are really just where do i want to run this workload
<hazmat> fwereade, fair enough. although that tempts me to  resurrect all the discussion on why that's broken ;-)
<fwereade> haha
<hazmat> thats an interesting perspective though
<hazmat> fwereade, do you have any other examples of placement as an end run around constraint
<hazmat> i guess --to is enough by itself, but its quite specialized.
<hazmat> interestingly enough --to and manual placement are the basis for juju behaving like chef or puppet.
<hazmat> er.. manual provisioning
<hazmat> fwereade, thanks for the discussion, happy to continue another day
<fwereade> yeah, it does definitely end up somewhere quite similar if you focus on those in particular, but I still see them as building blocks for more interesting stuff... we're just not doing it yet
<fwereade> hazmat, cheers :)
<wallyworld> fwereade: hey, where did you get to with the instance type selection?
<fwereade> wallyworld, nowhere I'm afraid -- I got caught up in talk of logging, and decided that that's upsetting more people right now
<wallyworld> fair enough
<wallyworld> we can target 1.18 then
<wallyworld> i assume we are still on track for 1.16 today?
<natefinch> fwereade: I have some code towards an instance-type constraint for ec2 btw
<fwereade> wallyworld, yeah, I think so, I still hope to have something saner for the logging shortly
<fwereade> natefinch, cool -- we'll want it to work for openstack too, I think :)
<natefinch> fwereade: btw, I'd really like to get the ec2 root disk stuff into 1.16, but with the bot not able to merge it, I'm kinda stuck.  Not sure what to do about that
<bac> hi sinzui, charmworld has a dependency on ppa:ce-orange-squad/experimental which does not yet have saucy packages.  would it be possible to update that ppa?
<wallyworld> natefinch: what's the issue with the bot?
<fwereade> natefinch, crap, sorry, I was not paying proper attention -- but wallyworld I think *does*know the bot
 * wallyworld knows a little
<natefinch> wallyworld: it seems to not be updating a dependency in goamz that my branch needs
<sinzui> bac: I think so
<wallyworld> natefinch: i can try and manually update the goamz tree
<fwereade> I at least ought to know where to find out about the bot though... wallyworld, so we have useful instructions anywhere? is it a matter of trawling though old emails?
<bac> sinzui: that would be swell.  would allow 'make check' from a dev box that is saucified.
<wallyworld> fwereade: i think so. there's an ip address that i ssh in to
<sinzui> ah, I always build in precise so I haven't even used the raring
<wallyworld> fwereade: 10.55.32.52
<wallyworld> and from there you can su to the tarmac user etc
<wallyworld> i find the ip address via nova list
<wallyworld> using the correct credentials for canonistack
<wallyworld> natefinch: so you just need goamz updated to tip?
<natefinch> wallyworld: yeah
<sinzui> bac: I am attempting a quick copy, but expect Lp to smack me
<wallyworld> natefinch: ok, give me a sec
<natefinch> wallyworld: thanks for the help
<sinzui> bac, do you need elasticsearch as well as charm-tools?
<wallyworld> natefinch: now on rev 43
<davecheney> sidnei: ping
<wallyworld> see if that works
<natefinch> wallyworld: awesome, thanks
<wallyworld> np :-)
<bac> sinzui: i think just es-java
<sinzui> looks like this will work. I see a wait for publication
<bac> cool, thanks sinzui
<wallyworld> natefinch: looks like the dependencies.csv is not updated
<wallyworld> it only says rev 42
<wallyworld> so that is the issue i think, assuming folks have done the tarmac set up to look there for twhat revs to use
<wallyworld> ah, dependencies.tsv
<wallyworld> natefinch: so you should update that file in your current juju-core branch you are trying to land
<wallyworld> be sure to use the correct rev id also
<wallyworld> not just the rev number
<wallyworld> you can find the rev id using the --show-ids arg to bzr log
<sinzui> bac: still waiting. I see the charm-tools has already arrived
<sinzui> bac, I think everything is in saucy now
<natefinch> wallyworld: dependencies.tsv is correct in my branch
<wallyworld> ok
<wallyworld> must not be used yet then i guess
<sinzui> wallyworld, natefinch I don't think it is used but gobot yet. I use it for release tarballs and packaging building
<wallyworld> ah ok
<sinzui> I believe jam and mgz  tinker when they see deps change
<wallyworld> i updated goamz manually just before
<wallyworld> i miss python :-(
<wallyworld> i just don't get why Go folks bury their heads in the sand with dep management
 * sinzui is watching his new script build juju-core debs for 1.16.0 testing
<wallyworld> yay
<sinzui> And I have
<sinzui> 8 minutes from the moment we think we have a rev, I can have a deb that I can install to build tools and test
<davecheney> \o/!
<sinzui> wallyworld, in fact. I think fwereade believes this open bug is quite a bit of work: https://launchpad.net/juju-core/+milestone/1.16.0
<wallyworld> sinzui: the logging changes?
<davecheney> if it's to big
<davecheney> there is a simpler solution
<sinzui> wallyworld, davecheney: I can start testing tip of lp:juju-core/1.16 after dinner. I can know if tip is a viable rc candidate. Then we can decide if we want to include the logger fix.
<davecheney> fix the default logger so it uses this config
<wallyworld> ok
<davecheney> export JUJU_LOGGING_CONFIG='juju=DEBUG;juju.rpc=INFO'
<davecheney> job done
<sinzui> wallyworld, that bug is half-regression. We did mean to change it, but we need some behaviour put back
<wallyworld> ok, so davecheney's idea above may work?
<davecheney> it works
<davecheney> i use that line when deploying
<wallyworld> i'm not sure where to set the default logging config, but i can look
<davecheney> so where the current default logging config just says 'juju=WARNING'
<davecheney> just change it to
<davecheney> export JUJU_LOGGING_CONFIG='juju=DEBUG;juju.rpc=INFO'
<davecheney> to get the old behavior
<wallyworld> ok
<bigjools> not sure why someone thought this was a good idea, but constraining by host name in maas requires a tag with the hostname.  WTF
<davecheney> ....
<wallyworld> sinzui: i can do a branch for davecheney's change ready for after your dinner if you want
<sinzui> wallyworld, I appreciate your effort.
<wallyworld> np :-)
 * wallyworld creates a branch and sets to work
<natefinch> bigjools: we added support for maas tags, that's all that's different from 1.14.
<sinzui> I am off to feed children, but I leave you with my rediscovery of an old commandL
<sinzui> bzr cat -d lp:juju-core/1.16 dependencies.tsv -r -1
<fwereade> davecheney, wallyworld: there is an even simpler solution if it comes to it -- just log at INFO level by default, that'll get you hook output
<bigjools> natefinch: ok, I was told that juju core expects tags matching the hostname for hostname constraining, which is a bit crackful.
<wallyworld> fwereade: i'm not across it fully. it seems though that davecheney needs debug level in places?
<fwereade> davecheney, wallyworld: but the trouble is that we don't really want to drop the rpc logging
<fwereade> davecheney, wallyworld: and what we *really* need I think is the ability to filter debug-log a bit more sanely
<natefinch> bigjools: I think that was my mistake... someone asked about making tags for hostnames in maas
<fwereade> davecheney, but I may be misunderstanding your particular use case
<wallyworld> fwereade: won't using JUJU_LOGGING_CONFIG='juju=DEBUG;juju.rpc=INFO' as per dave's suggestion work? as a short term 1.16 fix? until get get it right for 1.18?
<bigjools> natefinch: yeah we're missing maas-name, or equivalent
<fwereade> wallyworld, davecheney: jam and rogpeppe are in agreement that there are several bugs they would have had serious trouble with if they hadn't had the rpc spam to look at after the fact
<wallyworld> ah developer vs user friction :-)
<natefinch> bigjools: yeah, we don't have that.  Someone asked me to do maas tags, so I did that.
<fwereade> bigjools, natefinch: ehh, I thought we'd agreed to drop maas-name some time back
<wallyworld> fwereade: tim has put in a way to dynamically change logging
<fwereade> bigjools, natefinch: if we need it, I think it's a placement request
<wallyworld> fwereade: why can't we default to sane values for user
 * natefinch ducks and lets fwereade and bigjools duke it out
<wallyworld> and allow devs to change as necessary
<davecheney> fwereade: right, so at the moment you have no RPC logging data at all
<jpds> fwereade: Filing one now.
<davecheney> how does your solution to #1237151 resolve jam and rogers concerms ?
<fwereade> davecheney, this is true
<bigjools> fwereade: people are depending on placement using hostnames in the python app
<bigjools> so this is a juju-core adoption blocker for them
<fwereade> davecheney, we log at DEBUG by default; badge unit log output with names; and make debug-log a little bit closer to python by implementing (at minimum) --include
<jpds> Yeah, pyjuju did maas-name, and I'm filing a bug for maas-name in juju-core now.
<davecheney> fwereade: please clarify
<davecheney> you would LIKE to log at DEBUG
<davecheney> or you think we currently log at DEBUG ?
 * wallyworld gets the popcorn
<fwereade> davecheney, there is some degree of upset that we don't, on the basis that debug logging of rpc has been repeatedly helpful in diagnosing problems via user-submitted logs
<davecheney> fwereade: please read https://bugs.launchpad.net/juju-core/+bug/1237151
<davecheney> the deafult logging level was raised to WARNING for the agents
<davecheney> this caused me to table flip
<davecheney> i want it back where it was
<davecheney> i think you do as well
<fwereade> davecheney, I do, the problem is setting rpc logging to TRACE
<fwereade> davecheney, what debug output were you after in particular though?
<davecheney> fwereade: i don't want to workship solutions
<davecheney> I just want the 1.14.0 behavior back
<davecheney> it wasn't perfect
<davecheney> but everyone knows how to read the logs in that format
<davecheney> as of now
<davecheney> when my environment fucks out
<davecheney> I have zero logging
<davecheney> tim said he would fix it last week
<davecheney> he forgot
<fwereade> davecheney, ok then, I agree, now is probably not the time to aim for perfect, let's just treat it as a regression and go back to DEBUG across the board for now?
<davecheney> fwereade: agreed
<wallyworld> davecheney: fwereade: so JUJU_LOGGING_CONFIG='juju=DEBUG;juju.rpc=INFO' then?
<wallyworld> or debug everywhere?
<fwereade> wallyworld, 'juju=DEBUG' please
<wallyworld> ok
<davecheney> fwereade: wallyworld thanks
<wallyworld> np
<fwereade> bigjools, jpds: ok, so, maas-name is a bad constraint because it makes no sense at a service level
<bigjools> I agree
<bigjools> there are almost certainly better ways
<fwereade> bigjools, jpds: would you object to placement syntax? eg `juju deploy whatsit --to envname:hostname`?
<jpds> fwereade: Right, but I can do that to bootstrap?
<bigjools> I think hostname constraints generally are weird in a cloud
<jpds> It's not a cloud.
<jpds> It's deploying a cloud on top of MAAS.
<fwereade> jpds, if you could, would that suffice?
<bigjools> MAAS gives you a cloud, so yeah, it is
<jpds> I want to be able to say, install mongodb on this little VM I prepared and don't touch my precious metal.
<jpds> fwereade: Yes.
<jpds> s/mongodb/bootstrap node/
<fwereade> jpds, ok, great, thanks; and if that were internally icky, how would you feel about putting it in environment config? say, `bootstrap-placement: hostname`
<jpds> fwereade: Well, it's not just the bootstrap I care about.
<davecheney> fwereade: it's more comlex than that
<davecheney> all the ceph nodes have to land on the machines with two luns
<wallyworld> fwereade: i've not had much to do with logging set up - i can't find where to change the default logging level for juju from warning to debug
<davecheney> all the compute node have to NOT land on any of the machines with two luns
<jpds> fwereade: I know that I have servers with two disks, to I need to be able to deploy to those specifically.
<fwereade> jpds, ok,sorry I misled, I was only fretting about the interaction between bootstrap and --to
<davecheney> wallyworld: http://paste.ubuntu.com/6215921/
<davecheney> this is what I have so far
<jpds> For things like cinder, swift, ceph like davecheney said.
<fwereade> wallyworld, so at the moment it comes from loggo itself
<wallyworld> oh really????
<wallyworld> the library????
<wallyworld> :-(
<fwereade> jpds, assuming `--to envname:hostname` works everywhere except bootstrap, would using env config for bootstrap hurt too badly?
<jpds> fwereade: I opened https://bugs.launchpad.net/juju-core/+bug/1237709 if you want to add your thoughts to it.
<fwereade> jpds, thanks
<davecheney> wallyworld: nah, that can't be it
<davecheney> we control the root logger in cmd/logging.go
<wallyworld> davecheney: so you are saying the by changing the cmd logging, when the agents have workers started, the correct logging will be used?
<jpds> fwereade: Shouldn't.
<fwereade> wallyworld, davecheney: it so is. if it's not set there it falls all the way back to loggo's default, which is WARNING
<davecheney> wallyworld: i don't understand what you said
<fwereade> wallyworld, set-env logging-config="blahblahblah"
<fwereade> wallyworld, will work after the fact
<davecheney> if you don't want to touch the code
<davecheney> add an env var to the upstart scripts
<davecheney> ok, that needs touching code and tests as well
<fwereade> jpds, cheers
<fwereade> davecheney, no, please do *not* touch the upstart scripts
<fwereade> davecheney, wallyworld: there's an environment config setting
<wallyworld> davecheney: you said you wanted agents to use debug, so i was reasoning that your pastebin did that by changing cmd default logging level cause that's how agent workers are started
<fwereade> davecheney, wallyworld: that will work just fine
<wallyworld> but changning cmd logging also affects user cli commands
<wallyworld> doesn't it?
<wallyworld> which we don't want
<wallyworld> why can't we just have agent workers started with debug flag
<davecheney> wallyworld: yes, it will probably revert some behavior
<wallyworld> or, make the loggo api call to set the agent logger to use debug
<wallyworld> ie in agent.go we have var logger = loggo.GetLogger("juju.agent")
<davecheney> sgtm
<wallyworld> if i make that logger default to debug, that should fix it?
<davecheney> wallyworld: my patch proposed above fixes the bug
<davecheney> the cli is not affected
<wallyworld> davecheney: so it's effectively these 2 lines
<wallyworld> +	// quick fix for LP #1237151
<wallyworld> +	level := loggo.INFO
<wallyworld> but that's info, not debug?
<wallyworld> and it will affect cli commands as well
<davecheney> it does affect them
<davecheney> but is invisible without -v
<davecheney> which sets the logging level to debug anyway
<wallyworld> maybe we need to change jujud
<wallyworld> -v is deprecated
<wallyworld> i think jujud is the correct approach isn't it, since that is used to launch the agent workers
<wallyworld> so cli not affected
<fwereade> wallyworld, agents run a logger worker that takes it from environment config
<fwereade> wallyworld, that's where it needs to be set
<wallyworld> so jujud registers these commands
<wallyworld> 	jujud.Register(&BootstrapCommand{})
<wallyworld> 	jujud.Register(&MachineAgent{})
<wallyworld> 	jujud.Register(&UnitAgent{})
<wallyworld> 	jujud.Register(&cmd.VersionCommand{})
<wallyworld> if we can find a way to set up logging for those as we want, i think that seems best? and isolated from other things?
<fwereade> wallyworld, environs/config/config.go:107
<fwereade> wallyworld, that's where we take it from loggo if not otherwise set
<wallyworld> fwereade: so if i change jujud to set the env variable?
<fwereade> wallyworld, it won;t work
<fwereade> wallyworld, the logger worker will overwrite it with what's in env config
<wallyworld> or poke it into logging-config
<wallyworld> i am -1 on doing something that also affects the cli
<davecheney> wallyworld: it doesn't affect the cli
<davecheney> it is masked/reset/something but other forces
<davecheney> https://codereview.appspot.com/14595043
<davecheney> use it or throw it out the window
<wallyworld> davecheney: it looks like your change will have the cli use debug instead of warning by default
<fwereade> davecheney, wallyworld: except, fuck *that* looks like it's affected by what's been set up in cmd/logging.go, by means of spooky action at a distance
<wallyworld> the cli was changed so that it only showed warning, unless showlog was epcified, then it showed info
<davecheney> wallyworld: lucky(~) % juju status
<davecheney> environment: ap-southeast-2
<davecheney> machines:
<davecheney>   "0":
<davecheney>     agent-state: started
<davecheney>     agent-version: 1.17.0.1
<davecheney>     dns-name: ec2-54-253-151-61.ap-southeast-2.compute.amazonaws.com
<davecheney>     instance-id: i-8646ffba
<davecheney>     instance-state: running
<davecheney>     series: precise
<davecheney>     hardware: arch=amd64 cpu-cores=1 cpu-power=100 mem=1740M root-disk=8192M
<davecheney> services: {}
<davecheney> yet surprisingly, it didn't
<wallyworld> the above is with your change?
<davecheney> yup
<davecheney> lucky(~) % juju deploy mysql
<davecheney> lucky(~) %
<davecheney> silent, as the grave
<wallyworld> that's because show log was not specified
<wallyworld> if you specify show log, then you get debug
<wallyworld> but you don't want that, you want info
<wallyworld> unless debug is asked for
<wallyworld> so, the behaviour is:
<wallyworld> anything >= warning, print to stderr
<wallyworld> info only show if showlog = true
<wallyworld> or, if debug is specified, show that if showlog = true
<wallyworld> your change makes it so that debug is the default rather than info for the case when show log is asked for
<davecheney> it sure does
<wallyworld> so we need for confine the changes to only stuff running on the nodes ie launcged by jujud
<davecheney> ok, lets to fwereade 's ida
<davecheney> ida
<davecheney> idea
<wallyworld> we don't want debug for cli
<wallyworld> by default
<fwereade> davecheney, wallyworld: if we could reliably tell when the logging config was the loggo default, I think we could just handle it in config.go where I pointed you
<wallyworld> sounds ok i think. i was just trying to justify why i was -1 on the changes being done in cmd/logging
<wallyworld> fwereade: we could add a default schema value for "logging-config" ie <root>=WARNING;juju.agent=DEBUG;juju.rpc=DEBUG
<davecheney> wallyworld: lets try that
<fwereade> wallyworld, I'd prefer <root>=DEBUG, I don;t think that affects the CLI, doesit?
<wallyworld> cause line 107 for config.go - by default loggo ignore loggers with unspecified levels
<wallyworld> fwereade: not sure. probably not actually
<fwereade> wallyworld, so the problem with a default is that we want to take it from the env if it's set
<wallyworld> fwereade: we will do
<wallyworld> i think
<wallyworld> ah no
<fwereade> wallyworld, it can be done but will take work and testing, I think
<fwereade> wallyworld, well everything will, but it's not entirely trivial, I mean
<wallyworld> so order of precedence is: 1. ENV, 2. yaml ?
<wallyworld> yaml = logging-config
<wallyworld> in which case i can flip the check in config
<fwereade> wallyworld, I *think* that we expect bootstrap's --logging-config > yaml > env
<fwereade> wallyworld, and the problem is that the path from bootstrap into envconfig is via loggo rather than passed directly
<wallyworld> fwereade: so - in jujud, why can't i just set the env when the commands are launcged
<wallyworld> so logging-config still takes precedence
<fwereade> wallyworld, because the logger worker will go and get the real value out of env config and set it
<fwereade> wallyworld, ha
<fwereade> wallyworld, you know what we could do, I think
<wallyworld> but logging-config will be "" by default no?
<wallyworld> so won't my method work because of that?
<fwereade> wallyworld, the config will be whatever cameout of loggo in the absence of other instructions
<fwereade> wallyworld, ie <root>=WARNING
<wallyworld> ah. i thought it would be empty because loggers by default have level=unspecified
<wallyworld> but maybe it sets root regardless
<fwereade> wallyworld, the root logger starts out as WARNING
<wallyworld> would it be too dirty to check for that?
<fwereade> wallyworld, I'm starting to feel tempted :(
<wallyworld> yeah :-(
 * natefinch contemplates raid 0 w/ 3 SSDs just to speed up his tests
<fwereade> wallyworld, ok, I think it's *slightly* less bad than the alternative, which would be to add *more* spooky action at a distance so config.go could tell whether any logging had been set explicitly
<fwereade> wallyworld, ok, do it, and assign thumper a bug to unfuck it when he comes back
<wallyworld> fwereade:  ok. so i'll dick with config.go to leave logging-config env setting "" if loggo info comes back as "root=<WARNING>" (or whatever)
<wallyworld> and then i need to change jujud to set logging config in env to juju=DEBUG
<natefinch> When someone gets a chance - constraint to specify ec2 instance types by name: https://codereview.appspot.com/14523052/
 * natefinch is well past EOD
<natefinch> g'night all
<fwereade> wallyworld, do not touch jujud
<wallyworld> yeah, just figured that out
<wallyworld> i'm changing config.go
<fwereade> wallyworld, if loggo info comes back as default set something less crazy in logging-cnfog
<wallyworld> 	if c.asString("logging-config") == "" {
<wallyworld> 		if environmentValue := os.Getenv(osenv.JujuLoggingConfig); environmentValue != "" {
<wallyworld> 			c.m["logging-config"] = environmentValue
<wallyworld> 		} else {
<wallyworld> 			loggoConfig := loggo.LoggerInfo()
<wallyworld> 			if loggoConfig != "<root>=WARNING" {
<fwereade> er, you know what I mean
<wallyworld> 				c.m["logging-config"] = loggoConfig
<wallyworld> 			} else {
<wallyworld> 				c.m["logging-config"] = "<root>=DEBUG"
<wallyworld> 			}
<wallyworld> 		}
<wallyworld> 	}
<wallyworld> does the above look ok?
<fwereade> yeah, sounds good
<wallyworld> i'll add todo and bug # etc
<fwereade> wallyworld, perfect, you anticipate my clumsy typing :)
#juju-dev 2013-10-10
 * fwereade wishes axw was here, I've suddenly thought it would be a really good idea to drop null providers from environments.yaml entirely
<fwereade> it's probably time for me to go to bed, if it's actually sane I'll probably remember it in the morning
<wallyworld> hopefully you'll wajke up and release will be good :-)
<wallyworld> davecheney: any chance of confirming that this works for you? https://pastebin.canonical.com/98846/
 * davecheney look
<davecheney> ha, and you called my hack gross
<davecheney> :)
<davecheney> give it a go
<davecheney> can't make it any worse
<wallyworld> this hack won't change cli though
<wallyworld> davecheney: if it works: https://codereview.appspot.com/14592044
<marcoceppi_> anyone around?
<hazmat> marcoceppi_, whats up?
<marcoceppi_> hazmat: any idea of `juju bundle` is going to be used at all in the near future?
<marcoceppi_>  s/of/if/
<hazmat> marcoceppi_, there's some discussion for bundle v2 @ the sprint
<hazmat> marcoceppi_, afaik no use, what were you thinking of?
<marcoceppi_> hazmat: okay, I've got plans to land a juju-bundle plugin I might hold off on that name for now
<hazmat> marcoceppi_, bundle plugin that would export a bundle?
<hazmat> marcoceppi_, there's already one of those in deployer fwiw
<marcoceppi_> hazmat: I'm adding bundle support to charm-tools, ie `juju charm create --bundle`, wanted to make `juju bundle create` an alias of charm with --bundle flag
<hazmat> marcoceppi_.. hmm.. i think there's likely some confusion there with exporting a bundle
<hazmat> marcoceppi_, not sure i understand the value of an empty bundle though
<hazmat> its not just a single file but the metadata format that gui is pushing?
<hazmat> ie. a dir and icon and export file?
<marcoceppi_> hazmat: this would create a directory with a README on how to create bundles from the gui
<marcoceppi_> hazmat: gui stuff
<marcoceppi_> hazmat: also adding proof support for GUI bundles
<marcoceppi_> I feel bundles are too commonly used for too many things
<hazmat> marcoceppi_, that's bad because?
<marcoceppi_> hazmat: the bundle name, not the deployer config stuff itself :)
<hazmat> marcoceppi_, re the dir thing and readme, that sounds like it belongs the in the new gui help system
<hazmat> ie i don't generally go to the commandline to seek help on the gui
<hazmat> or perhaps in the docs
<hazmat> marcoceppi_, fair enough.. i'd love to see them back with their original name.. 'stack'
<marcoceppi_> hazmat: good point, I might defer that portion and simply update proof to proof bundles. As for creating, I orininally spec'd that for charm-tools since there was mention of README, icon.svg, and bundle.yaml
<hazmat> but sadly too much baggage atm re stack
<marcoceppi_> but the icon.svg req was dropped
<hazmat> marcoceppi_, yeah... the only place that spec is used is the unpublished/undocumented used internally by charmworld afaik
<hazmat> almost every real world bundle is just a yaml/json file
<hazmat> argh.. grammar fail late night
<marcoceppi_> hazmat: right, up until next week when bundles land in the gui and they expect a readme and bundle file
<hazmat> marcoceppi_, ic. hence the proof support
<hazmat> marcoceppi_, the create thing sounds like it would be better done as an addition to the doc
<marcoceppi_> hazmat: right, proof and promulgation support
<hazmat> the gui has some significant issues with its bundle exports last i looked
<hazmat> marcoceppi_, the promulgation should just work the same as charms afaicr
<marcoceppi_> hazmat: I'm going to defer the create stuff, since both promulgate and proof can determine if it's a charm or bundle automatically
<marcoceppi_> hazmat: right, with the exception it's lp:~charmers/charms/bundles/<name>/bundle
<marcoceppi_> hazmat: which is easy enough to sort out
<hazmat> namely it exported default config as explicitly set config, and some exporting of empty constraints.
<hazmat> marcoceppi_, cool
<marcoceppi_> hazmat: I'm not aware of any of those issues
 * marcoceppi_ just needs to make sure proof and prom work
<hazmat> marcoceppi_, try an export in the gui, view the output, notice that all the config options are set
<hazmat> even though none where actually changed from the charm default
<marcoceppi_> hazmat: will give it a go
<hazmat> just filed a bug on it
<hazmat> bug 1237739 fwiw
<hazmat> the empty constraints thing looks like its solved on comingsoon though not on jujucharms.com
<davecheney> juju lucky(~/src/launchpad.net/juju-core) % juju ssh p1/0
<davecheney> ERROR unit "p1/0" has no public address
<davecheney> getting this quite a lot lately
<sinzui> wallyworld, did you say that sync-tools can send tools to a testing location, such as /juju-dist/testing/tools?
<wallyworld> sinzui: i think user specified destination can currently only be a local directory
<wallyworld> otherwise it uploads to control bucket in the "proper" location
<wallyworld> let me quickly check
<sinzui> wallyworld, that's all fine. I will use swiftclient to deliver the tools to /juju-dist/testing as I did last week
<wallyworld> ok
<wallyworld> we can add something to the tool if needed
<wallyworld> davecheney: is the logging change any good?
<davecheney> wallyworld: try it now
<davecheney> wil be a while
<davecheney> everything is slow
<wallyworld> :-(
<axw> sinzui: 1.16.0 isn't cut yet right? fwereade OK'd merging in https://bugs.launchpad.net/juju-core/+bug/1235716 overnight
<davecheney> wallyworld: what si the MP for that change ?
<sinzui> axw, you can add to it.
<davecheney> again
<davecheney> sorry
<axw> bueno
<wallyworld> davecheney: https://code.launchpad.net/%7Ewallyworld/juju-core/default-debug-logging/+merge/190282
<davecheney> ta
<davecheney> testing now
<jpds> Guys, is there any away I can tell juju to just download the precise-amd64 tools AND NOTHING ELSE?
<jpds> I'm not deploying saucy-armhf any time soon.
<wallyworld> jpds: this is when bootstrapping i assume
<jpds> wallyworld: Yes.
<wallyworld> the logic to get all series has been around forever. it's not currently possible afaik
<wallyworld> how critical is it?
<jpds> Not critical, I'm just on a super slow link here and I just need one of the tools.
<wallyworld> yeah, understand
<wallyworld> you could download just the one and then sync-tools manually
<wallyworld> then when you bootstrap, it will find it and not attempt to download anything else
<jpds> So, how do I do that?
<davecheney> wallyworld: works perfectly
<davecheney> thank you for fixing it
<wallyworld> davecheney: \o/
<wallyworld> jpds: get the tarball, copy it to <someplace>/tools/releases, then run juju sync-tools --source=<someplace>
<wallyworld> davecheney: if you can lgtm that would be great
<wallyworld> jpds: that will upload that one tarball to your control bucket
<wallyworld> which is one of the places bootstrap looks
<jpds> https://pastebin.canonical.com/98849/
<wallyworld> so it will be found
<davecheney> soordered
<wallyworld> wow 18kbps
<wallyworld> fast, not :-(
<wallyworld> sinzui: i am about to land a logging fix for davecheney. is it too late to include that in 1.16?
<jpds> wallyworld: It can trail at <1kB/s.
<wallyworld> wow
<wallyworld> jpds: this should make it so no outside net access is reuired
<jpds> Yeah, it'll make my life so much easier.
<jpds> Now the customer is complaining that juju needs to download too much stuff from the internet.
<wallyworld> only the tools, and this will fix that
<wallyworld> jpds: the doco for all this obviously needs to be written
<sinzui> wallyworld, no.
<wallyworld> we need a "how do i set up my private cloud" faq
<jpds> I want to complaing about places in sub-Saharana Africa having a better net connections.
<wallyworld> sinzui: i've proposed for trunk. i'll put up a mp for 1.16 and then land
<jpds> wallyworld: That would be nice.
<jpds> All this stuff is just shoved in my head at the moment.
<wallyworld> jpds: as with these things, the code comes first to make ti woek, then the doc follows
<wallyworld> we are all scrambling to make the 1.16 cut off
<wallyworld> we do know the doc needs to be written
<wallyworld> maybe one day we can shove a usb cable in our neck and it will all just flow out
<davecheney> all: if I do relation-set in a hook
<davecheney> then relation-get in the same hook without commiting the hook
<davecheney> will I be able to see the value I just set ?
<wallyworld> sinzui: the logging change has merged
<sinzui> thank you wallyworld
<wallyworld> hopefully everything works ok :-)
<wallyworld> dave tested it so should be good
<jpds> wallyworld: Is 'juju sync-tools --source=' only in devel?
<wallyworld> jpds: it was included before 1.16 branched
<wallyworld> what version are you running?
<jpds> 1.14.
<jpds> stable.
<wallyworld> oh. i'm not sure what's in that version. give me a sec to have a look
<wallyworld> jpds: actually for 1.14, i'm not sure that using the private storage is even an option :-(
<jpds> I have a /home/ubuntu/tools/releases/juju-1.14.1-precise-amd64.tgz
<wallyworld> i'll check
<wallyworld> jpds: a quick look tells me that --source should work
<jpds> $ juju sync-tools -v --source=/home/ubuntu/
<jpds> error:: no available tools.
<wallyworld> 1.14 may have required the tools just to be under tools not tools/releases
<wallyworld> try putting the tarball in /home/ubuntu/tools
<wallyworld> the releases location is a simplestreams thing
<jpds> There we go.
<wallyworld> and 1.14 did not fully support that when it was created
<wallyworld> jpds: i have to go to the school - my kid has left his viola at home. do you need to ask anything or is it going ok with the tools uploaded? i'll be back soon anyways
<jpds> wallyworld: Yes, it's bootstrapping.
<jpds> Thanks!
<wallyworld> great!!
<wallyworld> np
<bradm> davecheney: any ideas on a ETA for the squid charm merges I proposed?  I'm needing to do other work on the charms, and it'd be nice to not have to work around the unmerged code
<fwereade> davecheney, yes, you should be able to
<rogpeppe> mornin' all
<axw> morning rogpeppe
<axw> and fwereade
<mattyw> fwereade, morning, don't suppose you can elaborate on the meaning of this comment? https://codereview.appspot.com/14389043/diff/9001/cmd/juju/helptool_test.go#newcode37
<axw> mattyw: he loves it? :)
<fwereade> mattyw, it means "thank you, I sincerely appreciate you fixing this"
<fwereade> axw, rogpeppe: heyhey
<mattyw> fwereade, axw  ok thanks :)
<fwereade> axw, so, I'm sorry about the sudo password thing, I should probably have just approved it straight off... did you get an opportunity to get it in for 1.16? (and... eh, can *you* think of any way to use the values we ought to..? at all?)
<axw> fwereade: not a problem - it's merged
<axw> fwereade: I haven't come up with any good ideas
<axw> fwereade: "any way to use the values we ought to?"
<axw> non capisco
<fwereade> axw, sorry, to use the command context's stdin/out/err
<axw> oh right
<axw> fwereade: the only thought I had was to have some central package in juju, akin to "os", that has a pseudo Stdin/Stdout/Stderr
<axw> but... not really sure if that's any better than using os :)
<fwereade> axw, eh,replacing one global with another ain't so great really
<axw> indeed
<fwereade> axw, there may be a case to be made for requiring a cmd.Context (OSContext or something maybe?) in a whole bunch more places... eg access to env vars is not usefully restricted, and they're even more global than globals
<fwereade> axw, tedious to have to thread that through everywhere though ofc
<axw> fwereade: hmm yeah good point on env vars.
<axw> fwereade: when is ctx.Stdout ever going to be something other than os.Stdout? only for tests?
<fwereade> axw, in jujuc commands it's not
<fwereade> axw, they don't create envs/env configs though, so not really very relevant
<axw> fwereade: ah yes.
<axw> fwereade: not sure if it was quite obvious, but sshstorage's requirement is extra special: stdin must be a tty/pty
<axw> for remote sudo
<fwereade> axw, ha, yes
<axw> and its use of stdout only makes sense if it's interactive
<fwereade> axw, rogpeppe, wallyworld: ok, I'm now thinking about the manual-bootstrap default-series problem
<wallyworld> i'm not familiar with the issue
<axw> wallyworld: https://codereview.appspot.com/14433058/
<fwereade> axw, rogpeppe, wallyworld: it fails if default-series doesn't match the actual series of the bootstrap machine
<axw> wallyworld, rogpeppe : gist is: null provider's bootstrap machine should dictate what the boostrap tools are
<axw> but bootstrap tools are always default-series
<fwereade> axw, rogpeppe, wallyworld: and this is because we restrict the possibleTools we pass in by that series, even though there's no real reason to
<wallyworld> yeah, i've always had doubts about our approach
<fwereade> axw, rogpeppe, wallyworld: and ISTM that this is because we didn't manage to finish getting tools out of that interface entirely
<fwereade> axw, rogpeppe, wallyworld: they should never have been there in the first place, but turning it completely inside out was too much work
<wallyworld> i sorta kept them in the interface because that's how it was and i didn't want to change too much
<fwereade> axw, rogpeppe, wallyworld: yeah, indeed, but I think we're now at the point where we have to push further there
<wallyworld> +1 to that
 * wallyworld is about to eat dinner, bacl later
<fwereade> wallyworld, enjoy
<rogpeppe> fwereade: sorry, i'm not quite with you here. what are you proposing to change?
<fwereade> rogpeppe, I think we shouldn't be passing possibleTools into bootstrap (or really StartInstance for that matter)
<axw> fwereade: one problem I can think of immediately, is how to ensure tools are in storage
<fwereade> rogpeppe, does that make approximate sense, independent of the question of how we might get there from here?
<axw> that fun bit of code we were looking at yesterday
<fwereade> axw, yeah, I think that code is just in the wrong place
<rogpeppe> axw: that's a good question
<rogpeppe> axw, fwereade: what if StartInstance itself was responsible for ensuring the tools are there?
<fwereade> rogpeppe, I'm not 100% sure about StartInstance, but Bootstrap almost certainly should be
<axw> this would have to lead to --upload-tools disappearing
<rogpeppe> axw: really?
<axw> and rely on sync-tools if you want to do those kinds of things
<rogpeppe> axw: i'm not sure i see the reasoning there
<axw> rogpeppe: what would --upload-tools do if Environ.Bootstrap is responsible for ensuring tools are in storage?
<axw> or you have the logic in two places
<axw> I suppose
<rogpeppe> axw: they can potentially re-use some of the same logic
<fwereade> axw, rogpeppe: I think that --upload-tools could probably still work -- putting the tools is conceptually independent of choosing them I think
<fwereade> axw, rogpeppe: but the auto-sync logic would need to move inside bootstrap
<fwereade> axw, rogpeppe: and I think we'd need to untangle upload from sync tomake that sane
<axw> fwereade, rogpeppe: fair enough. you could upload in the command, and do the rest in bootstrap
<rogpeppe> fwereade: agreed. that would be a good thing anyway imho
<fwereade> rogpeppe, +1000
<fwereade> bugger, I need to help cath a bit
<axw> and I need to get dinner on
<axw> bbs
<rogpeppe> it would be nice if StartInstance would be responsible for fetching tools on demand; then we wouldn't need to push all that stuff up front
<rogpeppe> however, i'm not sure it's possible, as i don't think there's any way of syncing tools concurrently
<jamespage> morning folks! hows 1.16.0 looking?   milestoned bugs list looks great to me
<fwereade> jamespage, I have not heard from sinzui since last night, but he said he was doing preliminary tests without the last (logging config) bug -- so I hope that if there had been problems I would have heard
<mattyw> fwereade, do you have a moment?
<fwereade> mattyw, sure
<mattyw> fwereade, thanks. just looking at your comment here https://codereview.appspot.com/14389043/diff/9001/worker/uniter/context.go#newcode69. I like the idea of putting all the args into a stuct, but it would look very similar to the HookConext struct which it builds anyway
<mattyw> so I'm wondering if there's much value in taking the args in as a struct?
<fwereade> mattyw, ha, yes, it would, wouldn't it
<fwereade> mattyw, the value is in documenting the meanings of the params, I guess
<fwereade> mattyw, but I'm not sure it's that valuable really
<fwereade> mattyw, just one client
<fwereade> mattyw, but then it is an exported func
<fwereade> mattyw, how about unexporting it? and adding a NewHookContext passthrough in export_test?
<fwereade> mattyw, or we could just say "this bit's a bit smelly" and move on
<fwereade> mattyw, if the unexport idea doesn't immediately strike you, personally, as a good idea, feelfree to leave it as it is
<mattyw> fwereade, I'll let the marinate for a bit, in the meantime I'll propose the changes I've made for the other comments if that's ok?
<fwereade> mattyw, sounds great, thanks
<axw> someone on #juju having issues with OS X, if anyone has the time/knowledge
<axw> he's trying trunk to get past the EC2/VPC bug
<teknico> how do I get juju to show more output from hook errors?
<fwereade> TheMue, you're probably the man to respond to axw's suggestion
<axw> fwereade: I think it's an impasse
<axw> he's trying to run 1.17.0
<fwereade> teknico, if the env is already running, `juju set-env logging-config="<root>=DEBUG"`
<axw> well, trunk
<axw> which can't find released tools, so tries and fails to upload
<fwereade> axw, hum, yeah, that's need tools built
<mattyw> fwereade, whenever you're ready https://codereview.appspot.com/14389043/ thanks
<teknico> fwereade: thank you, trying that
<TheMue> axw: will contact him
<fwereade> mattyw, ha, I just spotted something, I feel dumb now -- micro-review sent
<mattyw> fwereade, if I put it in the service doc doesn't that mean it goes into mongo? I thought we were trying to avoid that for the moment?
<fwereade> mattyw, the method can still return what we return from the service though, right?
<fwereade> mattyw, no need for a field yet
<fwereade> mattyw, I'm just thinking of where we put the logic
<mattyw> fwereade, do you mean I could just do OwnerTag: svc.GetOwnerTag() in megawatcher.go:134?
<mattyw> and then make GetOwnerTag() a method on serviceDoc - until we implement it properly?
<fwereade> mattyw, essentially, if you just put that method onto serviceDoc instead of Service
<fwereade> mattyw, yeah exactly
<fwereade> mattyw, you can even keep it on serviceDoc indefinitely I think
<fwereade> mattyw, just call through from Service
<mattyw> fwereade, there don't seem to be any other method on serviceDoc, so this would be the first?
<fwereade> mattyw, it would -- I'm not sure whether it's better as a method or a freefunc
<rogpeppe> fwereade: you've gone...
<mramm> https://launchpad.net/juju-core/+milestone/1.17.0
<fwereade> rogpeppe, so it seems, I have been having some luck lately with sshuttle, thought it was goodenoughfornow
<rogpeppe> fwereade: :-(
<mattyw> :b3
<mattyw> ^^ that was meant for vim
<mattyw> fwereade, sorry me again :), is this basically what you had in mind? http://pastebin.ubuntu.com/6217438/
<mattyw> fwereade, you probably missed me asking if I roughly had the right idea here  http://pastebin.ubuntu.com/6217438/
<natefinch> TheMue: do you drink coffee?  trying to plan for how much coffee to bring to SF.  I roast my own coffee and don't want to have to live off hotel coffee for a week.
<TheMue> natefinch: hey, cool. yes, i like coffee very much. sounds fantastic.
<natefinch> TheMue: awesome.   I'll make sure to bring a few different kinds.
<TheMue> natefinch: having a coffee gourmet as roomie has been a good decision. :)
<natefinch> TheMue: haha yep.  I have like 10 different varieties of beans at home right now., I buy them from here:  http://www.sweetmarias.com/prod.greencoffee.mvc.php
<mgz> natefinch: that's slightly unbelievably coffee-snobish :)
<natefinch> mgz: it's a hobby.  it helps that buying unroasted beans is 1/2 the price, and they're shelf stable for up to a year
<natefinch> :)
<mgz> it sounds fun, right up to the point where you have to start travelling with roasting and grinding apparatus :)
<natefinch> well, I won't be roasting in SF... it produces a bunch of smoke, I think the hotel would get mad at me... that and the roaster is rather large.
<natefinch> I haven't decided what to do about grinding. Ideally you grind immediately before brewing, but my grinder weighs like 8 pounds
<natefinch> I may have to suck it up and grind ahead of time like some sort of plebe
<TheMue> jam, fwereade: unset is now exposed. a review of https://codereview.appspot.com/14439053 please
<TheMue> or anyone else
<fwereade> TheMue, LGTM, thanks
<jam> fwereade: TheMue: reviewed
<jam> TheMue: it would be nice to have 1 more test, but I wouldn't block landing that for 1.16.
<TheMue> jam: unset has so few options, had no idea what to test more ;)
<jam> TheMue: just 'juju unset --help"
<jam> which is the only way *I've* found of reliably asserting you've actually registered the command.
<TheMue> jam: sound good, yes, will keep it in mind. just oriented at the set command
<TheMue> fwereade: need some help, merging it in trunk is well known to me. but how do i get it in 1.16?
<TheMue> jam: or you?
<jam> TheMue: "lbox propose -for lp:juju-core/1.16"
<jam> but you need to build it from "lp:juju-core/1.16"
<jam> and then the same "Mark Approved, set commit message"
<TheMue> jam: thx, and shit, built it from trunk
<jam> TheMue: so generalyl something like "bzr branch lp:juju-core/1.16 $SOMEWHERE"; bzr switch $SOMEWHERE; bzr merge $EXISTING_BRANCH -c -1, bzr commit -m "Just the cmd_unset change"; lbox propose -for lp:juju-core/1.1.6
<jam> the merge might need to be a bigger range
<jam> if you have more than one commit
<TheMue> jam: no, just one commit
<TheMue> jam: will try now, thx
<TheMue> jam: don't get it with the switch, where do i have to call it? because from where i've done the branching or inside the branch it doesn't work
 * TheMue feels like a bazaar noob
<jam> TheMue: use whatever mechanism you usually do to create a new branch, just base it from "lp:juju-core/1.1.6"
<jam> `1.16" of course
<TheMue> jam: yeah, branch is created
<jam> TheMue: so for the merge you can do: bzr merge -c -1 lp:~themue/juju-core/052-expose-unset
<jam> you don't have to do the switch, it is just what *I* do to create a new branch and point src/launchpad.net/juju-core to it
<TheMue> ic
<TheMue> "All changes applied successfully." welcome to the world of branching and merging.
<TheMue> jam: so, now proposed. now approving with message.
<TheMue> jam: done. and for trunk? simply approve the first one too?
<jam> TheMue: so we can do it that way. The way *I*'ve done it on other projects was to just continually merge a stable branch => trunk (so once your patch lands on 1.16, then merge 1.16 => trunk)
 * TheMue has to write this down in his work process document
<jam> TheMue: but there is a bit of discussion as to whether we are landing on trunk and backporting
<jam> or landing on stable and merging up
<jam> *I* like the latter, but I was part of developing bzr :)
<TheMue> I prefer the simpler way. so if you you got one with one command and another one with two or more I'll take the first one. ;)
<jam> TheMue: well what you just did was cherrypick the trunk proposal back to 1.16, so it is obviously more straightforward for *this* patch to just land it on trunk
<jam> but going forward
<jam> if you know we want a change in the stable branch
<jam> it doesn't require thinking about it as much to just commit to something branched from 1.16 and merge it up
<jam> TheMue: does that make sense/
<jam> ?
<TheMue> jam: yes, absolutely
<jam> you don't have to think about how to cherrypick which is sometimes confusing to get the right bits
<TheMue> jam: so for this time i'll take the simple way but next time i start with the according branch directly. has been my fault.
<jam> TheMue: well the team as a whole hasn't done much stable + dev development
<jam> in Bazaar, we had 3+ stable branches for various Ubuntu releases
<jam> imagine if we were supporting 1.12, 1.14 and 1.16 for 6+ months
<jam> being able to land in 1.12
<jam> and then just merge into 1.14 and then 1.16
<jam> was very nice
<TheMue> jam: gna, so multiple switches the whole time every day. fantastic
<TheMue> jam: in my former jobs we mostly worked on trunk and only tagged it (no product business, all linear projects)
<TheMue> jam: only once in good old cvs times worked on a product which has to be maintained
<jam> TheMue: to be fair, that is pretty close to what we're doing in juju-core so far
<jam> TheMue: yeah, CVS made it very hard to manage multiple ongoing branches
<TheMue> jam: but what will change
<TheMue> jam: I'm happy that I now get more and more into git workflows for my private projects
<jam> TheMue: yeah, I'm not sure how much we'll actually commit to a "stable" juju. I would hope we commit to a stable *client* for Ubuntu 14.04
<jam>  
<TheMue> jam: but to be more philosophical, what in the world is stable? ;)
<jam> TheMue: committing that you'll make changes that can be rolled out directly to clients that won't accidentally introduce regressions because of introducing new functionality they aren't using
<TheMue> jam: that's how it should be
<TheMue> jam: btw, thanks, branch has landed in 1.16
<jam> sinzui: TheMue's patch for exposing "unset" has landed in 1.16, so you should be good to go
<sinzui> !
<sinzui> thank you very much jam
 * TheMue => lunch (wife just called)
 * fwereade needs to look after laura for a while, she's about to get back from school and cath isn't yet; bbl
<rogpeppe> a simple review for anyone that wants to take a look - no logic changes, just a bit of type juggling: https://codereview.appspot.com/14529050
 * rogpeppe goes for lunch
<mgz> looking
<natefinch> anyone interested in looking at my new constraint that covers ec2 instance names?cd ../
<natefinch> heh oops
<natefinch> here's the CR for ^^ https://codereview.appspot.com/14523052
<TheMue> *click*
<TheMue> natefinch: what is when you're using a type which is not matching the hardware constraints?
<natefinch> TheMue: not sure I understand the question. But if you're asking about types other than EC2 instances.... right now we don't support it, and it would be a constraint that can't be fulfilled, like if you asked for a machine with 10T of memory
<natefinch> TheMue: we probably should just error out early and tell the user they've specified a constraint that isn't valid... but I wasn't sure if I should special case the type constraint, or if we should write a general constraint validator (since other constraints can also just be plain wrong... like tags that don't exist in MaaS)
<natefinch> TheMue: it's also possible to mix type constraints with other constraints to produce an impossible combination.... like asking for an m1.small with 4 CPUs.  We could error out early on that, as well.
 * sinzui really must optimise what is uploaded to azure for testing
<TheMue> natefinch: that's what I meant. I could use constraints regarding mem and cpu that doesn't match the type
<TheMue> natefinch: and I'm not sure what has precedence
<TheMue> natefinch: or how it is detected and reported as problem
<natefinch> TheMue: right now it is not specifically reported as a problem... what it will do is simply not match any existing instance types, so the procurement will fail
<natefinch> TheMue: type is just another attribute, like number of CPU cores.  It just happens to be an attribute that is unique to a single type of EC2 machine.
<natefinch> TheMue: so it would be like asking for a machine with 8 CPUs and 32G of RAM, if EC2 didn't have such an instance - it would fail to find any matching instance
<TheMue> natefinch: ok, so I can use the type to ease my work but I also can add more constraints with the risk that the command fails
<natefinch> TheMue: correct.  For example, root-disk will work just fine with it.  You can ask for a m1.medium with 20G of root-disk, and that'll work.
<natefinch> TheMue: I think sanity checking constraints before sending them off to the server is something we should do, but it involves more than just the instance type constraint I just wrote.  It probably should be done as a separate changelist
<TheMue> natefinch: ok, thx, got it
<rogpeppe> mgz: did you get around to reviewing that CL? https://codereview.appspot.com/14529050
<mgz> rogpeppe: sorry, still have that tab open, resuming
<rogpeppe> mgz: np
<rogpeppe> fwereade, mgz: FYI this is the mechanism i'm implementing to enable filtering of RPC log events: http://paste.ubuntu.com/6218476/
<mgz> rogpeppe: lgtm'd
<rogpeppe> mgz: ta!
<fwereade> natefinch, fwiw axw has a prechecker mechanism that will enable just that
<natefinch> fwereade: swet
<natefinch> sweet too
<fwereade> natefinch, but I *think* that we need some additional complexity around instance-type vs mem/cpu/etc
<fwereade> natefinch, in python we had the notion of conflicting constraints
<fwereade> natefinch, ie you cannot specify both mem and instance-type together
<fwereade> natefinch, and if you have an instance-type env constraint, and a mem service constraint, the mem constraint masks out the instance-type one when you combine them
<fwereade> natefinch, but *that* then introduces root-disk confusion
<fwereade> natefinch, because openstack instance types do mask them, but ec2 ones don't
<sinzui> jamespage, fwereade I am blessing lp:juju-core/1.16 r1969. I will make the tarball
<fwereade> sinzui, <3
<natefinch> fwereade: I think we need per-provider constraint resolution... because it's entirely possible that constraints could be easily detectable as impossible on EC2 and easily detectable as ok on azure
<fwereade> natefinch, we do indeed, but while we're still using one provider per environment it's not too bad
<natefinch> sinzui: I just merged a branch to 1.16 btw, after 1969
<fwereade> natefinch, as we move towards multi-provider environments, we'll need to disambiguate by provider
<sinzui> oh? I don't need to panic
<natefinch> sinzui: it's just docs changes, but it would be good to have in 1.16
<sinzui> natefinch, okay. I don't see it in lp yet: https://code.launchpad.net/~go-bot/juju-core/1.16
<mgz> natefinch: it seems to... what sinzui said
<natefinch> mgz, sinzui: I forgot to put in a commit message, should be going now
<sinzui> As I need make a last minute change to roll tarballs from non-trunk. I will not worry about r1970 yet
<natefinch> fwereade: I just meant, we can't have checks that don't take the provider into consideration, because what works on one might not work on another
<natefinch> fwereade: even aside from provider-specific constraints
<sinzui> abentley, do you have a few minutes to hangout and talk about bzr reconfigure tricks to makr tarballs without undermining "go get"?
<fwereade> natefinch, the checks are delegated to the provider itself
<abentley> sinzui: Okay.
<fwereade> natefinch, but we're going to have to do a bunch more of that as well, indeed
<mgz> sinzui: can't we just undermine go get?
<natefinch> fwereade: good, yeah, that's what I was thinking - ask the provider if it thinks the constraints make sense.
<sinzui> mgz I have diligently worked to ensure the tarballs and deb cannot be infected by my setup. I really like using "go get" I just realised though that I am not releasing trunk so the wrong branch is gotten. I need lp:juju-core/1.16 to be lp:juju-core
<natefinch> sinzui, mgz:  that's the problem with go get.  Very common.  There's a whole google group created to discuss how to fix it: https://groups.google.com/forum/#!forum/go-package-management
<natefinch> sinzui: for now, go get won't work to build juju.  maybe someday we'll be able to fix that
<natefinch> (well, go get won't work to build juju except at trunk, and assuming all third party packages are ok to be built from trunk/tip/etc)
<jamespage> sinzui, great news!
<sinzui> natefinch, I see and accept your change. I am retagging 1.16 r 1970 as juju-1.16.0 and remaking the tarball
<natefinch> sinzui: awesome
<mgz> `bzr pull --overwrite-tags` all
<jamespage> sinzui, are there any new binaries we need to be building as part of the packaging?
<sinzui> jamespage, no
<jamespage> (preparing for upload now)
<jamespage> sinzui, marvellous
<sinzui> jamespage, for 1.17.0, I will make the unstable recipe's base closer to your packaging branch. It would then be obvious when something has/must change
<jamespage> sinzui, you could just use the distro packaging branch for the unstable recipe
<jamespage> lp:ubuntu/juju-core
<sinzui> yep, that's the one
<sinzui> I have just had my best idea today. bootstrap the juju 1.14.1 now while the packages build. no waiting for azure
<natefinch> sinzui: 1.16 windows installer - http://ubuntuone.com/1aRNVzEv0VDUS5s9Exh53B
<sinzui> thank you natefinch . I think we want to ask IS to sign it. Do you have a place it is normally delivered to?
<natefinch> sinzui: to arosales ;)
<jamespage> sinzui, uploaded to saucy and backports to stable PPA now in flight
<natefinch> sinzui: after that it's all magic and unicorns as far as I know
<sinzui> thank you very much jamespage
<jamespage> I tested upgrade for maas and openstack provides and bootstrapped and tested with klocal provider as well
<jamespage> nice work guys!
<natefinch> yay, we didn't break stuff! :)
<sinzui> Again thank you jamespage . I don't need to worry about that upgrade test
<jamespage> sinzui, the load on my maas server is back under control again - 1.15.1 had it ticking over at 2.0
<sinzui> there is still time. I can put all the public tools in the wrong place or corrupt them.
<sinzui> ouch
<jamespage> sinzui, re armhf in PPA's; we have an issue with golang compilation I'm looking at
<sinzui> :(
<jamespage> the arm PPA builders run everything under qemu which I think is the issue
<jamespage> I can't reproduce locally
<natefinch> rogpeppe: you around?
<rogpeppe> natefinch: i am
<jamespage> sinzui, out for a few hours - everything is building now
<natefinch> rogpeppe: I had a bootstrapped environment on windows under 1.14 and now I'm getting  "environment has no bootstrap configuration data"  when I try to use the environment
<jamespage> sinzui, even if a juju-core builds on armhf in the stable ppa don't use it - it will be using older golang I think
<sinzui> oh, thank you for the warning
<rogpeppe> natefinch: try removing the .jenv file for that environment (or moving it to somewhere else, in case it might contain important info)
<natefinch> rogpeppe: ahh crap
<natefinch> C:\Users\Nate>juju bootstrap
<natefinch> ERROR cannot create environment info "amazon": cannot rename new environment info file: rename C:\Users\Nate\AppData\Roaming\Juju\environments\561471723 C:\User
<natefinch> s\Nate\AppData\Roaming\Juju\environments\amazon.jenv: Cannot create a file when that file already exists.
<rogpeppe> natefinch: you're going to get that error if a Prepare has been done on a legacy environment (the client can't really know any better tbh)
<rogpeppe> natefinch: ah frick
<natefinch> rogpeppe: I just renamed the old amazon.jenv to something else
<natefinch> rogpeppe: so it definitely wasn't there before bootstrap
<rogpeppe> natefinch: what are you trying to do with the env?
<natefinch> rogpeppe: just testing bootstrap etc
<natefinch> rogpeppe: I can try starting from scratch, see if that fixes anything
<rogpeppe> natefinch: um, i think i misunderstood your problem
<rogpeppe> natefinch: i thought you were trying to reconnect to an already bootstrapped environment
<natefinch> rogpeppe: my fault... mispoke
<rogpeppe> natefinch: which is when you might come across the problem that my instructions were trying to solve (it would happen if you had an existing environment, but bootstrapped anyway)
<rogpeppe> natefinch: i'm a bit concerned by the above error though
<rogpeppe> natefinch: what's the accepted way to atomically replace an existing file under windows?
<natefinch> rogpeppe:  move /y in the CLI
<rogpeppe> natefinch: not very helpful from Go :-)
<natefinch> rogpeppe: haven't had to do it in Go.  Let me look
<rogpeppe> natefinch: thanks
<natefinch> rogpeppe: uh so... Go right now doesn't have a facility to do that.  We'd have to wrap our own windows API call to do it (which is not a big deal).
<rogpeppe> natefinch: ah. if we want the client to work on windows, we need to do that, unless there's another possibility
<rogpeppe> natefinch: because that's the way .jenv files are written such that the client can work concurrently with issue
<rogpeppe> s
<natefinch> rogpeppe: that's the only way I can see to replace a file atomically.  There's basically two windows API calls that can do it, and neither of them are currently used in syscall for windows
<rogpeppe> natefinch: rats
<natefinch> rogpeppe: it's really not a big deal to write it ourselves, it's just kind of annoying
<rogpeppe> natefinch: how much work do you think?
<rogpeppe> natefinch: (i'd think we'd make a utils/file package containing a ReplaceFile function, or something similar)
<arosales> natefinch, thanks for creating the installer
<arosales> natefinch, could you open an RT to have it signed>
<natefinch> rogpeppe: yeah.. no big deal, like an hour, if that
<natefinch> arosales: sure
<rogpeppe> natefinch: cool. it would be great if we could get that working, especially as it might rebound on us
<arosales> natefinch, thanks may need to reference RT 64638
<adam_g> is there an ETA on when 1.16.0 tools will be available ? stable PPA has 1.16.0 but no tools are available for bootstrapping
<natefinch> adam_g: there was just some talk about that internally.  The tools should be available shortly, but we realize pushing 1.16 to the ppa before the tools were available was a mistake.
<TheMue> good night
<natefinch> TheMue: g'night
 * sinzui sighs
<sinzui> the saucy packages has a different version than I expected.
 * sinzui hacks the script
<natefinch> sinzui: there's a bug in the windows client that I am fixing, but for now, i can't give the installer to IS to sign, since... it's broken.  Like totally.  Can't bootstrap.
<sinzui> natefinch, and that isn't because I have not put the tools there yet?
<natefinch> sinzui: nope, totally a problem of the code itself trying to do something that just doesn't work on windows.
<natefinch> sinzui: it's trying to replace file a with file b by just renaming a to b.... which evidently works on linux, but returns an error on windows.
<sinzui> :(
<natefinch> rogpeppe: so... basically anywhere we use os.Rename, we should probably use this new Replace function, assuming the idea is that it's supposed to always overwrite an existing thing.  Is that correct?
<rogpeppe> natefinch: pretty much, yes
<rogpeppe> natefinch: it would be worth having a quick audit
<rogpeppe> natefinch: in some places we use mv, but those places are already hopelessly parochial
<natefinch> heh
<natefinch> rogpeppe: trying to decide if I should just replace every instance, or only those ones that might possibly run on the client.   FWIW for non-windows, the new function just passes through to os.Rename
<rogpeppe> natefinch: for the time being, perhaps do only those that might possibly run on the client
<natefinch> rogpeppe: the only one I'm not really sure about - charm/repo.go   func (s *CharmStore) Get(curl *URL) (Charm, error)
<natefinch> rogpeppe: does the charm get downloaded to the client before getting uploaded, or is that only ever called by the agent?
<rogpeppe> natefinch: that's definitely client side
<natefinch> rogpeppe: ok, thought so
<rogpeppe> natefinch: it's probably server side too, FWIW
<natefinch> rogpeppe: ok
<sinzui> once again hp/openstack spits in my face
<natefinch> prorgamming would be so much easier without all these pesky "environments"
<natefinch> rogpeppe: once my VM finished rebooting after installing windows updates, I'll be able to test the changes.  Luckily, I had done the windows API wrapping before, otherwise it would have been a pain in the butt to figure out.
<rogpeppe> natefinch: yeah, thanks!
<rogpeppe> natefinch: that's why you're here :-)
<natefinch> rogpeppe: actually, trivia fact, it's what got me this job.  Gustavo asked me to write a named pipe implementation for windows, probably for use in a windows agent of some sort
<sinzui> bugger, none of the tools arrived at HP
 * rogpeppe has a lovely vision of a bag of hammers lost in transit
<natefinch> rogpeppe: and thus I wrote this: https://github.com/natefinch/npipe/
<natefinch> rogpeppe: lol
<rogpeppe> natefinch: nice. it would be nicer if it provided a portable interface that worked cross platform, of course :-)
<natefinch> rogpeppe: blah blah :)  He specifically asked for windows and gave me the weekend to do it, so I'm gonna call it pretty good.  You're welcome to contribute ;)
<rogpeppe> natefinch: lol, yeah
<sinzui> I am dropping anvils on sync-tools
<natefinch> sinzui: very looney tunes of you
 * rogpeppe tried to lift an anvil once, so feels sorry for the poor sync-tools
<sinzui> :)
<sinzui> I watched the Anvilainia episode of Animanics over the weekend
<natefinch> nice nice, love the animaniacs
<natefinch> rogpeppe: tests are running... had to stop my vm so I could run the tests... was getting OOM errors.  I'd add more RAM if I wasn't planning on replacing my laptop soon.  Running on 4G right now
 * rogpeppe has never heard of the Animanics
<natefinch> rogpeppe: cartoon... I doubt it's running anymore
<sinzui> rogpeppe, It is in the vain of classic Looney Tunes.
<rogpeppe> sinzui: cool
<sinzui> Looks like 1.16.0 is genuinely released and works on the public clouds. I need to retest canonistack though since it too did not get 1.16.0 with my initial upload
<natefinch> rogpeppe: http://www.youtube.com/watch?v=4YzTXxt9oOY
<natefinch> ok, all tests pass, I'll test the fix on windows
<rogpeppe> natefinch: ha, that doesn't go so well with the sounds that are currently playing here... (http://open.spotify.com/artist/6LtwY43IVeNpnireOVay0H)
<rogpeppe> g'night all
<natefinch> rogpeppe: night.  Thanks for the link to the music, sounds great
<rogpeppe> natefinch: it's one of the albums of the year for me
<natefinch> rogpeppe: very cool.  I don't have a spotify account, but I can find some live versions on youtube.  Pretty cool
<rogpeppe> natefinch: i think spotify should give you a certain amount of free listening
<natefinch> rogpeppe: I've been resisting signing up... but whatever, I'm a sucker, especially if I can't get the music anywhere else.... it's pretty great.
<rogpeppe> natefinch: i'm not a big fan of subscription services in general, but i think spotify are pretty special
<rogpeppe> natefinch: their metadata is brilliant because it was all provided by the labels themselves
<natefinch> rogpeppe: I have a google music account which so far has had everything I've asked for... but it doesn't have this :)
<rogpeppe> natefinch: (with the exception of dates of albums, annoyingly, which often come up as the date they were digitised)
<rogpeppe> natefinch: hey, buy it off amazon :-)
<natefinch> rogpeppe: buying music is so 90's
<rogpeppe> natefinch: better quality though
<natefinch> rogpeppe: yeah, and it's good to support the artists more directly
<rogpeppe> natefinch: indeed
<rogpeppe> natefinch: particularly below-the-radar stuff
<natefinch> rogpeppe: absolutely
<natefinch> rogpeppe: https://codereview.appspot.com/14502058
<natefinch> super simple if you have time
<rogpeppe> natefinch: i think utils.Replace should probably be utils.ReplaceFile
<rogpeppe> natefinch: there's nothing about "utils" which suggests that Replace might be a file-oriented operation
<natefinch> rogpeppe: good point
<natefinch> rogpeppe: I was kind of going by os.Rename... but since we're in utils, not os, that's fair
<rogpeppe> natefinch: when might UTF16PtrFromString fail?
<natefinch> rogpeppe: If s contains a NUL byte at any location, it returns (nil, EINVAL)
<rogpeppe> natefinch: i think we should perhaps panic in that case, or at the very least we should return a more descriptive message
<natefinch> rogpeppe: we could... but really I'm just doing the exact same thing all the rest of the syscalls do, which is just return the error.  I think the cases where it can fail are vanishingly small.
<rogpeppe> natefinch: hmm, i think that vanishingly unusual cases are just the place where you want something more than just "invalid argument"
<rogpeppe> natefinch: although i suppose in this case that's a little more appropriate than the millions of places that usually return it....
<natefinch> rogpeppe: actually, looking at os.Rename for windows, it wraps the error in a LinkError struct
<natefinch> rogpeppe: I should do the same thing
<rogpeppe> natefinch: or return "filename contains null bytes" ?
<natefinch> rogpeppe: I strongly prefer doing things the same way as os.Rename, so our code can be a drop-in replacement
<rogpeppe> natefinch: fair enough
<natefinch> rogpeppe:  reproposed with the change to wrap the error
<rogpeppe> natefinch: the name's still the same though...
<rogpeppe> natefinch: did the fslock tests pass under windows?
<rogpeppe> natefinch: reviewed
<natefinch> rogpeppe: doh, name, right, sorry
<natefinch> rogpeppe: it does work with directories as well, which is a little... confusing
<rogpeppe> natefinch: there are a few places where "file" is used to mean either file or directory
<rogpeppe> natefinch: os.File being a good example
<natefinch> rogpeppe: fair enough
<rogpeppe> natefinch: reciprocal review? https://codereview.appspot.com/14531048/
<natefinch> rogpeppe: certainly
<rogpeppe> natefinch: thanks
<rogpeppe> natefinch: not quite as small as yours, i'm afraid
<natefinch> rogpeppe: haha no problem.
<natefinch> rogpeppe: btw, I haven't run any of the tests under windows.  you can cross-compile, but running the tests still has to be done on the native OS
<natefinch> rogpeppe: and I don't have my environment quite set up on windows yet.  Getting there though.
<rogpeppe> natefinch: for this change, the fslock tests would be good to try to run, i think, as they they're quite dependent on the functionality
#juju-dev 2013-10-11
<bigjools> does charm config give an easy way to have multiple sets of the same named values?
<davecheney> bigjools: as in
<davecheney> names: ["dave", "julian"] ?
<bigjools> davecheney: not quite
 * bigjools thinks of example
<bigjools> set1: ["foo": "bar", "fizz": "bang"] and set2: ["foo": "bar2", "fizz": "bang2"]
<bigjools> just wondered if there was anything built in to help
<bigjools> or if I need to slap it all in manually
<davecheney> i think config suports, string, bool and int
<davecheney> so sets have to be modeled as json/yaml encoded strings
<davecheney> i guess
<davecheney> it might support sets
<davecheney> but i've never seen a case of it in the wold
<davecheney> anyone else knw ?
<davecheney> mramm: ping
<dpb1> Hi davecheney, thanks for your email on the haproxy charm, got a sec to chat?
<davecheney> dpb1: sure
<davecheney> off and on
<davecheney> small water heater issue in our house right at the moment
<dpb1> heh
<dpb1> OK, I'll write back over email.  I guess I was not aware that the interface was so strictly defined.  Much work around that explanation has been done in the past 6 months (gets and sets in the metadata.yaml file being one example).
<dpb1> kj:q
<dpb1> blah
<davecheney> wow, i had forgotten just how slow puppet is
<axw> wallyworld: "--source is needed because it allows people (and the release tools) to put the
<axw> tools tarballs in a local directory and upload from there."
<axw> can't you do the same thing with sync-tools though?
<wallyworld> i thought that's what you were talking about? sync-tools?
<axw> wallyworld: I'm saying remove it from "juju bootstrap", leave it in "juju sync-tools"
<axw> it's in both.
<wallyworld> oh. i didn't realise it was in juju bootstrap
<wallyworld> i've never used it from bootstrap at all
<axw> cool, I'll take that as +1 to remove then ;)
<wallyworld> sounds like it :-)
<wallyworld> i'll reply to the email in a sec. gotta attend quickly to some cooking
<axw> ta
<axw> btw, I see your point about upload. it's a bit ugly, but I guess not having it is going to be a PITA
<rogpeppe> mornin' all
<axw> morning rogpeppe
<rogpeppe> axw: i'd appreciate a review of this, if you have a few moments: https://codereview.appspot.com/14531048/
<axw> looking
<rogpeppe> axw: it'll allow us to log rpc messages but lose the pings
<axw> sounds excellent
<rogpeppe> axw: it's a compromise (the message information can't be quite as accurate as the original logging) but i *think* it's the right compromise
<axw> rogpeppe: where's the logging currently?
<axw> or has it been removed
<rogpeppe> axw: in rpc/jsoncodec
<axw> ah yes, ta
<rogpeppe> axw: i'm going to leave it there, but reduce it to trace level
<axw> ok
<rogpeppe> axw: because i think it might be still useful to have lower level logging sometimes
<axw> certainly, for tracing :)
<rogpeppe> axw: in particular, the new logging won't show unrecognised fields, because it's called *after* the marshalling has taken place
<axw> rogpeppe: unknown fields? do you mean after *un*marshalling?
<rogpeppe> axw: ah yes, sorry
<axw> ok
<axw> just making sure I understand the proble
<axw> m
<axw> rogpeppe: not sure if you saw my lgtm. I think there might be a missing call to ServerRequest
<axw> but otherwise lgtm
<rogpeppe>  axw: ah, i haven't yet
<rogpeppe> axw: thanks
<axw> nps
<rogpeppe> axw: good catch about the missing ServerRequest
<rogpeppe> axw: (not that that case can ever happen with the jsoncodec that we always currently use)
<axw> cool
<axw> wallyworld_: environs/sync.DefaultToolsLocation will change when we have <something>.canonical.com for tools, right?
<wallyworld_> yes
<axw> should be the same as environs/tools default URL?
<wallyworld_> let me check
<axw> wallyworld_: sync points at s3, tools points at juju.canonical.com
<wallyworld_> axw: yeah, the whole dependency on s3 thing will go away
<axw> ok
<axw> cool.
<arosales> davecheney, what is the latest gc toolchain needed by juju?  1.1.2?
<arosales> or is gc toolchain needed now that juju starting with 1.15 builds with gccgo?
<arosales> ref bug 1222636
<_mup_> Bug #1222636: juju should compile with gccgo <juju-core:Fix Released by dave-cheney> <https://launchpad.net/bugs/1222636>
* ChanServ changed the topic of #juju-dev to: https://juju.ubuntu.com | On-call reviewer: see calendar | Bugs: 3 Critical, 162 High - https://bugs.launchpad.net/juju-core/
<natefinch> morning all.  Quiet day today
<TheMue> natefinch: morning nate, yes, all busy or simply not here ;)
<rogpeppe> natefinch: hiya
<rogpeppe> anyone fancy a review?
<rogpeppe> this mutes the Ping logging spam: https://codereview.appspot.com/14609043/
<natefinch> rogpeppe: I think I owe you at least 2 I said I'd do and then never completed
<rogpeppe> natefinch: well, here's your opportunity :-)
<rogpeppe> natefinch: thanks!
<natefinch> rogpeppe: to make it 3? ;)
<rogpeppe> natefinch: lol
 * TheMue clicks too
<TheMue> rogpeppe: log noise reduction? already this deserves a double +1 and lgtm
<rogpeppe> TheMue: well, i know the princple is ok - but is the implementation appropriate? :-)
<TheMue> rogpeppe: that is checked now
<rogpeppe> TheMue: thanks
<wallyworld_> mgz:  it's Friday night here, I've had too many drinks, so i'll miss the standup. i won't make much sense anyway
<mgz> wallyworld_: sure :)
 * wallyworld_ opens another bottle
<mgz> enjoy your weekend
<wallyworld_> i've got 2 branches for review. that's my day essentially
<wallyworld_> will do :-)
<natefinch> rogpeppe: also reviewed w/ LGTM
<rogpeppe> wallyworld_: maybe you might make *more* sense... :-)
<wallyworld_> ha de ha ha
<rogpeppe> TheMue: i don't see your review BTW
<TheMue> rogpeppe: still in progress, so far only one question in there.
<rogpeppe> TheMue: ah, sorry, i've just approved the branch. will address any concerns in another branch.
<TheMue> rogpeppe: ok
<natefinch> rogpeppe: standup?
<TheMue> rogpeppe: reviewed too, and as nate said: standup
<rogpeppe> hazmat: Ping messages are now elided from the log in trunk
<TheMue> \o/
<hazmat> rogpeppe, awesome.
<rogpeppe> hazmat: the new scheme also provides us with the potential flexibility to avoid showing secrets too.
<hazmat> rogpeppe, how's that?  you mean internal principal secrets not hook output.. and changing all the locations that might log that to tracef instead of debugf.
<hazmat> as you say its flexibility not enforced though
<rogpeppe> hazmat: yeah. i don't think it's possible to avoid logging secrets entirely
<rogpeppe> hazmat: in the function that does the logging we have access to all the call data and its parameters
<rogpeppe> hazmat: so we could do call-specific munging
<rogpeppe> hazmat: (not that i've done that yet)
<hazmat> rogpeppe, no worries.. this is nice win by itself..
<hazmat> rogpeppe, the only way to not log principals secrets is to not log principal secrets ;-)
<rogpeppe> mgz: are the openstack tenant-name and region name attributes linked to userpass authentication in any way?
 * rogpeppe goes for lunch
<rogpeppe> natefinch: could you take another look at https://codereview.appspot.com/14426046 please? i've made some more changes.
<rogpeppe> natefinch: this is what juju init --show produces now: http://paste.ubuntu.com/6222546/
<natefinch> rogpeppe: on it
<jcastro> ok guys, I'm asking around for people to give manual provisioning a shot.
<natefinch> jcastro: ooh ooh.. I actually have been meaning to try that
<jcastro> next step is I'd like to document deploying to a container
<jcastro> https://juju.ubuntu.com/docs/charms-deploying.html
<jcastro> it's something like a flag to deploy to a container isn't it?
<natefinch> --constraints container=lxc
<jcastro> aha!
<jcastro> man, someone landed that on the container's doc page!
<jcastro> I was all ready to come in here and make fun of someone.
<natefinch> also - https://juju.ubuntu.com/docs/reference-constraints.html
<jcastro> yeah so I think I'll add that also to the manually provisioning page as an example
<natefinch> cool... the more documentation the better
<jcastro> where does juju cache things? I want to bootstrap to another null node but the client isn't picking up my changes to the config file
<jpds> jcastro: ~/.juju/environments/<foo>.jenv
<natefinch> jcastro: I think that's rogpeppe's new change.... environments.yaml is just a starting point, the realtime configs are ^^^
<jcastro> ok so do I blow those away to regen new ones or edit them by hand?
<natefinch> jcastro: you can edit them.. but if you just want a second null provider you can copy & paste a new on in environments.yaml and just give it a different name
<natefinch> null2 : type : null
<jcastro> blowing it away also worked
<jcastro> I just wanted to see what would happen there
<jcastro> 2013-10-11 15:04:45 DEBUG juju.state open.go:88 connection failed, will retry: dial tcp 127.0.0.1:37017: connection refused
<jcastro> I get that over and over when trying to manually provision a bootstrap node
<jcastro> ah, after about 5 minutes it just gave up
<mattyw> rogpeppe, ping?
<rogpeppe> mattyw: ping
<rogpeppe> mattyw: pong even :-)
<rogpeppe> mattyw: did you have a question?
<mattyw> rogpeppe, I don't suppose you have 5 minutes spare for a hangout?
<rogpeppe> mattyw: sure
<mattyw> I've got an odd error in my core branch
<mattyw> rogpeppe, https://plus.google.com/hangouts/_/01877422202d7d4982e8a135c06bbe6058de9ed0
<TheMue> so folks, good night and have a nice weekend
<natefinch> TheMue: have a good weekend!
<TheMue> natefinch: enjoy your time too, greetings to the family and have a good coffee ;)
<rogpeppe> natefinch: could you try running tests against trunk, testing launchpad.net/juju-core/juju, please
<rogpeppe> ?
<natefinch> rogpeppe: sure
<natefinch> rogpeppe: getting errors related to mgo
<rogpeppe> natefinch: could you paste the full output of go test -gocheck.vv in that dir please?
<natefinch> rogpeppe: sure thing
<rogpeppe> natefinch: ta
<natefinch> rogpeppe: http://pastebin.ubuntu.com/6223130/
<rogpeppe> can anyone else replicate the test failure?
<rogpeppe> mgz: ^
<mgz> looking
<mgz> seems to be a tear-down error with sockets? that's a known intermittent failure, no?
<mgz> (cd juju&&go test -gocheck.v) passes here
<rogpeppe> mgz: it's happening consistently for mattyw
<mattyw> rogpeppe, thanks for your help
<rogpeppe> mattyw: np
<rogpeppe> mattyw: mgz might have some ideas
<mattyw> I'm going to take a short break but I'll still be here
<rogpeppe> mgz: i've been in a hangout with mattyw trying to work out what the issue is - you might want to carry on trying to help. unfortunately i've got to go now.
<rogpeppe> happy weekends all
<mgz> mattyw: you're running just that same subset of tests right?
<mgz> bug 1236931 is the same symptom
<_mup_> Bug #1236931: juju: sporadic test failure in TestDeployWithForceMachineRejectsTooManyUnits <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1236931>
<mgz> and the same test even
<mgz> I guess the next step, with reliable repo, would be to just run a single test that triggers if (if possible) with -gocheck.f= and change the teardown failtf into a something that gives us more debug possibilites
<mgz> perhaps just letting it hang to it can be straced or similar
<rogpeppe> mgz: the problem never happens when running just a single test
<mgz> that's what I thought
<mgz> but if we can trigger it running two, it's a bit better than the whole suite
<mgz> feels like a mongoy isolation racey something :)
<mattyw> mgz, sorry, yes, just the juju tests
<mattyw> mgz, as in juju-core/juju/...
<mattyw> mgz, I'm using go 1.1.2 right?
<mgz> mattyw: yeah, that's what I'm using
<mgz> what mongo version?
<natefinch> jcastro: for the record, I can't get null provider to work at all.   I hit this bug: https://bugs.launchpad.net/juju-core/+bug/1235717
<_mup_> Bug #1235717: null provider bootstrap fails with error about sftp scheme <juju-core:In Progress by axwalk> <https://launchpad.net/bugs/1235717>
<jcastro> natefinch, I just hit that
<jcastro> you need to blow away /var/lib/juju on the node
<jcastro> it happens if you partially bootstrap and it fails
<jcastro> it doesn't know how to resume
<natefinch> jcastro: I did that.. lemme retry
<natefinch> jcastro: are you using --upload-tools or just letting it find the S3 tools?
<jcastro> letting it find the tools
<jcastro> Get http://162.243.42.173:8040/provider-state: dial tcp 162.243.42.173:8040: connection refused
<jcastro> is the next problem I am running into though
<natefinch> is that IP localhost or something else?
<jcastro> I just signed up for digital ocean to try it
<jcastro> I bet I need to open some port or something
<natefinch> (local to the bootstrap node that is)
<natefinch> I would expect that juju would do that itself.  You shouldn't need to open stuff manually, else, why even bother?
<jcastro> that was a guess
<jcastro> I have no idea what I am doing here, heh
<natefinch> haha
<natefinch> I'm trying to use trunk and not getting anywhere... keeps telling me my version number is 0.0.0  :/
<gary_poster> hi.  we thought that juju core has the bootstrap node directly download charms, but we are getting evidence that they are being downloaded to a user's box and then uploaded the way pyjuju used to do.  what's the expected story?
<natefinch> gary_poster: I think that's what it does right now, yeah.  Don't ask me why, I don't know, personally.
<gary_poster> natefinch, so IOW you are not surprised that they are downloaded to the user's machine first?
<gary_poster> I mean, not surprised to hear me say it :-)
<mattyw> mgz, mongo version is 2.2.0
<natefinch> gary_poster: correct. There's a folder under .juju called charmcache that appears to be a cache of charms (not surprisingly).  Why we download them to the local machine and then upload to the bootstrap node, rather than just directly download to the bootstrap node, I don't know.
<gary_poster> natefinch, :-( but thanks for clarification
<mgz> mattyw: hm, same here, must be something more environmental
<mattyw> mgz, I'm going to call it a night, will look again on monday, thanks for your help
<ahasenack> gary_poster: yeah:
<ahasenack> -rw------- 1 andreas andreas 44M Out 11 15:32 .juju/charmcache/cs_3a_precise_2f_juju-gui-77.charm
<natefinch> gary_poster, ahasenack: it's possible that the folder there is just for the local provider, so it's not a smoking gun, however, the logs do seem to look like we're downloading charms before uploading
<ahasenack> well, the timestamp matches
<ahasenack> natefinch: look at these: http://pastebin.ubuntu.com/6223761/ (they are in utc for some reason, sounds like a bug)
<ahasenack> 10min to download, 24min to upload, that's my guess
<natefinch> hot damn that is slow
<ahasenack> yeah, I have the worst isp of Brazil for sure
<natefinch> but yes, we're definitely downloading and then uploading the charms.  There doesn't seem to be any reason why we couldn't just tell the bootstrap node to go get them itself, and save one leg of the trip (and reduce the use of the local bandwidth in the case of bad Brazilian ISPs)
<natefinch> however, there may be a good reason I'm not aware of.
<jcastro> hey sinzui
<sinzui> hi jcastro
<jcastro> https://bugs.launchpad.net/juju-core/+bug/1238934
<_mup_> Bug #1238934: manual provider doesn't install mongo from the cloud archive <juju-core:Confirmed> <https://launchpad.net/bugs/1238934>
<jcastro> kapil and I ran into this
<jcastro> it basically makes the manual provider not work
<jcastro> natefinch,  ^^ this is the issue I was having earlier
<sinzui> jcastro, The issue looks familiar. We discussed this scenario last week in regards to lxc/local provider
<natefinch> jcastro: doh
<sinzui> jcastro, your do mean manual provider here, not local?
<jcastro> sinzui, tldr, we're yeah
<jcastro> I am trying to manually provision on a VPS
<jcastro> and I couldn't figure it out so hazmat figured it out while trying the same thing
<hazmat> natefinch,  https://bugs.launchpad.net/juju-core/+bug/1238934
<_mup_> Bug #1238934: manual provider doesn't install mongo from the cloud archive <juju-core:Confirmed> <https://launchpad.net/bugs/1238934>
<sinzui> I will tentatively say 1.17 and let our leads scream if I am asking too much
<natefinch> seems like installing mongo shouldn't be a big deal, but I'm not really the right person to ask
<jcastro> sinzui, ok so basically we should not announce that the manual provider is ready?
<sinzui> yeas, that is right.
<jcastro> natefinch, well, you'd have to ssh in, stop stuff, manually add the archive
<jcastro> start stuff
<jcastro> I'd rather say "wait one more stable release" than have people doing that crap
<natefinch> jcastro: what, it's just code, right? ;)
<natefinch> jcastro: absolutely. I definitely would not announce it yet.  As far as I can tell, it's quite a bit away from being ready to go
<jcastro> so close! I can taste it!
<natefinch> jcastro: I'm pretty psyched for it, but far better to make people wait for a great experience than give them a bad experience a couple weeks sooner.
<jcastro> indeed, I'll go ahead and make a note on the docs
<hazmat> natefinch, sorry that was meant for axw..
<hazmat> or sinzui
<hazmat> effective the manual installation path is a totally separate installation path than what juju normally does
<hazmat> it needs explicit testing before any release.. as the bit rot chance is much higher
<hazmat> in this case its missing the cloud archive install before using mongodb
<hazmat> for precise to work
<natefinch> hazmat: I hope it's not totally separate... but it definitely needs a lot of testing before the first release, and certainly it's enough of a special case to warrant at least some minimal testing each time
<hazmat> natefinch, well its not using cloudinit..
<natefinch> hazmat: yes, there is that. And that is a pretty big difference.
<hazmat> the command stream serialized in cloudinit should be roughly the same though
<hazmat> but the envelope is different, so anything done by cloudinit outside of the command stream needs replicating
<hazmat> jcastro, if we switch out to 13.04 instances it should work afaicr.. those distro mongos have ssl support.. but without the cloudarchive there's no mongodb version/feature normalization across series
 * hazmat tries it out
<natefinch> hazmat: FWIW, I tried with 13.04 and had difficulties... but I was doing it with trunk, so there may be other factors there
<hazmat> natefinch, i'm also on trunk
<hazmat> the issue was the same for 12.04 as what jcastro hit namely juju-db didn't startup
<natefinch> hmmm, the problem I had was that it couldn't find tools, even if I used upload-tools.  I didn't really look into it beyond that, though
<hazmat> hmm.. manual provider does some client side caching..
<hazmat> ie. changing the ip address of state server and bootstraping bombs out that provisioning has already happened
<hazmat> and you can't destroy-environment on a manual env
#juju-dev 2013-10-13
 * thumper headdesks
<bigjools> o/
<thumper> hi bigjools
<bigjools> good holiday thumper?
<thumper> lovely to not read work emails
<thumper> although quite a few to go through this morning
<thumper> with some dupes...
<bigjools> until you get back
<thumper> bigjools: looking at the maas bootstrap issue that rvb had
<thumper> can't reproduce it locally...
<thumper> in the test
<bigjools> I've not delved that far into my email yet, but I did see something fly past
<thumper> wondering how it got created...
 * thumper thinks
<bigjools> will discuss it when I get there
<thumper> bigjools: I'm going to go to the gym shortly
<thumper> so perhaps we could talk when I get back
<bigjools> ok
<thumper> also would like to chat about other maas issues
<thumper> specifically  abusing tags and alternative options
<bigjools> urgh my connection is crap this morning
<davecheney> arosales: yes 1.1.2 is required
#juju-dev 2014-10-06
<hazmat> jet laggers in the house?
<waigani> rogpeppe: can we grab you for a sec
<rogpeppe> waigani: sure
<rogpeppe> waigani: where are you?
<mgz> ericsnow: lp:jenkins-github-lander is what the bot runs, the github.py there could easily have a reviewboard.py analouge
<mgz> anastasia: https://github.com/orgs/juju/people?query=anastasia
<katco> marcoceppi: hey, do you have time to update the status spec with me? just a few quick tweaks
<mgz> cmars: http://juju-ci.vapour.ws:8080/job/publish-revision/
<jcw4> TheMue: are we planning on hanging out today?  I'm going to be online for another hour or so, and then travelling
<TheMue> jcw4: no problem. I would like you to talk to bodie_ about the current status and send me a short note later so that I'm on plan too
<jcw4> TheMue: sure, bodie_ just reached out to me a couple minutes ago
<bodie_> cool
<jcw4> I'm working on getting the juju-gui working in devel mode against a live local juju
<TheMue> great
<jcw4> TheMue: bodie_ is working on the CLI
<TheMue> jcw4, bodie_: so the API is evaluated from client and gui side, fine. same order of functions?
<jcw4> yes, I think both gui and cli will use the same API functions
<TheMue> jcw4: they mostly should
<TheMue> ;)
<bodie_> TheMue, jcw4 I think there will probably be a few differences.  for example the default actions "do" in CLI will probably wait for a result
<bodie_> TheMue, jcw4 while the web-gui will likely default to async
<TheMue> only try to coordinate with each other, so code implemented for the gui hasn't to be changed when the cli wants to use it
<bodie_> it's all in the spec doc, I will also be putting some energy into fleshing out the actions doc
<jcw4> bodie_: even so, the API calls will be the same I'm sure
<TheMue> bodie_: fine
<sebas5384> We are having some problems with pending machines in juju-local ?
#juju-dev 2014-10-07
<davecheney> morning
<davecheney> how's brussles
<davecheney> ?
<jrwren> tis lovely
<Spads> a lovely shade of pigeon
<Spads> http://www.benjaminmoore.com/en-us/paint-color/pigeongray
<wwitzel3> jam: so I looked at the code for the NonValidatingHTTPClient in juju/utils and it just sets the InsecureSkipVerify flag to true. So apparently that IS the fix we went with before.
<jam> wwitzel3: so we need to discuss this, as saying "ok, just don't validate SSL connections ever" is going to be a good way forward
<perrito666> davecheney: morning, I was hoping you would be here
<sinzui> jamespage, https://plus.google.com/hangouts/_/canonical.com/juju-qa-meeting?authuser=1
<perrito666> sinzui: hey, do you have a link to the new link to whatever you use to create custom streams? my streams are suddenly not working so I presume something changed
<dimitern> who can review a simple fix for bug 1373385 http://reviews.vapour.ws/r/154/ ?
<mup> Bug #1373385: juju needs to support the MAAS API's not_tags constraint <constraints> <maas-provider> <orange-box> <juju-core:In Progress by dimitern> <https://launchpad.net/bugs/1373385>
<perrito666> sinzui: sort of ping
<sinzui_> hi perrito666
<perrito666> hi, can you tell me if thisformat is correct? http://people.canonical.com/~hduran/juju-dist/
<sinzui_> perrito666, that looks good. I will try validating it
<sinzui_> perrito666, that looks correct. wallyworld changes streams in trunk to use tools-stream to select the kind of tools. the default is "released" and that is what I see in the index.json
<sinzui_> perrito666, I also confirm I can download the tar.gz and the sum matches
<sinzui_> perrito666, There were issues last year where juju failed to download the tools (in cloud-init-output.log) and it was reported as a shasum mismatch instead of truncated file
<perrito666> sinzui_: well I ssh into the machine and then sha256sum it and the checksum is also correct
<sinzui_> perrito666, maybe you want to try setting the tools-metadata-url to http://juju-dist.s3.amazonaws.com/testing/tools to verify something is pickup
<perrito666> I will thank you
<perrito666> sinzui_: so I upgraded my juju and created a new set of tools version: juju-1.21-alpha2-trusty-amd64 but now it seems to be ignoring me
<perrito666> RROR cannot bootstrap new instance: Juju cannot bootstrap because no tools are available for your environment.
<perrito666> You may want to use the 'tools-metadata-url' configuration setting to specify the tools location.
<sinzui_> perrito666, juju get-env tools-metadata-url
 * perrito666 destroys the whole things and starts over
<sinzui_> by default is is not set
<sinzui_> perrito666, but we can change it using set-env
<perrito666> sinzui_: just scrapped the whole thing and re bootstraping, so far seems to be working
<sinzui_> perrito666, which cloud are you deploying too? I just bootstrapped in aws with you stream and the juju 1.20-alpha2 that was built by CI a few hours ago
<perrito666> aws
<sinzui_> perrito666, http://pastebin.ubuntu.com/8514364/
<sinzui_> perrito666, I created a trusty instance in us-east-1
<perrito666> sinzui_: can I gent your .jenv conf for that particular conf?
<sinzui_> perrito666, http://pastebin.ubuntu.com/8514414/
<mgz> cmars: http://pastebin.ubuntu.com/8514413/
<perrito666> sinzui_: Ill take a look
<perrito666> sinzui_: it worked
<sinzui_> perrito666, ? what is different in your yaml?
<perrito666> sinzui_: region is different and default series is not present, yet I have done no changes
<perrito666> sinzui_: Ill keep an eye for this happening again
<sinzui_> perrito666, I set default-series because I don't trust juju to pick and LTS. Did your failures have a trusty machine?
<perrito666> they did
<fabrice> jaasteam: if someone could review : https://github.com/CanonicalLtd/blues_browser/pull/94
<fabrice> wrong channel sorry
<ev> hazmat: is there any way to tell deployer about availability zones? I have a prod and dev host aggregate. I can juju boostrap --to zone=production, but then running deployer tries to throw everything on the nova AZ (dev).
<bodie_> I know everyone's super busy, but I'm trying to find info on juju w.r.t. docker
<bodie_> I noticed a very stale issue for docker integration and container networking from last year on launchpad
<bodie_> I'd really like to do some work in that direction if nobody else is, or if others are, maybe I could help out a bit, or something
<ev> hazmat: nevermind; working around it with juju add-machine zone=production for now.
#juju-dev 2014-10-08
<mgz> I catch *all* my go exceptions
<waigani> menn0: b7f60f7f9073f9db3d1c4a9665fcc392e3436eb2
<waigani> menn0: http://pastebin.ubuntu.com/8519673/
<hazmat> ev, hmm.. no there's no zone support in deployer. in core it will auto roundrobin effectively units of a service across zones
<ev> hazmat: so am I doing it wrong, or is juju missing a distinction between spreading across AZs for redundancy and for specialising the environment (giving different cpu overcommit for prod and dev)?
<waigani> men
<waigani> menn0
<waigani> menn0: hello
<katco> https://www.reviewboard.org/downloads/rbtools/
<katco> if you see anastasiamac online or in person, please congratulate her on her FIRST JUJU BUG FIX!
<katco> she unblocked the trunk :)
<jcw4> yay anastasiamac!
<katco> she's not online right now, but bug her later ;)
<jcw4> yep will do
<jcw4> :)
<mgz> jcw4: actions status question, how demoable is it atm?
<jcw4> hmm;
<mgz> jcw4: marco is pretty keen to show off some things next week, wondered how far along you were
<jcw4> there is a demo branch  that can do a hello echo
<mgz> are the cli bits only in the demo branch?
<jcw4> mgz: yeah, bodie_ is working on the official CLI bits this week
<mgz> also bug me if you need reviews for that
<bodie_> sure thing
<jcw4> mgz: and I'm filling in the official API
<jcw4> the demo branch has unofficial CLI and API stuff\
<bodie_> mgz, if marcoceppi_ has a shopping list we can definitely prioritize specifics
<jcw4> mgz, bodie_ +1
<mgz> I gathered from him he had a shoddy charm and wondered if he could cheat around it a bit, so would need a bit more than echo
<bodie_> mgz, the docker charm, I suspect
<jcw4> haha
<jcw4> mgz: I don't think it's too far but next week might be a bit early
<bodie_> mgz, I'll do what I can to push forward on the CLI basics and bug you for reviews.  I should have some basic PRs up by EOD for you to look at tomorrow.
<bodie_> mgz, is there an email I should use?
<jcw4> bodie_: I'll try to get the API methods filled in ASAP so you can test end to end too
<mgz> mgz: either just the juju-dev list, or my canonical email for extra buggage
<bodie_> mgz, I'd also like to stay in sync with marcoceppi_ so we have a clear picture of what he needs to put in the charm (actions.yml) and how he can tie in the hook interactions for the actions
<bodie_> sure thing
<mgz> ...why am I talking to myself
<bodie_> lol
<jcw4> I was wondering
<hazmat> is there some trick to get juju to detect new addresses attached to a machine
#juju-dev 2014-10-09
<jcw4> Is anyone available to review my trivial PR to the names package? https://github.com/juju/names/pull/28
<jcw4> Also, http://reviews.vapour.ws/r/158 which will depend on the names package PR too
<jcw4> thanks mgz
<mgz> jcw4: no probs
<jcw4> anastasiamac: congrats on your first blocker bug fix yesterday :)
<mgz> jcw4: it's not quite done yet sadly :)
<mgz> anastasiamac: poke poke :)
<anastasiamac> it was not me fixing it tho :-) I was just a PA that typed it
<jcw4> anastasiamac: lol
<jcw4> mgz: it's still not working?
<jcw4> mgz: was that your ordering bugfix?
<mgz> jcw4: no, that's amusingly an unrelated regression that was masked by the fact that other bug was already causing the test job to fail
<jcw4> mgz: I see.
<mgz> once both land the ppc64el job might be happy again
<jcw4> anastasiamac: my first big PR was step by step laid out for my by thumper - I was pretty much a PA too.
<jcw4> mgz: hopefully
<jcw4> TheMue: I responded to your comments on https://github.com/juju/names/pull/28
<jcw4> mgz: wrt. to the un-exercised error case you mentioned in the review... the only error that would get returned is if the mgo iterator returns an error on closing in a defer block (state/state.go:1620)
<jcw4> mgz: I'm not quite sure how to cause that error
<jcw4> mgz: suggestions? or should I make a fake ActionReceiver and force a bogus error return there just to exercise that line?
<mgz> jcw4: you can fake, but if it's that edgy it's likely not worth bothering with
<jcw4> okay; thanks mgz - shall I drop that issue?
<jcw4> mgz: dropped - as soon as I get my names PR landed I'll update this one with the dependencies.tsv change.  Should I take "looks great overall" as an LGTM or wait for an official one? :)
<mgz> jcw4: I may bug someone else as well for the shipit
<jcw4> mgz: ta
<mgz> axw, anastasiamac: https://code.google.com/p/go/issues/detail?id=8654
<anastasiamac> mgz: yep
<huwshimi> 346022
<huwshimi> 250720
<huwshimi> 962491
<huwshimi> 279768
<huwshimi> 376365
<huwshimi> Erm
<perrito666> huwshimi: ok
<mgz> axw: bug 1379380 - can you go/gccgo compile and compare goyaml.Marshal(map[string]string{{"pew": "pew\npew\n"}}) - something very odd is happening...
<mup> Bug #1379380: Test failure on gccgo in RelationGetSuite <juju-core:Triaged> <https://launchpad.net/bugs/1379380>
<jcw4> mgz: axw fwiw, I was seeing the same error with go 1.3.3 recently
<axw> looking
<axw> anastasia had this issue before, but I couldn't repro
<mgz> with gccgo? maybe it's some other thing, like wrong goyaml version
<axw> yes, with gccgo
<axw> testing again now anyway
<axw> gccgo 4.9.1 vs. gc 1.3 makes no diff to me
<mgz> worth asking anastasiamac what goyaml she hath
<mgz> jcw4: if you can repo, what goyaml?
<axw> yes, I was going to ask to do godeps, but then got sidetracked
<jcw4> mgz:  I'd have to reboot my mac into ubuntu since it was on an experimental dual boot instance
<jcw4> mgz: but I'm pretty sure it was a stock godeps -u dependencies.tsv version of goyaml
<mgz> jcw4: 's not urgent
<jcw4> mgz, axw  my goyaml was at a different revision - confirming the goyaml suspicion
#juju-dev 2014-10-10
<mgz> dimitern: http://reviews.vapour.ws/r/162/
<jcw4> mgz, TheMue any chance of an LGTM on https://github.com/juju/names/pull/28 ?  Thanks!
<mgz> jcw4: there's one flag thing still open, but shipit from me
<jcw4> mgz: tx!
<jcw4> mgz: what is the flag thing?
<jcw4> mgz: oh, are you talking about reviewboard 158?
<jcw4> I see
<mgz> jcw4: I was
<mgz> because I can't read
<jcw4> haha
<jcw4> :)
<jcw4> thanks mgz
<mgz> jcw4: why did you send *me* the email about bug 1379380 rather than the bug?
<mup> Bug #1379380: Test failure on gccgo in RelationGetSuite <juju-core:Invalid> <https://launchpad.net/bugs/1379380>
<mgz> (thanks for verifying it :)
<jcw4> mgz: I guess I thought my comment was more relevant to our discussion than the bug
<jcw4> mgz: I can re-send to the bug :)
<mgz> that would be nice :)
<jcw4> makes sense now... sorry :)
<mgz> jog: http://paste.ubuntu.com/8533161/
<jcw4> another Actions API PR : http://reviews.vapour.ws/r/164
<bodie_> TheMue, mgz, have a pr for apiserver charminfo actions
<bodie_> http://reviews.vapour.ws/r/165/
#juju-dev 2014-10-11
<stokachu> in the api client code for Status what is 'patterns' supposed to look like?
<stokachu> calling Status or FullStatus without any params seems to return an empty hash
<stokachu> haha nope, fat fingered Requst
#juju-dev 2015-10-05
<rogpeppe> i'd much appreciate a review of this if anyone has some time. it cleans up quite a bit of cruft: http://reviews.vapour.ws/r/2828/
<voidspace> TheMue: stdup?
<TheMue> voidspace: yeah, already started hangout
<dooferlad> yay, I love it when someone kills my ec2 instances when I am debugging a problem :-(
<natefinch> ericsnow: you around?
<ericsnow> natefinch: yep
<natefinch> wanna standup?
<ericsnow> natefinch: sure
<rogpeppe> wwitzel3: are you OCR? if so, would you be able to review this, pretty please? http://reviews.vapour.ws/r/2828/
<rogpeppe> or anyone else... :)
<natefinch> rogpeppe: he's in Seattle
<rogpeppe> natefinch: ok
<rogpeppe> natefinch: fancy a review? :)
 * rogpeppe thinks that there should be at least one OCR on any given day...
<natefinch> rogpeppe: yeah, I can
<rogpeppe> natefinch: that would be great, thanks
<natefinch> rogpeppe: I agree.. it's tricky when there's a sprint, but it's something we should plan for.
<rogpeppe> natefinch: +1
<mup> Bug #1502935 opened: TestWaitMinionNeverBecomeMinion fails <ci> <intermittent-failure> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1502935>
<mup> Bug #1502935 changed: TestWaitMinionNeverBecomeMinion fails <ci> <intermittent-failure> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1502935>
<rogpeppe> natefinch: are you planning to do that review? (np if not, but i'll start badgering other people, 'cos i've got another PR in the pipeline that depends on it)
<mup> Bug #1502935 opened: TestWaitMinionNeverBecomeMinion fails <ci> <intermittent-failure> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1502935>
<natefinch> rogpeppe: sorry, doing it, just got sidetracked
<rogpeppe> natefinch: np
<voidspace> mgz: ping
<mgz> hey
<voidspace> hey, hi
<voidspace> has anyone done any work on bug 1499571
<mup> Bug #1499571: Restore failed: error fetching address <backup-restore> <blocker> <ci> <regression> <reliability> <retry> <juju-core:Triaged> <juju-core 1.24:Triaged> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1499571>
<voidspace> which is currently blocking 1.24 & 1.25
<voidspace> I recently changed the way that public address was "calculated"
<voidspace> and I hope that's not the cause
<voidspace> it really shouldn't be the cause!
<voidspace> can't see how it could be, but I'm suspicious...
<natefinch> rogpeppe: reviewed.  Sorry for the delay
<rogpeppe> natefinch: tyvm!
<mgz> voidspace: I think cherylj had a look, but it does need picking up still
<voidspace> mgz: ok
<voidspace> mgz: looking at it a bit
<voidspace> restore is using the right state methods to update addresses, so shouldn't be bypassing the new logic for setting public / private addresses
<natefinch> ericsnow: you available?
<ericsnow> natefinch: yep
<natefinch> ericsnow: back in moonstone
<ericsnow> k
<mup> Bug #1503029 opened: juju plugins which exit > 0 report a subprocess ERROR <juju-core:New for charmers> <https://launchpad.net/bugs/1503029>
<mup> Bug #1503039 opened: JUJU_HOOK_NAME does not get set <juju-core:New> <https://launchpad.net/bugs/1503039>
<mup> Bug #1503039 changed: JUJU_HOOK_NAME does not get set <juju-core:New> <https://launchpad.net/bugs/1503039>
<voidspace> perrito666: ping
<mup> Bug #1503039 opened: JUJU_HOOK_NAME does not get set <juju-core:New> <https://launchpad.net/bugs/1503039>
<mup> Bug #1503039 changed: JUJU_HOOK_NAME does not get set <juju-core:New> <https://launchpad.net/bugs/1503039>
<mup> Bug #1503039 opened: JUJU_HOOK_NAME does not get set <juju-core:New> <https://launchpad.net/bugs/1503039>
<voidspace> perrito666: I blame you! Another blocker on restore... (my fault this time actually)
<voidspace> (and slightly fwereade's fault too...)
<perrito666> oh, voidspace is unfair, blaming me and running
<dimitern> TheMue, dooferlad, o/
<dimitern> sprint's been going fine so far - in case you're wondering, we'll demo the spaces stuff tomorrow, we did a few changes to the bundle to overcome issues with the mediawiki-mysql not working properly in different subnets
<dimitern> it michael around today?
<dimitern> *facepalm* it's rather late I've just realized :/
#juju-dev 2015-10-06
<beisner> o/ hi core friends
<beisner> any idea if https://bugs.launchpad.net/juju-core/+bug/1500613 can be targeted to something (way) earlier than 1.26a1?
<mup> Bug #1500613: configstore should break fslock if time > few seconds <amulet> <openstack-provider> <tech-debt> <uosci> <juju-core:Triaged> <https://launchpad.net/bugs/1500613>
<mup> Bug #1261780 opened: api.Client().AddLocalCharm() uses utils.GetNonValidatingHTTPClient() <security> <tech-debt> <juju-core:In Progress> <https://launchpad.net/bugs/1261780>
<mup> Bug #1261780 changed: api.Client().AddLocalCharm() uses utils.GetNonValidatingHTTPClient() <security> <tech-debt> <juju-core:In Progress> <https://launchpad.net/bugs/1261780>
<mup> Bug #1261780 opened: api.Client().AddLocalCharm() uses utils.GetNonValidatingHTTPClient() <security> <tech-debt> <juju-core:In Progress> <https://launchpad.net/bugs/1261780>
<mup> Bug #1261780 changed: api.Client().AddLocalCharm() uses utils.GetNonValidatingHTTPClient() <security> <tech-debt> <juju-core:In Progress> <https://launchpad.net/bugs/1261780>
<mup> Bug #1261780 opened: api.Client().AddLocalCharm() uses utils.GetNonValidatingHTTPClient() <security> <tech-debt> <juju-core:In Progress> <https://launchpad.net/bugs/1261780>
<voidspace> TheMue: dooferlad: morning
<voidspace> TheMue: dooferlad: not really any worse than last night, so I'm "in" today
<voidspace> see you at standup
<voidspace> cherylj: I think you looked at the "Restore failed: error fetching address" critical issue
<voidspace> cherylj: just to let you know that I've got it (and it's almost certainly my fault anyway)
<voidspace> dooferlad: TheMue: if you have a chance, it's a nice short diff but should fix the problem
<voidspace> dooferlad: TheMue: http://reviews.vapour.ws/r/2835/
<voidspace> The added tests fail without the change in addmachine, so pretty sure it's the right fix. (Restore creates a new machine as the bootstrap machine and then immediately asks for its address - which currently isn't set.)
<TheMue> voidspace: looks good
<dooferlad> voidspace: what is the parameter from network.SelectInternalAddr that you are ignoring?
<voidspace> dooferlad: it's an error - which will be a "no address" error if the slice of addresses is empty
<voidspace> dooferlad: in that case the returned address will be empty - and setting the preferred address to an empty address is then the right thing to do
<dooferlad> voidspace: sounds good
<voidspace> dooferlad: so we can safely ignore the error - even if an error is returned we still use the empty address that is also returned
<voidspace> TheMue: thanks
<dooferlad> voidspace: maybe just add a comment about that logic? Then it is good to go.
<voidspace> dooferlad: ok
<TheMue> dooferlad: good idea
<voidspace> dooferlad: done
<rogpeppe> i hadn't realised just how broken reviewboard was until now
<voidspace> rogpeppe: just how broken is it?
<voidspace> screwed up diffs?
<rogpeppe> voidspace: unimaginably
<rogpeppe> voidspace: can't click on comments. can't close tabs.
<voidspace> wow
<voidspace> rogpeppe: to be fair, that sounds like firefox (or whatever) is broken
<rogpeppe> voidspace: it takes some serious screwups not to be able to close a tab in Chrome
<rogpeppe> voidspace: i don't think i've ever encountered that before
<voidspace> runaway javascript I guess
<rogpeppe> voidspace: the machine's not spinning...
<voidspace> rogpeppe: I thought tabs in chrome were supposed to be isolated so that couldn't happen
<rogpeppe> voidspace: indeed
<voidspace> it still sounds like a browser bug
<voidspace> I've never had that with reviewboard
<rogpeppe> voidspace: and it's just screwed up by design. the fact that if you want to reply to a review comment, it takes you to a new page.
<rogpeppe> voidspace: i've worked out how i did it, i think
<rogpeppe> voidspace: i didn't like the above behaviour (i wanted to keep my inline comment context and still reply to a review comment) so i right-clicked "open in new tab" on the Reply button.
<rogpeppe> voidspace: instant screwup
<TheMue> rogpeppe: a new page? hmm, never had this
<voidspace> rogpeppe: nice
<voidspace> TheMue: rogpeppe: submitting an inline comment takes you away from the review page
<rogpeppe> TheMue: so if you want to reply to a review comment (not adding a new comment which is what i was doing initially without knowing it), you click on "reply" and it doesn't take you away from the full diff context?
<voidspace> which is annoying
 * TheMue checks
<rogpeppe> voidspace: also, it appears that you're not able to reply to issues that have been fixed or dropped
<voidspace> not ideal
<TheMue> rogpeppe: somehow I only find the link "Add comment"
<rogpeppe> TheMue: that's in the comment overview page
<rogpeppe> TheMue: i'm almost always in the "View Diff" page
<TheMue> rogpeppe: yes, I'm mostly in the "View Diff" too
<TheMue> everyday something new ...
<voidspace> ericsnow: hey
<ericsnow> voidspace: hey
<voidspace> ericsnow: a package arrived for me today
<voidspace> ericsnow: thank you
<ericsnow> voidspace: you are quite welcome! :)
<voidspace> ericsnow: looks like there's a lot of good stuff in there, I've only dipped in so far
<perrito666> Voidspace you where blaming me for something but you disconnected too soon
<voidspace> perrito666: restore was broken
<voidspace> perrito666: my fault this time (originally I pinged you because I wanted some help - but then I worked it out)
<voidspace> I mainly blame fwereade though, my original approach didn't have that bug :-p
<voidspace> perrito666: http://pad.lv/1499571
<perrito666> Lol if what you did broke restore, then it was probably good
<voidspace> perrito666: the actual fix was only a few lines though
<voidspace> perrito666: it nicely proves our CI system works - it was a genuine bug only caught by the CI tests
<ericsnow> voidspace, perrito666: FYI, a LXD-based provider will allow you to test restore locally
<perrito666> Indeed
<ericsnow> hopefully it is coming sooner rather than later :)
<perrito666> Ericsnow anyway it is rather annoying to run that particular test but we could have a bootstrap test which would be nice
<perrito666> Lunch bbl
<voidspace> o/
<abentley> ericsnow: I had no idea an lxd provider was coming until I watched Stephane Graber's Container Camp presentation.
<ericsnow> abentley: yeah, the LXD folks have done a great job, reducing the complexity of building a Juju provider around LXD
<abentley> ericsnow: I'm excited that we're getting LXD support.  Especially if that means machine 0 will not be my actual host.
<ericsnow> abentley: yeah, that's what has me most excited as well :)
<voidspace> yep +1
<ericsnow> abentley: we'll see how it pans out, but there are a number of other possibilities that a LXD provider would open up
<voidspace> abentley: master is not blocked on bug 1499571
<ericsnow> abentley: e.g. running across multiple hosts; snapshots
<mup> Bug #1499571: Restore failed: error fetching address <backup-restore> <blocker> <ci> <regression> <reliability> <retry> <juju-core:In Progress by mfoord> <juju-core 1.24:Fix Released by mfoord> <juju-core 1.25:Fix Committed by mfoord> <https://launchpad.net/bugs/1499571>
<perrito666> Not having to sysadmin my machine back into working a couple of times a week is already a big plus
<voidspace> abentley: which surprises me as it blocked 1.24 and 1.25
<voidspace> abentley: I've committed a fix to both those branches but master is blocked on something else
<voidspace> abentley: ok for me to JFDI my fix onto master, or is it better if I wait?
<abentley> voidspace: sinzui demoted it to Medium.
<voidspace> right
<voidspace> I don't know why
<ericsnow> also, having multiple distinct Juju dev environments on different series (or even distros) all at the same time
<abentley> voidspace: It's better if you wait.  That way, when you land, it will be clear whether your branch was blessed or cursed.
<voidspace> abentley: ok, no problem
<dimitern> voidspace, o/
<voidspace> dimitern: hey, hi
<voidspace> dimitern: how's it going?
<dimitern> voidspace, so far so good :) we're demoing today - well, frobware will be driving it actually
<voidspace> dimitern: cool, good luck
<dimitern> voidspace, how about that 1.9 bug?
<dimitern> voidspace, thanks!
<voidspace> dimitern: only just getting to it really
<voidspace> dimitern: got hit fixing a critical blocker on 1.24 - master
<voidspace> dimitern: plus my maas is borked and no-one seems available to help fix it
<voidspace> dimitern: but I have maas 1.9 in a kvm I can use
<voidspace> dimitern: just looking at the cloudinit code now
<dimitern> voidspace, right, ok then - please, post updates on the bug as you go to show we're handling it
<voidspace> dimitern: so the bit I actually need to fix is the *bash script* in bridgeConfigTemplate
<voidspace> oh joy :-)
<dimitern> voidspace, yes, that was the crux of the issue IIRC
<voidspace> cool
<voidspace> looks like some fun playing with sed and grep
<voidspace> and some test scripts
<dimitern> voidspace, just don't drop the "auto eth0" if we haven't rendered the bridge config
<dimitern> in /e/n/i
<dimitern> voidspace, also if /e/n/i already has a static config for the primary nic, we should move it into the juju-br0 config, and still set ethX to manual
<voidspace> right
<voidspace> dimitern: what do you mean by "if we haven't rendered the bridge config"
<voidspace> dimitern: bridgeConfigTemplate checks to see if the bridge already exists and exits
<dimitern> voidspace, so in the script there's a pipeline like grep 'iface ethX inet dhcp' && sed ... & cat << EOF
<voidspace> yep
<dimitern> voidspace, if that first grep fails, the cat is not done, and /e/n/i not modified with the bridge config
<voidspace> ah, but then we still remove the auto
<dimitern> voidspace, yeah
<voidspace> so that should be guarded too
<dimitern> voidspace, yes, but just that won't cut it
<voidspace> we still need to create the bridge
<voidspace> do we leave the primary interface definition in place and copy it into the bridge definition?
<dimitern> voidspace, we need to check if /e/n/i has iface ethX inet static, and copy the rest of the section into the section for iface juju-br0 inet static (e.g. address, netmask, etc.) in addition to bridge_ports
<voidspace> so leave the auto in place *plus* copy the details
<voidspace> that should be fine, need to get this kvm booting instances - it's been a while since I set it up
<dimitern> voidspace, let me paste you an example, just a sec
<dimitern> voidspace, http://paste.ubuntu.com/12697581/
<voidspace> dimitern: cool, thanks - that's clear
<dimitern> voidspace, cheers :)
<voidspace> the fun part is using bash to detect the *whole* of the section to copy
<voidspace> not entirely sure how to express "copy everything up to the dedent" in bash :-)
<voidspace> I'll work it out though
<dimitern> voidspace, yeah, that's a bit fiddly with bash I guess
<voidspace> worst case I use Python :-p
<voidspace> but I'll try not to
<dimitern> voidspace, I don't mind - if you do, add it to the packages to install ;)
<voidspace> ok
<dimitern> voidspace, however, I've just realized it might not work, as this is called too early in the boot process, likely before the packages are installed - hence, bash unfortunatelly
<voidspace> dimitern: kk
<frobware> voidspace, you could refactor what's there a bit and turn some of into functions
<voidspace> frobware: that sounds eminently sensible
<frobware> voidspace, it would help to determine what to call based on what you discover
<voidspace> yep
<voidspace> and some good names could make it a bit clearer what's being done
<mgz> abentley: https://github.com/ericsnowcurrently/juju/tree/lxd-provider-full-provider
<voidspace> frobware: good luck with the demo
<frobware> voidspace, thx
<voidspace> frobware: did you do the devices demo already?
<frobware> voidspace, nope
<voidspace> ah
<voidspace> cool
<frobware> voidspace, does work for me though
<voidspace> great
<voidspace> that's a good sign...
<voidspace> it might actually work then ;-)
<rogpeppe> ales, anyone else: this PR adds macaroon authentication to the streaming API endpoints: http://reviews.vapour.ws/r/2838/
<rogpeppe> review much appreciated
<mgz> voidspace: any news on bug 1499571? it is due to the address stability change?
<mup> Bug #1499571: Restore failed: error fetching address <backup-restore> <blocker> <ci> <intermittent-failure> <regression> <reliability> <retry> <juju-core:In Progress by mfoord> <juju-core 1.24:In Progress by mfoord> <juju-core 1.25:In Progress by mfoord> <https://launchpad.net/bugs/1499571>
<voidspace> mgz: it was, fix committed on 1.24 and 1.25 - master is blocked on something else and that bug was only marked as "medium" on master
<voidspace> mgz: so not landed there yet
<mgz> voidspace: aha, I misunderstood then
<voidspace> mgz: misunderstood what?
<mgz> the state of the bug
<voidspace> ah :-)
<voidspace> I misunderstood your misunderstanding
<voidspace> mgz: just seen your comment
<voidspace> mgz: is your comment out of date then?
<voidspace> mgz: it was intermittent because as soon as SetMachineAddresses or SetProviderAddresses was called on the bootstrap machine the PublicAddress and PrivateAddress would become available
<voidspace> mgz: with the fix already landed they should be available immediately
<voidspace> mgz: have they failed since my fix landed?
<mgz> voidspace: I had not realised you had landed something,
<voidspace> ah
<voidspace> :-)
<mgz> you had the bug in progress and CI auto-marked fix released because 1.24 was blessed
<voidspace> kk
<voidspace> mgz: it marked fix released because prior to the bless I marked the 1.24 milestone as fix  committed
<voidspace> same with 1.25
<voidspace> but master is still in progress because I can't land there yet
<voidspace> mgz: so you put them back to "in progress"?
<mgz> right, I didn't see you'd landed anything.
<voidspace> :-)
<voidspace> fair enough
<mgz> so, can put back to fix released, and make it so you can land change on master too.
<voidspace> mgz: mark it as critical?
<voidspace> that should allow me to land it
<voidspace> I'm not sure why it was set as medium
<voidspace> it's still a regression on master
<mgz> voidspace: done and done
<voidspace> mgz: thanks
<mgz> voidspace: thanks for the fix, sorry for the confusion :)
<voidspace> mgz: np :-)
<voidspace> EOD
<voidspace> g'night all
<mgz> sleep well :P
<mup> Bug #1496972 changed: juju bootstrap fails to successfully configure the bridge juju-br0 when deploying with wily 4.2 kernel <hs-arm64> <kernel-da-key> <network> <Linux:Triaged by jsalisbury> <https://launchpad.net/bugs/1496972>
<ericsnow> cmars: FYI, I have a patch up that merges master into the lxd-provider branch: http://reviews.vapour.ws/r/2834/
<ericsnow> cmars: mind if I fold in those deps from your patch?
<perrito666> oh god why status is so cruel
<mup> Bug #1503449 opened: easy bug reporting for juju - e.g. sosreport <juju-core:New> <https://launchpad.net/bugs/1503449>
<katco> ericsnow: awesome. wwitzel3 ^^^ what ericsnow said
#juju-dev 2015-10-07
<mup> Bug #1499571 changed: Restore failed: error fetching address <backup-restore> <blocker> <ci> <regression> <reliability> <retry> <juju-core:Fix Released by mfoord> <juju-core 1.24:Fix Released by mfoord> <juju-core 1.25:Fix Released by mfoord> <https://launchpad.net/bugs/1499571>
<mup> Bug #1499571 opened: Restore failed: error fetching address <backup-restore> <blocker> <ci> <regression> <reliability> <retry> <juju-core:Fix Released by mfoord> <juju-core 1.24:Fix Released by mfoord> <juju-core 1.25:Fix Released by mfoord> <https://launchpad.net/bugs/1499571>
<mup> Bug #1499571 changed: Restore failed: error fetching address <backup-restore> <blocker> <ci> <regression> <reliability> <retry> <juju-core:Fix Released by mfoord> <juju-core 1.24:Fix Released by mfoord> <juju-core 1.25:Fix Released by mfoord> <https://launchpad.net/bugs/1499571>
<mup> Bug #1499571 opened: Restore failed: error fetching address <backup-restore> <blocker> <ci> <regression> <reliability> <retry> <juju-core:Fix Released by mfoord> <juju-core 1.24:Fix Released by mfoord> <juju-core 1.25:Fix Released by mfoord> <https://launchpad.net/bugs/1499571>
<mup> Bug #1499571 changed: Restore failed: error fetching address <backup-restore> <blocker> <ci> <regression> <reliability> <retry> <juju-core:Fix Released by mfoord> <juju-core 1.24:Fix Released by mfoord> <juju-core 1.25:Fix Released by mfoord> <https://launchpad.net/bugs/1499571>
<voidspace> TheMue: dooferlad: grabbing coffee, with you in 5 minutes
<voidspace> sorry
<mattyw> TheMue, ping?
<TheMue> mattyw: pong!
<mattyw> TheMue, just reviewing http://reviews.vapour.ws/r/2820. On the whole it looks great, couple of questions though
<mattyw> TheMue, 1) the juju/helptopics stuff. that documentation is available through the command line right?
<TheMue> mattyw: exactly, and I'm currently porting and extending it to jujucharms.com
<mattyw> TheMue, 2) There didn't seem to be any documentation about how to get units to spread across availability zones. is that documentation there somewhere (did I miss it or is it missing)
<mattyw> TheMue, you mean so the docs will appear on jujucharms.com?
<TheMue> mattyw: btw, 1.25 already as this branch in. has been a start on master, a backport to 1.25 due to needs, and then adopting the changes in master again
<TheMue> mattyw: yes, they will
<mattyw> TheMue, ok - my comments are only minor spelling things, and one place I'd like a bit more of an example, I'll let you work out where best to merge those in
<TheMue> mattyw: the spreading is done via constraints, but it's a good hint if you want them more detailed
<TheMue> mattyw: you're looking on it as a user :)
<mattyw> TheMue, I'm published my comments so far
<mattyw> TheMue, regarding the spreading across zones. I'd like to see examples of command line commands - and what you end up with. similar to my comment regarding the dmz/cms subnets example
<mattyw> I like seeing examples that say "if you run these commands at the command line this is the setup you end up with"
<TheMue> mattyw: ok, will add it
<mattyw> TheMue, you're great, thanks very much. let me know when there's a pr for jujucharms.com. Very excited about seeing that
<TheMue> mattyw: great, so I've already got a reviewer for it, thx
<voidspace> dooferlad: ping
<dooferlad> voidspace: pong
<voidspace> dooferlad: bridgeConfigTemplate in provider/maas/environ.go
<voidspace> dooferlad: it starts by attempting to detect if the bridge has already been setup
<voidspace> dooferlad: the first line is:
<voidspace> grep -q "iface {{.Bridge}} inet dhcp" && exit 0
<voidspace> dooferlad: it seems to me that's buggy, as it's missing the filename argument to grep (?)
<voidspace> I believe it should be
<voidspace> grep -q "iface {{.Bridge}} inet dhcp" {{.Config}} && exit 0
<voidspace> would you concur?
<dooferlad> voidspace: Yes
<voidspace> dooferlad: thanks
<dooferlad> voidspace: np
<dooferlad> voidspace: though grep will wait for stdin if you don't pass it a file
<dooferlad> voidspace: don't know if the script assumes it will be piped the current file?
<voidspace> dooferlad: right, but I don't think that's how we're using it (and it's executed on a non tty)
<dooferlad> voidspace: seems buggy anyway
<voidspace> dooferlad: we pass in .Config to the template, so I don't see why we would also pass in the file
<voidspace> a later invocation of grep uses {{.Config}}
<voidspace> dooferlad: I'll try and double check we're not passing the file
<voidspace> dooferlad: the script is executed via cloudcfg.AddRunCmd which has no mechanism for passing a file (unless it's done via the command itself - which it isn't)
<voidspace> I guess that invocation of grep always terminated with a non-zero exit code so it was effectively always ignored
<dooferlad> voidspace: how can that possibly work then? Grep will just sit waiting for input...
<voidspace> on a non-tty won't it just immediately exit with an error
<voidspace> as it can't get any input
<dooferlad> voidspace: ah, yes
<ericsnow> cmars: FYI, I've merged master into the lxd-provider branch and added the missing dependencies
<cmars> ericsnow, awesome stuff
<ericsnow> cmars: yeah, keep an eye on that branch
<dooferlad> natefinch: Hi, do you know much about why the uniter hook execution lock is file based and isn't reset on startup? I thought we only had one jujud running, so the choice seems odd.
<natefinch> dooferlad: I don't know.
<dooferlad> natefinch: thanks. Will reach out on list.
<natefinch> dooferlad: I have some guesses, because we do run things in different processes occasionally, but I don't know the details.
<dooferlad> natefinch: Ah, didn't know we had >1 process running sometimes. I have been tracking down a problem where the lock doesn't get released because the host is rebooted. Clearly nothing has it at that point!
<mup> Bug #1503740 opened: Storage should be behind a feature flag in 1.24 <juju-core:Triaged by axwalk> <https://launchpad.net/bugs/1503740>
<mup> Bug #1503740 changed: Storage should be behind a feature flag in 1.24 <juju-core:Triaged by axwalk> <https://launchpad.net/bugs/1503740>
<mup> Bug #1503740 opened: Storage should be behind a feature flag in 1.24 <juju-core:Triaged by axwalk> <https://launchpad.net/bugs/1503740>
<mgz> does anyone actually understand bug 1501563?
<mup> Bug #1501563: 1.26-alpha1 client gets connection shutdown deploy 1.22 server <blocker> <ci> <regression> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1501563>
<mup> Bug #1474588 changed: Many hook failures after upgrade <canonical-bootstack> <regression> <juju-core:Incomplete by menno.smits> <juju-core 1.24:Incomplete> <https://launchpad.net/bugs/1474588>
<mup> Bug #1474588 opened: Many hook failures after upgrade <canonical-bootstack> <regression> <juju-core:Incomplete by menno.smits> <juju-core 1.24:Incomplete> <https://launchpad.net/bugs/1474588>
<mup> Bug #1474588 changed: Many hook failures after upgrade <canonical-bootstack> <regression> <juju-core:Incomplete by menno.smits> <juju-core 1.24:Incomplete> <https://launchpad.net/bugs/1474588>
<perrito666> mgz: nope, not really clear what is going on there
<perrito666> mgz: it would seem that apiserver abruptly cuts off while colocating a charm
<mgz> perrito666: would --debug logging help?
<mgz> tasdomas: looks like dafe43b5683c9b22af86ee744a8ea7f088b087b6 caused bug 1501563 - any idea why?
<mup> Bug #1501563: 1.26-alpha1 client gets connection shutdown deploy 1.22 server <blocker> <ci> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1501563>
<natefinch> ericsnow: I'm just going to write a new command from scratch, unless you object?   Seems like all the work we did for the base command stuff does not apply to the new command structure
<ericsnow> natefinch: yep
<perrito666> mgz: might, but not sure
<perrito666> mgz: depends if we actually put debug messages around the broken part
<mgz> perrito666: indeed, which is why I was asking if you had an inkling :)
<mgz> we could also just try removing tasdomas' change
<mgz> seems to have another branch on top of it though
<ericsnow> rogpeppe: is there anything special that needs to be done relative to adding a new top-level section to metadata.yaml?
<ericsnow> rogpeppe: as far as I can see it's pretty straight-forward
<natefinch> ericsnow: see katherine's response in email - it's about the code not the yaml
<ericsnow> natefinch: yeah, that's what I'm asking about
<rogpeppe> ericsnow: technically you don't need to do anything in the charm package at all
<ericsnow> rogpeppe: does it ignore unrecognized entries?
<rogpeppe> ericsnow: yeah, you can have any files you like in a charm
<rogpeppe> ericsnow: it's just a zip file
<ericsnow> rogpeppe: I mean unrecognized entries in metadata.yaml itself
<rogpeppe> ericsnow: i thought you were adding a new file
<ericsnow> rogpeppe: no longer
<rogpeppe> ericsnow: what's this for?
<natefinch> same thing, new name
<ericsnow> rogpeppe: same feature (now called "payloads"
<ericsnow> )
<ericsnow> rogpeppe: we have to move it back into metadata.yaml
<rogpeppe> ericsnow: i'm pretty sure that metadata.yaml isn't the right place - that's the public interface of the charm
<rogpeppe> ericsnow: and this isn't actually something that's part of that, right?
<ericsnow> rogpeppe: it's not my call
<ericsnow> rogpeppe: agreed
<rogpeppe> ericsnow: so wtf?
<natefinch> rogpeppe: this is the direction we have been given from the highest levels of the echelon
<rogpeppe> natefinch: well if he wants the model broken, i guess we'll have to break it
<natefinch> rogpeppe: yep
<natefinch> rogpeppe: we argued it and other things for a month or so, and at the sprint the law was laid down.
<rogpeppe> ericsnow: so anyway, charm.ReadMeta does ignore extra members
<rogpeppe> ericsnow: so you can easily add more if you want to
<ericsnow> rogpeppe: in that case we could avoid adding any code to the charm repo
<ericsnow> rogpeppe: we'd just parse out the extra bits we care about
<rogpeppe> ericsnow: i'm not sure
<ericsnow> rogpeppe: I'm fine either way
<rogpeppe> ericsnow: i think that the metadata.yaml parsing code in charm should parse all the metadata
<rogpeppe> ericsnow: otherwise you'll need to do nasty things like re-parse and second-guess the format
<ericsnow> rogpeppe: k
<rogpeppe> ericsnow: that's not to say it has to be specified in total detail with all allowed keys etc. it's sometimes possible to allow a more general format and add more specific checks at higher levels
<rogpeppe> ericsnow: it depends really
<sinzui> cherylj: https://bugs.launchpad.net/juju-core/+bug/1468349
<mup> Bug #1468349: discoverySuite.TestDiscoverServiceLocalHost: invalid series for wily and vivid <centos> <test-failure> <unit-tests> <vivid> <wily> <juju-core:Fix Released by dooferlad> <juju-core 1.24:Won't Fix> <https://launchpad.net/bugs/1468349>
<cherylj> thanks, sinzui
<katco> rogpeppe: what natefinch says is true
<rogpeppe> katco: i believe it
<perrito666> katco: that was a powerful statement
<katco> perrito666: lol didn't catch that
<perrito666> katco: ?
<katco> perrito666: i didn't mean to say that everything nate says is true ;)
<katco> perrito666: because now i have introduced a tautilogical fallacy. as soon as he says "this statement is false", i am wrong
<perrito666> oh great, now you broke the universe
<mgz> rogpeppe: pokÃ©
<mgz> rogpeppe: any idea where the bits and pieces you ran to clean up txn db things for 1.23 issues are?
<rogpeppe> mgz: oh twats, i saw that email out of hours and forgot to reply this morning
<rogpeppe> mgz: one mo
<mgz> rogpeppe: merci
<rogpeppe> mgz: so the main thing is https://github.com/rogpeppe/mgo/pull/2/files
<rogpeppe> mgz: which is a patch to mgo (not accepted)
<rogpeppe> mgz: and then there's mgopurge.go: http://paste.ubuntu.com/12708968/
<rogpeppe> mgz: which actually does the job
<mgz> wallyworld: bug 1452422 - but see also bug 1271744
<mup> Bug #1452422: Cannot boostrap from custom image-metadata-url or by specifying metadata-source <sts> <juju-core:Fix Released by wallyworld> <juju-core 1.24:Fix Released by wallyworld> <https://launchpad.net/bugs/1452422>
<mup> Bug #1271744: bootstrap on maas with --metadata-source fails <bootstrap> <maas> <maas-provider> <upload-tools> <juju-core:Triaged> <https://launchpad.net/bugs/1271744>
#juju-dev 2015-10-08
<mup> Bug #1503990 opened: apiserver/http: attachment functions not tested <juju-core:New> <https://launchpad.net/bugs/1503990>
<mup> Bug #1503992 opened: apiserver: Digest SHA header is incorrectly formed <juju-core:New> <https://launchpad.net/bugs/1503992>
<mup> Bug #1503992 changed: apiserver: Digest SHA header is incorrectly formed <juju-core:New> <https://launchpad.net/bugs/1503992>
<mup> Bug #1503992 opened: apiserver: Digest SHA header is incorrectly formed <juju-core:New> <https://launchpad.net/bugs/1503992>
<dooferlad> TheMue / voidspace: If you have a moment: http://reviews.vapour.ws/r/2854/
 * dooferlad is done for the day
<rogpeppe> thumper: hiya
<perrito666> Is the core team going to happen ?
<natefinch> perrito666: ericsnow and I were there
<natefinch> now just me
<perrito666> I tried, no one was when I asked
<mgz> the core team is always happening
<frobware> voidspace, ping
<frobware> voidspace, any progress with the 1.9 MAAS bug?
<mup> Bug #1504272 opened: Errors from rpc do not appear in logs <api> <observability> <juju-core:Triaged> <https://launchpad.net/bugs/1504272>
<mup> Bug #1504272 changed: Errors from rpc do not appear in logs <api> <observability> <juju-core:Triaged> <https://launchpad.net/bugs/1504272>
<mup> Bug #1504272 opened: Errors from rpc do not appear in logs <api> <observability> <juju-core:Triaged> <https://launchpad.net/bugs/1504272>
<mup> Bug #1504272 changed: Errors from rpc do not appear in logs <api> <observability> <juju-core:Triaged> <https://launchpad.net/bugs/1504272>
<voidspace> frobware: I have a fix committed (in a branch)
<voidspace> frobware: needs a test
<voidspace> frobware: manually confirmed to work fine with maas 1.9
<frobware> voidspace, point me at the branch - I have a local 1.9 install and could try it.
<frobware> voidspace, (thx!!!)
<voidspace> frobware: however, I would still like to confirm the bug - maas 1.9 seems to work fine for me and I have yet to reproduce
<voidspace> frobware: https://github.com/voidspace/juju/tree/1494476-maas-networking
<mup> Bug #1504272 opened: Errors from rpc do not appear in logs <api> <observability> <juju-core:Triaged> <https://launchpad.net/bugs/1504272>
<voidspace> frobware: works fine, including deploying containers, with MAAS Version 1.9.0 (alpha1+bzr4064+4128+417~ppa0~ubuntu14.10.1)
<voidspace> frobware: dimitern: dooferlad is firmly of the opinion that the right cause of action is to remove juju-br0 altogether (even when addressable containers is off)
<voidspace> frobware: dimitern: and that it's relatively easy to do - even for deployed environments that have machines with an existing juju-br0
<voidspace> (leaving it in place for machines that have it - and so the PrepareContainerInterfaceInfo api call will need to know the bridge to use. Easy to add a new version of that api that takes the extra arg)
<dimitern> voidspace, dropping juju-br0 without replacing it with something else that will ensure container addressability is not an option unfortunately
<mgz> cmars: bug 1504272 btw, not sure if you want anything worded differently
<mup> Bug #1504272: Errors from rpc do not appear in logs <api> <observability> <juju-core:Triaged> <https://launchpad.net/bugs/1504272>
<voidspace> frobware: tell dimitern that the containers setup their own bridges that we can use
<voidspace> frobware: he's dropped off irc
<voidspace> frobware: anyway, a conversation for when you all return
<frobware> voidspace, back now
<cmars> mgz, thanks
<frobware> voidspace, when the container starts it sets up its own bridge?
<voidspace> frobware: no, installing kvm and lxc puts a bridge in place - containers will do that aspect of the networking themselves if we let them
<voidspace> if you have lxc installed you'll have a bridge it created
<voidspace> anyway, dooferlad can make his case for this when you return
<frobware> voidspace, can you recreate the problem?
<voidspace> frobware: not yet
<voidspace> just putting the daughter to bed - may get a chance in a bit
<voidspace> at the very least the new code works with maas 1.9 and when I run the script on the example that was screwed up it "does the right thing"
<voidspace> so it *should* work :-)
<frobware> dimitern, https://github.com/voidspace/juju/tree/1494476-maas-networking
<mgz> abentley: we're in the hangout
<mgz> http://reviews.vapour.ws/r/2860/ <- review please?
<mgz> menn0: http://reviews.vapour.ws/r/2860/
<sinzui> cherylj: https://bugs.launchpad.net/ubuntu/+source/juju-core/+bug/1267393
<mup> Bug #1267393: [MIR] juju-core, juju-mongodb, gccgo, golang <gccgo-5 (Ubuntu):Fix Released> <gccgo-go (Ubuntu):Invalid by doko> <golang (Ubuntu):Incomplete> <juju-core (Ubuntu):Confirmed> <juju-mongodb (Ubuntu):Incomplete> <https://launchpad.net/bugs/1267393>
<cherylj> hey dimitern, do you know where mfoord is with bug 1494476?
<mup> Bug #1494476: MAAS provider with MAAS 1.9 - /etc/network/interfaces "auto eth0" gets removed and bridge is not setup <addressability> <lxc> <maas-provider>
<mup> <network> <juju-core:Triaged by mfoord> <juju-core 1.24:Triaged by mfoord> <juju-core 1.25:In Progress by mfoord> <https://launchpad.net/bugs/1494476>
<dimitern> cherylj, he has a fix proposed for 1.25, I'm working on a simpler fix for 1.24
<cherylj> dimitern: awesome, thanks
#juju-dev 2015-10-09
<natefinch> ericsnow: thanks for the hard work.
<voidspace> dooferlad: grabbing coffee
<voidspace> 5 mins
<dooferlad> voidspace: ok
<voidspace> dooferlad: right, omw
<dooferlad> voidspace: are you in https://plus.google.com/hangouts/_/canonical.com/sapphire?authuser=0 ?
<voidspace> dooferlad: I'm using the link for today
<dooferlad> (that authuser=0 may not be 0 for you depending on how many google accounts you are signed into and in what order)
<voidspace> dooferlad: yes
<dooferlad> voidspace: looks like the standup ended then!
<voidspace> dooferlad: you've frozen
<voidspace> dooferlad: not sure if it's me or you :-)
<voidspace> dooferlad: but yes, reinstalling from scratch sounds good
<voidspace> dooferlad: I think so
<voidspace> dooferlad: have a good day :-)
<dooferlad> voidspace: and you
<rogpeppe> if anyone wants to take a look at this fairly simple review, that would be great. https://github.com/juju/names/pull/55/files
<rogpeppe> also, some test additions to juju (feature branch) http://reviews.vapour.ws/r/2852/
<voidspace> dooferlad: trusty installed, new maas 1.9 installed, now configuring maas (well, fetching boot images currently)
<voidspace> rogpeppe: looks like you already got your reviews
<rogpeppe> voidspace: yup, thanks for taking a look
<voidspace> rogpeppe: heh, np
<voidspace> rogpeppe: how's tricks?
<rogpeppe> voidspace: pretty good thanks
<rogpeppe> voidspace: been working quite a bit on juju-core recently for a change. just like old times :)
<voidspace> rogpeppe: same here, glad it's the weekend
<voidspace> rogpeppe: life is easier when the team leads are on a sprint
<voidspace> rogpeppe: it's been a good week, actually got some work done!
<rogpeppe> voidspace: yeah, we've got lots done too :)
<rogpeppe> voidspace: as for the weekend, i have some beef vindaloo marinating for saturday and my mouth is watering already
<voidspace> rogpeppe: ooh, nice
<voidspace> now you're making me hungry
<rogpeppe> voidspace: haven't done this particular recipe before but the marinade looks awesome
<voidspace> a nice vindaloo is a thing of wonder
<voidspace> I should learn to cook
<rogpeppe> voidspace: this is the recipe: http://www.munamahdzarfadaaq.com/2013/07/beef-vindaloo.html
<voidspace> it's on my list
<voidspace> (learning to cook that is - I rely to much on my wonderful but long suffering wife)
<rogpeppe> voidspace: curries are pretty straightforward actually
<voidspace> rogpeppe: they'd be good to start with as my wife won't cook hot curry :-)
<voidspace> although we have a great curry house nearby
<rogpeppe> voidspace: not too much subtle chemistry involved, unlike quite a bit of european
<voidspace> hehe
<voidspace> right
<voidspace> I don't do subtle at the best of times...
<voidspace> rogpeppe: I've bookmarked the recipe, I'm very tempted
<voidspace> thanks
<voidspace> rogpeppe: and enjoy :-)
<rogpeppe> voidspace: i will
<rogpeppe> voidspace: have a good weekend!
<voidspace> rogpeppe: we have a big church (from all across the country) celebration this weekend in Sheffield
<voidspace> rogpeppe: they're always great fun, lots of life and good to see everyone
<voidspace> rogpeppe: so it should be a good one
<rogpeppe> voidspace: cool
<voidspace> rogpeppe: and tonight I'm off to see the martian with a friend
<rogpeppe> voidspace: i need to see that
<voidspace> rogpeppe: I've heard mostly good things about it...
<voidspace> dooferlad: and after remembering to configure the network properly in maas, node enlisting worked and commissioning in process
<voidspace> I may need to do the "setting the power type" dance before I can attempt to bootstrap and see if I can reproduce the bug
<voidspace> ah no, can bootstrap without setting power type if I manually start the vms
<voidspace> maas used to refuse to let me do that
<rogpeppe> voidspace: here's another small review for you if you care to look - we changed our minds on the earlier one: https://github.com/juju/names/pull/56
<voidspace> rogpeppe: looking
<rogpeppe> voidspace: tal
<voidspace> dooferlad: hmmm... looks like I still can't repro the bug with 1.9 alpha 2 from dailybuilds-qa-ok
<voidspace> dooferlad: bootstrap is supposed to fail and it works fine!
<voidspace> rogpeppe: looks like a nice improvement
<rogpeppe> voidspace: thanks
<voidspace> LGTM
<voidspace> dammit
<voidspace> I have maas 1.7 I think
<rogpeppe> voidspace: and... here's the juju-core changes resulting from that names package change: http://reviews.vapour.ws/r/2866
<voidspace> rogpeppe: up to my knees in broken and confusing maas
<voidspace> rogpeppe: will probably be able to get to it in a bit though
<rogpeppe> voidspace: np
<rogpeppe> voidspace: that would be nice, thanks
<voidspace> rogpeppe: why the change to the github.com/juju/utils entry?
<voidspace> rogpeppe: the revision hasn't changed but the timestamp has
<rogpeppe> voidspace: i know, that's really really weird
<voidspace> :-)
<rogpeppe> voidspace: i've seen that happen a few times
<rogpeppe> voidspace: and i don't know how it can, 'cos AFAIK the time stamp is an immutable property of the commit log entry
<rogpeppe> voidspace: it's time-zone related somehow
<rogpeppe> voidspace: anyway, the commit hash is the only thing that really matters
<voidspace> rogpeppe: ok
<voidspace> rogpeppe: got a friend coming for lunch in ten minutes or so - but will carrying on reading until then
<voidspace> and pick it up again afterwards
<rogpeppe> voidspace: thanks
<voidspace> although straightforward changes so far...
<voidspace> rogpeppe: it's *all* comment changes, swapping Username for Id and removing of @local
<voidspace> rogpeppe: LGTM
<rogpeppe> voidspace: yeah, thanks
<voidspace> hmmm... except the backwards compatibility issue (potential) noted by mattyw
<voidspace> I have yet another screwed maas install *sigh*
<voidspace> a vanilla maas 1.9 alpha 2 from dailybuilds doesn't seem to work
<voidspace> and that's the only way to get a recent enough maas to reproduce this bug
<voidspace> so I give up and will just write my test anyway and leave manual testing for maas to be fixed
<mup> Bug #1504578 opened: upgrade fails because the api server cannot get port <blocker> <ci> <regression> <upgrade-juju> <juju-core:Incomplete> <juju-core chicago-cubs:Triaged> <https://launchpad.net/bugs/1504578>
<cherylj> cmars: ping?
<cherylj> rogpeppe: Could you take a look at bug 1504578?
<mup> Bug #1504578: upgrade fails because the api server cannot get port <blocker> <ci> <regression> <upgrade-juju> <juju-core:Incomplete> <juju-core chicago-cubs:Triaged> <https://launchpad.net/bugs/1504578>
<cherylj> rogpeppe: cmars volunteered you :)
<rogpeppe> cherylj: how privileged am i? :)
<rogpeppe> cherylj: i'll take a look
<rogpeppe> cherylj: i don't know what the status of critical bugs on feature branches is
<rogpeppe> cherylj: does it means we can't do any more work on the feature branch until it's resolved?
<rogpeppe> cherylj: (i think probably not)
<mup> Bug #1504602 opened: haproxy charm is stuck in an executing state preventing relation changes <juju-core:New> <https://launchpad.net/bugs/1504602>
<rogpeppe> does anyone here know about juju version upgrade state migrations?
<rogpeppe> we're trying to fix a critical bug here
<mgz> rogpeppe: commented on bug 1504578
<mup> Bug #1504578: upgrade fails because the api server cannot get port <blocker> <ci> <regression> <upgrade-juju> <juju-core:Incomplete> <juju-core chicago-cubs:Triaged> <https://launchpad.net/bugs/1504578>
<mgz> rogpeppe: as I understand it, the second bug from your comment was introduced by that change,
<mgz> so we could file another bug for the upgrade not doing what it apparently should
<mgz> cherylj: bzrlib portable process life detection code:
<mgz> http://bazaar.launchpad.net/~bzr-pqm/bzr/bzr.dev/view/head:/bzrlib/osutils.py#L2531
<mgz> http://bazaar.launchpad.net/~bzr-pqm/bzr/bzr.dev/view/head:/bzrlib/win32utils.py#L604
<mgz> bug 220464
<mup> Bug #220464: Bazaar doesn't detect its own stale locks <lockdir> <locking> <Bazaar:Fix Released by mbp> <https://launchpad.net/bugs/220464>
<cherylj> dooferlad: ^^^
<cherylj> dooferlad: it's relevant to my review comments for your fslock changes
<mup> Bug #1504637 opened: storage: error message for unknown storage is terrible <juju-core:Triaged> <https://launchpad.net/bugs/1504637>
<mup> Bug #1504637 changed: storage: error message for unknown storage is terrible <juju-core:Triaged> <https://launchpad.net/bugs/1504637>
<mup> Bug #1504637 opened: storage: error message for unknown storage is terrible <juju-core:Triaged> <https://launchpad.net/bugs/1504637>
<mup> Bug #1504658 opened: storage: cannot upgrade charms with additional required storage <juju-core:Triaged by axwalk> <juju-core 1.25:Triaged by axwalk> <https://launchpad.net/bugs/1504658>
<mup> Bug #1504658 changed: storage: cannot upgrade charms with additional required storage <juju-core:Triaged by axwalk> <juju-core 1.25:Triaged by axwalk> <https://launchpad.net/bugs/1504658>
<mup> Bug #1504658 opened: storage: cannot upgrade charms with additional required storage <juju-core:Triaged by axwalk> <juju-core 1.25:Triaged by axwalk> <https://launchpad.net/bugs/1504658>
<mup> Bug #1501563 changed: 1.26-alpha1 client gets connection shutdown deploy 1.22 server <blocker> <ci> <regression> <juju-core:Fix Released by cmars> <https://launchpad.net/bugs/1501563>
<cmars> mgz, cherylj did we ever get a fix landed for the chicago-cubs issue? was there a bug opened on it?
<mgz> cmars: I didn't see a response from rog, but I've been juggling
<cmars> mgz, is there a bug open on it?
<mgz> cmars: I see no commits
<cmars> mgz, ok, thanks
<mgz> cmars: bug 1504578
<mup> Bug #1504578: upgrade fails because the api server cannot get port <blocker> <ci> <regression> <upgrade-juju> <juju-core:Incomplete> <juju-core chicago-cubs:Triaged> <https://launchpad.net/bugs/1504578>
#juju-dev 2015-10-10
<mup> Bug #1504821 opened: Please switch dependency from gopkg.in/yaml.v1 to gopkg.in/yaml.v2 <juju-core:New> <https://launchpad.net/bugs/1504821>
#juju-dev 2016-10-10
<axw> wallyworld: I think you misunderstood me about the API
<axw> I just meant to combine them in the CLI code
<wallyworld> ah, damn
<axw> i.e. create an interface that you can satisfy by embedding both the facades into one struct
<wallyworld> oh well, what's there is ok i think?
<wallyworld> we use that same pattern elsewhere too
<axw> wallyworld: I think it's fine
<axw> wallyworld: ModelManager makes more sense for this command anyway I think
<wallyworld> yeah, agree
<rick_h_> wallyworld: is the model destroy blocking something communicated to other teams?
<rick_h_> wallyworld: coukd tgat imoact things like the gui?
<wallyworld> rick_h_: it was a stakeholder bug from OIL I think. I am not sure who else knows
<wallyworld> i am not sure about GUI - I'll send an emil
<wallyworld> rick_h_: but the blocking is ony CLI
<rick_h_> wallyworld: right, i'm +1000 on the change, but want to make sure we chat with ither clients
<rick_h_> wallyworld: ah ok
<wallyworld> the api behaves the same
<rick_h_> wallyworld: nvm then
<wallyworld> fair enough question to asl
<wallyworld> *ask
<menn0> thumper: i've fixed the racy pingerSuite tests: https://bugs.launchpad.net/juju/+bug/1625214
<mup> Bug #1625214: pingerSuite.TestAgentConnectionsShutDownWhenAPIServerDies write: broken pipe <ci> <intermittent-failure> <regression> <unit-tests> <juju:In Progress by menno.smits> <https://launchpad.net/bugs/1625214>
<menn0> thumper: what I meant to paste: https://github.com/juju/juju/pull/6410/commits/f9ee5dba43e0fa032fe67d14d1c89cdc1c1e1a55
<thumper> looking
<thumper> approve
<menn0> thumper: cheers
<menn0> wallyworld: https://github.com/juju/juju/pull/6411 please
<wallyworld> menn0: sure, just finishing another
<menn0> wallyworld: ignore the first commit. that one is merging as part of another PR now.
<wallyworld> ok
<wallyworld> menn0: could you take a look at this one by christian? i think your +1 would be valuable. I've already had a look https://github.com/juju/juju/pull/6408
<wallyworld> also, your 6411 pr has conflicts
<menn0> wallyworld: will do. he emailed me about it too.
<wallyworld> ta
<menn0> wallyworld: conflicts fixed
<wallyworld> ta
<axw> menn0: are you investigating that cert issue?
<menn0> axw: I was about to take a short break and then stuck into it
<menn0> axw: I still have a controller up where that's happened
<menn0> axw: have you seen it too?
<axw> menn0: ok. would it be helpful if I looked in parallel?
<axw> menn0: no, but it didn't sound hard to repro
<menn0> axw: could be. if you could take a look too that would be great.
<axw> menn0: ok. I'll let you know if I find anything
<menn0> axw: here's what the failure looks like: http://paste.ubuntu.com/23301342/
<axw> okey dokey
<axw> menn0: and all you did was bounce the controlelr agent?
<menn0> axw: I stopped the lxd machine and started it again
<axw> ah yes
<axw> ok
<menn0> the controller instance
 * menn0 will be back in 15 mins
<axw> wallyworld: what did you want me to test with add-model? something like this? bootstrap rc2, upgrade to my branch, then add-model foo && add-model bar ap-southeast-2
<wallyworld> axw: yeah, check that the endpoint fixing is done for addmodel also
<wallyworld> axw: or even first confirm that it is broken without your fix
<wallyworld> so we can be sure that the fix is relvant nd valid
<axw> wallyworld: it would be broken if you tried to upgrade from rc2
<axw> wallyworld: but if you added a model to a running rc2 without upgrading, it should work. anyway, I'll try what I described
<wallyworld> ta, i think we need to be 10000% sure
<axw> menn0: I strongly suspect -> https://github.com/juju/juju/commit/0d64508f8005ea84c141ed392389fbaef0e70b30
<axw> menn0: in particular, "info.Cert = existing.Cert" without also setting Key
<blahdeblah> Hi everyone, which ppa should I be using to get the RC version of juju 2.0 so I can try https://docs.google.com/document/d/1yT5pvS38g9Z9SviI9NPmhIZrsO8drwHDPNWAMSmCedE/edit# ?
<menn0> axw: looks like there was already a ticket for this issue: https://bugs.launchpad.net/juju/+bug/1631145
<mup> Bug #1631145: rc2 upgrade to rc3 failed with cannot start api server worker: cannot set initial certificate: cannot create new TLS certificate: crypto/tls: private key does not match public key <juju:New> <https://launchpad.net/bugs/1631145>
<menn0> axw: i've just bumped the priority etc
<axw> okey dokey
<menn0> axw: any luck replicating?
<axw> menn0: not yet
<axw> blahdeblah: according to release email,  ppa:juju/devel
<blahdeblah> axw: cool - thanks
<wallyworld> axw: with maas, do you know what happens if you attempt to acquire a node without specifying an architecture? does it just pick some arbitary machine so long as any other constraints match?
<axw> menn0: hrm. I am seeing "getsockopt...connection refused" without the other error though
<axw> and status isn't working now
<axw> wallyworld: I don't recall, sorry
<menn0> axw: that's what you'll see at the client
<menn0> axw: what about in the controller's logs?
<wallyworld> maybe menn0 recalls?
<axw> menn0: this is in the controller machine log
<menn0> axw: hmm ok
<axw> menn0: it's failing to connect to mongo...
<menn0> axw: possibily a different problem?
<axw> yes, possibly
<anastasiamac> wallyworld: r:maas arch.. yes, i beleve so.. at lest this s what i ve seen
<menn0> axw: not ideal
<axw> menn0: although... do we use the same cert/key for mongo?
<wallyworld> menn0: ta, ok
<menn0> wallyworld: sorry, not sure about maas
<wallyworld> np, i'll ask the maas qguys
<wallyworld> guys
<menn0> axw: maybe we do use the same cert/key... worth checking
<axw> menn0: it does appear to be the case
<axw> when we update the cert, we write out to server.pem
<menn0> axw: ok, so probably a different manifestation of the same issue
<axw> yup
<menn0> axw: i'm tracing through where the cert and key that apiserver.NewServer is given come from
<menn0> axw: ok so that's out of StateServingInfo in the agent config
<menn0> axw: perhaps one is being updated without the other?
<menn0> axw: this could be it: https://github.com/juju/juju/blob/master/cmd/jujud/agent/machine/servinginfo_setter.go#L67-L74
<menn0> axw: should existing.PrivateKey get copied too?
<menn0> axw: I'd also be a lot happier if most of what the stateservinginfo_setter did happened /inside/ the ChangeConfig
<menn0> axw: same for the method which does cert updates for the apiserver
<menn0> looks racy otherwise
<menn0> axw: the more I look, the more I think this is the bug
 * menn0 fixes
<axw> menn0: sorry, I just realised you lost connection after I pasted the commit before :/
<menn0> axw: did you come to the same conclusion?
 * menn0 has been having wifi problems
<axw> [11:40:32] <axw> menn0: I strongly suspect -> https://github.com/juju/juju/commit/0d64508f8005ea84c141ed392389fbaef0e70b30
<axw> [11:40:57] <axw> menn0: in particular, "info.Cert = existing.Cert" without also setting Key
<menn0> axw: ha! did see that. we independently found the same thing
<menn0> axw: s/did/didn't/
<menn0> axw: do you also agree that much of the logic in stateservinginfo_setter should be /inside/ the ChangeConfig func?
<menn0> axw: same for MachineAgent.upgradeCertificateDNSNames
<axw> menn0: yes, I think so
<axw> anything looking at current config
<menn0> axw: ok, let's fix it.
<menn0> axw: have you already started?
<axw> menn0: nope, eating lunch atm
<wallyworld> blahdeblah: i so want to mark your bug as invalid. using firefox as default browser should be invalid :-)
<menn0> axw: ok, i'll start on servinginfo_setter.go.
<menn0> axw: i'm going to have to stop soon but will be back on later this evening
<blahdeblah> wallyworld: Feel like taking the gloves off today, eh? :-)
<axw> menn0: ok. let me know where you leave off, I can pick it up
<menn0> axw: sounds good
<wallyworld> blahdeblah: i used to run firefox until it started to grind everything to a halt with many tabs open and other performance things
<blahdeblah> wallyworld: It has never done that for me.  I've heard many claims of such, but never seen it personally, despite having 4 windows with 100-odd tabs open right now.  I thank NoScript for that.
<wallyworld> blahdeblah: yeah, that may well be it. sadly lots of sites require js, including lp. well lp doesn't need it but uscks without it
<blahdeblah> I use js on lots of sites; I just don't let it do that by default
<wallyworld> i find it too hard to whitelist everything. well maybe am too lazy
<blahdeblah> anyhew, my bug is valid, and I'll fight you for it! :-P
<wallyworld> adblock works a treat though
<wallyworld> oh i know it is valid
<wallyworld> was just stirring
<blahdeblah> I see your stirring, and raise you lp:1587644, which got me out of bed at least once this weekend
<anastasiamac> menn0: wallyworld: considering u've re-viewed the fix for bug 1625774
<mup> Bug #1625774: memory leak after repeated model creation/destruction <ateam> <eda> <oil> <oil-2.0> <uosci> <juju:In Progress by 2-xtian> <https://launchpad.net/bugs/1625774>
<anastasiamac> menn0: wallyworld: is it possible/wanted/needed on 1.25 to solve some of the memory leakage there too?
<wallyworld> not sure, depends on when state pool was introduced. not sure if it was for multi model or not
<wallyworld> would have to dig in and look
<menn0> anastasiamac: it could be applied there but I don't think multi-model was supported without a feature flag was it? the fix would have minimal utility if that's the case.
<wallyworld> yes, multi model in 1.25 is ff
<anastasiamac> wallyworld: even server-side stuff? we usually ff at cli level...
<wallyworld> not for multi model
<wallyworld> IIRC
<menn0> yeah, I'm pretty sure the feature flag was applied throughout the server too
<menn0> axw: I have the ssi setter fixes done, pushing now
<axw> cool
<menn0> axw: can I leave the MachineAgent.upgradeCertificateDNSNames fix to you?
<axw> menn0: yep sure
<menn0> axw: https://github.com/juju/juju/pull/6413
<axw> menn0: I think the only thing to do there is to not use "si.Cert", but get fresh value inside ChangeConfig?
<axw> ta
<mup> Bug #1630728 changed: remove user  needs better message that user is made inactive <usability> <juju:Triaged> <https://launchpad.net/bugs/1630728>
<menn0> axw: shouldn't cert.NewDefaultServer be called inside the ChangeConfig though?
<axw> menn0: yes, I mean everything from "Parse the current certificate to get the current dns names." down should be inside
<menn0> axw: +1 that's what I was thinking
<menn0> should be easy
<menn0> axw: just trying to QA this PR before I get called to help out with other stuff
<axw> menn0: except maybe the mongo bit. might want to do that after writing agent conf
<axw> menn0: okey dokey, thanks
<menn0> axw: maybe... it might be ok to do it in the changeconfig though
<menn0> axw: actually, i'm out of time. if you could review that PR, I'll check back in later and do the QA before merging
<axw> menn0: approved
<axw> menn0: with one late comment
<menn0> axw: I made that change and QAed. Merging now.
<axw> menn0: thanks
<axw> menn0: I'm just QAing my change now
<menn0> axw: cool. i can review when it's up. i've got to do dishes etc but will check back in later.
<axw> menn0: https://github.com/juju/juju/compare/master...axw:lp1631145-upgradecertificatednsnames?expand=1 if you want to take quick look
<axw> ok, no woprries
<axw> wallyworld: got anything smallish you'd like help?
<wallyworld> axw: not off hand. I am finishing a partial list-clouds fix pending input from rick et al. i think there's one remaining bug on alexis' list to do with agent restart but not sure how big it is
<axw> wallyworld: sounded big, I'll take a look
<wallyworld> yeah, it did at first look
<wallyworld> axw: is your cert fix just because we need to recover from a borked cert cause by rc3?
<axw> wallyworld: yes. and also we should be doing stuff inside ConfigChanged in general. doesn't matter in this case because the function is only called at startup
<axw> but good to keep it clean, to avoid copy&paste errors
<wallyworld> +1
<wallyworld> lgtm
<hoenir> could someone review my PR ? https://github.com/juju/juju/pull/6414
<hoenir> also note that this PR will be the foundations of a new feature I wish to unlock on juju, enabling manual provisioning for windows machines.
<hoenir> foundation*
<mup> Bug #1420996 opened: Default secgroup reset periodically to allow 0.0.0.0/0 for 22, 17070, 37017 <canonical-is> <juju-core:New> <https://launchpad.net/bugs/1420996>
<dooferlad> mgz: is the github jujubot happy? It seems to be ignoring my $$merge$$ on https://github.com/juju/juju/pull/6406
<babbageclunk> wallyworld: ping?
<wallyworld> yo
<babbageclunk> wallyworld: I think I agree with you about Release vs Put - do you think menn0 will sulk if I change it?
<wallyworld> babbageclunk: he can sulk, i'll get the popcorn ready :-D
<babbageclunk> :)
<wallyworld> babbageclunk: or even Done()
<babbageclunk> wallyworld: Done isn't verby enough for my taste. Get/Done doesn't seem right.
<wallyworld> fair point, i was trying to find an alternative to Release() to aleviate objections :-)
<babbageclunk> always the peacemaker
<wallyworld> oh, i have been known to enjoy poking the hornet's nest :-D
<babbageclunk> wallyworld: do you think I should rip the non-strict model uuid checking out of validateModelUUID?
<babbageclunk> wallyworld: There are still a few places that use it.
<wallyworld> babbageclunk: i *think* so - maybe in a followup after checking with tim/menno. i am pretty sure it was just to support really old clients
<babbageclunk> wallyworld: Ok - I'll do it as a separate change.
<wallyworld> sgtm
<mup> Bug #1631899 opened: juju show-controller --show-password does not show the password <juju-core:New> <https://launchpad.net/bugs/1631899>
<babbageclunk> wallyworld: Want to take another look at that PR, or are you ok for me to merge it?
<wallyworld> babbageclunk: i can take a quick look, menno was happy with it
<babbageclunk> wallyworld: cool thanks - I think I've gone through all of your comments.
<wallyworld> babbageclunk: i think it looks good to go. thanks for doing the fix; was a challenege to get all the bit lined up so you did well
<babbageclunk> wallyworld: cheers!
<beisner> hi all, last week we started seeing sha256 mismatches when units try to download the charm from the controller (1.25.6).  it's prevalent in openstack ci.  ie. shutting down: ModeInstalling ... failed to download charm ... expected sha256 FOO, got BAR
<beisner> is there a known issue or are we special? :)   more detail @ http://pastebin.ubuntu.com/23302619/
<mup> Bug #1631899 changed: juju show-controller --show-password does not show the password <juju-core:New> <https://launchpad.net/bugs/1631899>
<mup> Bug #1631899 opened: juju show-controller --show-password does not show the password <juju-core:New> <https://launchpad.net/bugs/1631899>
<mup> Bug #1631899 changed: juju show-controller --show-password does not show the password <juju-core:New> <https://launchpad.net/bugs/1631899>
<mup> Bug #1541482 opened: unable to download local: charm due to hash mismatch in multi-model deployment <2.0-count> <juju-release-support> <juju:Fix Released by menno.smits> <juju-core:New> <https://launchpad.net/bugs/1541482>
<mup> Bug #1629951 changed: cannot specify subnet to  create controller in on bootstrap <juju:Triaged> <https://launchpad.net/bugs/1629951>
<mup> Bug #1630029 changed: models should inherit vpc-id from controller  <juju:Triaged> <https://launchpad.net/bugs/1630029>
<dooferlad> rick_h_: 1:1 today?
<rick_h_> dooferlad: /me checks thought he was in it
<rick_h_> dooferlad: oh hmm, stuck at "requesting to join the video call"
<rick_h_> katco``: dimitern voidspace mgz natefinch ping for standup
<mgz> omw
<dimitern> omw
<rick_h_> oh right, US away so ignore me katco`` and nate
<voidspace> rick_h_: omw
<frobware> jamespage: do you often run into this issue: https://bugs.launchpad.net/juju/+bug/1600546
<mup> Bug #1600546: lxd subnet setup by juju will interfere with openstack instance traffic <2.0> <network> <sts> <juju:Triaged by rharding> <nova-compute (Juju Charms Collection):New> <https://launchpad.net/bugs/1600546>
<voidspace> rick_h_: ping
<rick_h_> voidspace: pong
<rick_h_> dooferlad: ping
<dooferlad> rick_h_: hello
<rick_h_> dooferlad: have more time to chat?
<dooferlad> rick_h_: yes
<rick_h_> dooferlad: k, meet you back in the 1-1 room
<voidspace> rick_h_: 30 seconds for a bikeshed needed if you have it
<voidspace> rick_h_: last comment on bug 1602192 is the output I've implemented
<mup> Bug #1602192: when starting many LXD containers, they start failing to boot with "Too many open files" <lxd> <juju:In Progress by rharding> <lxd (Ubuntu):Confirmed> <https://launchpad.net/bugs/1602192>
<rick_h_> voidspace: otp
<rick_h_> voidspace: will look in a sec
<voidspace> rick_h_: ok, np - I'll PR it and the reviewer can bikeshed it
<jamespage> frobware, I've never run into that issue
<frobware> jamespage: thanks. just trying to understand the severity and whether we address it "now".
<jamespage> but apparently trent has hit it alot
<frobware> jamespage: any feeling whether this is just happening for just the training setup?
<jamespage> frobware, I'm not quite close enough to know the answer to that
<jamespage> sorry
<frobware> rick_h_: ^^ fyi
<voidspace> dimitern: if you have any ideas on how to test this, I'm all ears
<voidspace> dimitern: https://github.com/juju/juju/pull/6419
<dimitern> voidspace: looking
<voidspace> dimitern: except providing something that stubs out PrintLn.
<voidspace> dimitern: the important bit is in provider/common/bootstrap.go
<dimitern> voidspace: how about adding an interface with BootstrapMessage() method and then check in common/bootstrap if it's implemented by the Environ ?
<dimitern> voidspace: then you could test it with the dummy provider, but no need to change all providers?
<voidspace> dimitern: right, but what does that enable me to test?
<voidspace> dimitern: I could then pass in a fake environ with a custom message, but what do I test?
<voidspace> dimitern: unless I replace the call to fmt.Println with something that can be mocked - which doesn't seem very useful
<voidspace> dimitern: I'm kind of arguing that as this is a ui change it needn't be tested
<voidspace> dimitern: maybe I can look in featuretests to see if something like this is covered
<dimitern> voidspace: only that common.Bootstrap() calls the optional method, when it's there, and checking the ctx.Stdout() to ensure it's there?
<rick_h_> voidspace: wfm for now, thank you
<voidspace> rick_h_: cool, thanks
<rick_h_> frobware: yea, so I'm -1 on that being the biggest issue atm
 * rick_h_ goes to get lunchables
<voidspace> dimitern: in which case I can test that *already*
<voidspace> dimitern: and put a non-empty string in dummy provider to test it
<voidspace> dimitern: let me look - thanks
<voidspace> dimitern: if we're calling fmt.Println will that be in ctx.Stdout ?
<dimitern> voidspace: alternatively, you could go with BootstrapContext.Infof() only called in provider/lxd ?
<dimitern> voidspace: since it's very specific and not needed everywhere
<dimitern> voidspace: check e.g. PrepareForBootstrap in provider/ec2 and the code around validateBootstrapVPC()
<dimitern> voidspace: (bootstrap)ctx.Infof() is already used for similar messages during bootstrap in some providers
<voidspace> kk
<voidspace> dimitern: using Stderr on the context I can test that the message is output
<dimitern> voidspace: \o/ nice!
<rogpeppe1> katco``: i've made a bunch of comments and changes in response to your review of https://github.com/juju/juju/pull/6407. PTAL.
<rick_h_> rogpeppe1: she's on US holiday today
<rick_h_> rogpeppe1: and with the EU folks EOD'ing will have to catch tomorrow it looks like sorry for the delay
<arosales> If any folks are have access to s390 we are seeing https://bugs.launchpad.net/juju/+bug/1632030
<mup> Bug #1632030: juju-db fails to start -- WiredTiger reports Input/output error <juju> <juju-db> <mongodb> <s390x> <juju:New> <https://launchpad.net/bugs/1632030>
 * rick_h_ goes to get the boy home from school, biab
<veebers> alexisb: Ina normal (test) run, should I see the log message: "ERROR cmd supercommand.go:458 creating API connection: ..."? (I'm seeing this error in the failing tests: ERROR cmd supercommand.go:458 creating API connection: EOF)
<alexisb> not on a normal test run you are expecting to pass without error
<veebers> I never see "creating API connection" in a passing test run (for grant-revoke)
<alexisb> veebers, when did you start seeing the new failures?
<veebers> alexisb: what is it a symptom of? One sec will check times
<veebers> alexisb: looks to be about Oct 7 (Jenkins time, so UTC?) had 2 passes of 12 runs since then, the failures all include that error message
<alexisb> menn0, ^^^
<alexisb> I am wondering if this lines up with this merge: https://github.com/juju/juju/pull/6400
<menn0> veebers: where are you see that message? from the juju client or in agent logs?
<menn0> veebers: a certain amount of such messages are expected as when controllers come up and when workers restart after new controllers are added
<veebers> menn0: um, I'm pretty sure the client? (I'm seeing it in the log output from juju command)  http://reports.vapour.ws/releases/4467/job/functional-grant-revoke/attempt/721 if you search for "connection: EOF"
 * menn0 looks
<veebers> menn0: this is after a controller comes up, users have been added etc.
<veebers> menn0: Would this be related to the ping changes to? http://reports.vapour.ws/releases/4467/job/run-unit-tests-race/attempt/1949 (FAIL: monitor_internal_test.go:69: MonitorSuite.TestLaterPingFails)
<menn0> veebers: that's a new test. i'll take a look.
<menn0> veebers: the other issue appears to be macaroons related. here's the related bit in the controller's logs:
<menn0> 2016-10-10 16:17:39 INFO juju.apiserver request_notifier.go:70 [1E] API connection from 10.0.8.1:33086
<menn0> 2016-10-10 16:17:39 INFO juju.apiserver admin.go:102 login failed with discharge-required error: verification failed: no macaroons
<menn0> 2016-10-10 16:17:39 INFO bakery service.go:366 server attempting to discharge "eyJUaGlyZFBhcnR5UHVibGljS2V5IjoieVZlUEU3cUNJMm5nZWZuUXo2NlFPWVEwanlMVnVDanMwZGhMa0VUa3dRST0iLCJGaXJzdFBhcnR5UHVibGljS2V5IjoiNzF5NnNIOExNNUU1OHYxaGpGek9JdCtCZGRLRHR6SE5pN0ExQ1dBWHZqND0iLCJOb25jZSI6InpoMllTNnZ1bFFHNUlPdVRucXY4U0hVNW5MQWlSaE43IiwiSWQiOiJlSHM0Y3dERDRiOCs5TXBDUG1RakwvenVFdXZhV3dSbmhkY0tXWmtGQjA5MENaMXYxcnBYZ0RNczRsTTUrc2FSS2l2RGt0OFhlOG5BZTNxVG5
<menn0> 4Y1YwMXhIUWZxTXRmWHk4TUdnNmRvSWZlZWpSZmJnby9pdjIxSmZiU2FkQjNwdXR6ZmJ2ekRvbGh1aDNPcGdCbTRQd2pSRGx4cGFEQnZaL0lwZGhoL0lQb3dMdXc9PSJ9"
<menn0> 2016-10-10 16:17:39 INFO juju.apiserver request_notifier.go:80 [1E]  API connection terminated after 28.244335ms
<menn0> 2016-10-10 16:17:39 INFO juju.apiserver request_notifier.go:70 [1F] API connection from 10.0.8.1:33096
<menn0> 2016-10-10 16:17:39 INFO juju.apiserver request_notifier.go:80 [1F] user-admin API connection terminated after 43.24719ms
<menn0> veebers: i'll dig a bit more into that one to see if I can figure it out
<veebers> menn0: I see a macroons change about 3 days ago, "remove macaroons on logout"
<menn0> veebers: ok, could be related. those messages might also be normal. I'll do some digging.
<veebers> menn0: Hmm, it's possible that the test misbehaves with those recent changes. Just looking now and if it's running on lxd (which this test is) it doesn't handle entering the password.
<veebers> menn0: Did that behaviour change recently? (i.e. lxd not need password)
<menn0> veebers: hmm, ok. that could be it.
<menn0> veebers: I have no idea.
 * veebers builds latest juju to test
<veebers> menn0: let me test that and I'll get back to you
<rick_h_> veebers: hmm, yea the branch nate did made sure to clear cookies when you logout
<rick_h_> veebers: menn0 which meant that you had to actually login again after a logout for a change
<veebers> rick_h_: that's probably it then, I should be able to confirm shortly
<menn0> veebers: is there a ticket yet for the TestLaterPingFails intermittent failure?
<veebers> menn0: not yet, I can make one right now
<menn0> veebers: ok thanks. i've just reproed it here
<menn0> grrr. even with injected clocks it's still too easy to introduce these intermittent timing issues ....
<veebers> menn0: fyi https://bugs.launchpad.net/juju/+bug/1632105
<mup> Bug #1632105: Test MonitorSuite.TestLaterPingFails fails  <ci> <regression> <unit-tests> <juju:Confirmed> <https://launchpad.net/bugs/1632105>
<menn0> veebers: cheers
<menn0> veebers: ok i've got the fix for TestLaterPingFails done... will propose shortly
<veebers> menn0: sweetbix
<menn0> veebers: thanks for pointing it out
<veebers> menn0: heh, no worries, it popped up in the 'unknowns' that I'm wrangling
<menn0> good to get on top of these intermittent failures quickly
<veebers> agreed
<menn0> thumper: easy review pls: https://github.com/juju/juju/pull/6420
 * thumper looking
<thumper> +1
<veebers> menn0: ugh, sorry for the noise earlier, the fix for the grant/revoke/macroons stuff was in the CI test.
<menn0> veebers: ok good to know
<menn0> thumper: it's probably too late to expose the HighAvailabilty facade for controller logins isn't it?
<thumper> menn0: yes and no
<thumper> you could create a new one
<thumper> but still need to support the old one
<menn0> thumper: i'm working on the ticket to allow juju enable-ha to just work regardless of the current model
<thumper> I guessed
<menn0> thumper: the client side work is a piece of cake but you then end up with a controller login
<menn0> thumper: the alternative is to do something tricky in the client so it logs into the controller model
<thumper> Add the new facade
 * thumper thinks
<menn0> I don't think that would be hard to do and it's possibly lower risk
<thumper> you need to have admin access in the controller model to do it
 * menn0 checks what the current story with access is
<thumper> or perhaps just write?
<menn0> you currently need superuser access
<menn0> that probably makes sense TBH
<menn0> thumper: i don't *think* a new facade is required, the existing HighAvailability facade just needs to be exposed for controller logins and (controller) model logins (for compatibility)
<thumper> ok
<menn0> thumper: the client will have to try both approaches (in case a newer client is talking to an older server)
 * thumper nods
<menn0> thumper: with a preference for a controller login
<menn0> ok
<menn0> thumper: thanks rubber duck :)
<veebers> thumper: you recently worked on some unit tests wrt to certificates/keys etc? Fyi I just filed this bug which may be of interest: https://bugs.launchpad.net/juju/+bug/1632127
<mup> Bug #1632127: Unittest "MachineSuite.TestCertificateDNSUpdatedInvalidPrivateKey" fails on multiple archs <ci> <unit-tests> <juju:Confirmed> <https://launchpad.net/bugs/1632127>
<thumper> veebers: as much as I love fixing bugs, I'm focusing on a different piece of work this week
<veebers> thumper: ack, only wanted to bring it to your attention incase my assumption of you working in that area was correct :-)
<menn0> veebers: TestCertificateDNSUpdatedInvalidPrivateKey was added by axw yesterday
<menn0> it just failed for me on a merge attempt tooo
<veebers> menn0: ack thanks, oh he's now on leave right?
<menn0> veebers: ah yes... could be
<menn0> veebers: assign it to me... I was working with him in that area yesterday
<anastasiamac> menn0: veebers: Roger filed a bug for the failure.. i've marked Chris's as a duplicate...
<menn0> anastasiamac: ok cool. bug number?
<veebers> menn0: anastasiamac points out I filed a dupe bug
<anastasiamac> menn0: veebers: https://bugs.launchpad.net/bugs/1631990
<mup> Bug #1631990: cmd/jujud/agent: sporadic test failure in MachineSuite.TestCertificateDNSUpdatedInvalidPrivateKey <ci> <unit-tests> <juju:Triaged> <https://launchpad.net/bugs/1631990>
<veebers> ah, beat me to it :-)
<menn0> veebers: yep got that... I was after the original ticket
<menn0> anastasiamac, veebers: thanks
<jamespage> hey menn0 - I see you picked up bug 1541482
<mup> Bug #1541482: unable to download local: charm due to hash mismatch in multi-model deployment <2.0-count> <juju-release-support> <uosci> <juju:Fix Released by menno.smits> <juju-core:Triaged by menno.smits> <https://launchpad.net/bugs/1541482>
<menn0> jamespage: yep. I fixed the same/similar issue for 2.0 so I seemed like the right person to deal with it for 1.25.
<jamespage> menn0, awesome
<jamespage> menn0, its killing our final release testing atm
<jamespage> menn0, any ideas what triggers the race that causes it?
<jamespage> we don't see it all of the time
<menn0> jamespage: IIRC it's to do with the apiserver's on disk charm cache
<menn0> jamespage: the solution is to remove the cache. it's not necessary any more (already done for 2.0)
<jamespage> menn0, yeah - I saw the fix you did for 2.0
<menn0> jamespage: there might be some other related fixes in that area too. I need to review previous PRs.
<jamespage> menn0, ok - I'll leave it with you
<menn0> jamespage: actually, I just checked and I've already done that
<menn0> jamespage: those changes just haven't been released yet
<menn0> jamespage: we need a 1.25.7 release
<jamespage> yp!
<menn0> jamespage: if it would help I can supply binaries for you to use in the mean time (to validate that the problem is indeed fixed for you)
<thumper> jamespage: how urgent is the need for 1.25.7 for your team?
<jamespage> thumper, we release thursday; currently we're doing final sweep of amulet testing tidy including disabling old release combos and enabling new ones
<jamespage> thumper, but that final sweep has been going for alot longer than normal as we can't get a full amulet run through consistently atm
 * thumper nods
<jamespage> thumper, we tried a workaround, but we're having to unpick that now
<jamespage> thumper, as that did something quite different to what we expected
<jamespage> thumper, so fairly urgent - I'll just see if we can splice in custom binaries to our Charm CI or not
<thumper> jamespage: well, there is no way it is happening this week :(
<thumper> so, sorry
<jamespage> that was my guess
<alexisb> menn0, ping
<anastasiamac> jamespage: plz email alexisb and rick_h_re:urgency for 1.25.7 to come out \o/
<mwhudson> alexisb: well i started on that bug and now i'm reading the kernel source :)
<thumper> menn0: are you working on https://launchpad.net/bugs/1631990 now?
<mup> Bug #1631990: cmd/jujud/agent: sporadic test failure in MachineSuite.TestCertificateDNSUpdatedInvalidPrivateKey <ci> <unit-tests> <juju:Triaged by menno.smits> <https://launchpad.net/bugs/1631990>
<menn0> thumper: I haven't gotten to it yet
<menn0> thumper: did you want to pick it up, or were you just bitten by it?
#juju-dev 2016-10-11
<alexisb> thumper, can you please do the technical reviews on wallyworlds list-clouds and list-regions PRs
<thumper> sure
<menn0> wallyworld or thumper: here's the "juju enable-ha" improvement: https://github.com/juju/juju/pull/6421
<wallyworld> thumper: thanks for review, i've updated list-regions pr
<wallyworld> looking
<wallyworld> menn0: done. i also just wanted to make sure you tested using an rc3 client to check the model facade call works still
<menn0> wallyworld: good call, will do
<menn0> wallyworld: rc3 client works as expected although there are some errors due to unqualified user names: http://paste.ubuntu.com/23305923/
<menn0> wallyworld: is this ok?
<wallyworld> menn0: those errors should not come up in practice for other clients. the main takeaway is tha the enable ha call worked. the error is because the 2.0 server returned a user without @local which rc3 client did not understand
<menn0> wallyworld: yep that's what I figured. just wanted to check with you that that's expected. all good. merging now.
<menn0> (other issues fixed)
<wallyworld> awesome ty
<wallyworld> menn0: if you are stuck for work, i have a small improvement i can pass on. not sure what you have in your plate
<menn0> wallyworld: I was going to tackle that not-so-intermittent cert update test failure which was introduced yesterday
<wallyworld> ok
<menn0> wallyworld: but what else do you have?
<anastasiamac> wallyworld: menn0: thumper: I feel like bug 1631438 is not for point release (2.0.1) but for minor (2.1.0) since it inverts current behavior? agree?
<mup> Bug #1631438: Switch default behaviour of 'gui' w.r.t. browser <juju:New> <https://launchpad.net/bugs/1631438>
<anastasiamac>  i need for executive 2nd opinion :D
<wallyworld> menn0: easier via quick hangout to give context
<wallyworld> standup?
<menn0> wallyworld: see you there
<menn0> anastasiamac: yep, that seems like more of a 2.1 thing
<anastasiamac> menn0: u r awesome \o/
<thumper> menn0, anastasiamac: hmm... I'm not so certain, given that it is a very 2.0 thing
<thumper> personally it isn't something that I've had to deal with yet
<thumper> and I feel that most others won't have hit it either
<thumper> however it is very late in the day to be changing this
<anastasiamac> thumper: i can propose a change if we are just talking about code.. dunno who it'll break.. considering 2.0.0 is freezing soon, there are only couple of hrs eft really..
<thumper> my preference is to change nothing
<anastasiamac> left*
<thumper> I think I talked myself out of saying we should change now
<anastasiamac> thumper: mine too... hence 2.1.0
<anastasiamac> :D
<anastasiamac> thumper: since I have u... isn't this a dupe of the one u did last week? https://bugs.launchpad.net/juju/+bug/1576342
<mup> Bug #1576342: `juju status` should show leadership primitives <juju-release-support> <leadership> <status> <tech-debt> <juju:Triaged> <https://launchpad.net/bugs/1576342>
<thumper> yes, fixed the bug
<anastasiamac> thumper: \o/
<mup> Bug #1632159 opened: juju 1.25.6 HA - Causing unit state issues. <canonical-bootstack> <juju:Incomplete> <juju-core:Incomplete> <juju-core 1.25:Incomplete> <https://launchpad.net/bugs/1632159>
<wallyworld> thumper: list-regions PR? you happy with it now?
<thumper> getting there, just dropped kiddo down to BJJ
<anastasiamac> wallyworld: PTAL - https://github.com/juju/juju/pull/6422 thank you for the wording
<wallyworld> give me a minute
 * anastasiamac is very giving today 
<anastasiamac> :D
<wallyworld> thumper: i added details to regions for yaml and json because in general with juju, yaml and json can provide extras not ordinarily excluded from list. i realise show-cloud can be used, but my thinking was 1. that gives more than just region details, 2. I wanted to not have 2 commands to present region info (it's more natural for the user to flip output format for the regions command). I originally didn't have yaml or json because
<wallyworld> just having []string for those didn't seem to pay its way so adding those formats I wanted to make it worth it
<wallyworld> s/excluded/included
<thumper> ok\
<wallyworld> anastasiamac: not sure if you might have time to review the table header changes? https://github.com/juju/juju/pull/6423
<anastasiamac> wallyworld: looking
<wallyworld> ta
<anastasiamac> nps. let me know when u want a stamp :)
<wallyworld> anastasiamac: thanks for picking up those things i missed, i think it's good now?
<anastasiamac> looking
<wallyworld> anastasiamac: gotta run to do school pickup, if it's ok, can you $$merge$$ for me?
<wallyworld> bbiab
<anastasiamac> wallyworld: run coz I think "paylod class" needs to b changed too...
<anastasiamac> k \o/
<anastasiamac> i'll finish another review by the time u r back
<wallyworld> anastasiamac: ty, fixed that last one
<anastasiamac> wallyworld: \o/
<hoenir> it's the same rule that one two +1 = $$merge$$ no?
<babbageclunk> Hey, thumper, can I bug you for a quick opinion?
<thumper> sure
 * thumper is online to talk to uros
<hoenir> if that rule is the same, could someone take a look on my latest PR? https://github.com/juju/juju/pull/6414
<babbageclunk> Ah ok - no worries if you're busy
<thumper> babbageclunk: not busy, he isn't around
<babbageclunk> consolation chat!
<hoenir> let me know if I have something to improve even further or it's safe to merge.
<thumper> hoenir: to be honest, we probably won't be looking at things for a few days as we prep for 2.0 GA
<thumper> hoenir: we won't merge it until after GA
<thumper> bug fixes only just now
<thumper> GA is Thursday, US time
<thumper> so soon
<hoenir> thedac, GA? what is GA? could you explain a little more please? and thanks for the response
<babbageclunk> thumper: So I've worked out the remaining goroutine leaks - they're the read- and write-loops of http transports created by the lxd provider.
<hoenir> GA stands for?
<thumper> general availability
<thumper> so the full 2.0 release
<thumper> no more release candidates
<dimitern> it's finally come to that..
<hoenir> thumper, thanks for clearing this up, I should ping here before the GA release.
<thumper> o/ dimitern
<thumper> probably just after :)
<dimitern> hey thumper :)
<hoenir> thumper, thanks a lot !
<babbageclunk> thumper: If I disable keepalives they go away, but that's probably bad.
<thumper> hmm
<thumper> how long is the keep alive set for?
<thumper> can we just tweak that?
<babbageclunk> No, they don't timeout
<thumper> and is it just lxd that we leak?
<thumper> oh
<babbageclunk> Probably not
<thumper> at all?
<babbageclunk> Actually, I'll try putting a timeout in to see what happens. Would be easier than adding a .Close to EnvironProvider.
<thumper> where does the http transport live?
<babbageclunk> inside the lxd client, which is in the lxd provider.
<thumper> and does the code create a new transport if the old has gone?
<babbageclunk> I think when a new model added is we create a new client/new transport.
 * babbageclunk yoda'd
<thumper> heh
<babbageclunk> The docs say we should keep one http.Client to avoid leaking - I think the problem is that we create a new one for each lxd client.
<thumper> hmm...
<babbageclunk> So I could contrive to pass it in somehoe
<babbageclunk> uh, somehow
<thumper> that sounds like a good idea
<babbageclunk> The other option would be to call .CloseIdleConnections which will stop the loops - but that probably requires EnvironProvider.Close.
<babbageclunk> I'll try injecting the http client in from further up.
<babbageclunk> ok thanks!
<thumper> np
<babbageclunk> had fun mangling the stdlib to work out where the leak was coming from.
<babbageclunk> hey dimitern, how was Florence?!
<dimitern> babbageclunk: awesome! :) and all of Tuscany in general - better than Rome for sure hehe
<babbageclunk> menn0: So it turned out that the leaked connections are from the lxd clients inside environs.
<babbageclunk> menn0: also, hi!
<menn0> babbageclunk: hi :)
<menn0> babbageclunk: ok, interesting. easily fixable?
<babbageclunk> menn0: Well, setting IdleConnTimeout in the transport when creating the http client works. I'm going to talk to the lxd people to see if they mind doing that. Do you think there'd be a downside to just doing that all the time?
<babbageclunk> menn0: Or should it be configurable?
<menn0> babbageclunk: depends if we expect to have connections not doing anything for a long time I guess
<menn0> babbageclunk: configurable might be best
<babbageclunk> menn0: it'll reopen them if they timeout and there's another request - it's purely a performance thing.
<menn0> babbageclunk: ok, in that case, I don't see any downside, especially if the timeout is a few minutes or something
<babbageclunk> menn0: Cool - I'll discuss it with Stephane and see if he has any objections.
<babbageclunk> menn0: Thanks!
<menn0> babbageclunk: np. nice work figuring it out.
<menn0> babbageclunk: reivew please: https://github.com/juju/juju/pull/6425
<menn0> review even
<menn0> I'm off for now
<menn0> so no rush
 * menn0 is hoping he can still land this before the release branch
<babbageclunk> menn0: I can look at it quickly now if you want?
<menn0> babbageclunk: no need. I still need to QA the changes with MAAS.
<babbageclunk> menn0: ah, ok
<menn0> babbageclunk: just updated the description to show what the new output looks like
<menn0> the change is the last line
 * menn0 is exhausted
<menn0> babbageclunk: good night
<babbageclunk> menn0: night!
<mgz> jamespage: bug 1580418
<mup> Bug #1580418: Cached local charms should be deleted when their service is removed <juju:Fix Released by wallyworld> <juju-core:Fix Released by wallyworld> <juju-core 1.25:Fix Released by wallyworld> <https://launchpad.net/bugs/1580418>
<jamespage> mgz, ta
<rick_h_> dimitern: running a sec late
<rick_h_> dimitern: omw
<rick_h_> dimitern: ok, ping if you're around later
<dimitern> rick_h_: oops sorry - omw
<voidspace> dooferlad: ping
<dooferlad> voidspace: pong
<dooferlad> voidspace: sorry, I have been in meetings for the last 24 hours
<voidspace> dooferlad: ouch :-)
<dooferlad> voidspace: I guess you want me to look at some code!
<dooferlad> voidspace: it's fine
<voidspace> dooferlad: please, it's already merged - but it should have had a follow up review
<dooferlad> voidspace: in one now
<voidspace> dooferlad: https://github.com/juju/juju/pull/6419
<voidspace> dooferlad:  if there are changes needed I'll do a follow up
<voidspace> dooferlad: when you have time - thank you very much!
<dooferlad> voidspace: thanks, will do
<voidspace> mgz: ping
<rick_h_> dooferlad: did you get anywhere with the test failures in your PR https://github.com/juju/juju/pull/6406 ?
<dimitern> dooferlad, voidspace, frobware, babbageclunk: I'd appreciate a review on https://github.com/juju/juju/pull/6426 guys - needed to fix bug 1616098
<mup> Bug #1616098: Juju 2.0 uses random IP for 'PUBLIC-ADDRESS' with MAAS 2.0 <4010> <cpec> <juju:In Progress by dimitern> <https://launchpad.net/bugs/1616098>
<perrito666> morning
<voidspace> perrito666: o/
 * dimitern steps out for <1h
 * voidspace launches 30 containers and goes for coffee
<voidspace> mgz: ping again if you're around
<anastasiamac> perrito666: hi :D
<mgz> voidspace: yo
<rogpeppe1> natefinch: i replied to https://github.com/juju/utils/pull/245 if you wanna take another look
<lazyPower> well, it was really contrived and took some effort on my part, but i found a way to break the juju controller in a way that i dont know how to recover from
<lazyPower> so, let me check some assertions first before i label it as such -   does the juju controller probe interfaces on the host and add those to the controller config? specifically the juju-db service?
<natefinch> rogpeppe1: cool
<dooferlad> voidspace: have looked at your branch - no issues from me
<dooferlad> rick_h_: there was a Windows build failure. I didn't have a Windows VM around so I have been setting that up. It should be a case of providing a stub for Windows, but I don't want to code by throwing changes at the merge bot!
<rick_h_> dooferlad: ok, just making sure we're on top of it since today is the last day to get code in for GA
<rick_h_> dooferlad: so want to make sure we can get this change in
<mgz> you can also just ssh into the ci windows machine and build stuff there
<dooferlad> rick_h_: we will be lucky to.
<dooferlad> mgz: details?
<mgz> dooferlad: in the cloud-city branch
<dooferlad> mgz: ohh, thanks!
<voidspace> dooferlad: cool, thanks
<rick_h_> katco: dimitern frobware dooferlad ping for standup
<rick_h_> mgz: ^
<mgz> rick_h_: I'm deep in rebuilding packages so don't want to reboot for ho
<mgz> can give update here
<dimitern> omw
<babbageclunk> alexisb: ping?
<dimitern> frobware: interestingly port 49151 is indeed unreachable on windows2012r2, but wininit.exe does listen on 49152-49159
<frobware> dimitern: heh
<voidspace> macgreagoir: we have a parent-teacher thing, but when I'm back from that I'll grab a review
<macgreagoir> voidspace: Cheers!
<rick_h_> mgz: anything you can do to help voidspace and his vsphere connection woes is <3 so we can validate this isn't a bigger 2.0 issue.
<rick_h_> mgz: understand on the packaging and thanks for pushing through on that
<rick_h_> dimitern: heads up this is showing conflicts now: https://github.com/juju/juju/pull/6426
<rick_h_> dimitern: oh, nvm...read that too fast heh
<rogpeppe1> here's a PR that fixes this critical juju bug: https://bugs.launchpad.net/juju/+bug/1631449. a review would be much appreciated please: https://github.com/juju/juju/pull/6427
<mup> Bug #1631449: External users don't have access to models unless everyone@external is granted <juju:Incomplete> <https://launchpad.net/bugs/1631449>
<mgz> rick_h_: gotcha, thanks
<alexisb> perrito666, can you review rogpeppe1 PR
<perrito666> sure I can, havent seen his message sorry
<katco> macgreagoir: +1 on your pr
<macgreagoir> katco: woo hoo! :-)
<macgreagoir> Cheers
<katco> macgreagoir: great job sticking with it! i think we got it to a much better place
<macgreagoir> Thanks for the help.
<rogpeppe1> perrito666: thanks
<rogpeppe1> katco: would you be able to take another look at https://github.com/juju/juju/pull/6407 ? i've addressed the most significant issues but i haven't added fixes for unrelated tests.
<katco> rogpeppe1: yes sure... will be just a bit but i will get to it today
<katco> rogpeppe1: sorry for the wait
<rogpeppe1> katco: thanks
<katco> rogpeppe1: and thanks for the discussion
<rogpeppe1> katco: np
<perrito666> rogpeppe1: I am not completely sold on the allow model access thing, what kind of access are you allowing?
<rogpeppe1> perrito666: if you've been granted permission to a model, you get access to that model
<rogpeppe1> perrito666: even if you haven't got any access rights to the controller itself
<dooferlad> voidspace / dimitern / frobware / babbageclunk: https://github.com/juju/juju/pull/6406 has been updated to exclude LXD code on Windows. The rest of the code already has a +1.
<babbageclunk> dooferlad: looking
<dooferlad> rick_h_: maybe that container fix will land after all... ^^
<rick_h_> dooferlad: oooh
<dimitern> rick_h_: it seems I won't manage to fix the public address bug tonight - as discussed with frobware, more detailed and specific connection checker tests are needed
<dimitern> rick_h_: but that should be OK, right? I mean even once in GA mode we can still fix bugs?
<rick_h_> dimitern: ok, yes we can fix bugs but we're not going to be doing weekly releases now
<rick_h_> dimitern: so it'll be some time before a follow up and such
<rick_h_> dimitern: that's why my push to get things in so that users don't feel stuck as much as possible, but we do what we can
<rick_h_> dimitern: right fix > quick fix
<dimitern> rick_h_: +100
<dimitern> :)
<dimitern> ok then, cheers rick_h_
 * rick_h_ goes to get lunch
<redir_afk> new kernel bbiam
<frobware> voidspace: what was the other 'thing' you changed for LXD? I have the examples from bug #1631038 but have since close the HO chat...
<mup> Bug #1631038: Need /etc/sysctl.d/10-juju.conf <juju-release-tools:Triaged> <https://launchpad.net/bugs/1631038>
<voidspace> frobware: fs.max-files
<frobware> voidspace: ty
<frobware> voidspace: any particular value that will take me over 18 containers?
<voidspace> frobware: I read the current value and then added a zero... :-)
<frobware> voidspace: should this be: fs.file-max ?
<voidspace> frobware: uhm, yes. Sorry. Dodgy memory.
<frobware> voidspace: and what's the setting that requires a reboot?
<rogpeppe1> alexisb: thanks for hitting $$merge$$ again for me! :)
<rogpeppe1> alexisb: i filed another "sporadic test failure" bug...
<alexisb> rogpeppe1, np, I will keeo an eye on it
<alexisb> rogpeppe1, yeah we have a bug for that one
<alexisb> but thank you
<rogpeppe1> alexisb: it's just landed, yay!
<alexisb> thank you rogpeppe1!
<rick_h_> voidspace: do you have a link to the bug you created for tracking the packaging?
<rogpeppe1> alexisb: ah, i didn't find the relevant bug, perhaps i should mark mine as dupe
<alexisb> yeah when a have a moment, I wil go cross check with our list
<voidspace> rick_h_: I'll do it now...
<frobware> rick_h_: https://bugs.launchpad.net/juju-release-tools/+bug/1631038
<mup> Bug #1631038: Need /etc/sysctl.d/10-juju.conf <juju-release-tools:Triaged> <https://launchpad.net/bugs/1631038>
<frobware> voidspace: is that it?
<rick_h_> voidspace: frobware ty
<voidspace> frobware: plus sysctl -p
<voidspace> frobware: and you need the max_user_instances set higher than the default 128 too
<voidspace> frobware: but that *should* work (worked for me)
<rogpeppe1> alexisb: p'raps we should have a standard format for the subject line of sporadic test failures
<rogpeppe1> alexisb: my bug is here FWIW https://bugs.launchpad.net/juju/+bug/1632412
<mup> Bug #1632412: apiserver: sporadic test failure in pingerSuite.TestAgentConnectionDelaysShutdownWithPing <juju:New> <https://launchpad.net/bugs/1632412>
<frobware> rogpeppe1: can we not dot it with tags?
<voidspace> rick_h_: https://bugs.launchpad.net/juju/+bug/1632414
<mup> Bug #1632414: Package sysctl.conf file with juju <juju:New> <https://launchpad.net/bugs/1632414>
<rogpeppe1> frobware: i guess so, although a standard form for the subject line would help too
<rogpeppe1> frobware: tbh i didn't even know that bugs could have a set of tags
<frobware> rogpeppe1: I add network often enough... :)
<voidspace> rick_h_: there is morre we could do on bug 1602192
<mup> Bug #1602192: when starting many LXD containers, they start failing to boot with "Too many open files" <lxd> <juju:In Progress by rharding> <lxd (Ubuntu):Confirmed> <https://launchpad.net/bugs/1602192>
<rogpeppe1> frobware: where do you see the tags in the bugs list?
<rick_h_> voidspace: right, just want to get what we're doing for 2.0 to move it forward
<voidspace> rick_h_: but should we mark is fix committed now we have the warning in and a separate bug for the system settings
<rick_h_> voidspace: and we'll look at next steps from there
<rogpeppe1> frobware: ah, i see on the lower right
<rick_h_> voidspace: hmm, not sure I want to mark it fix-committed
<voidspace> rick_h_: kk
<frobware> rick_h_, voidspace: given that I've been asking for your additional settings can we update the bug to contain more than the two entries currenty listed?
<rogpeppe1> frobware: there are enough "intermittent-failure" bugs that it's still hard to find the one that applies
<rick_h_> frobware: voidspace please update the bug to have all the info needed so they can move that
<rogpeppe1> frobware: i think the subject should at least mention the failing package and test.
<frobware> rogpeppe1: rather like a (good) commit message... :)
<rogpeppe1> frobware: exactly
<rogpeppe1> frobware: anyway, now that you've brought it to my attention, i've added the intermittent-failure tag
<rogpeppe1> frobware: thanks for that
<frobware> rick_h_: I want to punt the completion of that to voidspace as I'm cargo-culting... but I think voidspace and jam have the absolute things to change...
<rick_h_> frobware: k
<frobware> rick_h_: happy to validate because ATM, conjure-up openstack localhost fails for me.
 * frobware EOD
<voidspace> macgreagoir: looking at #6424
<voidspace> macgreagoir: actually, #6424 has existing review comments that need addressing
<voidspace> macgreagoir: looking at 6393
<voidspace> macgreagoir: I would go with rick_h_ for that PR :-)
<voidspace> macgreagoir: that one LGTM
<voidspace> rick_h_: there's a chance macgreagoir is EOD
<voidspace> rick_h_: shall I set "Change the grant admmodel permission to add-model" to merge?
<voidspace> rick_h_: https://github.com/juju/juju/pull/6393
<rick_h_> voidspace: looking, did it get the fallback going?
<voidspace> rick_h_: yep, there's a test for it too
<rick_h_> voidspace: then yes please
<dooferlad> voidspace / rick_h_: https://github.com/juju/juju/pull/6406 -- last change was a comment fix so it should be good to go.
<voidspace> dooferlad: looking
<voidspace> then EOD
<rick_h_> voidspace: ty
<rick_h_> dooferlad: ty as well
<dooferlad> voidspace: thanks. EOD + 1h for me. This is back after the toddler has been put to bed for a last minute code poke.
<voidspace> rick_h_: you got there before I could review it
<voidspace> g'night all
<stokachu> frobware: whats the conjure-up failure from?
<stokachu> ive run it on yakkety and the charms fail because of ipv6 i think
<rick_h_> stokachu: is the ipv6 issue noted? I thought we had the ipv6 issues licked where you'd get an ipv6 default address?
<stokachu> rick_h_: so with a fresh yakkety and juju rc3 a default deploy assigns ipv6 to applications
<stokachu> rick_h_: i think juju does right by giving it ipv6 but the charms fail?
<stokachu> haven't had a chance to dig further into it
<rick_h_> stokachu: so ipv4 should be preferred, but if it has ipv6 then I guess that's ok except juju doesn't support ipv6 yet
<stokachu> rick_h_: i can boot another yakkety system here and run it again
<stokachu> rick_h_: will juju status show ipv6 even if it uses ipv4?
<rick_h_> stokachu: so it should pick ipv4 if it's there as the preferred and not show ipv6. It shouldn't show ipv6 if it's using v4
 * rick_h_ is trying to find the bug/pr we fixed along those lines recently
<stokachu> rick_h_: ok
<stokachu> rick_h_: also would that be in rc3
<rick_h_> stokachu: https://bugs.launchpad.net/juju/+bug/1624495 yes, would be in there.
<mup> Bug #1624495: operations fails on rackspace because of ipv6 address in dns-name <rackspace> <status> <juju:Fix Committed by mfoord> <juju-ci-tools:Won't Fix> <https://launchpad.net/bugs/1624495>
<stokachu> rick_h_: ohhh
<stokachu> so i should setup xenial to configure both ipv4 and ipv6 with lxd init
<stokachu> so ill do that in another test in addition to a default yakkety run
<rick_h_> stokachu: maybe? I don't want to have issues with juju 2.0 not working with conjure-up I guess
<rick_h_> stokachu: that's why I ask if you've filed a bug or something or what's up here. That's news to me that it's not working for you.
<rick_h_> stokachu: and we're in lock down for GA at EOD today so kind of :/
<stokachu> rick_h_: i just ran it this morning
<rick_h_> stokachu: ah ok
<rick_h_> stokachu: yea, please let me know if there was any ipv4 or if it was pure ipv6 and what versions of the stack we're looking at there. lxd tests on yakkety pass CI so curious if things aren't there for you as there's no father bug fixes in that area on the plate for tomorrow's freeze
<stokachu> rick_h_: ok im testing right now for yakkety and xenial
<rick_h_> stokachu: ty
<stokachu> rick_h_: normally on xenial i dont choose to configure ipv6
<stokachu> ill enable it though and see what happens
<rick_h_> stokachu: ah ok, so it's working sans ipv6 then?
<stokachu> yea that works
<rick_h_> stokachu: ah ok, I feel better then
<stokachu> but yakkety may be different i want to verify
<rick_h_> stokachu: and it should work ok with ipv6 and ipv4 enabled, but pure v6 won't work
<rick_h_> stokachu: k, ty
<natefinch> sinzui: I'm not really sure what the correct behavior for this bug is: https://bugs.launchpad.net/juju/+bug/1613858
<mup> Bug #1613858: generate-tools ignores 2.x agents and creates bad path <jujuqa> <metadata> <regression> <rteam> <simplestreams> <streams> <juju:In Progress by natefinch> <https://launchpad.net/bugs/1613858>
<sinzui> natefinch: we have a lot of Wiggle room. the goal is to allow a private cloud to assemple strreams to boostrap with. 1. stop ignoring agents with major version greater than 1, then make a sensible dir structure as I show 1.x generates
<natefinch> sinzui: I don't know what you mean when you say the tool ignores them.  as I said, I don't know what the intended behavior is
<sinzui> natefinch:  with juju 1, run "juju metadata generate-tools -d mystreams --stream devel". it makes streams with the agents in the dir. it also makes the correct dir tree.
<sinzui> natefinch: doing that with juju 2, it should also recognise the 1x and 2x agents.
<natefinch> sinzui: I don't know what the correct directory tree should look like.  Is it just a v2 where we have a v1 now?
<sinzui> natefinch: I think juju 1x should also recognise 2.x agents, but that is not the critical path to enbling a provate cloud
<sinzui> natefinch: The rules have not changed in 3 years!
<sinzui> natefinch: 2,x's output sould match 1x's output
<sinzui> natefinch: both jujus should include 2.x agents, but for today, we just need to get 2.x to make streams
<natefinch> sinzui: I really don't know how to say in any other ways that I don't know what the expected output should be.  When I run juju emtadata-tools, I get this: http://pastebin.ubuntu.com/23309748/
<natefinch> sinzui: what part of that is wrong, and what should it look like instead?
<natefinch> sinzui: assume I have no clue what streams stuff is supposed to look like... because I don't.
<sinzui> natefinch: that is why I keep suggesting to run with 1.x we used juju  1.15.0 to 1.25.2 to make streams with that command. the output works for juju. Juju 2 needs to make the same tree. Juju 1 and 2 need to bootstrap with that tree
<natefinch> ok
<sinzui> natefinch: that paste lookss right, BTW, except there are not agents shown in devel.
<natefinch> sinzui: I didn't have anything to put there, so I figured I'd just run it to see what it generates without anything
<sinzui> :), I would ahve done the same
<menn0> stupid irc client, not reconnecting lately
<menn0> thumper: so are we still able to land things?
 * thumper shrugs
<thumper> what thing?
<menn0> thumper: for the release? no branch cut yet as far as I can tell.
<thumper> menn0: what is the mongo update syntax for updating values inside a slice?
<thumper> alexisb: when are we freezing landings?
<thumper> menn0: and what are you wanting to land?
<thumper> if it is just a bug fix, I'd say that'd be fine
<menn0> thumper: a specific element in an array you mean?
<thumper> but we want to reduce churn on change
<thumper> menn0: relations, there is a slice of structures called "endpoints", inside those values is "service-id"
<thumper> I want to remove that one and add "application-id"
<alexisb> thumper, not until eod
<alexisb> for US
<menn0> thumper: the fix for this: https://bugs.launchpad.net/juju/+bug/1629194
<mup> Bug #1629194: Don't warn for default architecture on bootstrap <juju:In Progress by menno.smits> <https://launchpad.net/bugs/1629194>
<menn0> thumper: https://github.com/juju/juju/pull/6425
<menn0> thumper: actually, I'd appreciate your thoughts on the direction wallyworld and I took with this. See the example output in the PR description.
 * rick_h_ goes to get the boy from school
<alexisb> hml, ping
<hml> alexisb: pong
<alexisb> heya hml, happy tuesday :)
<alexisb> just checking in on how things are going
<thumper> menn0: looks good to me
<hml> alexib: going okay - i have the pr for the api version stuff out - resolved the git issues - hope to have the other code available tonight - but still no interwebs at home.
<menn0> thumper: one reservation I had is that we went with a syntax that's almost the same as the constraints format but is missing "arch=" for the architecture
<menn0> thumper: regarding your mongodb question, you can't do it in one operation
<menn0> thumper: http://stackoverflow.com/questions/4669178/how-to-update-multiple-array-elements-in-mongodb
<alexisb> ok hml do you need a review for the api version stuff?
<menn0> thumper: either update them one at a time, or replace the entire field/doc
<thumper> I know how many values there are
<thumper> I can explicitly identify them
<thumper> ah...
<menn0> thumper: well there is this to address a particular field: https://docs.mongodb.com/manual/reference/operator/update/position/#up._S_position
<hml> alexisb: axw did an initial review for the goose piece, i resolved his comment.  both PRs need review - details in email sent recently
<thumper> so...
<thumper> endpoint.0.service-id
<menn0> thumper: but that doesn't exist in 2.4
<alexisb> hml, ack
<menn0> thumper: actually ignore $position, not what you want anyway
<alexisb> aah hml I see the PR from an hour ago now, awesome!
<menn0> thumper: I would read out the field as a bson.D, update it in memory and then replace it
<redir> another PR https://github.com/juju/juju/pull/6429 pretty similar to https://github.com/juju/juju/pull/6399 which bring htem up to speed with https://goo.gl/yqrrPI
<redir> PTAL
<perrito666> redir: looking
<redir> perrito666: gracias
<perrito666> redir: you do not fool around when it comes to QA steps
<alexisb> redir, sorry I am stuck in a call
<redir> alexisb: np.
<redir> perrito666: :) Since there's 3 commands getting collapsed into one I wanted to make sure they actually work:|
 * katco falls back to a different computer
<alexisb> wallyworld, when you are online can you please take a look at this bug: https://bugs.launchpad.net/juju/+bug/1631529
<mup> Bug #1631529: "upload-tools strikes back" cannot upgrade with streams <jujuqa> <simplestreams> <upgrade-juju> <juju:Triaged by alexis-bruemmer> <https://launchpad.net/bugs/1631529>
<perrito666> redir: just being curious here, what is a not valid utf-8 string?
<redir> perrito666: preexisting code
<perrito666> redir: I am curious because I thought any thing was a valid utf string :p
<redir> perrito666: not so or there wouldn't be a need for the replacement character
<redir> ï¿½
<perrito666> redir: reviewed
<perrito666> redir: as I understand, there might not be a symbol but utf pretty much has a place for everything
<redir> perrito666: my best WAG is it is supposed to validate that one hasn't input non printables in the command line ^I etc..
<perrito666> redir: its possible, it was just a curiosity
<redir> perrito666: https://blog.golang.org/strings
<redir> perrito666: indeed
<perrito666> redir: well I had only a few comments
<perrito666> redir: great work
<redir> perrito666: you look at this too? https://github.com/juju/juju/pull/6399
<redir> same basic concept, different snowflake
<perrito666> redir: sure, have a call with alexisb now but ill read it after
<redir> k
<redir> perrito666: mg (spanish for ta?)
<perrito666> redir: mg?
<redir> perrito666: muchas gracias
<wallyworld> alexisb: updated bug - i claim it is invalid. juju is working as designed
<redir> wallyworld: I claim that all the time
<alexisb> wallyworld, thanks
<alexisb> wallyworld, hml also has some PRs up
<menn0> rick_h_: ping?
<rick_h_> menn0: pong
<menn0> rick_h_: i was about to merge this but wanted to run it past you: https://github.com/juju/juju/pull/6425
<menn0> rick_h_: see the output in the PR description
<menn0> rick_h_: I decided to go with the same format as the constraints syntax for consistency
<rick_h_> menn0: looking
<menn0> rick_h_: adding MAAS output for completeness
<rick_h_> menn0: cool LGTM
<menn0> rick_h_: ok thanks
<rick_h_> katco: ping, can you join the other #juju channel please to help out someone?
<katco> rick_h_: sure
<rick_h_> katco: they're asking if the lxd provider asks for zfs and I'm unsure of that behavior and have to run to take the boy to soccer atm
<rick_h_> katco: ty
<mup> Bug #1632477 opened: panic when deploying bundle when bundle relative path and bundle has relative paths and series is missing for an application <juju-core:New> <https://launchpad.net/bugs/1632477>
<mup> Bug #1632483 opened: juju says error: No selected controller. when there is a typo in the -m parameter it should say "could not find specified model" <juju-core:New> <https://launchpad.net/bugs/1632483>
<rick_h_> natefinch: ping
<mup> Bug #1632477 changed: panic when deploying bundle when bundle relative path and bundle has relative paths and series is missing for an application <juju-core:New> <https://launchpad.net/bugs/1632477>
<mup> Bug #1632483 changed: juju says error: No selected controller. when there is a typo in the -m parameter it should say "could not find specified model" <juju-core:New> <https://launchpad.net/bugs/1632483>
<mup> Bug #1632477 opened: panic when deploying bundle when bundle relative path and bundle has relative paths and series is missing for an application <juju-core:New> <https://launchpad.net/bugs/1632477>
<mup> Bug #1632483 opened: juju says error: No selected controller. when there is a typo in the -m parameter it should say "could not find specified model" <juju-core:New> <https://launchpad.net/bugs/1632483>
<katco> rogpeppe1: i don't suppose you're still on?
<rick_h_> wallyworld: can you forward me the email from natefinch  please?
<wallyworld> sure
<rick_h_> ty
<mup> Bug #1632477 changed: panic when deploying bundle when bundle relative path and bundle has relative paths and series is missing for an application <juju-core:New> <https://launchpad.net/bugs/1632477>
 * alexisb goes to pick up kid, biab
<alexisb> veebers, ping
<veebers> alexisb: hey o/
<alexisb> veebers, you joininh us?
<veebers> alexisb: sorry not today (unless it's really urgent) this standup time is a little awkward for me on Mon,Tues and Fri due to existing commitments
#juju-dev 2016-10-12
<anastasiamac> alexisb: m in 1:1
<alexisb> anastasiamac, yep I will be there soon
<anastasiamac> alexisb: k. thnx :)
<alexisb> thumper, you back yet?
<menn0> wallyworld or anastasiamac: https://github.com/juju/juju/pull/6431
<wallyworld> looking
<menn0> plz :)
<anastasiamac> menn0: lgtm
<menn0> anastasiamac: thanks
<wallyworld> menn0: question
<menn0> wallyworld: yes?
<wallyworld> in the test, we are passing the same clock value for clock and pingclock
<wallyworld> so how is that doing anything different?
<menn0> wallyworld: no we're not... in these tests, PingClock is the test's clock, Clock is WallClock
<menn0> wallyworld: sampleConfig() returns a config where Clock is WallClock
<wallyworld> menn0: oh right ok, i didn't realise sampleConfig() returned a clock
<menn0> wallyworld: np
<wallyworld> so it's ok to use wall clock
<wallyworld> we werenm'e before
<wallyworld> weren't
<menn0> wallyworld: it doesn't matter for these tests
<wallyworld> ok
<menn0> wallyworld: the injected clock is used for the code being tested
<menn0> wallyworld: the other goroutines can use wallclock and do what they like
<wallyworld> righto, sorry for dumb question
<menn0> wallyworld: not at all, glad you're being suspicious
<thumper> alexisb: am now
<menn0> wallyworld: you happy with these then?
<menn0> (that PR)
<wallyworld> menn0: yeah
<wallyworld> i figured you'd hit merge
<thumper> wallyworld: FYI capital case should be "Last Connection" not "Last connection"
<thumper> IMO
<wallyworld> thumper: it's FirstLetter case
<wallyworld> not CapitalCase
<thumper> fugly
<alexisb> "hi Jay, I missed you today, I love you"
<alexisb> "mommy I dont love you"
<menn0> alexisb: :(
<alexisb> "you dont, why?"
<alexisb> "because your working mommy"
<alexisb> :(
<menn0> alexisb: that sucks
<menn0> alexisb: last night I got "I hate you daddy, I hate you all and I want to live somewhere else"
<menn0> bed time related, not work related though
<anastasiamac> ouch :(
<alexisb> wow
<alexisb> did you tell her to count to 182?
<menn0> bed time is not usually that tough
<thumper> heh
<anastasiamac> usually, when kids tell their parents "I hate u" they really mean "I hate the power u have over me" ...
<wallyworld> huh, i get "i hate you ian" from everyone all the time
<perrito666> wallyworld: yup
<anastasiamac> u must be very powerful :)
<blahdeblah> I don't hate you, wallyworld; I just hate the fact that sensible-browser isn't, and you still use it anyway. :-P
<wallyworld> ah firefox vs chrome, vi vs emacs, star trek vs star wars
<wallyworld> i'd use firefox still if it weren't so slow and bloated
<blahdeblah> No, nothing that religious; I was just a bit bemused that they could call something sensible-browser that is so demonstrably not so.
<mup> Bug #1632530 opened: 1.25.6 cannot deploy yakkety lxc units on metal <uosci> <juju-core:New> <https://launchpad.net/bugs/1632530>
<mup> Bug #1632530 changed: 1.25.6 cannot deploy yakkety lxc units on metal <uosci> <juju-core:New> <https://launchpad.net/bugs/1632530>
<mup> Bug #1632530 opened: 1.25.6 cannot deploy yakkety lxc units on metal <uosci> <juju-core:Invalid> <juju-core 1.25:Invalid> <https://launchpad.net/bugs/1632530>
<mup> Bug #1632530 changed: 1.25.6 cannot deploy yakkety lxc units on metal <uosci> <juju-core:Invalid> <juju-core 1.25:Invalid> <https://launchpad.net/bugs/1632530>
<menn0> thumper: you might like this one: https://github.com/juju/juju/pull/6432
<veebers> menn0, anastasiamac: I believe you mentioned that Christian was working on the memory leak bug? I saw some fixes from him land so I re-ran the long run test and saw a big difference in the overall memory usage (I have commented on the bug 1625774).
<mup> Bug #1625774: memory leak after repeated model creation/destruction <ateam> <eda> <oil> <oil-2.0> <uosci> <juju:In Progress by 2-xtian> <https://launchpad.net/bugs/1625774>
<veebers> http://people.canonical.com/~leecj2/perfscalemem/ vs http://people.canonical.com/~leecj2/perfscalemem2/
<menn0> veebers: looking
<thumper> menn0: +1
<menn0> thumper: thanks
<anastasiamac> veebers: \o/ this is awesome!!!
<veebers> anastasiamac: it seems that there are a bunch less mongodb actions happening as well
<anastasiamac> veebers: yeah both memory and mongodb graphs look neater \o/
<anastasiamac> veebers: m sure alexisb would love this too :D
<alexisb> yay!
<alexisb> wallyworld, it works
<alexisb> thanks
<alexisb> I will report to aaron in the morning
<wallyworld> awesome
<wallyworld> thanks for testing
<wallyworld> i knew it would work :-D
<alexisb> wallyworld, well really you tested I pressed buttons
<alexisb> but you are welcome :)
<alexisb> with that all I am out
<wallyworld> alexisb: you definitely saw a message saying an update occurred?
<alexisb> yep
<wallyworld> good :-)
<alexisb> wallyworld, ding
<alexisb> ding
<alexisb> ding
<alexisb> ;)
<wallyworld> dong
<veebers> thumper, wallyworld: huh, this isn't good: http://reports.vapour.ws/releases/4474/job/run-unit-tests-xenial-s390x/attempt/667
<veebers> ../../../golang.org/x/sys/unix/flock.go:18: undefined: Flock_t et. al about 5 undefined in total
<natefinch> ruh roh
<thumper> :(
<veebers> filing a bug now
<thumper> just in time for GA
<thumper> wonderful
<mup> Bug #1632483 changed: juju says error: No selected controller. when there is a typo in the -m parameter it should say "could not find specified model" <usability> <juju:Triaged> <https://launchpad.net/bugs/1632483>
<veebers> fyi bug 1632541
<mup> Bug #1632541:  Build fails on s390x due to undefined variables <build> <ci> <regression> <s390x> <unit-tests> <juju:Confirmed> <https://launchpad.net/bugs/1632541>
<alexisb> thumper, wallyworld we need to get on veebers bug asap ^^^
 * wallyworld is finishing addmodel upgrade steps, after that i can look
<alexisb> thanks wallyworld
<natefinch> wallyworld, thumper: should just be a godeps update to newer version of golang/x/sys
<wallyworld> ok, ta
 * natefinch goes back to twiddling with simplestreams
<mup> Bug #1629817 changed: [arm64] bad cpu-cores detection <arm64> <openstack-provider> <juju:Incomplete> <juju-core:Won't Fix> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1629817>
 * redir eod
<menn0> anastasiamac, thumper or wallyworld : https://github.com/juju/juju/pull/6433
<wallyworld> oooh, a test fix
<wallyworld> nice
<alexisb> alright all, I am eod
<alexisb> see y'all tomorrow
<wallyworld> menn0: not sur eif you have a moment for a small review. similar code to existing upgrade steps https://github.com/juju/juju/pull/6435
<wallyworld> or anastasiamac ? ^^^^
<menn0> wallyworld: give me a sec and i'll do it
<wallyworld> ty
<wallyworld> menn0: and also a one liner please https://github.com/juju/juju/pull/6436
<wallyworld> nfi how such an old version of that dependency go committed :-(
<menn0> wallyworld: it probably hasn't changed since 2015
<menn0> wallyworld: ship it
<wallyworld> ty
<wallyworld> menn0: i think it was a recent dep addition
<menn0> wallyworld: ok well that's dumb then
<wallyworld> whoever did it just used an outdated version perhaps
<wallyworld> menn0: i checked the commit history of that golang project and s390 was a recent addition, so i *think* it will fix it :-)
<menn0> wallyworld: seems plausible
<menn0> certainly worth updating
<wallyworld> if we had wanted to use that package before then we would have been screwed
<menn0> wallyworld: other one reviewed. a few suggestions but looks fine.
<wallyworld> menn0: ty, appreciate it, will fix the issues
<wallyworld> menn0: i see you suggestions - that will teach me just to copy and tweak existing code without thinking too hard
<menn0> wallyworld: haha
<wallyworld> too much to do, sigh
<menn0> I think we've all done it
<wallyworld> menn0: also did you se email about moving tech board meeting forward 2 hours?
<menn0> wallyworld: yeah, I was just about to write about that
<menn0> wallyworld: works for me but I don't know about the jam
<wallyworld> it *should* do, it's his 10am
<menn0> wallyworld: jam has been idle for over 41 hrs... I wonder if he's on leave
<wallyworld> might just be me and you :-)
<menn0> which kind of defeats the point
 * menn0 checks calendar
<wallyworld> yeah
<wallyworld> although we do need to discuss at least the add-model change
<menn0> wallyworld: nothing in the calendar
<wallyworld> i did check yesterday i now recall
<menn0> wallyworld: there's certainly important things to discuss
<wallyworld> we can make a start even if he is not there. he may come on later for his SOD
<menn0> wallyworld: ok so meet 2 hours earlier then?
<wallyworld> yep, let's
<menn0> wallyworld: ok
<menn0> wallyworld: I'll reply to that email in case John notices that.
<wallyworld> sgtm, ta
<thumper-afk> menn0: replied
<thumper-afk> will remove critical logging :)
<jam> menn0: I just don't chat as much, just Lurk. :)
<jam> menn0: I'm there now, though it seems wallyworld is away again?
<menn0> jam: he emailed to say he had to pick up his kid. he'll be back at 20 past.
<menn0> jam: delay until then?
<jam> menn0: either way. I don't know what your day is like. Middle of my day so anytime is fine.
<menn0> works better for me to delay as kids are going mental right now
<menn0> can help my wife
<menn0> for a bit
<menn0> jam: see you in 15mins
<wallyworld> menn0: her enow
<thumper> oh... you had the tech board meeting well before what the calendar said
<thumper> oh well
<voidspace> frobware: ping
<frobware> voidspace: pong
<frobware> voidspace: this is happening because it is latest+greatest lxd (2.4.x)
<voidspace> frobware: is that right? ah, damn - it means it might not fix the vsphere issue at the same time
<voidspace> frobware: does the proposed fix work for lxd < 2.4 ?
<frobware> voidspace: trying now
<voidspace> heh, I could try it too
<frobware> voidspace: well... just started looking at this so my hypothesis may turn out to be wrong
<thumper> o/
<thumper> are we expecting babbageclunk?
<voidspace> thumper: as far as I know, yes
<voidspace> mgz: ping
<voidspace> frobware: works fine for me so far with lxd 2.0 (xenial default)
<voidspace> frobware: my containers (controller plus two units) have started fine with IPv4 addresses and are active
<voidspace> frobware: so the config change hasn't broken the "default case"
<frobware> voidspace: which bug did https://github.com/juju/juju/pull/6430 fix?
<voidspace> frobware: I'm not sure there's a specific bug to track this
<frobware> voidspace: my mistake: which bug did https://github.com/juju/juju/pull/6338 fix?
<voidspace> frobware: ah, looking
<voidspace> frobware: https://bugs.launchpad.net/juju-ci-tools/+bug/1624495
<mup> Bug #1624495: operations fails on rackspace because of ipv6 address in dns-name <rackspace> <status> <juju:Fix Committed by mfoord> <juju-ci-tools:Won't Fix> <https://launchpad.net/bugs/1624495>
<voidspace> frobware: and it seemed to fix that bug
<voidspace> frobware: that pull request should ensure we get IPv4 *if* we have an IPv4 address that matches the requested scope (public)
<voidspace> frobware: it won't help if we have a public IPv6 address and only a cloud local IPv4
<voidspace> frobware: which I assume is what's happening
<frobware> voidspace: so in this case the defaults [auto] for IPv6 is enabled when running lxd init on lxd 2.4
<voidspace> frobware: we can tweak that code to prefer a cloud local (fallbackScope) IPv4 over a public IPv6 if that's what we want
<voidspace> frobware: and that's new in 2.4?
<frobware> voidspace: looking at the steps in: https://bugs.launchpad.net/juju/+bug/1632440, I would say yes.
<mup> Bug #1632440: rc3, lxd 2.4.1, 16.04/16.10 use ipv6 if configured via lxd init <conjure> <ipv6> <oil-2.0> <uosci> <juju:Triaged by rharding> <https://launchpad.net/bugs/1632440>
<frobware> voidspace: the question I have (and I think rick_h_ too) is why didn't our 'prefer ipv4 over ipv6' choose the former?
<voidspace> frobware: see my comment above
<voidspace> frobware: that pull request should ensure we get IPv4 *if* we have an IPv4 address that matches the requested scope (public)
<voidspace> frobware: it won't help if we have a public IPv6 address and only a cloud local IPv4
<voidspace> frobware: we can tweak that code to prefer a cloud local (fallbackScope) IPv4 over a public IPv6 if that's what we want
<voidspace> I've added this as a comment to the bug
<frobware> voidspace: if we did as you suggest wouldn't that a) be better and b) catch other cases like this?
<voidspace> frobware: it would be better if we were happy to return a cloud local address when a public one is requested
<frobware> hmm
<voidspace> frobware: I can prepare a PR with that change
<voidspace> frobware: can you repro the bug?
<frobware> voidspace: was about to try
<voidspace> frobware: you could test my change to see if it fixes it
<frobware> voidspace: ok, let's do that. if you prepare a patch, I'll look into getting a machine with lxd 2.4 on it
<voidspace> frobware: cool
<frobware> voidspace: I cannot repro with 2.0. I changed my local LXD setup to have IPv6 so not sure why we run into this with lxd 2.4
<voidspace> frobware: *sigh*
<frobware> voidspace: I really want to understand the 2.0 case with IPv6 enabled first.
<frobware> voidspace: bbiab (10 mins)
<frobware> voidspace: oh... so the reason I cannot make it fail with lxd 2.0 on my machine is I disabled IPv6 for some other bug... :)
<dooferlad> frobware: yay. The future is here :-)
<frobware> voidspace: bug #1614953
<mup> Bug #1614953: hw csum failure when IPv6 interfaces configured in netdev_rx_csum_fault+0x38/0x40 <amd64> <apport-bug> <kernel-da-key> <kernel-fixed-upstream> <performing-bisect> <xenial> <linux (Ubuntu):In Progress by jsalisbury> <linux (Ubuntu Xenial):In Progress by jsalisbury> <linux (Ubuntu
<mup> Yakkety):In Progress by jsalisbury> <https://launchpad.net/bugs/1614953>
<frobware> dooferlad: ^^ :)
<frobware> dooferlad: most of the time I got the csum failure and occasionally my machine panic'd. so I disabled the whole shebang. I did leave myself a note in /etc/sysctl.conf.
 * frobware tries to recall what he was doing...
<babbageclunk> mgz: ping?
 * dooferlad tries to think through the haze of fluffy cold brain. Stupid viruses - we have a release!
<rogpeppe> an ultra-small PR to fix a bug (8 character change). could someone review please? https://github.com/juju/juju/pull/6438
<voidspace> rogpeppe: looking
<rogpeppe> voidspace: thanks
<voidspace> rogpeppe: hah, LGTM
<rogpeppe> voidspace: ta!
<macgreagoir> I was too slow to review, that was a great one! :-D
<rogpeppe> oh dammit, it causes tests to fail!
<rogpeppe> macgreagoir: great but flawed!
<macgreagoir> Will I look more sensibly slow when it breaks the tests?
<babbageclunk> macgreagoir: showoff
<macgreagoir> :-D
<rogpeppe> macgreagoir: this fixes the fix: https://github.com/juju/juju/pull/6438/files
<macgreagoir> *coughs* LGTM
<babbageclunk> rogpeppe: too long now, I'm withdrawing my approval.
<babbageclunk> I've got a dependency update PR! It's short! https://github.com/juju/juju/pull/6439
 * babbageclunk doesn't really withdraw his approval it was just a joke.
<macgreagoir> babbageclunk: lgtm, fwiw
<babbageclunk> macgreagoir: Thanks! You're a graduated reviewer in my heart!
<macgreagoir> I believe babbageclunk is already drinking in an NZ timezone.
 * rogpeppe never "graduated" as a juju reviewer
<frobware> voidspace: can repro with LXD 2.0 and -rc3 now: http://pastebin.ubuntu.com/23312073/
<frobware> voidspace: so baby steps. trying tych0's patch
<frobware> macgreagoir: your patch is tremendously helpful. http://pastebin.ubuntu.com/23312086/
<frobware> macgreagoir: we *must* land this for GA
<babbageclunk> voidspace, dooferlad, frobware can I get another +1? Or rogpeppe, even though you're not graduated? :)
<macgreagoir> frobware: You can only blame yourself! :-D
<frobware> voidspace: ^^ that's LXD 2.0 with tych0's patch
 * macgreagoir is working on it
<frobware> voidspace: I don't see tych0's patch having a positive impact on LXD 2.0... double-checking...
<frobware> voidspace: and that's because it only looks for the config enabled in the new style networking.
<frobware> babbageclunk: I previously ran into build issues when bumping the lxd dependencies - mgz has the detail.
<babbageclunk> frobware: Hmm. Good point - I'll confirm with him.
<babbageclunk> frobware: Maybe this will be better since there aren't API changes?
<frobware> babbageclunk: added to the PR comment.  but I think the issues are when they build the .debs.
<voidspace> frobware: does my PR help?
<frobware> voidspace: let me load that on top.
<frobware> voidspace: looking at your PR - this is quite a change "cloud local is picked over hostname"
<babbageclunk> I can't raise mgz - frobware do you know whether he's around today? Or is he just head down on release stuff?
<frobware> babbageclunk: given the release you might find he's starting his day later for overlap.
<babbageclunk> Ah right.
<voidspace> frobware: yes it is quite a change
<voidspace> frobware: does it work?
<frobware> voidspace: for a sample size of 1. :)
<frobware> voidspace: just looking at adding a model, adding 10 containers, verifying no IPv6, destroy model. Lather. Rinse. Repeat.
<voidspace> frobware: cool
<voidspace> frobware: it does mean that where we would have got a public hostname we can now get a cloud local IP address
<voidspace> frobware: I could fix that with a more complex change
<voidspace> but maybe we would usually prefer the IP address?
<frobware> voidspace: unless I screwed up... http://pastebin.ubuntu.com/23312299/
<voidspace> frobware: so I want to know (from the logs - probably need INFO or TRACE level) what IP addresses those machines have
<frobware> voidspace: looking
<voidspace> frobware: I'm pretty confident the address picking code works, so the problem is more likely to be that they *only* have IPv6 addresses
<frobware> voidspace: your branch has only 1 commit, correct?
<voidspace> correct
<voidspace> the production change is trivial
<frobware> voidspace: I see IPv4 and 6. http://pastebin.ubuntu.com/23312308/
<macgreagoir> If anybody has worked on status output, I'm looking for where indentation is specified, please.
<voidspace> frobware: right, but what does the machine agent see
<voidspace> frobware: are they still showing ipv6?
<voidspace> frobware: in juju?
<frobware> voidspace: http://pastebin.ubuntu.com/23312331/
<babbageclunk> macgreagoir: I've worked on that a little bit. Do you mean in yaml serialisation?
<frobware> voidspace: just adding the logs
<macgreagoir> babbageclunk: I think so. I need add some indentation to a new list in status and show-machine.
<macgreagoir> babbageclunk: https://github.com/juju/juju/pull/6424
<frobware> voidspace: http://pastebin.ubuntu.com/23312333/
<frobware> voidspace: the machine-0.log http://pastebin.ubuntu.com/23312338/
<frobware> voidspace: actually that log ^ is of little use
<voidspace> frobware: I have a failing test case where it is picking IPv6 over IPv4 - debugging
<macgreagoir> babbageclunk: Don't let me eat your time, though thanks. I'm just looking for a shortcut under time pressure, but I'm sure I can find it.
<voidspace> frobware: oh for goodness sake
<voidspace> frobware: network.NewAddress("10.229.221.170") creates an address with type "hostname"
<frobware> voidspace: controller log: http://178.62.20.154/~aim/controller-machine-0.log
<voidspace> frobware: so then juju treats it with equal priority to ipv6
<frobware> voidspace: ugh?
<voidspace> frobware: http://pastebin.ubuntu.com/23312355/
<voidspace> frobware: I believe this is the cause of the problem
<frobware> voidspace: we lack distinction between address and hostname, IMO.
<frobware> voidspace: addresses are bytes. should never be strings until you want to print them.
<voidspace> frobware: well, evidently... ;-)
<frobware> voidspace: wow
<rogpeppe> what's the status on landing stuff in juju master now? is it supposed to be only critical stuff, or has the release been forked now?
<frobware> rogpeppe: my understanding is only critical
<rogpeppe> frobware: thanks
<voidspace> frobware: gah, no - my mistake
<voidspace> frobware: I gave it the address 10.299.221.170
<voidspace> frobware: which is obviously invalid (299)
<voidspace> frobware: given the choice between 10.199.121.170 and the ipv6 address the address selection does work - even on current master without my change
<voidspace> frobware: so SelectPublicAddress is not being given that IPv4 address
<voidspace> coffee
<babbageclunk> macgreagoir: I think frobware's comment about the indentation is wrong (no offence frobware!). That's just how a list of items is represented in YAML. You can see the same format in the Projects section on yaml.org.
<rogpeppe> voidspace: FWIW I think a string is a perfectly reasonable representation of an address :)
<frobware> babbageclunk: fair enough. I just tried various /other/ indenters and they ... indented lists.
<frobware> rogpeppe: semantically broken
<rogpeppe> voidspace: but i don't think that Type should be part of Address - it should be a method
<rogpeppe> frobware: how so?
<voidspace> rogpeppe: agreed
<voidspace> rogpeppe: it shouldn't be possible for it to be wrong
<rogpeppe> voidspace: it's always possible for it to be wrong, even if it's just a set of bytes
<voidspace> I meant Type
<rogpeppe> voidspace: ah yes, indeed
<frobware> rogpeppe: because you only need it as a string for display. the type system should aid us to disntinguish between hostnames and addresses. if we need a hostname (string) then convert. but the I believe the base type of an IP address is []byte.
<voidspace> really coffee
<macgreagoir> babbageclunk: Cheers, I'm not seeing any standard flags to indent either. frobware, maybe this could be a nicety for later if we decide it's better?
<rogpeppe> frobware: it's really useful to be able to connect to an address whether it's a DNS name or IP address
<rogpeppe> frobware: we do it on the command line all the time
<rogpeppe> frobware: given that there's no syntactic overlap between the two, having a single string representation seems good to me
<frobware> rogpeppe: sure. it's more about the internal type.
<rogpeppe> frobware: why does it matter?
<frobware> rogpeppe: ^^ network.NewAddress("10.229.221.170") creates an address with type "hostname"
<rogpeppe> frobware: in my view, very little code should need to care about addresses as other than an opaque string
<rogpeppe> frobware: it doesn't
<rogpeppe> frobware: (i checked)
<frobware> rogpeppe: we've also given this problem to the charmers - when we say "address" they say I want to both "address" and "name"
<rogpeppe> frobware: ?
<frobware> rogpeppe: network-get address
<frobware> rogpeppe: it's not clear what that returns. they faff around trying IPaddress? symbolic-hostname? should I always convert?
<frobware> rogpeppe: if we had: network-get hostname and network-get address then it would be clearer
<rogpeppe> frobware: why should it matter?
<frobware> rogpeppe: apparently it matters to the. hadoop for one.
<babbageclunk> frobware: I think it's just a neither-a-bug-nor-a-feature of the particular yaml serialiser we use.
<rogpeppe> frobware: if you need an ip address and your software doesn't know how to resolve it, then you can resolve it easily
<rogpeppe> frobware: FWIW network-get seems to document that it returns an IP address
<frobware> rogpeppe: except for the times it returns a symbolic name
<rogpeppe> frobware: in that case the doc should be changed
<rogpeppe> frobware: (or the code changed)
<frobware> rogpeppe: the charmers want both 'name' and 'address'. where 'address' is always an IP address (in string format)
<rogpeppe> frobware: what would "name" return if there's no DNS name?
<frobware> rogpeppe: IP address.
<rogpeppe> frobware: ha ha
<frobware> rogpeppe: :)
<frobware> rogpeppe: but that's OK...
<frobware> rogpeppe: what needs fixing is DNS
<rogpeppe> frobware: so what's wrong with just returning something that's either a name (which can be resolved) or an IP address (if there's no name) ?
<rogpeppe> frobware: what's wrong with DNS?
<frobware> rogpeppe: semantically they want the distinction: if I call `network-get address` I know that will always be a dotted-quad.
<frobware> rogpeppe: nothing wrong with DNS except we don't always end up with a setup that resolves names. and that's just a bug.
<rogpeppe> frobware: if it's really too much hassle for charm software to resolve a DNS name then you could add a --resolved or --numeric flag to network-get
<frobware> rogpeppe: oh... I think the issue is more on the juju/maas/cloud side
<rogpeppe> frobware: ah, so it can return a DNS name that's unresolvable?
<frobware> rogpeppe: correct. bug.
<rogpeppe> frobware: if that's the case, how could the charm resolve it to return it?
<rogpeppe> s/the charm/juju/
<frobware> rogpeppe: they shouldn't have to. if DNS is working then `network-get address` and `network-get hostname` would just be correct. Assuming that's actually your question.
<rogpeppe> frobware: so the bug is really that it's returning invalid DNS names, not that people need numeric IP addresses per se?
<frobware> rogpeppe: certain s/w want both to forwars/reverse resolve correctly: http://sujee.net/2012/03/08/getting-dns-right-for-hadoop-hbase-clusters/
<frobware> rogpeppe: the java big data stacks seem to be the pickiest
<frobware> rogpeppe: (hopefully) no offense, but need to step out of this discussion and fix the IPv6 issue ...
<rogpeppe> frobware: np :)
 * rogpeppe undistracts
<frobware> voidspace: where are we... ? :)
<macgreagoir> dimitern: Just a reminder that you wanted to comment on https://github.com/juju/juju/pull/6424 too.
<voidspace> frobware: well, tych0's fix works for 2.4, but not for 2.0 for you - right?
<frobware> voidspace: I need to _actually_ try on 2.4.
<voidspace> frobware: and my fix doesn't help 2.0?
<frobware> voidspace: I understand why it doesn't.
<frobware> voidspace: not unless you've pushed since
<voidspace> frobware: I haven't
<voidspace> frobware: so the problem is not in address selection but in the addresses we see for the machine
<dimitern> macgreagoir: hey, yeah - sorry I got distracted, looking now
<macgreagoir> Cheers :-)
<voidspace> frobware: which I think is the same problem vsphere has
<voidspace> frobware: but I can't repro
<voidspace> fiddling with my lxd settings
<frobware> voidspace: to repro on LXD 2.0 you need to reconfigure LXD to use IPv6 - by default I was only using IPv4.
<voidspace> frobware: yeah, just done that with dpkg-reconfigure
<frobware> voidspace: and for LXD 2.4 this might be the easiest way: https://lists.ubuntu.com/archives/snapcraft/2016-October/001327.html
<rick_h_> frobware: voidspace ty for helping to chase this down. As lxd 2.4 is going to get backported I'm mostly concerned there.
<rick_h_> frobware: voidspace if we can get it working back in 2.0 that's a bonus imo
<frobware> rick_h_: the 2.0 case could be just a release note atm
<rick_h_> frobware: k, and we can go the route that folks need to configure an ipv4 only lxd to get it to work there, for instance.
<frobware> rick_h_: yep, it's essentially the message you would [now] get on LXD 2.4 with IPv6 enabled.
 * rick_h_ has to get the boy fed and dressed for school, biab
<rick_h_> macgreagoir: how's the upgrade steps looking?
<frobware> rick_h_: but having faffed around a bit, I think we could do the same change but detect in /etc/default/lxd-bridge
<macgreagoir> rick_h_: They tested OK for me. Sent you and wallyworld an email with details.
<rick_h_> frobware: k, just need something as solid as we can get this morning so we can build a GA today.
 * frobware goes back to actually trying LXD 2.4
<rick_h_> macgreagoir: k, please pull in any help to get reviewed/landed so it's not blocking GA
<macgreagoir> rick_h_: It's already merged, so I think we're grand.
<voidspace> with ipv6 enabled and lxd 2.0 I see containers with both IPv6 and IPv4 addresses, but juju picks (correctly) the ipv4 address
<voidspace> that's with current master
<voidspace> so I can't repro
<voidspace> yet
<frobware> voidspace: how do you determine that it correctly picks IPv4? from status, something else?
<voidspace> frobware: from status
<voidspace> frobware: in the problematic cases, status was showing an ipv6 address - status address comes from dns-name which is PreferredPublicAddress
<voidspace> which comes from address selection
<dimitern> macgreagoir: you've got a review on 6424
 * macgreagoir looks
<macgreagoir> dimitern: Cheers, I've just been looking at ScopeMachineLocal and ScopeLinkLocal in tests.
<dimitern> tych0: you've got a review on 6430 as well
<voidspace> mgz: ping
<mgz> voidspace: yo
<voidspace> mgz: I'm still trying to connect to vsphere and I wondered if you had any further ideas
<voidspace> mgz: I can ssh in now
<mgz> voidspace: okay, so now do exactly what the vsphere-deploy-trusty-amd64 job does?
<mgz> be jenkins, export JUJU_DEV_FEATURE_FLAGS=vsphere-provider
<mgz> and run the ci script?
<voidspace> mgz: ok, will try - otp right now
<voidspace> mgz: thanks
<rick_h_> natefinch: ping
<natefinch> rick_h_: yo
<rick_h_> natefinch: howdy, how goes the streams tool work? It's the last issue blocking GA atm.
<natefinch> rick_h_: I couldn't reproduce the problem last night.  It seems to work fine for me
<natefinch> rick_h_: I was just about to ping sinzui about it
<rick_h_> natefinch: please get with sinzui then as he's pretty confident it doesn't work. I think there's some misunderstanding about what works and what doesn't.
<katco> rogpeppe: hey, fyi discussion of your retry package was bumped from this week's tech board meeting due to time. it's on the docket for next week
<rogpeppe> katco: thanks. yeah, i heard from menno this morning.
<rogpeppe> katco: unfortunately i'll be away at a sprint next week
<katco> rogpeppe: I responded to your review yesterday with some questions. just a few; we should be able to get this landed
<rogpeppe> katco: i thought it might be nice to join that discussion
<katco> rogpeppe: agreed. i think it's in your morning?
<rogpeppe> katco: i'm not sure i should land it now as it's all frozen for GA, right?
<sinzui> natefinch: the bug was reported after you were trying to make streams with 2.x agents. If generate-tools now makes streams with 2.x agents, and juju can bootstrap with them, then the bug is fixed
<katco> rogpeppe: oh, has that happened already?
<rogpeppe> katco: it'll be at 10am CET I think
<rogpeppe> katco: i'm not sure. someone assured me we should only land critical bug fixes and i don't think that really counts as that.
<rick_h_> rogpeppe: katco +1
<katco> rogpeppe: hm, ok.
<rick_h_> rogpeppe: katco we've got one blocker landing atm and natefinch's is the last thing we're waiting for GA to go final
<katco> rogpeppe: well at any rate, happy to carry on discussion so we can land for 2.1
<katco> rick_h_: cool
<rogpeppe> katco: i actually wanted to get the next PR (dependent on that one) landed because that changes a controller config attribute name, but i think the delays to that PR have made that infeasible
<natefinch> sinzui: I'm not sure the repro steps in the bug are sufficient for me to be confident about the fix.  I trust that you tried it and it didn't work... I'm just not sure where the disconnect it
<natefinch> s/it/is/
<sinzui> natefinch: i have not tried in in weeks
<rogpeppe> katco: i guess we'll just have to live with the wrong config attribute name for 5 years :)
<rick_h_> rogpeppe: juju 2.0 isn't on a 5yr plan :P
<natefinch> sinzui: I would appreciate if you could give it a try.  I hope it was just a momentary thing that is now past... but I think it would be faster for you to just try it than to try to give me exact steps.
<rogpeppe> katco: thanks for the on-going reviews BTW
<rick_h_> rogpeppe: so I think only 18mo?
<rogpeppe> rick_h_: oh, that's good!
<rick_h_> rogpeppe: or we make it backward compatible. Accept both values, but move the docs/etc to the new value name
<rick_h_> rogpeppe: so you can push things forward in a clean way
<katco> rogpeppe: more than happy to do so. trying to find that balance between expediency and making sure we're moving the codebase in a good direction with every commit
<sinzui> natefinch: so you got streams with 2,x in them and juju bootstrapped with them. If, so the bug is fixed
<rick_h_> sinzui: this can be used with canonistack correct? to test it out?
<voidspace> having to reset network
<voidspace> bbiab
<rick_h_> natefinch: have you generated the file and attempted to use it as streams to bootstrap against?
<rick_h_> natefinch: sinzui this means sticking the file up at some url and referencing that via bootstrap config correct?
<natefinch> rick_h_: I generated the streams and they have 1.25 and 2.x in then... I don't actually know how to bootstrap with them
<sinzui> rick_h_: yep, any cloud and lxd. So long as your localhost and the controller machine can see the streams that agent-metadata-url point to, the test is valid. bootstrap with --debug so that juju will show you where it downloaded the agent from.
<rick_h_> natefinch: https://jujucharms.com/docs/2.0/howto-privatecloud
<rick_h_> natefinch: does that work with the file you get
<mgz> stand-de-up?
<rick_h_> natefinch: dimitern voidspace ping for standup
<natefinch> rick_h_: I don't have openstack.  I have a maas, not sure how to use these generated simplestreams with them though
<natefinch> rick_h_: figuring it out now... bootstrapping with tools-metadata-url IIRC
<sinzui> natefinch: The problem is the same. run generate-tools with some agents you downloaded. push the generate tree to a website iike people.canonical.com or to swift like canonistack, or to s3 in aws.
<sinzui> natefinch: use "agent-metadata-url" tools-* was obsoleted more than 6 months ago
<natefinch> sinzui: oh yeah, I was looking at old docs
<natefinch> sinzui: it's too bad, the old docs were a lot clearer
<tych0> dimitern: you around?
<sinzui> natefinch: https://jujucharms.com/docs/2.0/howto-privatecloud is not about agents.
<sinzui> natefinch: That is about image streams. So the new docs are missing 50% of the work
<sinzui> natefinch: https://jujucharms.com/docs/1.25/howto-privatecloud#tools-metadata is the relavent part from the old docs
<macgreagoir> frobware dimitern: PR 6424 has a new commit, whenever you're ready, please.
<natefinch> sinzui: when I specified agent-metadata-url ... do I specify the /tools directory, or the directory above that? I can never remember
<sinzui> natefinch: the directory that contains streams/
<dimitern> macgreagoir: one comment added, but I couldn't see tests with IPv6 addresses?
<macgreagoir> dimitern: https://github.com/juju/juju/pull/6393
<macgreagoir> dimitern: ignore ^
<macgreagoir> dimitern: https://github.com/juju/juju/pull/6424/files#diff-e3aecf740b74fe827fbd780d709c29e5R2605
<rick_h_> dooferlad: I'm going to head back from the coffee shop, should be back on in time for our meeting, but if I'm a min/two late be right there
<natefinch> aaahhhh $ juju credentials
<natefinch> ERROR removing secrets from credentials for cloud azure: auth-type "userpass" not supported
<katco> rogpeppe: your pr lgtm
<rogpeppe> katco: thanks
<katco> rogpeppe: ty
<rogpeppe> katco: BTW I don't always for two LGTMs from core, but I would never submit a PR after someone had asked for changes without getting their +1 first.
<rogpeppe> s/for/ask for/
<katco> rogpeppe: ack; that is very gracious of you
<katco> rogpeppe: i feel the same way
<rogpeppe> katco: isn't that, like, a thing
<katco> rogpeppe: i dunno, i've seen review-points get summarily dropped
<rogpeppe> katco: i'd be disappointed in general if someone did it to me. otherwise i could just go around to someone else if i didn't like the review someone had given me.
<katco> rogpeppe: yeah
<rogpeppe> katco: well, if that really happens, it's not ok
<natefinch> Hopefully no one just drops review points without a discussion.  Sometimes the discussion may happen outside the review tool, so it may not be obvious (which is I why I try to note it if it was - "per discussion with x...")
<katco> natefinch: saw it happen a few times ("disagree" & close)
<natefinch> rick_h_, sinzui: bootstrapping with the generated file and agents in 2.0 seems to work fine.
<sinzui> :)
<rick_h_> natefinch: <3 is that with any change or just as it is in trunk?
<natefinch> rick_h_, sinzui: as it is in trunk
<rick_h_> katco: can you please get with natefinch and provide a confirmation there please?
<sinzui> natefinch: you used 2.0.0?
<natefinch> sinzui: current master, yes
<sinzui> natefinch: maybe devel agents are the problem...I am not sure I care if juju cannot make streams for devel agents
<katco> rick_h_: natefinch: sure... what do you need me to do?
<natefinch> katco: I'm looknig at this bug: https://bugs.launchpad.net/juju/+bug/1613858
<mup> Bug #1613858: generate-tools ignores 2.x agents and creates bad path <jujuqa> <metadata> <regression> <rteam> <simplestreams> <streams> <juju:Incomplete by natefinch> <https://launchpad.net/bugs/1613858>
<rick_h_> katco: natefinch maybe best to hop on a HO and walk through it and make sure it's replicatable
<natefinch> katco: moonstone?
<natefinch> https://hangouts.google.com/hangouts/_/canonical.com/moonstone?authuser=2
<katco> natefinch: sure
<macgreagoir> dimitern: This time :-) https://github.com/juju/juju/pull/6424
<macgreagoir> dimitern frobware katco: I'm not sure if that PR qualifies as changing a published API (adding IPAddreses). ^
<dimitern> macgreagoir: strictly speaking, yes it does
<rogpeppe> natefinch: FWIW i replied to your comment and made some minor changes to https://github.com/juju/utils/pull/245.
 * macgreagoir takes a look to see if that can be avoided...
<dimitern> macgreagoir: LGTM
<katco> macgreagoir: hm, i don't know our stance on that. I don't think adding fields should break backwards compatibility
<katco> macgreagoir: but i'm not positive
<dimitern> macgreagoir: it can't be avoided, but adding stuff is less of an issue than changing/removing stuff
<katco> macgreagoir: don't those fields need comments?
<katco> dimitern: well, it can be avoided by bumping the facade revision
<dimitern> katco: sure :) .. if there was a release with the new api version so far
<katco> rick_h_: do you happen to know if adding fields to API params requires a facade revision bump?
<katco> dimitern: we have promised backwards compatibility from rc1 i think?
<rick_h_> katco: adding should not, the name of the field is new/different? or have we changed the type of an existing field?
<katco> rick_h_: macgreagoir correct me if i'm wrong, but looks like just new fields
<rick_h_> katco: as long as we add it as a new field and didn't remove the old then no, shouldnot require any bump
<macgreagoir> katco: You are not wrong.
<macgreagoir> Added
<katco> macgreagoir: cool, should be good then. please throw some comments on there. i don't know why we seem to be ignoring that rule in this file...
<macgreagoir> rick_h_: Aye, it is a new field, no existing have been changed.
<macgreagoir> katco: Let me add some comments...
<rick_h_> macgreagoir: <3
<natefinch> rick_h_: katco was able to generate the streams as well, do we need to do the whole upload to s3 thing from her machine as well?
<natefinch> sinzui: ^
<rick_h_> natefinch: a whole bootstrap. if we can bootstrap --debug and it's using the streams/etc then cool
<natefinch> rick_h_: ok
<katco> does anyone know how to use s3cmd to get the url for an s3 bucket?
<natefinch> if you can get the UUID of the bucket, it should just be  https://s3.amazonaws.com/<uuid>
<macgreagoir> katco: If you please, https://github.com/juju/juju/pull/6424/files#diff-8af0b17f03b1bac65594c1e20a5798ecR47
<katco> macgreagoir: look great, ta!
<macgreagoir> It's all my fault from here on in if the comments are misleading!
<katco> macgreagoir: lol
<redir> macgreagoir: I think your last comment was misleading
<macgreagoir> redir: It's not alwayas real hardware, I know.
<redir> apparently I need to work on my deadpan delivery
<macgreagoir> :-D
<redir> so I got an alert today that I am OCR next week, and I have no idea where that comes from.;
<frankban> hey rick_h_: who can we ping for problems bootstrapping juju on openstack provider?
<macgreagoir> rick_h_: The list-ip-addrs-in-status PR is approved. Is it good to merge or has it missed the deadline?
<frobware> rick_h_: in respect of the deadline I'm still doing the LXD fix
<rick_h_> frankban: what's up? file a bug and we'll see if we can see what's up
<rick_h_> macgreagoir: yes please land
<macgreagoir> rick_h_: ack
<frankban> rick_h_: we are trying to bootstrap juju on openstack in bastion, and it's stuck on "waiting for address", but at this point we don;t know if we missed something in the process or if it's a juju bug
<frankban> rick_h_: we'd like some help to unblock us, in preparation for demos
<redir> anyone seen this cannot start bootstrap instance: no "xenial" images in us-west-1 with arches [amd64]
<redir> ?
<katco> sinzui: how would i tell bootstrap to look for devel streams instead of released?
<rick_h_> frankban: k, you're vpn'd in or have access to the address that was allocated to the machine?
<katco> larger question, how do i enumerate the possible config values?
<rick_h_> frankban: k, the best folks might be other folks using bastion or openstack users there unfortunately. Maybe reach out to nice OS folks like beisner or thedac
<frankban> rick_h_: yes we'll ask them, ty
<natefinch_> katco: you mean bootstrap --config values?  (I may have missed a bit, had to log out and back in).
<sinzui> katco: add "--config agent-stream=devel" at boostrap
<natefinch_> katco: I don't think we have a way to list them... there's docs, but I don't know how up to date they are
<redir> katco: --config agent-stream=devel or something
<redir> doh
<katco> natefinch_: couldn't find docs =/
<katco> sinzui: ta, i was using image-stream
<natefinch_> yeah, the docs are well hidden
<natefinch_> katco: https://jujucharms.com/docs/2.0/models-config
<redir> so are the images
<redir> I can't deploy xenial or trusty to us-*-*
<sinzui> redir: is that aws us-west-1?
<natefinch_> I hate the way we call that model config even though almost all of those are controller-specific.  I mean, you can't have a different agent-metadata-url per model, can you?
<sinzui> redir: no, I think it is google
<sinzui> natefinch_: yes, I think juju should be more like snaps in this case. juju bootstrap --devel
<redir> sinzui aws us-west/east-1/2
 * redir removes custom clouds
<sinzui> redir us-west-1, us-west-2, and us-east-1 do have both xenial and trusty
<sinzui> in aws
<redir> sinzui: yea I have a custom cloud which makes the closest region to me the default and speeds up dev time waiting for it to spin up... It has worked since April and suddenly seems to have stopped
<redir> which is probably unrelated to streams other than the error
<redir> next I blame --build-agent
<frobware> rick_h_: https://github.com/juju/juju/pull/6430 is good to merge - did I miss the deadline?
<rick_h_> frobware: no, we're holding for athat and natefinch_'s work so please go ahead and merge
<rick_h_> natefinch_: katco how are we doing? is it still working properly and can bootstrap with the generated streams file?
<katco> rick_h_: i am still trying to bootstrap with the generated streams
<rick_h_> katco: k, ty for the update
<frobware> rick_h_: I need to run but $$merge'd.
<redir> looks like we changed things in rc2
<rick_h_> frobware: k, will watch it ty
<natefinch_> sinzui: do we have a CI test for this?  Seems like "two devs tried it and it sorta seemed to work" is not the best verification strategy ;)
<sinzui> natefinch_: we do not. We stopped using generate-tools in January. When you asked me how to make streams a few weeks ago, you reported that your 2.x agents were not found, when I ran the commands, I confirmed 2.x agents were skipped and that the directory structure was incorrect
<natefinch_> I just retried, making sure to use --agent-stream=devel and it still seems to work.  The log output is kind of hard to read, so it's possible it's not actually downloading from my S3 bucket... but I think it is: http://pastebin.ubuntu.com/23313802/
<natefinch_> oh, no, wait, it's not
<natefinch_> crud
<katco> sinzui: natefinch_: that's where i'm stuck at. this concerns me: found 0 packaged agent binaries
<natefinch_> No packaged binary found, preparing local Juju agent binary
<katco> natefinch_: that's the message i'm getting so i was triyng to figure out if i had it set up wrong or if it really just isn't
<sinzui> katco: does the com file for released or devel list the binaries you made
<natefinch_> sinzui: mine does: http://pastebin.ubuntu.com/23313815/
<natefinch_> I gotta run.  Doctor time.  Good luck guys, sorry I  can't be of more help debugging.
<natefinch_> s/guys/fellow engineers/
<redir> yup, it ignores old region endpoints, replacing it with new region endpoints from goamz. But then it must ignore that change or just use the now bad endpoint data to find instances. Not too sure, but updating endpoints in my custom config WFM.
<katco> sinzui: i don't think it does, just the 1.24.1 series, not 2.0?
<sinzui> katco: :(
<katco> sinzui: looking at nate's comment on the bug, it looks like it does? i was running master
<katco> sinzui: i.e.: "com.ubuntu.juju:16.04:amd64": {
<sinzui> katco: I will try with a recent juju and and a few agents
<katco> sinzui: sorry, i don't deal with this a lot. i might be fumbling somewhere
<sinzui> katco: I think that fact that "juju metadata generate-tools -d <tools_dir>" doesn't just work or doesn't work with --stream means we have an enterrpise problem
<katco> sinzui: if that is indeed the case, then i agree
<sinzui> katco: We are special though. We have lots of jujus and possibly juju-metadata plugins. I am removing everything juju juju-2.0 from my host
<katco> sinzui: ah i had the dir structure wrong when i generated, retrying the bootstrap
<sinzui> katco: indeed I think I did that too. I just say juju-dist/tools/tools appear
<katco> sinzui: i'm still getting: 12:42:20 INFO  juju.environs.bootstrap tools.go:74 found 0 packaged agent binaries
<katco> sinzui: but i still don't trust my result. i have a feeling i'm doing something wrong
<sinzui> katco: do you see tools/streams/v1/index2.json and does its point to a com file with the agents you have? the end agent-metadata-url you set needs to end with the "tools"
<katco> sinzui: yes, the generated stuff looks correct. the url is not as you say. let me append it with the path up to and including "tools"
<katco> sinzui: yay: 12:46:29 INFO  juju.environs.bootstrap tools.go:74 found 1 packaged agent binaries
<katco> sinzui: so i think it's working as intended?
<sinzui> katco: I am getting bogus results
<sinzui> katco: the com file has every puiblished agent ever, but not just the 3 agents I want in my cloud's streams
<katco> sinzui: mine has just the agents i specified
<katco> sinzui: when i didn't have the directory structure correct, it had every agent ever
<katco> so now i have mystreams/tools/devel/*.tgz
<katco> juju metadata generate-tools -d mystreams --stream devel --debug
<katco> sinzui: and that works for me
<sinzui> katco: This is what I did https://pastebin.canonical.com/167729/
<katco> sinzui: can you do a --debug on metadata generate-tools ?
<sinzui> katco: https://pastebin.canonical.com/167730/
<sinzui> katco: per the bug, I think I can instread pass juju-dist/tools and the agents will be found, bug I get a tools/tools dir :(
<katco> sinzui: i just pass the parent of tools
<katco> sinzui: can you name your root something more boring? foo or something?
<sinzui> katco: the debug out shows streams.canonical.com was called. the mirrors is only on that host
<sinzui> katco: yes, foo it will be
<katco> sinzui: and what version of juju is this?
<sinzui> katco: juju version
<sinzui> 2.0.0-xenial-amd64
<sinzui> ^ katco froma  few hours ago
<katco> sinzui: from tip?
<sinzui> it was tip a few hours ago
<katco> sinzui: cool, same here
<sinzui> katco: you don't pass -m?
 * sinzui wonders if juju is reading something like his racksapce model and deciding that streams.canonical.com is to be used
<sinzui> katco: with foo and again streams.canonical.com was used https://pastebin.canonical.com/167732/
<katco> sinzui: i get the same result you do if i don't pass a --stream
<sinzui> katco: okay, I will pass --stream released
<katco> sinzui: that may be the bug
<sinzui> that is different :)
<katco> sinzui: so that works?
<sinzui> katco: yes: https://pastebin.canonical.com/167733/
<katco> sinzui: ok, and then i assume a bootstrap will work as well
<sinzui> katco: I think we hit a problem with juju 1's old "releases" dir. I am going to reset and rename released => release and see if it finds the agents
<sinzui> yep, . the bug is that there is a juju-1 ism in it
<sinzui> katco: https://pastebin.canonical.com/167734/ shows that juju defaults/requires tools/releases, btu the help says "released" and the product file (com file) is also "released"
<bodie__> so I was just now using dlv debugger to step through an io.ReadAtLeast where len(buf) and min both were 0
<bodie__> but once I was inside the func, printing the value of the int return value ("n") gave a result I didn't expect: 2325760
<bodie__> can anyone help me understand how this could happen?
<bodie__> it was being called from io.ReadFull
<sinzui> katco: --stream devel or beta or pting works. So I think --stream is required, not an option. Since juju assumes the stream name is "released" I think we should fix juju to look for that dir.
<bodie__> damn.  this was intended for #go-nuts -- sorry!
<rick_h_> bodie__: :P
 * bodie__ awkwardly takes his leave :D
<katco> sinzui: sorry got a phone call (otp)
<sinzui> katco: the juju 1 docs for private cloud do wook for juju2 because it tell uses to put agents in "releases" it makes no mention of --streams.
<perrito666> is there anybody from the tech board awake?
<katco> perrito666: o/ but i need to get lunch
<perrito666> katco: defer ping()
<katco> perrito666: do you have a question i can mull while i lunch?
<perrito666> katco: sure, typing
<katco> sinzui: can we clean up your bug to be more about the streams flag default not working?
<katco> sinzui: and do you feel that represents the problem wholly?
<perrito666> our s390 build is broken because of the introduction of golang.org/x/sys/unix which requires some extra steps by hand to compile (which include cgo) I believe I might be able to remove said dependency since its used for a very small thing (calculating fs free space in a more accurate way than previously done iiuc)
<perrito666> so my question is
<perrito666> nothing is worth the pain of cgo right?
<katco> perrito666: almost certainly not, but let me mull
<perrito666> katco: thanks for the shared head procesing :)
<sinzui> katco: yes, I think the --streams is the extent of the issue now
<sinzui> katco: I updated the bug https://bugs.launchpad.net/juju/+bug/1613858
<mup> Bug #1613858: generate-tools ignores 2.x agents and creates bad path <jujuqa> <metadata> <regression> <rteam> <simplestreams> <streams> <juju:In Progress by natefinch> <https://launchpad.net/bugs/1613858>
<perrito666> I would really appreciate a rather urgent review https://github.com/juju/juju/pull/6440
<alexisb> katco would appriciate a review on https://github.com/juju/juju/pull/6440
<thumper> perrito666: s390x fix approved
<perrito666> thumper: tx, merging
<thumper> perrito666: did you test it on s390 already?
<perrito666> thumper: its a revert
<thumper> ah
<thumper> ok
<perrito666> thumper: that method was the way it was done before
<thumper> ack
<perrito666> np
<perrito666> brb, small errand before the sky falls on our heads
<rick_h_> katco: ping, do you have a sec to fill me in on the bug update and where we go from here please?
<katco> rick_h_: yep
<rick_h_> katco: meet you in standup meeting?
<katco> rick_h_: ok
<babbageclunk> thumper: ping
<thumper> babbageclunk: hey
 * thumper joins hangout
<menn0> perrito666 (and thumper): I see you merged the change to remove the golang.org/x/sys dep
<menn0> wallyworld updated that to the latest version yesterday
<menn0> the newer version claimed to include s390x support
<menn0> we were using a rev from 2015 before
<menn0> did that not help?
<wallyworld> the commit logs lied it seems :-(
<menn0> wallyworld: or it wasn't complete s390 support
<menn0> thumper: I see you hit a few intermittent test failures... I'm making sure there's bugs for them
<thumper> menn0: thanks
<menn0> thumper, wallyworld: do you know why Juju prints this at bootstrap? "Launching controller instance(s) on maas2..."
<menn0> why plural on controllers?
<menn0> instances even
<menn0> is there ever a case where bootstrap will result in more than once instance
<menn0> ?
<thumper> no
<thumper> seems stupid
<redir> looks like model-defaults is broken. when supllying a region it now complains "error: No selected controller"
<alexisb> redir, we need to get on top of that
<redir> :|
<katco> last remaining blocker (an honor) for 2.0 need review. very simple: https://github.com/juju/juju/pull/6441
<menn0> katco: looking
<katco> menn0: ta
<menn0> katco: done. just some help tweaks.
<katco> menn0: tal
<perrito666> wallyworld: menn0  the commit logs where true-ish, for support there are a set of steps required to be run
<perrito666> wallyworld: menn0 we are a bit on the edge to add all that to the build procedure
<perrito666> it depends on cgo
<menn0> perrito666: that's fine. just making sure you and wallyworld weren't tackling the same problem in different ways.
<katco> menn0: addressed your comments, thanks
<alexisb> menn0, when you are done with the review i am on our 1x1 HO
<menn0> katco: looks good but I noticed one minor problem.
<katco> menn0: sure
<menn0> katco: oh, and you didn't do the "#" change?
<katco> menn0: sorry, the "#" change?
<katco> menn0: lol sorry just saw it
<alexisb> so redir what command exactly is failing?
<menn0> katco: sorry, I shouldn't have mixed up 2 things in the one comment
<katco> menn0: no worries should have caught that
<katco> menn0: should it still be a bulleted list? or just #?
<menn0> katco: the examples for other command seem to just use "#"
<katco> menn0: k i'll fix it up
<katco> menn0: k try that out
<redir> juju model-defaults <region> ...
<redir> alexisb: ^
<alexisb> redir, looking at the help for the model-defaults I am not seeing anything about specifying regions
<redir> then the help is out of date:/
<redir> it looks like something changed in modelcommand, so that now the command doesn't correctly get the current controller
<redir> which appears to have changed in https://github.com/juju/juju/commit/a5f95c441ac29ed1a98c8efafbb71869ed89e7b6
<wallyworld> menn0: thumper: it prints that at bootstrap because a certain someone asked for it
<thumper> pfft
<redir> alexisb: it is in the Usage string but not the detailed help.
<alexisb> redir, I am confused
<alexisb> we should chat
<alexisb> will ping whne I am done
<redir> OK
<redir> alexisb: ^
<alexisb> redir, https://hangouts.google.com/hangouts/_/canonical.com/alexis-ian
<alexisb> when you are ready
<redir> brt
<anastasiamac_> katco: in ur recent charm related work, have u come accross this? https://bugs.launchpad.net/juju/+bug/1606684
<mup> Bug #1606684: upgrade-charm fails to upgrade local charm <regression> <juju:Triaged> <https://launchpad.net/bugs/1606684>
<katco> rick_h_: there were failing tests
<katco> anastasiamac_: tal
<katco> anastasiamac_: that looks like a dupe of bug 1609463 ?
<mup> Bug #1609463: upgrade-charm command is inconsistent with deploy when using local charms <2.0> <usability> <juju:In Progress by cox-katherine-e> <https://launchpad.net/bugs/1609463>
<katco> anastasiamac_: if you agree, please flag it as such
<anastasiamac_> katco: awesome :)
<katco> anastasiamac_: hello btw
<anastasiamac_> katco: \o/
<anastasiamac_> rogpeppe: wallyworld: do u know if bug 1614010 has already been addressed as part of othr movements in the area?
<mup> Bug #1614010: juju register: cannot register a user when controller already exists <juju:In Progress by rogpeppe> <https://launchpad.net/bugs/1614010>
<wallyworld> what do you mean by fixed? there's disagreement about what's needed. the bug comments explain it a bit
<wallyworld> there was a functionality change. but there's disagreement now whether the change is for the best
<anastasiamac_> wallyworld: k. i guess i was asking whether rogpeppe has done anything in the area one way or another. nm
<wallyworld> not that i know of
<wallyworld> i think it'
<wallyworld> s something we should discuss for 2.0.1
<wallyworld> i sort of agree with roger tbh
<wallyworld> but since there's disagreement, it needs to be hashed out
<anastasiamac_> k
<katco> alexisb: rick_h_: fix landed for bug 1613858... last blocker, yes?
<mup> Bug #1613858: generate-tools ignores 2.x agents and creates bad path <jujuqa> <metadata> <regression> <rteam> <simplestreams> <streams> <juju:Fix Committed by cox-katherine-e> <https://launchpad.net/bugs/1613858>
<alexisb> katco we found another just a minute ago wallyworld has picked up
<alexisb> but  katco thanks for the fix!
<katco> wallyworld: wth! release blocker
<katco> wallyworld: \/
<katco> alexisb: k i'm off to dinner. gl
<alexisb> wallyworld, always causing trouble
<katco> so true
<wallyworld> there's always one more :-)
<alexisb> bye and enjoy
<katco> pfffbt
<wallyworld> wot? me?
<alexisb> redir, I gave you a task on teh release notes
<redir> :)
<alexisb> :)
<alexisb> redir, if you get things down I will review
<redir> k
<alexisb> anastasiamac_, wallyworld ping
<alexisb> redir, ping
<wallyworld> shit sorry, was talkin g to heather
<hml> alexisb: hi - do you have a date for 2.0.1?
#juju-dev 2016-10-13
<mup> Bug #1632530 opened: 1.25.6 cannot deploy a charm to a Yakkety lxc unit (image-stream: daily) <uosci> <juju-core:New> <juju-core 1.25:New> <https://launchpad.net/bugs/1632530>
<wallyworld> menn0: here's that fix for model-defaults https://github.com/juju/juju/pull/6442
<wallyworld> i want to make one more change which i'll do now in a separate pr
<wallyworld> menn0: sorry, one more, follow up to the previous one https://github.com/juju/juju/pull/6443
<menn0> wallyworld: ok
<menn0> wallyworld: review done
<wallyworld> tyvm
<wallyworld> anastasiamac: pretty please? https://github.com/juju/juju/pull/6444
<thumper-cooking> wallyworld: what does bootstrap lxd do?
<thumper-cooking> as long as it is just "lxd" I'm ok
<wallyworld> thumper-cooking: lxd-localhost
<thumper-dogwalk> ick
<wallyworld> as localhost is the region
<wallyworld> i don't mind it tbh
<wallyworld> you are always free to specify controller name
<wallyworld> cause we could lxd lxd not localhost later
<wallyworld> menn0: not sure if you're around? i need  a huge favour - a review of the PR to swap bootstrap args https://github.com/juju/juju/pull/6444 . CI has been updated so I just need to land this and it should be good
<anastasiamac> wallyworld: was school pick-uping.. u always ping around this time... m thinking on purpose? :D
<anastasiamac> wallyworld: looking now
<wallyworld> yay, tyvm
<wallyworld> anastasiamac: i ping because the NZ folks are EOD (with daylight saving) and with other away on leave, there's no one else to bother :-)
<wallyworld> and the europeans aren't on yet
<anastasiamac> wallyworld: reviewed :D
<wallyworld> tyvm
<thumper> wallyworld: perhaps just not add a region if there is only one?
<thumper> anyway
<thumper> I don't really care too much
<wallyworld> thumper: yeah, that could work too. i'm ambivalent about that bit
<wallyworld> thumper: i guess update-clouds could add another region and then you'd be confused if you had already bootstrapped
<thumper> wallyworld: I don't care that much
<thumper> if I did, I could provide a name
<thumper> wallyworld: see ya tomorrow
<dooferlad> mgz: the latest build seems to be a mess. What is the plan? Want me to take a look at anything?
<menn0> wallyworld: I'm here now
<wallyworld> menn0: hey. it got reviewed and landed and is almost through CI
<menn0> wallyworld: excellent
<wallyworld> lots of green :-)
<menn0> wallyworld: I tried to repo the functional-storage failure but can't get assess_storage.py to get past the bootstrap
<menn0> wallyworld: so no local repro yet
<wallyworld> hmmm, is it intermittent? I think so?
<menn0> wallyworld: I fear the functional-storage ain't going to pass as it hasn't in ages
<menn0> wallyworld: http://juju-ci.vapour.ws/job/functional-storage/
<menn0> wallyworld: and it's a strange/alarming one - not being able to find a trusty image on AWS apparently
<menn0> but why only that test?
<wallyworld> yeah, have no idea off hand
<wallyworld> but surely that error is not related to storage functionality
<wallyworld> we probably need to audit the bootstrap set up
<menn0> wallyworld: the bootstrap is pretty conventional. I've checked the bootstrap config and it's all pretty standard stuff.
<menn0> wallyworld: the test fails when deploying a charm which uses storage
<wallyworld> and yet that test's bootstrap fails and others don't
<wallyworld> so bootstrap works
<menn0> wallyworld: bootstrap only failed for me locally so that could be environment
<wallyworld> i'll take a look
<menn0> wallyworld: when the test runs in CI it fails deploying that charm
<menn0> but it's the machine that fails to come up
<wallyworld> ok
<menn0> so before the charm even gets used
<wallyworld> so hard to see then based on that how storage is involved
<menn0> wallyworld: indeed
<menn0> wallyworld: the test also runs create-storage-pool a few times before deploying the charm
<menn0> wallyworld: hard to see why that would matter though
<wallyworld> yeah
<wallyworld> and it seems when the machine does come up, the test pases
<menn0> wallyworld: and when I do roughly what the test does manually, it all works
<wallyworld> yeah, sure does seem like a test artifact
<macgreagoir> dooferlad: fwiw, both the failures in 'run-unit-tests-race/build-1967' look like obvious races on magic numbers in time.
<frankban> wallyworld, rick_h_: trivial https://github.com/juju/juju/pull/6445
<wallyworld> sure
<wallyworld> frankban: doh! i missed that
<wallyworld> thanks!
<frankban> wallyworld: np, it was quite confusing earlier ;-)
<wallyworld> was all done in a bit of a rush
<frankban> shipping it
<frankban> wallyworld: oh you did, thanks
<wallyworld> yeah, keen to see it land
<wallyworld> so we can get another good Ci run
<mgz> dooferlad: seems 4485 is back to something closer to normal again
<frankban> wallyworld: ci merge job failed: not sure if it's spurious: http://juju-ci.vapour.ws:8080/job/github-merge-juju/9504/
<wallyworld> probs is let me look
<wallyworld> yup
<wallyworld> remerge
<frankban> wallyworld: ok done
<mgz> [LOG] 0:10.015 ERROR juju.worker.dependency "metric-sender" manifold worker returned unexpected error: Access is denied.
<mgz> wallyworld: we don't seem to have a bug filed for this?
<wallyworld> i haven't seen one
<mgz> oh, we do
<mgz> bug 1632006
<mup> Bug #1632006: cmd/jujud/agent: sporadic test failure in UnitSuite.TestWorkers <intermittent-failure> <juju:Triaged> <https://launchpad.net/bugs/1632006>
<wallyworld> oh, an intermittent failure
<mgz> ^you can metoo that frankban
<mgz> it is a pretty new one
<perrito666> meh, the docs for https://github.com/juju/loggo are out of date, some of the samples dont work anymore
<natefinch> Given that Go has runnable examples, that's a shame
<rick_h_> natefinch: call?
<natefinch> rick_h_: oops, sorry, coming
<rogpeppe> mgz: hiya
<mgz> rogpeppe: yo
<rogpeppe> mgz: i've been looking at network.ResolveOrDropHostnames (part of poring over PrepareEndpointsForCaching logic yet again) and see that it doesn't preserve address scope information. is that deliberate?
<mgz> it does not sound deliberate
<rogpeppe> mgz: i thought not too. i'll add a TODO and raise a bug.
<mgz> but I don't think the logic there has changed in a good while either
<rogpeppe> mgz: it's only used in one place
<mgz> a TODO and bug sounds good
<rogpeppe> mgz: https://bugs.launchpad.net/juju/+bug/1633089
<mup> Bug #1633089: network: ResolveOrDropHostnames doesn't preserve address meta-information <juju:New> <https://launchpad.net/bugs/1633089>
<mgz> rogpeppe: thanks!
<mgz> dimitern: ^also of interest I think
<dimitern> rogpeppe, mgz: what's the case you're hitting?
<rogpeppe> dimitern: i've just been trying to understand the code, and that logic just seems wrong to me
<dimitern> rogpeppe: Scope is auto-discovered usually, it should only affect tests
<rogpeppe> dimitern: if we already have scope information, why should we throw it away?
<dimitern> rogpeppe: because it might be different?
<rogpeppe> dimitern: why would the scope be different after we've resolved an address?
<dimitern> rogpeppe: e.g. resolving "www.google.com" which was using ScopeCloudLocal (for some reason), will yield multiple
<dimitern> rogpeppe: IPs, with likely different scopes
<rogpeppe> dimitern: if the scope is always derived from the address, why store it in the Address structure at all?
<dimitern> rogpeppe: well, it's not always derived - only when using NewAddress() (which calls NewScopedAddress() with ScopeUnknown)
<rogpeppe> dimitern: well, it's derived when we call ResolveOrDropHostnames - the original info is just thrown away
<dimitern> rogpeppe: and it seems for Type==HostName Scope is preserved as-is
<dimitern> rogpeppe: so, fair enough - ResolveOrDropHostnames() should be
<dimitern> (why I keep pressing enter :/)
<dimitern> rogpeppe: should be preserved
<rogpeppe> dimitern: so for example if we've got a numeric IP address with Scope=CloudLocal, that information will be thrown away unless it's actually an address in the private-ip-address range
<mgz> yeah, that's the wrong bit
<dimitern> rogpeppe: nope, we only try to resolve when Type==HostName (line 154)
<rogpeppe> dimitern: ha, that assumes that Type is set correctly (I think that the Type field should be dropped and made into a method tbh)
<rogpeppe> dimitern: actually, no you're right, it'll work even when Type==""
<dimitern> rogpeppe: well, network.Address and HostPort were always intended to be plain old data structs
<rogpeppe> dimitern: nothing wrong with a plain old data struct with some methods that return derived information
<rogpeppe> dimitern: there's no need to denormalise info
<rogpeppe> dimitern: it just provides more ways for things to go wrong
<rogpeppe> dimitern: do you think there's any way a cloud-local host name can legitimately resolve to a public ip address?
<dimitern> rogpeppe: of course :)
<dimitern> rogpeppe: it's whatever the dns resolver returns
<rogpeppe> dimitern, mgz: hmm, ok, well maybe ResolveOrDropHostnames is correct then.
<dimitern> rogpeppe: but usually, without specially configured DNS it won't happen
<mgz> hm, I don't know of any case that should happen
<dimitern> the (almost) same thing happens if you put "www.google.com  127.0.0.1" in /etc/hosts
<mgz> but we do get hostnames in aws that resolve to differently scoped ip addresses
<mgz> depending on the context of the request
<rogpeppe> dimitern: so i think what you're saying is that for host names, the scope is important because we can't tell any other way, but for IP addresses, we can always derive the scope from the address itself.
<dimitern> rogpeppe: overriding the scope is useful in tests, but it's almost always derived otherwise
<rogpeppe> dimitern: what about when we get the addresses from a provider instance? do providers always derive the scope too
<rogpeppe> ?
<dimitern> rogpeppe, mgz: yeah, good point - split-horizon DNS servers like on AWS can resolve "ip-x-y-z.ec2-compute.internal" to local-cloud address inside the VPC or to the public IP outside of it
<dimitern> rogpeppe: not all of them, some are explicit (vsphere, joyent, a few others)  - but then they don't use NewAddress(), they use NewScopedAddress() or create the Address directly
<rogpeppe> dimitern: an address like that will never resolve to the public ip address outside, will it? 'cos ".internal" isn't a valid TLD.
<rogpeppe> dimitern: ok, right
<rogpeppe> dimitern: so the scope can be important information that's not derived
<dimitern> rogpeppe: it's not a gTLD, but you still *can* change your AWS DNS config to resolve "xx.yy.ec2-compute.internal" to e.g. "8.8.8.8"
<rogpeppe> dimitern: sure
<dimitern> rogpeppe: there was no easy way to figure out if a hostname is public or not - there are a few golang/x/net packages that potentially can be used for such detection, but wasn't needed so far
 * dimitern afk for a bit
<rogpeppe> dimitern: thanks for the help
<rogpeppe> mgz: i closed the bug
<mgz> rogpeppe: gotcha, sorry, some of the subtleties here had gone past me
<rogpeppe> mgz: me too :)
<rogpeppe> dimitern, mgz: i'm thinking that we should get rid of "unresolved-api-endpoints" too, and replace it with "resolved-api-endpoints".
<dimitern> rogpeppe: as long as it doesn't slow down each run of `$ juju ...` +1
<dimitern> rogpeppe: as the reason to have both was to lazy-resolve-when-changed
<rogpeppe> dimitern: well, tbh the DNS resolution thing should be done in a totally different place. it really clutters up that higher level logic.
<rogpeppe> dimitern: anyway, what's wrong with using the usual DNS cache?
<dimitern> rogpeppe: the reason for resolving before trying to connect to the apiserver was mostly to work around broken maas setups
<rogpeppe> dimitern: how does it help there?
<dimitern> rogpeppe: it somewhat does.. not too much though
<rogpeppe> dimitern: how? surely net.Dial does the resolve anyway?
<dimitern> rogpeppe: it works because we have both IPs and hostnames coming from the maas provider
<rogpeppe> dimitern: i'm still not getting it
<rogpeppe> dimitern: (it's very useful to have the conversation now because i'm really really trying hard to simplify the impenetrable logic in PrepareEndpointsForCaching...)
<dimitern> rogpeppe: but (IIRC) if your nameserver(s) in /etc/resolv.conf are misconfigured - e.g. unreachable or worse - blackholed, the first time net.Dial tries to resolve a hostname causes at minimum 3s delay (TCP-stack-related quirk) and some times up to several minutes
<rogpeppe> dimitern: it causes a delay even to a concurrent dial request made to an IP address a moment later?
<dimitern> rogpeppe: I'm actually working on fixing the juju ssh (and later bootstrap) behavior with multiple HPs available to choose from
<dimitern> rogpeppe: and I'm doing this by proactively trying to connect to all HPs in parallel
<dimitern> (unfortunately your parallel.Try didn't work as I expected)
<rogpeppe> dimitern: so, if slow-DNS-resolution is the problem, a better solution would be to implement a DNS cache at the bottom level rather than cluttering up the top level, I think.
<rogpeppe> dimitern: what didn't work as expected about parallel.Try?
<rogpeppe> dimitern: with a DNS cache at the bottom level, we wouldn't need to bother munging the addresses in juju/api.go at all
<dimitern> rogpeppe: I'll try to explain another day :)
<rogpeppe> dimitern: Try was designed to fit just this kinda problem FWIW
 * dimitern really needs to finish that connection checker
<rogpeppe> dimitern: np
<dimitern> rogpeppe: thanks for the discussion though - it's useful and we should chat more about it soon
<dimitern> ;)
<rogpeppe> dimitern: i should have a PR for your delectation before too long
<dimitern> rogpeppe: nice! (learned a new word today ;)
<marcoceppi> uh, what does this mean?
<marcoceppi> juju enable-ha
<marcoceppi> ERROR unsupported with hosted models
<marcoceppi> wat
<rick_h_> marcoceppi: juju switch controller && juju enable-ha ?
<rick_h_> marcoceppi: I thought that was fixed to auto do the right thing though, is this rc3?
<marcoceppi> y
<marcoceppi> yes
<marcoceppi> BLEH
<marcoceppi> switch worked, this is rc3
<rick_h_> marcoceppi: k, maybe it landed post-rc3, but yea models aren't HA, controllers are
 * rick_h_ goes to make sure that was landed post-rc3
<marcoceppi> lame
<rick_h_> marcoceppi: totally, why we have landed a chance to correct that
<marcoceppi> I know models aren't ha ;) I want my controller ha bruv ;)
<marcoceppi> cool, thanks
<rick_h_> totally
<rick_h_> bruh
 * rick_h_ goes to grab lunchables
<natefinch> marcoceppi: yeah, I hate that error message
<natefinch> marcoceppi: also, it shouldn't even be an error... HA only applies to the controller, not models
<natefinch> marcoceppi: so it should just do the right thing.
<marcoceppi> I think an INFO might be better
<rick_h_> marcoceppi: natefinch https://github.com/juju/juju/pull/6421
<natefinch> at the very least it should say "please run this command on the controller model"
<rick_h_> "juju enable-ha now just works without requiring the controller model be selected."
<natefinch> rick_h_: awesome!
<natefinch> man I hated that error message... "hosted model" is just such a confusing term
<marcoceppi> it caught me off guard, someone asked about HA
<marcoceppi> and I was like I WILL SHOW YOU
<marcoceppi> and I couldn't show him :'(
<marcoceppi> but I made up some other bullshit
<marcoceppi> so it's fine
<frobware> dooferlad: ping
<dooferlad> frobware: sorry, about to go out. Super urgent?
<frobware> dooferlad: which PR fixed https://bugs.launchpad.net/juju/+bug/1623480
<mup> Bug #1623480: Cannot resolve own hostname in LXD container <lxd> <network> <juju:Fix Released by dooferlad> <https://launchpad.net/bugs/1623480>
<dooferlad> frobware: sorry, don't have that to hand (on phone). Only two PRs from me in the last week or so - should be findable
<frobware> rick_h_: fyi - https://bugs.launchpad.net/juju/+bug/1633126
<mup> Bug #1633126: can't resolve lxd containers by fqdn <juju:New> <https://launchpad.net/bugs/1633126>
<alexisb> I need a voluteer[told} to help with relesae blocker bug
<alexisb> anyone available?
 * perrito666 sighs
<perrito666> o/
<perrito666> alexisb:
<rick_h_> frobware: :(
<alexisb> I need another voluteer for a blocker bug
<rick_h_> natefinch: ^
<rick_h_> alexisb: what's up?
<alexisb>  https://bugs.launchpad.net/juju/+bug/1626187
<mup> Bug #1626187: metricsManagerSuite.TestMeterStatusSuccessfulSend got false <ci> <intermittent-failure> <regression> <unit-tests> <juju:Triaged by rharding> <https://launchpad.net/bugs/1626187>
<alexisb> ^^^ is no longer intermitten
<alexisb> and minimum we need to show it is a required test update
<natefinch> I can take it
<alexisb> thanks natefinch
<mup> Bug #1632909 opened: ceilometer-api fails to start on xenial-newton (maas + lxd) <maas-provider> <uosci> <OpenStack AODH Charm:New> <juju-core:New> <ceilometer (Juju Charms Collection):New> <https://launchpad.net/bugs/1632909>
<mup> Bug #1632909 changed: ceilometer-api fails to start on xenial-newton (maas + lxd) <maas-provider> <uosci> <OpenStack AODH Charm:New> <juju-core:New> <ceilometer (Juju Charms Collection):New> <https://launchpad.net/bugs/1632909>
<mup> Bug #1632909 opened: ceilometer-api fails to start on xenial-newton (maas + lxd) <maas-provider> <uosci> <OpenStack AODH Charm:New> <juju-core:New> <ceilometer (Juju Charms Collection):New> <https://launchpad.net/bugs/1632909>
<alexisb> natefinch, anything thoughts on the test failure?
<rick_h_> cmars: ping, got a sec?
<rick_h_> cmars: can you pair with natefinch on the https://bugs.launchpad.net/juju/+bug/1626187 failure to help unblock GA?
<mup> Bug #1626187: metricsManagerSuite.TestMeterStatusSuccessfulSend got false <ci> <intermittent-failure> <regression> <unit-tests> <juju:Triaged by natefinch> <https://launchpad.net/bugs/1626187>
<natefinch> rick_h_: yeah, looking through it... I'm sure it's sjust a timing issue.  Might be able to restructure the test so that we don't depend on the time.
<rick_h_> natefinch: k, I just want to make sure that if there's anything in that topic that cmars might be able to help
<natefinch> rick_h_: definitely
<natefinch> need to reboot, brb
<cmars> rick_h_, natefinch sure
<cmars> one sec
<natefinch> cmars: btw, I was able to reproduce that failure if I added a 2 second sleep to the beginning of that test... testing with your fix and a 2 second sleep to make sure it does the right thing (which I'm sure it does, since the mock clock doesn't care about real-world time)
<cmars> natefinch, awesome, that supports my hypothesis about the sequence of events leading to failure
<cmars> natefinch, initial tests have passed, i've $$merge$$'d
<alexisb> ug!  model-defaults is still the old table heading format
<natefinch> cmars: yeah, double checked that the fix works even with a sleep at the beginning of the test. Awesome.
<cmars> natefinch, thanks!
<cmars> natefinch, is the build bot stuck? it doesn't seem to pick up my $$merge$$
<cmars> rick_h_, any idea why this isn't landing? https://github.com/juju/juju/pull/6446
<alexisb> cmars, master is blocked for the release
<cmars> lol
<cmars> release is blocked on this PR
<cmars> so... deadlock? :)
<alexisb> cmars, I spoke with QA and given you have shown it is a test failure they are good wth the release as is
<alexisb> cmars, natefinch thank you!
<natefinch> __JFDI__ ?
<cmars> oh, ok, sure
<redir> brb
<rick_h_> redir: back from school duties. you still heading up from the bottom?
<rick_h_> redir: should I start at the top and work down?
<redir> rick_h_: I started at the top and am working down
<rick_h_> redir: ah ok, should I start at the bottom then?
<redir> sure
<rick_h_> k, /me starts cutting/pasting
<alexisb> rick_h_, redir, I am going to change locations while you cut and paste :)
<alexisb> then finish my audit of the table
<redir> word
<rick_h_> alexisb: rgr safe travels
<redir> i mean ack
<redir> rick_h_: back in HO
<rick_h_> redir: k, omw
<menn0> veebers: are you aware of this: https://docs.google.com/document/d/1sDPlv7AxyPTfaVfh0Yj6Pwr1I_jKOcbh4fQs-yB8GtY/edit
<menn0> veebers: sorry wrong link :)
<menn0> veebers: http://juju-ci.vapour.ws:8080/job/github-merge-juju/9506/console
 * veebers looks
<veebers> menn0: I'll bring it up with the team now
<veebers> menn0: balloons will take care of you :-)
<balloons> menn0, no more PR's against master. But since you made one, I'll let you play guenia pig on the message
<balloons> give me 30 mins, I'll ping you.. otp
<menn0> balloons: I have a PR against master?
<balloons> menn0, ohh I just assumed
<balloons> ahh it is develop.. no worries
<menn0> balloons: and it's not mine, I just noticed it
<balloons> anyway, it's disabled for release still
<menn0> balloons: it's a fix for the only thing that failed in the last CI run :)
<natefinch> I was going to be happy that juju bootstrap lxd works.... except that juju bootstrap lxd && juju show-controller lxd ... does not work *sad trombone*
<alexisb> natefinch, what do you mean?
<alexisb> I have been bootstraping all day
<rick_h_> alexisb: it's called localhost vs lxd
<rick_h_> alexisb: so you can boostrap lxd, but the name will localhost
<alexisb> well yeah unless you rename it
<alexisb> I may be coming into th emiddle of a conversation where I dont have the context
<natefinch> nope, that was basically it
<natefinch> if I juju bootstrap azure... what's will the name of the controller be?  I have no clue.
<perrito666> natefinch: do try, you have credentials for it
<menn0> natefinch: you can always specify the name if you want to be sure
<natefinch> I'm not saying I can't figure it out, I'm saying it's bad UX
<perrito666> menn0: I have OSs for answers like that :p
<natefinch> but I'm also aware that there are Those Who Disagreeâ¢
<wallyworld> natefinch: the controller name is cloud-region
<wallyworld> if you don't want that, just supply the name as was the case before, your choice
<wallyworld> and it tells you what the name of the controller is when it bootstraps, so you would have seen the controller was not called just lxd
<natefinch> wallyworld: I stick by my point that it's bas ux.  I don't know what the default region for every cloud is. I shouldn't have to analyze the mound of output generated from bootstrap to figure out what the controller name is.  And the region is not such critical information that it deserves to go into the name, IMO... especially since it's always the same per cloud.
<natefinch> s/bas/bad
<natefinch> but anyway, I gotta run for dinner.  I know it's not going to change.
<perrito666> dinner? man, I just had afternoon tea and I am one hour ahead of you
<balloons> can I get a +1? https://github.com/juju/juju/pull/6448
<alexisb> I need a volunteer who enjoys reading
<thumper> OMG, a bless
 * thumper hides from alexisb
<alexisb> NICE!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
<alexisb> ok thumper, menn0 you guys need to play rock paper scissors
 * perrito666 throws holy wather at the build
 * thumper just played, and he won
 * thumper hands to menn0
<thumper> :)
<alexisb> I see menn0 is hiding
<menn0> menn0: is writing a carefully craft bike shedding email
<alexisb> menn0, when you have a moment: https://hangouts.google.com/hangouts/_/canonical.com/core
<balloons> wallyworld you are quiet. Can you +1 this to bump the revision number? https://github.com/juju/juju/pull/6448
<wallyworld> sure
<alexisb> menn0, ??
<wallyworld> balloons: lgtm
<bdx> congrats everyone on the release!
<bdx> thanks for all your hard work!
<bdx> party time
<mup> Bug # changed: 1318378, 1328269, 1349854, 1632530
<mup> Bug #1589680 opened: Upgrading to cloud-archive:mitaka breaks lxc creation <canonical-bootstack> <juju-core:Triaged> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1589680>
<mup> Bug #1589680 changed: Upgrading to cloud-archive:mitaka breaks lxc creation <canonical-bootstack> <juju-core:Triaged> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1589680>
<mup> Bug # opened: 1318378, 1328269, 1349854, 1632530
<mup> Bug # changed: 1318378, 1328269, 1349854, 1632530
<mup> Bug #1589680 opened: Upgrading to cloud-archive:mitaka breaks lxc creation <canonical-bootstack> <juju-core:Triaged> <juju-core 1.25:Triaged> <https://launchpad.net/bugs/1589680>
<balloons> cmars, if you're still about, I restarted your merge. Things should be open now
<cmars> balloons, no rush, but thanks. just thought y'all needed it earlier for the release
<balloons> cmars, ack. We had to cut and go with the info we had it
<cmars> balloons, sure, it was just mocks and clocks :)
<menn0> wallyworld: the release notes say that Juju now uses the AWS instance type API directly but that's not quite right is it?
<wallyworld> no
<menn0> wallyworld: it looks like axw introduced some tooling to automatically generated the hardcoded instance types
<menn0> wallyworld: i'll remove that
<wallyworld> ok, ta
<wallyworld> menn0: yes, there's tooling there, but as discussed this week, we want to improve things a bit
<menn0> wallyworld: that's what I thought, hence my surprise at the release notes item
<menn0> fixed now
<menn0> wallyworld: there is an awful lot of stuff in 2.0.
<rick_h_> menn0: my fault
<rick_h_> menn0: asked for real inout buy folks were away so i made it up :p
<wallyworld> menn0: there is a shit tonne
<alexisb> http://www.vermontcountrystore.com/store/jump/productDetail/65022?searchid=7SPFGPLA&feedid=googlenonbrand&product_id=65022-RSF--2X&adpos=1o1&creative=84728444658&device=c&matchtype=&network=g&gclid=CM6mhPTv2M8CFVQHvAodUH4Biw
<alexisb> us version of a 'jumper' - wallyworld ^^
<perrito666> alexisb: I can definitely see wallyworld wearing that
<alexisb> so long as it has a AC/CD logo
<wallyworld> i'll wear it if it comes in purple
<alexisb> menn0, reed you guys done with your read through on release notes?
<alexisb> if so do you want to meet early to go through comments?
<alexisb> then we can be done with them
<redir> alexisb: yes I think so
<redir> and early
<redir> I'll joing
<alexisb> redir, lets just jump on bluejeans
<redir> alexisb: are we done with the release notes?
<alexisb> looks to be done
<alexisb> anastasiamac, you want to meet me in bluejeans
<anastasiamac> alexisb: k
 * thumper heads out for lunch
#juju-dev 2016-10-14
<rick_h_> ty redir for all the help!
<redir> np. I made a joke on standup about joining the docs team. Nobody laughed.
<redir> seriously though, always happy to help with whatever.
<anastasiamac> redir: i did not laugh. i thought it was scary :D
<redir> anastasiamac: :)
<anastasiamac> but seriously - awesome job on release notes! tyvm to all involved \o/
<redir> Use the source, Luke.
<anastasiamac> redir: haha :D this is good. no lightsabers plz
<alexisb> wallyworld, do we need to add a blurp in the "whats new for GA" on new bootstrap command
<alexisb> sequence
 * redir eod
<alexisb> redir, thank you for the release notes help today
<alexisb> I owe you a beer in december
<alexisb> or wine :)
<redir> I'll take the beer or wine:) but it really wasn't a big deal.
<alexisb> wallyworld, ping
<alexisb> thumper, I may need you if wallyworld is mia
<alexisb> menn0, maybe??? someone??
<menn0> alexisb: hello?
<wallyworld> thumper: here now
<wallyworld> alexisb: sorry, was afk getting coffee
<wallyworld> bootstrap is all new in 2.0 anyway, compared with 1.25
<wallyworld> not sure we need anything special for yesterday's change
<thumper> wallyworld: ping
<wallyworld> thumper: hey, just in a call
<thumper> ack
<thumper> I'll idle in the hangout
<wallyworld> thumper: hopefully will be there soon
<alexisb> thumper, wallyworl is occupied
<thumper> :)
<alexisb> feel free to join the party if you want :)
<thumper> where's the party?
<alexisb> wallyworld, thumper you guys need to stay close
<wallyworld> close to each other?
<thumper> hmm...
<alexisb> well you guys do that naturally ;)
<alexisb> close to the channel
<menn0> thumper: https://github.com/juju/juju/pull/6449
<menn0> thumper: that's one of the intermittent failures
 * thumper looks
<menn0> thumper: crap. wrong branch. proposing again.
<thumper> :)
<menn0> thumper: with the new workflow does the bot check if someone's reviewed it?
<alexisb> ah wallyworld did you see my request about new for GA release notes
<wallyworld> alexisb: the bootstrap stuff? did you see my reply?
<alexisb> no
<alexisb> well so something new is new fore GA
<alexisb> and I htink it would be good to note that the this main command as changed
<wallyworld> isn't there info already that bootstrap is new?
<wallyworld> i'll take a look
<alexisb> k
<alexisb> wallyworld, there could be, the notes all blur after multiple days of looking at them
<wallyworld> understood, i'll take a look
<wallyworld> still be held hostage by thumper
<alexisb> ok all, looking for haiku ideas for the release announcement
<wallyworld> alexisb: looks like someone, probs rick, has already edited all the bootstrap examples in the notes
<wallyworld> i like haikus but am bad at making them up
<alexisb> wallyworld, thumper is there  an easy way to get # of bugs fixed since 11/6/2015
<alexisb> or 6/11/2015 if you are not a US persone ;)
<alexisb> from launchpad
<wallyworld> you mean the rest of the world?
<wallyworld> i think if you do a filtered query and specify fix committed, it tells you the count
<wallyworld> in the results
<wallyworld> but we'll need to include juju-core also
<wallyworld> not just juju
<wallyworld> as we switch projects
<alexisb> OK, haiku idea to kick us off:
<alexisb> Juju 2
<alexisb> So Amazing
<alexisb> Models Big Software
<alexisb> F*%$&#% Amazing
<alexisb> Model big software, easy!
<alexisb> Juju 2 is here!
<balloons> alexisb, I used http://reports.vapour.ws/dashboard/juju/all/Critical,High,Medium,Low,Undecided,Wishlist/2015-11-06/2016-10-13
<balloons> giving 845.. not sure if I feel good or bad about that number
<alexisb> seems like it should be higher :)  and yeah
<alexisb> man this timezone is boring
<balloons> ::yawn::
<alexisb> :)
<balloons> ^^ it's a pun.. bored and tired!
<balloons> boring is good though.. Feels less chaotic
<alexisb> yes
<thumper> alexisb: Today we are done, Thank God Juju 2 is out, now where is my drink!
<alexisb> :)
<alexisb> there we go
<alexisb> 2.0, so good
<alexisb> models, controllers, oh may!
<alexisb> big software for all
<cmars> service application
<cmars> model controller switcharoo
<cmars> cant stop juju 2
<balloons> thumper, wallyworld, any thoughts on # of external contributors?
<balloons> or other interesting number nonsense?
<wallyworld> hmmmm
<balloons> at this hour, it's all handwavy
<thumper> more than one
<alexisb> lol
<alexisb> yeah contributers will be a be low given the size of the project
<wallyworld> and complexity
<alexisb> cmars, I love switcharoo
<alexisb> lines of code, maybe??
<balloons> I should do an infographic
<balloons> man, if things weren't so crazy, this would be more fun
<alexisb> :)
<thumper> :)
<thumper> it is friday afternoon and I have run out of fucks to give
<thumper> time to go choke someone
<alexisb> wow
<thumper> BJJ
<thumper> what were you thinking?
<alexisb> trumper ??
<thumper> haha
<alexisb> balloons, I just lost power
<alexisb> it is likely to go out again
<balloons> ohh fun
<alexisb> wallyworld is here to help with announcement review
<alexisb> when you are ready
<alexisb> just in case
<balloons> I was waxing on about my prose
 * wallyworld is waiting with baited breath
<alexisb> :)
<alexisb> so close
<balloons> we are close -- updating mirrors and waiting for packages to appear
<balloons> have at it wallyworld. I keep sounding like an informercial or a corny old man
<balloons> I think it should be light and have a touch of fun in it
<wallyworld> yeah
<balloons> Mostly we're linking to the docs
<balloons> So no need to rehash or sell it
<wallyworld> balloons: i reworded "providers" as there's a push to move away from that term
<wallyworld> best not to poke the hornet's nest
<balloons> semantics semantics!
<balloons> ty, I'll add that to my mental list of banned words
<wallyworld> balloons: when will juju 2.0 drop in the official xenial/trusty archives?
<wallyworld> should we mention that? mybe not?
<balloons> *soon* TM
<balloons> no, don't jinx me please
<balloons> NO promises
<wallyworld> i would imagine most users would not want to use a ppa
<balloons> rc3 is in yakkety.. it's sitting in proposed in xenial
<balloons> it would have been REALLY nice to have that in xenial also
<wallyworld> yeah, it's a shame that we can't issue release notice after juju is properly in the archives
<wallyworld> not just a ppa
<wallyworld> balloons: do we add a link to the official release notes in this announcement?
<wallyworld> i know it's in introducing page
<wallyworld> but that's a step away
<alexisb> wallyworld, I think that requires it to be publised, which will need the docs team
<balloons> hehe.. we ran 180m000 test suites over 2.0
<balloons> wallyworld, we can link to anything that's on the site now. It's what was linked from the PR
<wallyworld> so the introducing link in your doc above goes to https://jujucharms.com/docs/2.0/temp-release-notes
<wallyworld> which is rc3
<wallyworld> so if we issue this notice, people will be directed to old info
<balloons> wallyworld, yep, so it goes
<wallyworld> that doesn't look very professional of us
<balloons> stable ppa has the packages
 * menn0 is done
<alexisb> wallyworld, agreed on that one
<balloons> including precise alexisb.. just for you!
<alexisb> sigh
<wallyworld> how quickly can we get the proper 2.0 release notes up
<wallyworld> and can we delay any notice of release until then
<balloons> alexisb, and precise is gone now ;-)
<alexisb> balloons it is the doc team that does the release notes commit right?
<alexisb> can we send them a note and have someone on teh QA team that is on during EUR hours send the announcement?
<balloons> I believe they publish -- even landing it doesn't do anything
 * alexisb looks at the rc3 notes
<alexisb> rc3 announcement
<wallyworld> balloons: alexisb: also, the instructions for snap say --beta. but this is GA
<wallyworld> people will be very confused IMO
<balloons> yep, we can't go to stable channel without leaving devmode :-(
<balloons> which of course would mean it wouldn't work, so ...
<wallyworld> oh
<balloons> I agree it's confusing though
<wallyworld> everything is awesome
<balloons> and non-stable snaps don't appear in search
<alexisb> yeah in that case I think we are OK though given the juju snap is development
<alexisb> but not have published release notes seems like an issue for announcement
<wallyworld> +1 to that
 * balloons notes this wouldn't be the first time
<alexisb> true
<balloons> did you noticed before? ;-)
<wallyworld> but for 2.0 GA it sort of matters
<alexisb> heh no, but we know how to use it
<wallyworld> 2.0 GA will get a lot of attention and we will be judged on stuff like this. perception and all that
<balloons> honestly, I would expect them to follow along with the site update shortly tomorrow. Outside of the "temp-release-notes", I don't think rc3 is mentioned
<wallyworld> it says RC3 all over the page
<alexisb> balloons, lets get the anouncement sent to the Juju managers (myself, rick, torsten and brett)
<alexisb> as if you would send it to the list
<balloons> wallyworld, if it helps, you are already announced: http://insights.ubuntu.com/2016/10/13/canonical-releases-ubuntu-16-10/
<alexisb> I will take it from there
<wallyworld> and all the bootstrap examples are wrong
<balloons> that is technically true
<wallyworld> balloons: yeah i saw, which is a bit premature
<balloons> never early or late.. precisely on time
<wallyworld> maybe no one reads release notes :-)
<alexisb> obviously we do not
<wallyworld> so those folks that run to get 2.0 will not be told the wrong info about bootstrapping
<wallyworld> and see outdated examples
<mup> Bug #1425669 changed: AllWatcher next internal failure for previously removed service. <api> <canonical-bootstack> <juju-core:Expired> <juju-core 1.25:Won't Fix> <python-jujuclient:Invalid> <https://launchpad.net/bugs/1425669>
<balloons> wallyworld, can you add the command to upgrade from rc. I will mention it
<wallyworld> to the doc? it's just juju upgrade-juju
<wallyworld> assuming the streams are all good
<balloons> wallyworld, if you didn't notice, I asked so I didn't have to think..
<balloons> those braincells are still in yesterday :-)
<balloons> ty!
<wallyworld> :-)
<balloons> ok thumper..
<balloons> wait, did thumper bail?
<balloons> ok, everyone who isn't thumper can party now
<wallyworld> balloons: you did it!
<balloons> wallyworld, WE did it!
<jose> OMGOMGOMG CONGRATULATIONS!
<wallyworld> if it goes well, you did it; if it goes poorly, alexisb did it
<balloons> wallyworld, <3
<alexisb> that is how it works :)
<balloons> I remember those who stayed the course
<balloons> and those who didn't :-)
<wallyworld> been a long road
<balloons> here's to the first hotfix!
<wallyworld> thanks jose, let's hope everything goes well
<alexisb> WOOT!
<jose> seriously, I'm very excited
<balloons> please don't break anything..
<balloons> not tonight anyway
<balloons> preferably not until next week
<balloons> a whole weekend of happiness please
<alexisb> :) thank you balloons for pushing through
<alexisb> jose, happy deploying with 2.0 GA :)
<alexisb> I am off to bed, see y'all later
 * balloons collapses too
<wallyworld> see ya
<jose> alexisb: oh I'm pretty sure all my live demos will just break
 * frobware celebrates 2.0 by upgrading the firmware on his router. I may even by online later... :-O
<dimitern> :D
<perrito666> lol I am celebrating by upgrading ubuntu... it broke
 * macgreagoir started on his family machine, not his work one ;-)
<perrito666> macgreagoir: my family machine is a mac
<macgreagoir> perrito666: Ssshhh!
<perrito666> k, finally end up upgrading with dist-upgrade since the upgrade tool just would not work
<rogpeppe> can we land stuff on master now without fear of breaking the new release?
<perrito666> rogpeppe: you can no longer land stuff in master, at all
<perrito666> rogpeppe: did you see rick email about?
<rogpeppe> perrito666: nope
<rogpeppe> perrito666: which mailing list?
<rogpeppe> perrito666: ah, new developer workflow?
<perrito666> yes
<rogpeppe> perrito666: ah, ok. so we can propose changes. that's good.
<rogpeppe> perrito666: i guess existing PRs will have to be recreated, right?
<perrito666> rogpeppe: sadly yes
<perrito666> mm, now I need to wait for the ppas I use to add y builds
<rick_h_> you can retarget an existing PR against a develop
<rick_h_> don't have to recreate it
<rick_h_> and morning and such
<perrito666> rick_h_: really?
<rick_h_> perrito666: go click the "Edit" button in the upper right
<perrito666> oh look, that actually works
<rick_h_> perrito666: you can change the title of the PR and the commit into branch
<perrito666> and seems develop looks enough like master so it looks decent
<rick_h_> perrito666: yes, all the new workflow branches were recreated from master after 2.0 lock yesterday
<perrito666> I feared it would show as if my branch had a ton of diffs
<perrito666> rick_h_: excelent job
<rick_h_> no, should be good to go
<rick_h_> perrito666: thanks for letting me know it wasn't common knowledge. Emailed the list with the nugget.
<rogpeppe> rick_h_: ah thanks! I never knew it was possible to retarget a PR.
 * perrito666 is  most likely the only juju dev that loves git
<rogpeppe> perrito666: git or github ? :)
<perrito666> git
<perrito666> rogpeppe: I am fond of github for floss projects but not so much of how many people  use it
<perrito666> rogpeppe: you do know you can view and link diffs in gh without making a pr right?
<rogpeppe> perrito666: yes, but that's not quite the same
<perrito666> how so?
<rogpeppe> perrito666: i want it to be a PR because that's what it will be - I'm just not sure that it's exactly the right thing until I actually see it.
<rogpeppe> perrito666: if I just view a diff, then I might make the PR and perhaps I've done it differently from the diffs I viewed.
<perrito666> how is this different form a pr https://github.com/juju/juju/compare/develop?expand=1
<rogpeppe> perrito666: it's not a PR :)
<rogpeppe> perrito666: it can't be commented on for a start
<perrito666> rogpeppe: you just changed your use case ;)
<rogpeppe> perrito666: did I? I explicitly mentioned "demo" in my email.
<rogpeppe> perrito666: there are a few use cases here.
<perrito666> I might be having a es-> en issue here  but demo is just show it
<rogpeppe> perrito666: if I show someone something, it's really useful if they can comment on it too.
<voidspace> anyone know the repo for the CI tests?
<voidspace> perrito666: rogpeppe: dooferlad: frobware: ^^^
<rogpeppe> perrito666: "here's this WIP that I hope to submit tomorrow - it's a bit broken for the moment, but you might want to comment on the API"
<perrito666> voidspace: bzr+ssh://bazaar.launchpad.net/+branch/juju-ci-tools/
<voidspace> perrito666: thanks
<perrito666> voidspace: sorry I dont know the short lp thinguie
<perrito666> rogpeppe: well in that case I understand you can still pr and the tests will run but that is orthogonal to your use of the PR
<perrito666> as a workaround I have on ocasions made a PR against my own master
<voidspace> perrito666: that's fine, i've got it - appreciated
<rogpeppe> perrito666: i think it's reasonable to make the tests triggered manually. they take a bunch of resources and the developer probably knows best about when best to trigger them.
<rogpeppe> perrito666: for example, if I've just updated a comment, there's really no point in running the tests again.
<rick_h_> rogpeppe: but the point is that it's short enough that it's harmless and having it auto means things can't slip by
<rogpeppe> perrito666: but if I've updated something more substantial, then they should.
<rogpeppe> rick_h_: 25 minutes isn't really very short...
<rick_h_> rogpeppe: ideally, sure noticing that, but for the most part it's not worth the effort
<rick_h_> rogpeppe: in the light of a PR, with a review and a QA and such, not horrible
<rogpeppe> rick_h_: particularly if its double that because the test bot started before I noticed a trivial issue
<rogpeppe> rick_h_: personally, I like to make the PR, check that everything looks OK, *then* publish it/run auto tests.
<rogpeppe> rick_h_: are the representative tests run for many PRs concurrently?
<rick_h_> rogpeppe: ok, well apologies it didn't match your personal expectations. Hopefully it's better than it was yesterday and we can smile at progress
<rogpeppe> rick_h_: i'm definitely +1 on having some way to run tests before hitting $$merge$$
<rogpeppe> rick_h_: do the auto-tests run every time the PR is updated?
<rick_h_> rogpeppe: I believe it shuold
<rick_h_> webhooks ftw
<rogpeppe> rick_h_: is there a way to trigger the auto-tests to run again (say if there's a test failure that might or might not be intermittent) ?
<perrito666> can annyone that knows enough bzr tell me how I diff my local copy with lp one?
<voidspace> mgz: ping
<balloons> rogpeppe, there isn't ATM -- no magic words for those tests
<rogpeppe> balloons: ok, thanks
<rogpeppe> perrito666: bzr diff lp:somewhere ?
<balloons> rogpeppe, please collect your feedback after trying it for a bit. We can tweak as we go
<balloons> rogpeppe, running tests on command for instance is certainly doable, so i don't want you to think it's locked in
<rogpeppe> balloons: it would be great if the $$merge$$ 'bot could recognise when it doesn't need to run the tests again because they've been run on the same code already by the try bot
<rick_h_> rogpeppe: but it also has to check that develop has had no other changes int he meantime
<rick_h_> rogpeppe: that's the main reason to rerun
<rogpeppe> rick_h_: that's what i'd expect it would do
<rogpeppe> rick_h_: keep track of whether tests had been run on a given tree hash
<rogpeppe> rick_h_: we could even run a microservice just to keep track of that.
<rick_h_> rogpeppe: yea, sorry. I'm just happy to get this far so bigger fish to fry atm. I completely agree it could get better over time.
<rogpeppe> rick_h_: yeah, it's really nice to get this far.
<rogpeppe> balloons: out of interest, does $$merge$$ run more tests than the auto-try check?
<rick_h_> rogpeppe: no
<rogpeppe> rick_h_: ok, thanks.
<katco> rick_h_: hey i'm going to attend the standup, but i can't talk atm
<rick_h_> katco: k
<rogpeppe> perrito666: ping
<perrito666> rogpeppe: pong
<rogpeppe> perrito666: we've run across a perm checking problem with AllWatcher
<rogpeppe> perrito666: the offending line of code is this: isAuthorized, err := auth.HasPermission(permission.LoginAccess, context.State().ControllerTag())
<rogpeppe> perrito666: in NewAllWatcher
<perrito666> aha, what issue are you getting?
<rogpeppe> perrito666: the question is: why is this checking for login access to the controller in order to be able to watch a model?
<rogpeppe> perrito666: ah, perhaps it's because it's used by both the WatchAll and WatchAllModels API calls?
<perrito666> rogpeppe: it comes from the assumption that users created should have login access (in order for us to be able to also take that permission out)
<rogpeppe> perrito666: but presumably the user wouldn't have been able to create the watcher if they weren't able to log in?
<perrito666> any model user should have controller login
<rogpeppe> perrito666: i'm wondering why the explicit check is there
<rogpeppe> perrito666: that's no longer true FWIW
<rogpeppe> perrito666: even if true, why did we feel the need to make that check there?
<rick_h_> voidspace: frobware ping for standup
<perrito666> rogpeppe: I cant really remember now, it is most likely the translation from the check in place previously or from someone reporting a bug
<rick_h_> dimitern: ping for standup
<dimitern> oops omw
<wallyworld> rogpeppe: all users accessing model indeed need controller login. tim landed that change last week
<rogpeppe> wallyworld: actually they don't
<rogpeppe> wallyworld: we landed a flag that turns off that requirement
<wallyworld> the code says they do unless it has changed in the last week
<rogpeppe> wallyworld: allow-model-users
<wallyworld> right but out of the box it does need it
<rogpeppe> wallyworld: yes, but the code can't make that assumption
<wallyworld> so it seems that check you are talking about doesn't honour that new flag
<rogpeppe> wallyworld: well, i'm wondering why that check exists in the first place
<rogpeppe> wallyworld, perrito666: it was added here http://reviews.vapour.ws/r/5430/diff/5/ (in watcher.go)
<wallyworld> not sure, don't know that bit of code directly
<rogpeppe> perrito666: i don't really understand the comment there
<perrito666> rogpeppe: reading
<voidspace> rick_h_: sorry, omw
<perrito666> rogpeppe: I would venture it is because its used by WatchAll*, the check might need to be in each WatchAll*
<rogpeppe> perrito666: i don't understand
<rogpeppe> perrito666: specifically in the comment i don't understand "this allows us to remove login permission for a user".
<rogpeppe> perrito666: that seems like something relevant to some other endpoint entirely.
<perrito666> rogpeppe: sorry was reading the wrong thing
<rogpeppe> perrito666: :)
<perrito666> rogpeppe: AuthClient seems to return true even if you have removed "login" from the permissions (or that I imply from my own comment)
<perrito666> so if I want to revoke login from you, I want the check to ban you from entering if I do
<rogpeppe> perrito666: but... if you've got access to the AllWatcher facade, you've already logged in by definition, right?
<perrito666> rogpeppe: we might have named this poorly
<rogpeppe> perrito666: named what?
<perrito666> but if you want to be fine grained, if you are there it means you have authenticated, not logged in
<perrito666> rogpeppe: the permission name
<rogpeppe> perrito666: what's the difference between authenticating and logging in?
<perrito666> rogpeppe: I was just trying to make a point, ignore me, what I meant is that we named the basic permission login, but it is a poor name
<rogpeppe> perrito666: it seems a reasonable name to me - without that permission, you can't log in to the controller
<rogpeppe> perrito666: (but, given allow-model-access, you can log in to a model even without that permission on the controller)
<perrito666> rogpeppe: you might then want to change HasPermission to honour "allow-model-access"
<rogpeppe> perrito666: i don't think we should be calling HasPermission in NewAllWatcher at all
<rogpeppe> perrito666: i think AuthClient would be just fine
<perrito666> rogpeppe: you should always be calling haspermission
<rogpeppe> perrito666: why?
<perrito666> rogpeppe: because its the interface endpoint for common.authenticator to tell you if you have the right credentials, ideally all should converge there
<rogpeppe> perrito666: i don't see that any other API facades are checking for controller login access
<rogpeppe> perrito666: and NewAllWatcher is somewhat special - it can only do something by virtue of the fact that watcher has already been created (and permission checks were done then)
<perrito666> rogpeppe: grep -r ".HasPermission" | grep -v _test | grep apiserver | wc -lc
<perrito666>      66    8256
<rogpeppe> perrito666: try grepping for LoginAccess
<perrito666> rogpeppe: if you are asking if the user has any of Login, AddModel, Superuser you are implicitly checking for login access
<rick_h_> natefinch: katco either of you have osx?
<perrito666> now, if you want to state that NewAllWatcher is special, and you can stand by that just change it and add a rich comment about it
<rick_h_> natefinch: katco I need a volunteer to help  sinzui with https://bugs.launchpad.net/juju/+bug/1633495 please
<mup> Bug #1633495: Panic MacOS Sierra <osx> <juju:Triaged> <https://launchpad.net/bugs/1633495>
<rogpeppe> perrito666: but surely when you're watching a model, the right thing to check would be the model perms, not the controller perms, right?
<katco> rick_h_: i do not, sorry
<katco> rick_h_: 100% ubuntu here
<rick_h_> voidspace: ? ^
<sinzui> rick_h_: katco: I am going to attempt to use https://github.com/juju/utils/commit/28c01ec2ad930d41fe5acd9969b96284eb61660b as a patch to unblock hombrew
<natefinch> rick_h_: I have no apples
<frobware> dimitern: let's sync on monday for the juju side of nss-juju
<dimitern> frobware: sure
<katco> sinzui: rick_h_: i think that should be easy to dx/fix though? no hw required other than for the final test?
<rick_h_> katco: yea, just figured it'd be good to get a verify. I thuoght bdx's patch would have taken care of it and it landed pre GA
<voidspace> rick_h_: sorry, I'm 100% ubuntu these days too
<sinzui> katco: indeed no hardware. pass it to me. I am seeing this issue as I prepare the homebrew request
<perrito666> rogpeppe: as I said before, when we assume that a user has Login by default  and we want to use that as a way to block the user from accessing then checkign for that permission makes sense. Remember that until a couple of weeks for a user to do anythiung, Login to controller permissino was a must
<voidspace> rick_h_: perrito666 has a Mac
<rogpeppe> perrito666: when i grepped for '\.HasPermission\(.*Controller', AllWatcher is the only one that seems to be checking controller perms when acting on a model
<rick_h_> katco: can you please help sinzui out and if we need a final verify use sinzui's hardware or perrito666's ?
<perrito666> voidspace: my wife has, but I might be able to check whatever that bug is)
<katco> rick_h_: sure. sinzui ramping up on bug
<rick_h_> ty
<rogpeppe> perrito666: well, actually it changed quite recently
<rogpeppe> perrito666: but anyway, i do think that watchers are special
<rogpeppe> perrito666: because a watcher facade can't do anything at all by itself
<rogpeppe> perrito666: i'll propose a change
<rogpeppe> perrito666: thanks for the useful discussion.
<perrito666> rogpeppe: propose a change, be very verbose on the comments since this overall knowledge is not shared across the team sadly
<perrito666> rogpeppe: sure, just ping me if you need to discuss it more
<rogpeppe> perrito666: https://github.com/juju/juju/pull/6453
<rogpeppe> balloons: y'know what would be really good - if we could abort an auto-test half way through if another update is seen
<rogpeppe> hmm, looks like it runs two jobs concurrently anyway, cool. http://juju-ci.vapour.ws/job/github-check-merge-juju/
<rogpeppe> perrito666: would you be able to review the above PR, please?
<perrito666> rogpeppe: otp will review in a moment
<rogpeppe> perrito666: ta
<rogpeppe> balloons: FYI I just pushed another commit to https://github.com/juju/juju/pull/6453 and it doesn't seem to be running the auto-test job again
<dooferlad> is anyone else having trouble with the magic auto-upload tools stuff not working? I have a different md5sum of jujud on my local machine to the one I just asked for by doing an add-machine...
<rogpeppe> dooferlad: it generally seems to work ok for me, but i haven't checked checksums much
<dimitern> I'd appreciate a review on https://github.com/juju/juju/pull/6454 (finally all seems to be working and tested exhaustively)
<dooferlad> rogpeppe: I thought I was going crazy when some logs didn't turn up that I expected :-|
<rogpeppe> dooferlad: i know the feeling well
<sinzui> katco: I just confirmed that the patch I made from the juju/utils Sierra commit does fix the issue.
<rogpeppe> dooferlad: that's why i wanted there to be a "please make sure you're really uploading local binary" flag to juju bootstrap
<katco> sinzui: cool; i'm working on a comprehensive fix
<sinzui> :)
<dooferlad> rogpeppe: yea, magic is fine until it doesn't work
<katco> sinzui: i had to pull in a newer version of juju/testing and we don't need to panic there anyway
<voidspace> mgz: ping
<mgz> voidspace: yo
<voidspace> mgz: you said I should run the bootstrap (for vsphere) in "the same way CI tests do"
<voidspace> mgz: I have juju-ci-tools and juju-release-tools
<voidspace> mgz: the readme doesn't say how to run tests (is that documented elsewhere) and I can't find the entry point
<voidspace> mgz: I can see a "bootstrap_with_env" function call, but that's from jujupy and I can't see which package provides that
<mgz> voidspace: as in, just look at the ci test on jenkins
<mgz> and literally run that script
<perrito666> rogpeppe: add QA Steps to your PR please
<voidspace> mgz: ah...
<rogpeppe> perrito666: ah yes, will do
<rogpeppe> perrito666: in a call right now; after that
<katco> need very short review of juju/utils which precludes fixing critical 2.0 bug: https://github.com/juju/utils/pull/246
<mgz> voidspace: I'm happy to just go over running a ci test on hangout in like 20 mins?
<mgz> if that helps?
<voidspace> mgz: I'll be going to fetch daughter then
<voidspace> mgz: I'll see how I get on and shout again, maybe sometime after that if you're free?
<mgz> sure, after pickup also fine
<katco> rogpeppe: what do you think of getting rid of that once all together?
<rogpeppe> katco: i'd take a look at where it's used
<katco> rogpeppe: it's used in various places around Juju, but the once functionality should be managed there, not in a lib
<rogpeppe> katco: tbh i'd probably keep it as was and just set it to "" or "unknown" on failure
<rogpeppe> katco: at least initially
<katco> rogpeppe: that sounds good for now
<katco> rogpeppe: see if you like that?
<rogpeppe> katco: when i said "as was" i meant "as it was before the PR"
<katco> rogpeppe: oh, no. i need the ability to not panic
<rogpeppe> katco: no need to panic. just initialise the variable to "unknown" on error.
<voidspace> mgz: where is the juju release build job on CI?
<katco> rogpeppe: that's what the pr is doing
<dimitern> mgz: it seems my PR #6454 pre-check job failed on trusty and lxd for unknown reasons
<dimitern> mgz: do I need to push another commit to re-trigger it?
<rogpeppe> katco: the PR is adding two exported functions
<voidspace> mgz: when I follow the links from reports.vapour.ws for release builds I get 404 and they don't show up on the jenkins dashboard (that I can see)
<katco> rogpeppe: HostSeries was "exported" previously via the global variable
<katco> rogpeppe: i'm making HostSeries not panic, and introducing MustHostSeries for those who want to panic
<mgz> voidspace: log in using the credentials in cloud-city consoles.txt
<rogpeppe> katco: i was thinking we could probably keep the global for the time being
<katco> rogpeppe: why? it serves no purpose
<voidspace> mgz: hah
<voidspace> mgz: ok, thanks
<rogpeppe> katco: easy patching :)
<katco> rogpeppe: hm, on juju we don't allow patching any longer
<katco> rogpeppe: is this used somewhere other than juju?
<natefinch> katco: when did we blanket stop allowing patching in juju?
<katco> natefinch: mm... i think around july? let me check
<katco> natefinch: june 8th
<rogpeppe> katco: ok, without patching you'll need to pass in HostSeries as an explicit dependency everywhere.
<rogpeppe> katco: good luck!
<katco> rogpeppe: that's the idea
<katco> rogpeppe: so, where does this leave us with this PR?
 * rick_h_ goes for lunchables, biab
 * dimitern really thinks we should have an option to say $$retry$$ on a PR when the pre-check job fails
<natefinch> katco: what's the title of the email about patching?  I'm trying to find it, but not seeing anything that looks relevant
<rick_h_> dimitern: there is a command for that. !!build!! i think
<dimitern> rick_h_: oh? trying now.. thanks!
<dimitern> rick_h_: yeah, that worked - it needs to be on its own as a comment though
<rogpeppe> katco: you do what you need to. if you're really going to pass it in explicitly, then you don't need anything more than a function that reads host series and doesn't cache it
<rogpeppe> katco: you can then pass in the series string as the explicit dependency
<katco> natefinch: sorry was otp: https://github.com/juju/juju/wiki/Boring-Techniques#do-not-use-global-variables
<katco> rogpeppe: yeah that's a great point: code at the edges shouldn't have to actively introspect series, just be told
<rogpeppe> katco: then you don't need to worry about caching the value
<katco> rogpeppe: yep, exactly!
<katco> rogpeppe: so sorry, i'm still unclear: what do i do with this PR?
<rogpeppe> katco: i'd change it to just expose HostSeries
<rogpeppe> katco: and dispense with all the globals
<katco> rogpeppe: including the once functionality?
<rogpeppe> katco: yup
<rogpeppe> katco: just a simple function to read the series
<katco> rogpeppe: i am in love
<rogpeppe> katco: :)
<katco> rogpeppe: ok updated PR
<dimitern> ok, so the pre-check job is broken with lxd, it's not just for me
<rogpeppe> katco: i added a few more comments :)
<katco> rogpeppe: ta! just one left to address (logger)
<perrito666> rogpeppe: your patch failed on windows
<katco> rogpeppe: ok, responded w/ new patch
<rogpeppe> katco: i don't understand your link response to "why isn't it concurrent safe?"
<katco> rogpeppe: sorry, i'll add more detail
<rogpeppe> katco: that function isn't called from readSeries
<katco> rogpeppe: indirectly it is
<katco> rogpeppe: readSeries -> updateSeriesVersionsOnce()
<katco> rogpeppe: oops, actually it is direct ;p i was thinking of HostSeries
<rogpeppe> katco: hmm, i see. that seems like a bug to me.
<rogpeppe> katco: readSeries could just call UpdateSeriesVersions
<katco> rogpeppe: i think it probably has something to do with OS's. looks like we have some OS specific files in here
<rogpeppe> katco: but tbh i think that given your policy decision, i think you should remove all the global state
<katco> rogpeppe: that would be ideal, but probably not prudent for this pr
<rogpeppe> katco: well in general in Go, global functions should be safe to call concurrently
<katco> rogpeppe: wow, really? i had never noticed that
<katco> rogpeppe: that's kind of awesome
<rogpeppe> katco: methods on a type are in general *not* safe to call concurrently unless documented as such
<rogpeppe> katco: tbh that whole package is a bit spaghetti-like and could use some love
<katco> rogpeppe: yeah i noticed that...
<katco> rogpeppe: there is a lot of redirection
<rogpeppe> perrito666: i'm pretty sure that windows failure is nothing to do with my change
<katco> rogpeppe: so +1 on pr?
<rogpeppe> katco: if you've made the function concurrent-safe, sure
<katco> rogpeppe: oy... i don't think i can do that in a short amount of time
<katco> rogpeppe: i don't think it was concurrent-safe before
<katco> rogpeppe: e.g. you would get different results depending on whether you were the first in
<rogpeppe> katco: there was no function to call, so it was
<katco> rogpeppe: ?? yes there was... it was exposed through that global variable `HostSeries`
<rogpeppe> katco: ha, no i lie
<rogpeppe> katco: but it *was* concurrency safe
<katco> rogpeppe: i don't think so? it had a RC
<rogpeppe> katco: it used a sync.Once
<katco> rogpeppe: right, so if you were 2nd in but 1st in hadn't completed, you'd get "", nil
<rogpeppe> katco: nope, that's not how sync.Once works
<katco> rogpeppe: ohhh yeah, i forgot subsequent calls block
<rogpeppe> katco: so given that HostSeries is using global state anyway, you could easily just continue to use a global variable.
<rogpeppe> katco: i hadn't realised about all the other global state stuff.
<katco> rogpeppe: yeah, sad state of affairs :(
<rogpeppe> katco: well to be fair, it is all about a global thing
<katco> rogpeppe: leaning towards taking the "once" out of "updateSeriesVersionOnce"
<katco> rogpeppe: wow nm... that really spider-webs out... you're right, everything is global
<rogpeppe> katco: i'm not sure i see the problem - you could just change hostSeries to return "unknown" rather than panicking.
<rogpeppe> katco: tbh i think that would be the better solution until you can fix the whole package
<katco> rogpeppe: ok i'll once again revert back to that
<katco> rogpeppe: do you think it should return an error every time?
<katco> rogpeppe: or just "unknown", err 1st and then "unknown", nil
<mup> Bug #1633554 opened: juju ssh uses old/invalid known_hosts data <juju-core:New> <https://launchpad.net/bugs/1633554>
<katco> rogpeppe: pr updated
<mup> Bug #1633554 changed: juju ssh uses old/invalid known_hosts data <juju-core:New> <https://launchpad.net/bugs/1633554>
<mup> Bug #1633554 opened: juju ssh uses old/invalid known_hosts data <juju-core:New> <https://launchpad.net/bugs/1633554>
<redir> crazy weather here in the easy bay, today.
<katco> rick_h_: what am i doing wrong? https://github.com/juju/juju/pull/6455
<rick_h_> katco: /me looks
<katco> rick_h_: for context, i am successfully rebased off of staging
<katco> rick_h_: no conflicts
<rick_h_> katco: so a conflict with something else on develop?
<rick_h_> katco: so this is a window where someone's landed something on develop, and it's not yet landed on trunk (staging) and that is causing a conflict?
<katco> rick_h_: i think that's the case?
<katco> rick_h_: but it looks like there's PRs included from Sep. 6th?
<rick_h_> katco: oh ic, sec
<katco> rick_h_: unless i'm reading this wrong
<rick_h_> katco: can you update staging from upstream, create a new branch from that, and cherry-pick your commit over to the new branch and see if that's better?
<katco> rick_h_: wait i think i see... my local staging was out of date
<rick_h_> katco: I'm wondering if it was something off between when you started and now?
<katco> rick_h_: which is weird... i just updated that
<katco> rick_h_: i think this is human error on my part
<rick_h_> katco: hmm, ok. If you find it's something too easy to do let me konw so we can adjust/warn folks
<katco> rick_h_: no i guess there was an error updating my local refs that i missed, or i just did something else wrong
<katco> looks like rogpeppe is off for the day; still need a +1 on https://github.com/juju/utils/pull/246
<rick_h_> natefinch: perrito666 can either of you please look at ^ ?
<rick_h_> redir: as well? ^
<perrito666> ill tal
<rick_h_> ty perrito666
<redir> tx perrito666 :)
<katco> ta perrito666
<katco> sinzui: also could use a review on this: https://github.com/juju/juju/pull/6455
<rogpeppe> rogpeppe: sorry, i was pairing getting a critical fix out
<rogpeppe> katco: ^
 * rogpeppe talks to himself once again
<katco> rogpeppe: oh sorry
<rogpeppe> katco: i'll take another look
<katco> rogpeppe: ta
<sinzui> katco: do you need me to build and run n Sierra?
<katco> sinzui: to test the fix, i think so
<sinzui> katco: okay, you fix matches my expectations. I will ping you back in about 15 minutes
<katco> sinzui: ta
<rogpeppe> katco: reviewed
<katco> rogpeppe: good call on the bug#, making those changes. ta for the review
<katco> perrito666: you are off the hook if you like
 * perrito666 unhooks himself
<perrito666> katco: shout if you want the review anyway
<katco> rogpeppe: do you think that should be a single meta bug, or one specific to those lines?
<katco> perrito666: ta
<perrito666> rogpeppe: approved https://github.com/juju/juju/pull/6453
<rogpeppe> katco: dunno :)
<rogpeppe> perrito666: ta!
<katco> rogpeppe: i'll make it a meta-bug for all of utils
<rogpeppe> katco: yeah
<rogpeppe> katco: no global loggers, eh?
<katco> rogpeppe: i think loggers might be OK, but once the framework for injecting things in is established, often it's just as easy to inject a logger
<rogpeppe> katco: injection is just a parameter, right?
<katco> rogpeppe: parameter, or member-variable
<rogpeppe> katco: ok
<katco> rogpeppe: or a type which can retrieve those things
<katco> rogpeppe: have you done anything with IoC before?
<rogpeppe> katco: tbh i've argued for non-global loggers before
<rogpeppe> katco: i think it's worth doing
<rogpeppe> katco: (that was in the context of go-kit)
<katco> rogpeppe: i don't know anything about go-kit, what is it?
<rogpeppe> katco: it might be nicer to do loggo a bit differently if that's what we're doing
<rogpeppe> katco: a set of packages for microservices
<rogpeppe> katco: loggo is oriented around global loggers
<katco> yeah
<rogpeppe> katco: but if it wasn't, i think you'd probably ask for a sub-logger of an existing logger
<rogpeppe> katco: anyway, i'm late, gotta go
<katco> rogpeppe: tc, ta
<sinzui> katco: LGTM
<katco> sinzui: cool... it says tests failed...
<katco> sinzui: but... i don't see any failing tests
<sinzui> katco: indeed I don't see a reson for the failure
<katco> sinzui: can you take that and run with it on your end? i'll go ahead and merge this into develop
<sinzui> katco: I cannot ^ balloons any insight into why the build check failed to unblock katco
<balloons> PR?
<sinzui> katco: balloons: is there a problem with the lxd test? http://juju-ci.vapour.ws/job/github-check-merge-juju/60/console
<balloons> See /var/lib/jenkins/workspace/github-check-merge-juju/artifacts/lxd-err.log
<katco> balloons: ah i see it now... is that anything the code can influence?
<balloons> the LXD test failed..
<katco> balloons: "Failed to copy file. Source: /var/lib/jenkins/cloud-city/jes-homes/merge-juju-lxd/models/cache.yaml Destination: /var/lib/jenkins/workspace/github-check-merge-juju/artifacts/lxd/controller"
<katco> balloons: i don't know what that means
<balloons> katco, that's not the issue here.. It's weird it shows there
<balloons> the lxd.err log is with full debug, so it's hard to parse imho
<katco> balloons: oh, so we look at trusty-out.log for unit test failures, and lxd-err.log for lxd failures?
<katco> balloons: hmmm i see a python exception... what is causing it? "juju --debug expose -m merge-juju-lxd:merge-juju-lxd dummy-sink" ?
<balloons> katco, I was just going to say the same thing
<balloons> that's why it failed
<balloons> now, as to what that error is about, well, let's see
<balloons> katco, ahh timeout. Look how long it was waiting
<balloons> juju --debug expose -m merge-juju-lxd:merge-juju-lxd dummy-sink never did anything
<katco> balloons: ah from the timestamps? 20m or so?
<balloons> yea.
<katco> balloons: so the question is immediately: is this intermittent? wondering if there's anything in the version of this lib i pulled in that would cause this
<balloons> katco, if you think it's spurious I can run again. And if it is, we'll need to investigate
<katco> balloons: well, i already did a $$merge$$ before you popped into the conversation, so i think we'll get our answer in a bit
<balloons> katco, so I kicked it again. We'll see and followup on our end, or you'll know on yours ;-)
<katco> hehe
<balloons> ohh, hheheh
<katco> balloons: sorry about that; got my other data point in sinzui's comment and went with it
<katco> balloons: looks like it timed out again... huh
<balloons> katco, you have another run still going of the pre-check
<balloons> they ran at the same time..
<katco> balloons: ah ok
<katco> balloons: guess we'll see what happens there. otherwise i'm in for some fun debugging
<balloons> katco, so just a datapoint, but looking like it's you ;-)
<katco> balloons: i dunno i'm looking at the changes from 406e7197d0690a3f28c5a147138774eec4c1355e to 28c01ec2ad930d41fe5acd9969b96284eb61660b for this library, and there's like... nothing
<balloons> katco, that gave me an idea -- I think I've solved it
<katco> balloons: in fact, here's the diff: https://github.com/juju/utils/commit/28c01ec2ad930d41fe5acd9969b96284eb61660b
<balloons> katco, it's the dreaded LXD too many container bug. I'll have to apply the sysctl values to this host too
<katco> balloons: oh? what was it?
<katco> ohh
<balloons> yea..
<balloons> sorry about that
<katco> balloons: i.e. too many tests running at once = run into this?
<katco> balloons: and since we're now running tests for every pr...
<katco> kablooey
<balloons> katco, exactly
<balloons> we're at magic number 13.. where things end
<katco> balloons: not even x^2 (shakes head)
<balloons> katco, sending $$merge$$ should land you now. Sorry for the trouble
<katco> balloons: no worries, thanks for the help
<katco> natefinch: https://twitter.com/i/moments/782600001541251073
<natefinch> katco: ha. Goats are the cutest, sorry puppy lovers
<cmars> goats get my votes
<katco> sinzui: your MacOS fix is committed
<sinzui> :)
<balloons> rogpeppe, remember when I said you can't ask for a build.. Seems like I lied
<balloons> rogpeppe, !!.*!! seems to be working
<natefinch> lol, it's like you're swearing at the bot
<balloons> !!ilovebots!!..
<balloons> or just !!build!!
<mup> Bug #1566450 opened: Juju claims not authorized for LXD <bootstrap> <ci> <intermittent-failure> <lxd> <juju-core:Triaged> <https://launchpad.net/bugs/1566450>
<natefinch> rick_h_: in your bug, you say "Accounts should only be available to the admin of the controller and not for anyone that can read the data", but the only account it shows is the one you're logged in as, so I presume it's ok to show your own account and ACLs.
<mup> Bug #1566450 changed: Juju claims not authorized for LXD <bootstrap> <ci> <intermittent-failure> <lxd> <juju-core:Triaged> <https://launchpad.net/bugs/1566450>
<mup> Bug #1566450 opened: Juju claims not authorized for LXD <bootstrap> <ci> <intermittent-failure> <lxd> <juju-core:Triaged> <https://launchpad.net/bugs/1566450>
<rick_h_> natefinch: +1
<natefinch> rick_h_: actually, it looks like non-admins can't see any controller details at all, which is kind of silly.
 * redir lunches
<perrito666> natefinch: yup that was by design
<natefinch> perrito666: most of it is information the client already has.... like api endpoint, list of models the user has read access to, what account you're using to connect to it, what your curent model is, etc
<natefinch> perrito666: the only information that definitely must be hidden is the ca-cert and models the user doesn't have read access to.  I don't think showing the cloud type or region is going to hurt anything
<natefinch> perrito666: also, just returning empty json is really ugly.
<perrito666> natefinch: in which context?
<perrito666> to users?
<perrito666> in that case any form of json is ugly :p
<natefinch> perrito666: I guess show-controller always outputs yaml, so empty json is fine.... we just should have had a tabular output that outputs a nice message when you don't have rights to see the controller info
<natefinch> rick_h_: so the long and the short of it is, I'm not sure what to do with this bug.  I think it's sort of a "it's already done slash not applicable" unless we want to change how the whole thing works to show some data to non-admins
<hackedbellini> hey! I'm trying to setup juju 2.0 with lxd/zfs following this: https://jujucharms.com/docs/stable/getting-started
<hackedbellini> When I run bootstrap, everything seems to go right. But at some point it fails in this line: 2016-10-14 21:53:20 DEBUG juju.tools.lxdclient client.go:199 connecting to LXD remote "remote": "192.168.99.4:8443"
<hackedbellini> that because 192.168.99.4 is not the machine that I'm running it (the machine is 192.168.99.3). That ip is from the broadcast machine
<hackedbellini> I don't know from where it is getting that config. Do someone have a clue on what is going on here?
<hackedbellini> it is something like this comment: https://bugs.launchpad.net/juju/+bug/1547268/comments/19
<mup> Bug #1547268: Can't bootstrap environment after latest lxd upgrade   <2.0-count> <juju:Triaged by rharding> <https://launchpad.net/bugs/1547268>
<hackedbellini> note that it is doing a GET on the gateway ip and not on the bridge address
<hackedbellini> anyone?
#juju-dev 2016-10-15
<redir> hackedbellini: how did you setup lxd?
<redir> more specifically how did you setup the network bridge for lxd?
 * redir eow
<hackedbellini> redir: here are my lxd-bridge and network/interfaces file:
<hackedbellini> network/interfaces: http://pastebin.ubuntu.com/23326622/
<hackedbellini> lxd-bridge: http://pastebin.ubuntu.com/23326614/
<hackedbellini> and here is the full output of the bootstrap command: http://pastebin.ubuntu.com/23326644/
<hackedbellini> the strange part is that it tried to "Get https://192.168.99.4:8443/1.0". But the 192.168.99.4 is another computer (the gateway). The computer where I'm running the bootstrap has an ip of 192.168.99.3, so I would expect that it tried to "Get https://192.168.99.3:8443/1.0" instead. I don't know where he is taking that ip from... The only place it is defined
<hackedbellini> is on the gateway of the br0 interface
<redir> hackedbellini: I was hoping it was a simple you need to run dpkg-reconfigure -p medium lxd answer
<redir> but it looks like you've done manual network setup on your system
<redir> I have to run, becasue I have dinner plans an won't be back for a while.
<redir> but LXD worked for me just running the reconfigure command and restarting the lxd services.
<redir> no network twiddling needed.
<redir> but I'm guessing you're trying to run lxd hosts on the same network as the host sysytem and gateway, rather than on it's own subnet.
<redir> good luck
#juju-dev 2016-10-16
<mup> Bug #1312013 changed: "cannot allocate memory" error when running `juju authorised-keys` <authorised-keys> <juju-core:Expired> <juju-core 1.25:Won't Fix> <https://launchpad.net/bugs/1312013>
<thumper> ugh
<thumper> need more coffee
<menn0> thumper: https://github.com/juju/juju/pull/6456 pls
#juju-dev 2017-10-09
 * babbageclunk goes for a run
#juju-dev 2017-10-10
<rick_h> wallyworld: ping around?
<wallyworld> rick_h: hey
<rick_h> wallyworld: nvm, I'm a moron. I was all focused on adding network-get to charm helpers I miss read that what I really need in the charm is relation-get
<rick_h> wallyworld: so I got it all working and realized...hmm...the address I need isn't in this new network-get output
<rick_h> wallyworld: updating to use relation-get now instead and will test out. Is that backward compatible with pre-2.3?
<wallyworld> rick_h: relation-get still contains the old private-address value, yes
<wallyworld> juju 2.3 will put in the new stuff also
<rick_h> wallyworld: gotcha, so 2.3 is ingress-address and if that's not there fall back to private-address for backwards compat
<wallyworld> yep
<wallyworld> so you want to get what the remote unit's address is?
<wallyworld> as opposed to using network-get to see the local unit's info?
<wallyworld> rick_h: I ask because i thought the prometheus charm was using unit-get
<wallyworld> hence the replacement for that is network-get
<rick_h> wallyworld: it's using: hostname = relation_data.get('hostname')
<rick_h> wallyworld: so I'm updating that line to build the newer address details
<wallyworld> ok. i also recall unit-get being used
<wallyworld> somewhere in a hook
<rick_h> wallyworld: yes, in the address of prometheus itself
<wallyworld> that should use network-get
<rick_h> wallyworld: so that'll need to be setup so that it's listening/sending the right info to things that relate to it (e.g. grafana)
<rick_h> wallyworld: right
<wallyworld> so your network-get charm helper work is still needed
<rick_h> wallyworld: so the work on network-get is not in vain, but atm just trying to get the telegraf->prometheus part working and that's the the relation-get update
<wallyworld> ok
<rick_h> wallyworld: anyway, basically I freaked out and ping'd and now I'm unfreaked out and on to working more. sorry for the noise
<wallyworld> rick_h: no worries, sorry i didn't see stuff earlier, was in sessions etc
<rick_h> wallyworld: completely figured as much. All good
<wallyworld> am excited to see this working
<rick_h> bdx: is as well. He was asking about it and is doing some demo material for work around it
<wallyworld> great
<rick_h> wallyworld: bdx has some questions on what gets returned when used in a vpc and such
<rick_h> so once I get the charms behaving I'll ask him to do some vpc testing and make sure things are looking cool
<wallyworld> ok. i can't tell you off hand what is returned. cmr just reads the public address recorded against the machine
<wallyworld> whatever that is
<rick_h> wallyworld: right, so there might be bugs filed because if he binds against vpc specific networks he won't want his stuff on the public network. His vpc is making sure different models can speak to each other
<wallyworld> right. that's not something we support yet
<rick_h> ok, good to know. bdx will love to test it out when it does :)
<wallyworld> not sure how much time we'll have this cycle
<wallyworld> we were told to cut scope to just support public addresses for now
<rick_h> k
<wallyworld> if there's a driver to do the next step, we can look at what we can do
<rick_h> well still, it'll be good to have bugs on the books with repro steps and users wanting things
<wallyworld> exactly
<bdx> theres a driver!
<bdx> me
<bdx> the public address stuff ..... I don't even have a use for (not that others won't)
<bdx> but like
<zeestrat> Count me in for use cases with non-public addresses.
<bdx> even the MAAS <-> AWS stuff I have planned all happens over VPG to our vpc
<bdx> so the MAAS instances have no idea the AWS instances aren't local
<bdx> well ok
<bdx> so the majority of the aws instances I will be connecting to don't have public ips
<bdx> they are all in private NAT subnets that dont get public ips
<bdx> so, I don't think I even have to worry about it for the most part
<bdx> because juju will just think (for aws instances in NAT subnets) the public ip is the private ip, because it will be the only ip
<bdx> wallyworld, rick_h: from what I can see, I will encounter an issue when I want to make a CMR from an instance with both public and private ips (AWS instance in IGW subnet that gets public ip auto assigned)
<bdx> but even then, I can just use "--via" right?
<rick_h> bdx: yea, we'll have to play with it and put together notes on what's there what's next.
<bdx> kk
<rick_h> bdx: so you can on the relate command, but what the charm reads from the hooks...
<bdx> ahh
<rick_h> bdx: it might be that it'll be more maas/openstack friendly at the start. JAAS has some work to support it as well.
<wallyworld> i think if there's no public address recorded it will fall back to private
<wallyworld> so long as those are routable between models it should work
<bdx> wallyworld: I can deal with that
<bdx> awesome
<wallyworld> *should*
<wallyworld> :-)
<bdx> wallyworld, rick_h, jpk: having an issue with standard lxd deploys to instances using aws provider and 2.3beta2, getting http://paste.ubuntu.com/25715842/
<bdx> ahhhh
<bdx> AWS isn't a supported provider for the default FAN type probably I bet
<bdx> per the release notes "On AWS, FAN works out of the box"
<bdx> :(
<wallyworld> bdx: sorry, in meeting, will respond soon if i can
<bdx> np
<bdx> thx
<jam> bdx: please file a bug, I have the feeling it is a problem with the interaction between a defined space and fan configuration
<jam> since 'common-nat' isn't a space name that Juju would have set up
<jam> so I'm guessing you added it
<bdx> jam: yeah
<bdx> https://bugs.launchpad.net/juju/+bug/1722661
<mup> Bug #1722661: FAN - unable to setup network <juju:New> <https://launchpad.net/bugs/1722661>
<jam> bdx: are you trying to test out fan, or are you wanting to just have a container behind nat?
<bdx> jam: I was just taking 2.3 for a test drive .... thought I would start out with testing what happens when I type in something familiar
<bdx> :)
<bdx> I like what I see for the non-space-constraint instance though
<bdx> jam: I wonder if it has anything to do with the fact that my common-nat space is a nat subnet
<jam> bdx: sounds possible
<bdx> the ips that the containers get from the provider are basically elastic ips right?
<bdx> I see
<bdx> so, this is a limitation then?
<bdx> the limitation being that provider type FAN networking for AWS is limited to subnets with routing tables that use an IGW
#juju-dev 2017-10-11
<axw> hml: will be a couple mins
<hml> axw: ack
<bdx> heyyyyy, some network-get findings
<bdx> juju status --format yaml <- http://paste.ubuntu.com/25722803/
<bdx> ok, now check this out
<bdx> http://paste.ubuntu.com/25722809/
<bdx> no public address at all
<bdx> all I'm doing is logging the dict returned by the new network_get() addition
<bdx> there doesn't seeem to be a public ip address anywhere in there
<bdx> jam, wpk, wallyworld, rick_h: ^
<jam> bdx: are you in a cross model relation?
<bdx> jam, no
<jam> otherwise I wouldn't expect to give a public address
<bdx> ahhh
<bdx> so juju chooses for you
<jam> cause everyone who wants to talk to you should talk to you on the private addresses
<bdx> what you get
<bdx> got it
<bdx> jam: http://paste.ubuntu.com/25722894/
<bdx> ^ thats a cross-model
<bdx> looks nice, only one comment
<bdx> possibly dedup the addresses list
<bdx> 'bind-addresses': [{'addresses': [{'cidr': '172.31.48.0/20', 'address': '172.31.51.59'}, {'cidr': '172.31.48.0/20', 'address': '172.31.51.59'}],
<bdx> :)
<bdx> other then that, super!
<bdx> so it looks like I can just use bind-addresses to get my private address to feed to my relation data then
<jam> bdx: can you file a bug about us giving the same address multiple times?
<jam> I don't know where that is coming from but definitely looks like a bug
<bdx> yea
<bdx> https://bugs.launchpad.net/juju/+bug/1722958
<mup> Bug #1722958: network-get: dedup bind-addresses addresses  <juju:New> <https://launchpad.net/bugs/1722958>
#juju-dev 2017-10-12
<mup> Bug #1723184 opened: Multiple units told they are leaders <juju-core:New> <https://launchpad.net/bugs/1723184>
<mup> Bug #1723184 changed: Multiple units told they are leaders <juju-core:New> <https://launchpad.net/bugs/1723184>
<mup> Bug #1723184 opened: Multiple units told they are leaders <juju-core:New> <https://launchpad.net/bugs/1723184>
<redir> how was NY?well coughin
<redir> wah!
<axw> externalreality: standup?
#juju-dev 2017-10-13
<mark-dickie> Hello all! I'm quite new to juju and am writing a charm which utilises layer:snap but it never seems to actually install the snap. Would this be the right place to talk to someone about that?
<rick_h> mark-dickie: I'd stick with the main #juju channel. This is more on the back end code services vs the charming itself
<mark-dickie> rick_h: Thanks for the tip. I'll go look there.
#juju-dev 2017-10-14
<Ichbins_> Ichbins, value?
<Ichbins_> Ich hab ich
